Sustainability-oriented evaluation metrics ofer a means to assess the quality of configuration systems beyond conventional metrics such as accuracy of personalized configurations or sales-related conversion rates. Aligned with the United Nations' Sustainable Development Goals (SDGs), these metrics enable a structured analysis of the environmental, social, and economic impacts of configuration systems. In this paper, we explore sustainabilityfocused evaluation metrics tailored to configurators and examine applications and implications.
Environmental sustainability metrics extend the evaluation of configuration systems beyond traditional performance-based measures by assessing their contribution to environmental objectives.
The carbon footprint of a set of configurations ℱ proposed to users over a specific time period measures the average greenhouse gas emissions (expressed, e.g., in tons of 2 (equivalent) produced over the full configuration lifecycle) associated with the configurations ∈ ℱ . Let CarF( ) denote the estimated overall carbon footprint of components in . Then, the average carbon footprint of ofered configurations (AvgCarFConf) can be defined as follows: 1
∑︁ | ℱ | conf∈ ℱ AvgCarFConf =
CarF(conf)
(
In this context, lower values of AvgCarFConf indicate that the configuration system tends to favor components with lower carbon footprints.
Energy consumption of configuration (i.e., the generation of configurations) refers to the energy required to compute configurations ( ∈ ℱ ) for users of a configuration system. Let configuration denote the total energy (e.g., in kilowatt-hours) consumed by the configuration system over a deifned evaluation period, and let conf be the total number of configurations generated. The Energy Consumption per Configuration (ECConf) can be defined as follows:
ECConf = Configuration conf
This metric is particularly relevant for large-scale configuration systems that involve complex reasoning and consistency management, where an optimization for energy eficiency is important.
Energy consumption of configuration model building refers to the total energy consumed during the construction of a configuration knowledge base. Let dev represent the cumulative energy consumed throughout the entire configuration model development process, and let versions denote the number of developed configuration model versions over a specific time period. The Energy Consumption per Model Version (ECModVer) can be defined as follows:
ECModVersion = dev versions
Such metrics are particularly relevant for evaluating the environmental and computational eficiency of diferent configuration environments.
Energy Savings through Configuration (ESTConf) refers to the reduction in energy consumption or resource usage achieved as a result of applying a configuration system [ 13]. Configuration systems can enhance eficiency by guiding users toward environmentally friendly component selections, reducing over-dimensioning, or optimizing system designs. Let baseline denote, for example, the energy consumption of a system (e.g., annual energy usage) designed without the support of a configuration system, and let withconf represent the energy consumption observed when a configuration system has been used (e.g., a configuration tool supporting energy-eficient component selection). Related energy savings can be expressed as follows:
ESTConf = baseline − withconf
baseline
(
Such metrics are particularly relevant in domains where configuration decisions can significantly influence consumption patterns [10, 13].
Environmental sustainability metrics reflect a paradigm shift from short-term optimization goals—such as maximizing user engagement—toward long-term ecological considerations. However, implementing these metrics in practice presents challenges such as limited access to reliable carbon footprint data and the lack of standardized definitions for the sustainability of components.
Social sustainability in configuration systems emphasizes the fair and inclusive generation of configurations. These metrics go beyond traditional performance measures to ensure that the system’s design and resulting configurations support social equity, accessibility, and community well-being. In this context, configuration processes should avoid bias, promote inclusive component ( ∈ ) selection, and ensure that all users can efectively engage with and benefit from the configuration system.
Related metrics can be used to assess whether configuration outcomes (components) are equitably distributed across diferent demographic groups. A commonly discussed fairness criterion is demographic parity, which requires that the share of selected components ( ∈ ) in generated configurations is similar across sensitive attributes (e.g., gender, age group). Let denote a set of demographic groups, and let () represent the probability that component is included in a configuration for users belonging to group ∈ . In this context, demographic parity is satisfied if the following condition holds:
ACC = ∑︀∈ (, , )
|| (comp) ≈ ′ (comp) ∀ , ′ ∈ ( ̸= ′)
To promote exposure to diverse perspectives, diversity in a configuration list ( ) presented to a user
is crucial. Configuration diversity ConfDiv for a user can be measured, for example, based on the
average pairwise similarity (sim) among configurations ({ , } ⊆ , ̸= ) presented to user
where similarity values (between two configurations) are assumed to be in the interval (
|| × (|| − 1)
In this context, sim(conf, conf ) denotes the similarity between configurations conf and conf (e.g., based on component equality). Higher values of ConfDiv indicate greater diversity in which reflects a lower average similarity among configurations. The metric ConfDiv would then represent the average calculated over all user-specific values (ConfDiv ).
Accessibility aims to ensure that both, configurators and configurations can be efectively used by users
with diverse abilities and backgrounds, represented by diferent groups ∈ . Let denote a set of
accessibility criteria (e.g., understandability of configuration steps, clarity of component descriptions, or
usability for users with visual or cognitive impairments), and let be a function that measures the
extent to which the components and/or user interface elements in a set satisfy these criteria for a
group on a scale from 0 to 1. Then, the accessibility score () for a group ∈ can be defined as
follows:
(
A higher value of ACC indicates better accessibility for group . Inclusivity is considered to be achieved when accessibility is fulfilled equitably across diferent demographic or ability-based groups ∈ :
ACC() ≈
ACC( )
∀ , ∈ ( ̸= )
(
Health Improvement through Configuration (HIConf) refers to the enhancement of individual or population health outcomes achieved through the use of configuration systems. Configuration systems can support healthier decisions by guiding users toward appropriate selections of components related, for example, to diet plans and training plans.
Let ℳwith denote a health outcome metric (e.g., average activity level or body mass index) for users of a configuration system, and let ℳwithout represent the same metric for users not using such a system. The health improvement across all users can be defined as:
HIConf = ℳwith − ℳ without (
ℳwithout
This metric is particularly relevant in application domains such as digital health platforms, wellness configurators, and preventive healthcare services, where personalized configurations can positively impact well-being [14, 15].
Economic sustainability metrics assess the role of configuration systems in fostering inclusive, resilient, and locally grounded economic ecosystems. These metrics extend traditional evaluation dimensions by considering how configuration systems influence market fairness and the visibility of small and/or local component suppliers. Such metrics help evaluate whether configurations promote equitable access to market opportunities and support economically sustainable choices.
This metric quantifies the proportion of components in the configuration knowledge base that originate from small or local businesses. Let ⊆ ℳ denote the components from local or small-scale providers available to user . The Local Business Promotion Rate (LBPR) can be defined as: LBPR = ∑︀∈ |{comp ∈ ℳ : comp ∈ }| | |
Higher LBPR values indicate that the configurator supports community-level economic development by promoting components supplied by small and/or local businesses.
Configuration systems can inadvertently concentrate exposure and revenue on a smaller subset of producers supplying specific types of components (i.e., we regard fairness in exposure as a contextdependent metric). The reasons behind could be, for example, specific variable (value) orderings specified in the underlying constraint solver. To foster economic fairness, we define fairness in the context of component producer exposure as Component Producer Exposure Fairness (CPEF):
CPEF = 2()
2()
(
In this context, denotes producers associated with components appearing in user configurations. The term 2() represents the average pairwise distance in exposure counts between two producers, while 2() is the maximum observed distance between any two producers in . Both are calculated based on how often a producer’s components are presented to users across all configurations.
The discussed economic sustainability metrics can ofer valuable insights into how configuration systems influence the distribution of economic value. These metrics help to evaluate whether a system promotes equitable market exposure and supports small or local businesses.
Cross-cutting sustainability metrics capture and assess the multifaceted efects of configurators spanning environmental, social, and economic aspects.
This metric evaluates sustainability-related user interaction behavior. Let represent the set (more
precisely, the bag) of user behaviors over a specific time period (of user ) when interacting with
the configurator (e.g., inspecting component details, reading explanations, or selecting a component).
Furthermore, let be a set of sustainable behaviors (e.g., selecting eco-friendly components). The
Sustainable Configuration Behavior Score (SCBS) can be defined as:
(
∑︁ interpret() | | ∈ |ℰ| ∈ℰ
Interpretability may be estimated on the basis of explicit user feedback, information complexity scores, or automated assessments (e.g., using large language models).
Life cycle impact analysis considers both, upstream and downstream efects in the production, distribution, usage, and disposal of configurations. Let LCIC(comp) denote the total estimated life cycle impact score for component comp (including aspects such as carbon footprint or the potential for reuse and recycling). The Average Life Cycle Impact of Configurations (AvgLCIC) can be defined as: AvgLCIC = ∑︀conf∈ ℱ ∑︀comp∈conf LCIC(comp)
∑︀conf∈ ℱ |conf|
Lower values of AvgLCIC indicate that the configuration system favors components with lower ecological and social burdens throughout their life cycles.
The deployment of such cross-cutting sustainability metrics also depends on the availability and reliability of life cycle metadata for the involved components.
SCBS = ∑︀∈ |∑{︀∈∈|:| ∈ }|
Higher SCBS values indicate a higher degree of sustainability-related user interaction behaviors.
Interpretability (IntCS) of configuration support is essential for enabling informed user decision-making. Let ℰ denote the set of explanations provided to user over a specific time period (e.g., justifications for selecting a particular component comp), and let interpret() quantify the interpretability of explanation . The Average Explanation Interpretability (IntCS) across all users can be defined as:
Despite growing interest in sustainability-aware configuration [ 6, 10, 11, 12], several challenges hinder a widespread adoption and evaluation.
The incorporation of sustainability goals into configuration systems often introduces trade-ofs between traditional performance metrics (e.g., e.g., accuracy with regard to recommended/included components of a configuration) and sustainability-related outcomes. Formally, this leads to the optimization of a vector-valued objective function: max
FOPT( ) = [Accuracy( ), Sustainability( )]
(
Most sustainability metrics rely on fine-grained metadata, such as the carbon footprint of a component, ethical sourcing labels, or the classification of vendors (e.g., local or small-scale). Let ℳ be the set of components available in the configuration catalog, and let comp be a binary sustainability label for a component comp ∈ ℳ. The share of labeled components is defined as: CLabelCov = |{comp ∈ ℳ : comp is known}|
|ℳ| Low CLabelCov values limit the applicability and accuracy of sustainability-related evaluations. (16)
Developers have to integrate sustainability indicators such as carbon footprint or component origin into evaluation workflows of the configuration environment. This includes activities such as extending existing logging frameworks with the goal to capture relevant data such as energy usage, component source information, and demographic data of users. In addition, configuration algorithms can be enhanced to prioritize sustainable choices, for example, by applying “green component variable value ordering” that favor environmentally friendly or ethically produced components. Beyond implementation, companies have the opportunity to promote transparency by reporting the sustainability performance of their configuration engines.
Sustainability-oriented evaluation metrics are essential for advancing configuration systems beyond conventional performance criteria. By embedding environmental, social, and economic considerations into system assessment, these metrics help to align the development of configuration systems with global sustainability goals, particularly those outlined in the United Nations Sustainable Development Goals (SDGs). Configuration systems have the potential to promote eco-friendly component selection, ensure equitable access to configurable solutions, and support local and responsible value chains. With this, these systems can contribute to more sustainable forms of product customization. A central focus of our future work will be to provide software components that will support the application of our proposed metrics in real-world contexts.
While preparing this work, the author(s) used ChatGPT-4 (GPT-4-turbo) and Grammarly to check grammar and spelling and improve formulations. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content.