Bias and other social responsibility challenges in AI arise from both the underlying Machine Learning (ML) model and the context in which it is used. AI systems, due to inherent model biases, can propagate these issues at scale, affecting numerous user applications [6] [15] [23] [24]. As AI systems are increasingly deployed for critical tasks, concerns about safety and security also escalate.
ML system design involves multiple stages, each with multiple decision points, and iterative cycles, including data gathering, feature engineering, ML model training, deployment, and user output. Responsible AI is an approach within ML-based that integrates fairness, transparency, and ethical considerations at each stage, ensuring that decisions are evaluated not only for technical effectiveness but also for their societal and ethical impact.
Supporting decision-making in iterative ML and Responsible AI cycles is crucial for refining models and improving accuracy by quickly identifying and addressing issues like overfitting or data drift. It ensures that each iteration adds value, ultimately leading to more reliable and robust outcomes. Goal-oriented conceptual modeling is a well-established technique to support systematic decision-making processes [2] [7] [8]. This approach argues that the rationale for system development lies outside the system itself, in the enterprise context. It enables modelers to evaluate goal satisfaction, compare design alternatives, inform requirements, validate design reasoning, and facilitate communication. Through goal refinement, business and Responsible AI goals are broken down into sub-goals and alternative tasks that can achieve goals. Quality objectives are treated as softgoals.
As goals are operationalized in terms of tasks, current goal modeling approaches do not consider how each task alternative may contribute differently to various softgoals, across iterative cycles. To deal with this problem, this paper proposes 3 new goal modeling constructs and examines how they aid in analyzing tradeoffs and conflicts in Responsible AI that are distributed throughout the ML lifecycle. Design decisions at each stage interact and contribute to goals at different stages, with issues like concept drift causing ML processes to evolve over time. Recognizing where tradeoffs occur is crucial, as focusing solely on technical decision points can lead to oversimplified solutions that do not meet other objectives. We explore how tradeoffs at the operationalization level can eventually impact those at the business level, emphasizing the importance of modeling processes and decisions. To illustrate relevant aspects of Responsible AI and their tradeoffs, we draw upon the literature [4] [22]. We use these sources to demonstrate the challenges of dealing with iterative cycles in ML model development and how our proposed Goal Modeling approach can help. We focus on Explainability, Fairness, Privacy, and Accuracy to analyze and demonstrate conflicts at different ML process stages. Our main contribution is enhancing current goal modeling approaches to improve decision-making in Responsible AI design by addressing conflicts between goals in iterative ML cycles.
In Section 2, we first consider tradeoffs between business goals and technical ML goals. Then in Section 3, we introduce the proposed goal modeling notation. In Section 4, we apply the modeling notation to deal with tradeoffs between responsible AI and technical ML goals in various stages of ML development. We discuss related work in Section 5 and conclude with future work in Section 6.
In the design of ML-based systems, technical ML objectives can often conflict with business objectives due to competing priorities, leading to tradeoffs [27]. For instance, in the development of a customer feedback system, a technical ML objective might be to prioritize the accuracy of sentiment analysis, which could be achieved by using a Support Vector Machine (SVM) algorithm that maximizes the margin between positive and negative feedback [28]. However, this could conflict with the business objective of improving the ease for customers to provide feedback, as the SVM algorithm might require a large amount of labeled training data to achieve high accuracy, which could be timeconsuming and costly to obtain. Additionally, the SVM algorithm might be sensitive to noise and outliers in the data, which could lead to a poorer user experience if customers are required to provide precise and detailed feedback to be understood. In this case, the team may need to make tradeoffs, such as using a simpler ML algorithm that balances accuracy with ease of use, or implementing additional features that help customers provide more effective feedback, such as natural language processing or sentiment analysis tools. This tradeoff would allow the team to meet the business objective of improving the customer experience while still achieving a reasonable level of performance in the ML model. While one can describe tensions and conflicts between various aspects of ML using narrative text, goal modeling supports decision-making, to help solve problems systematically, through incremental steps. For instance, there are known conflicts between explainability and accuracy [22]. Specific techniques, such as ad-hoc methods for explainability, can impede accuracy. But why? By utilizing goal modeling, we can see that specific techniques contribute to one or more softgoals, elucidating why the conflict occurs and at which point in the ML process.
The Goal model in Figure 1 demonstrates how we analyze the above example of conflicting priorities between technical ML and business objectives. The Goal "Customers are satisfied" is refined into two goals: "Ongoing customer satisfaction" (a business goal) and "Customer sentiment be predicted" (an ML goal).
Thus far, these goals have been business goals. To achieve the latter goal, ML goals are now required: "ML model be deployed" and "ML model be trained to predict customer sentiment". We can see how these goals are refined in Figure 1. As we conduct goal and task refinement on the ML goal, we analyze options for ML model techniques (SVM, Naive Bayes, and Decision Trees).
As we attach tasks to achieve the refined goals in Figure 1, the modeling techniques of SVM, Naive Bayes, and Decision Trees are represented as alternative Tasks. The tradeoff explained above is illustrated in this goal model using the positive and negative softgoal contributions among these Tasks. By conducting goal refinement, then identifying and analyzing conflicting contributions of task alternatives to softgoals, we can identify tradeoffs which occur during design decisions when choosing techniques, by deciding to prioritize between specific softgoals (e.g. interpretability of the model and improving the ease for customers to provide feedback).
Though the examples of goal modeling presented in this section is useful and allow us to identify simple tradeoffs, it does not allow us to identify tradeoffs that occur at different stages of the ML lifecycle. Specifically, the tradeoffs and decisions in these examples occur at the same stage of model training. In Figure 1, this can be seen as the tradeoff occurs at the same task refinement level, as we refine the goal of "ML model be trained to predict customer sentiment". This can lead to wrong or poor decisions because, through the goal modeling, we cannot trace the positive and negative effects that the softgoal contributions have to other stages in the ML lifecycle, such as data preparation, feature engineering, or business decisions such as cost effectiveness.
Decisions get made at different iterative cycles in the ML lifecycle. For example, in Figure 1, the primary goal of "ML model be trained to predict customer sentiment" would involve iterations where model training is continuously done until the stopping criteria is met. To achieve the success of "ongoing customer satisfaction", this goal will involve iterations of continuous monitoring. When we drill down and expand into these goals, further tradeoffs appear with respect to how computational, business, and Responsible AI goals must be simultaneously achieved. Traditional goal modeling does not allow us to analyze the distribution of goals and tradeoffs at different iterative cycles, as well as how they interact with each other. In the remainder of this paper, we take the goal modeling a step further by analyzing conflicts between technical ML and Responsible AI goals at decision points distributed through various stages of the ML lifecycle.
We propose three modeling constructs to help us improve our design of ML processes concerning appropriate decisions at each iterative cycle: Sensors, Actuators, and Iterative Loops (Figure 2). A metamodel is shown in Figure 3. In each iteration, based on the most recent information, decisions are made and actions taken, to incrementally get closer to meeting the objectives. Since there are multiple decisions aiming to meet multiple interacting objectives, it is important to position the decision points and their associated information collection and actions appropriately in the nested iterative structures. Together, the three modeling constructs aim to facilitate systematic reasoning that ensures that decisions at each iteration are purposeful and aligned with the overarching goals of ML model development.
The concepts of goals and tasks are drawn from i* [8] [33]. Sensors and Actuators are used in an abstract conceptual sense and do not refer to physical devices. Sensors collect information from the environment. Information can be collected through tasks, or when pursuing goals. Sensor variables are used as input for decisions. An Indicator [12] is one type of Sensor. They are associated with goals so as to indicate how well the goals are achieved.
Actuators are used to manipulate the environment through Tasks. Actuator variables are settings for parameters in tasks. They are outputs of decisions, and can be thought of as levers or knobs for adjusting values. A Task may manipulate multiple Actuators.
Iterative Loops are associated with goals. They repeat until a condition, the stopping criteria, is reached. When a goal that has an Iterative Loop is refined into a subgoal that also has an iterative loop, the latter loop is said to be nested inside the former loop. The latter loop is the inner loop and the former the outer loop. The inner loop is iterated multiple times for each iteration of the outer loop. Consider the following example. In the initial goal model below (Figure 4), we provide a detailed illustration of the various conflicts that can arise between different aspects of Responsible AI, specifically focusing on privacy and fairness at a high level. Below, we break down the initial set of useful features in this goal model.
In Figure 4, Fairness and Privacy are conveyed as separate goals with their own set of alternative tasks which can achieve the respective goal. For each of the goals of Fairness and Privacy, there is an Indicator which is used as a gauge to determine the success of the goal. To achieve the "Privacy Methods be Set" goal, the Indicator "Composite Privacy Score" is calculated to meet its desired threshold. To gauge whether this Indicator can be met, each of these alternative techniques, have a Sensor which senses a specific value to determine if the Indicator threshold has been met. For example, if the task T-closeness is chosen, the Sensor of "T-closeness" collects the T-closeness data value, which is used to gauge the success of the Indicator "Composite Privacy Score". Task refinement allows for representing conditional softgoal contributions to Responsible AI goals based on the choice of alternative techniques. For instance, K-value (a technique to achieve privacy) can negatively impact the softgoal "Balanced Accuracy."
Cycles in the ML Process
Let us consider the following challenge of Principal Component Analysis (PCA), which is a technique option that can help with feature dimensionality reduction but affects fairness. Traditional Principal Component Analysis (PCA) is not designed with fairness in mind and may perpetuate biases, leading to unequal reconstruction errors across different demographic groups, resulting in potentially harmful and unfair outcomes [20].
Let us consider the goals involved. In Figure 5, we present the following parent goals: Model algorithms be set and features be transformed. Together, these goals would eventually support the producing the prediction. Upon identifying the parent goals, refine the parent goals into further subgoals until we can identify alternative techniques (tasks) for accomplishing those goals. When refining the goals, we ensure that the topic is consistent. We refine the goal "Features be transformed" into the following sub-goals: "Features be normalized" "Features be encoded", and "Feature Dimensionality Reduction". We do not yet refine the goal Model algorithm(s) be set because we can identify alternative techniques for this goal.
Next, we identify the alternative tasks that can accomplish each of the sub-goals. Upon identifying these tasks, we attach softgoal contributions (help and hurt) to each softgoal. By identifying the softgoal contributions, we can visualize tradeoffs that arise as a result of choosing one alternative technique over another.
Finally, we attach the Actuators and Sensors to each Goal where they apply in Figure 5 to identify specifically where the tradeoff occurs and why. Toward the right of the model, we can see that we can see that the tradeoff between PCA and LDA can negatively affect the success of Balanced Accuracy, which in turn eventually affects Fairness, through softgoal contributions. This is helpful because it gives us a visual breakdown of why choosing PCA can eventually hurt fairness while providing technical benefits in feature generalization and noise reduction. However, how does this conflict then affect the larger ML development process? At what point in the iterative loops involved does this conflict occur and how does it affect other stages?
The conflict occurs at the "feature transformation" stage, where the PCA technique can be chosen as a dimensionality reduction technique for achieving feature transformation. This technique then negatively affects the group fairness softgoal "mortality prediction output be fair across groups". The Sensors "Explained Variance Ratio" and "Discriminant Power" are used as inputs to determine when Dimensionality Reduction has successfully converged (the Iterative loop stopping criteria for this goal). These Sensors serve as inputs for consideration to adjust the "number of components" Actuator for each of the PCA and LDA techniques.
In this section, we introduce the context of a recent case study [4] that we will draw upon to build on the previous model, toward a comprehensive goal model that captures conflicts between interpretability, explainability, accuracy, and fairness, using a case study example. In this case study, the authors conducted empirical research on conflicts arising between healthcare stakeholders due to ethical concerns with ML applications in healthcare. The authors map the relationships between stakeholders and potential "values-collisions," identifying several themes of conflict. For our purposes, we focus on the following themes: The ML model evaluated in the case study aims to identify what individuals might benefit from advance care planning by addressing a proxy problem: predicting the probability of a given patient passing away within the next 12 months, to aid in palliative care consults. Based on the outcome of the mortality prediction, patients will have the option of being notified if they ought to be considered for advance care planning based on the mortality prediction.
In the following subsections, for the purpose of our goal modeling in this paper, we will focus on the prediction of mortality rates, and build upon the initial goal model (Figure 4). This model will use the case study [4] to illustrate specific aspects of fairness, accuracy, privacy, and explainability appearing in various goals. Each of these Responsible AI goals has been further refined, and conflicts are observed at different goal refinement points, representing various stages of the broader ML process. This approach enables us to illustrate how different Responsible AI challenges are distributed and interact throughout the ML lifecycle, using an empirical case study.
Conflicts often arise not only between technical ML priorities and Responsible AI objectives but also among different aspects of Responsible AI itself [22]. For instance, a model prioritizing fairness may compromise transparency, as fairness metrics might involve complex calculations that are challenging to interpret. Regarding explainability and interpretability, a model emphasizing explainability might sacrifice interpretability, as explanations could necessitate simplifications or approximations that obscure the original model's nuances.
In the preceding sections, we explored (1) conflicts between Responsible AI and technical ML objectives (e.g., feature generalization vs. fairness) and ( 2) conflicts between distinct Responsible AI goals (e.g., explainability vs. fairness). In Figure 6, we illustrate the broader ML lifecycle as it relates to the case study. In this goal model, the nested iterative stages are represented in Loop 1.1 and Loop 1.2 nested within Outer Loop 1. This initial goal model provides us with a breakdown of where each functional goal and set of tasks exist with respect to the nested iterative loops they are a part of.
Building on this model, Figure 6 below introduces softgoal contributions and illustrates a goal model encompassing the broader context of technical ML, business, and Responsible AI goals throughout the ML lifecycle. This figure provides a comprehensive view, analyzing the interactions among three critical aspects of Responsible AI: interpretability, explainability, and fairness.
In this example, we identify two primary conflicts: (1) between fairness and interpretability and (2) between fairness and explainability. Regarding the first conflict, depicted on the left side of the goal model, fairness and interpretability are at odds because interpretability can enhance "Tolerance to outliers," which adversely affects the goal of "mortality prediction output being fair across groups," thereby undermining group fairness. Through softgoal refinement, it becomes evident that interpretability indirectly impacts fairness by enhancing tolerance to outliers. This figure allows us to visualize conflicts that can occur across different stages in the ML development process.
The second conflict, between fairness and explainability, arises because the Fairness softgoal of "mortality prediction output being fair across groups" may compromise decision trustworthiness and, consequently, the softgoal of "explainable prediction result." When considering these tradeoffs among different Responsible AI aspects represented as softgoals, the model designer must evaluate and prioritize these objectives accordingly.
Principle-based approaches are utilized within specific guidelines and ethical frameworks for Responsible AI. These approaches are often prescriptive to specific contexts and issues, rather than being universally applicable to the broader spectrum of Responsible AI. Current methodologies are constrained to addressing a finite set of ethical concerns, such as explainability, fairness, privacy, and accountability. They lack inclusivity regarding the various sub-concepts, perspectives, and interpretations of Responsible AI. Translating a list of ethical objectives into actionable steps poses significant challenges, including determining the most appropriate metric or technique for each use case.
Principle-based approaches, standards, and guidelines (e.g., [13]) are designed to be universal, aiming to apply to all projects. However, requirements are inherently project-specific. Often, principles may conflict with one another, and some may not be relevant or meaningful within the project's specific context. Through Goal Modeling, principles (represented as softgoals) can be refined according to specific Responsible AI contexts, rather than adhering to a finite set of principles applied uniformly across all contexts.
Goal modeling facilitates the clarification and operationalization of vague or ambiguous requirements through goal refinement. Our approach extends the advantages of goal modeling by offering a reasoned and systematic methodology for making design decisions at various stages of the ML lifecycle.
Initial research on fairness predominantly concentrated on formulating quantitative definitions of fairness (see, e.g., [9], [11], [31]) and developing technical methods for 'debiasing' AI models in accordance with these mathematical formalizations (see, e.g., [1], [3], [34]).
As the application of computational techniques proves valuable in addressing challenges within this domain, the notion of Responsible AI is increasingly recognized as contextual. This necessitates greater attention to the varying definitions and needs of Responsible AI, alongside the specific practices and requirements of practitioners. The inherent complexities and contextual nuances of fairness make it impractical to fully de-bias an AI system or guarantee its fairness [14], [21]. The primary objective, therefore, is to mitigate fairness-related harms and other adverse outcomes to the greatest extent possible ( [16], [19]).
It is crucial to approach ML as a holistic process, actively considering the diverse social perspectives, stakeholders, and interactions involved. For example, Srivastava et al. [30] discovered that competing definitions of fairness often do not align with established mathematical definitions. Current computational techniques (e.g. [10]) and tools (e.g. [9]) provide conceptual frameworks that facilitate decision-support for data-driven applications. However, these tools lack critical reasoning capabilities, such as tradeoff mechanisms, goal refinement processes, and the operationalization of those goals.
Kuwajima and Ishikawa [13] proposed a goal-oriented conceptual modeling approach that adheres to the Ethics guidelines for trustworthy AI set forth by the European Commission. While this approach is methodical, it is constrained by its narrow focus on a singular dimension of Responsible AI. It does not encompass the diverse interpretations of Responsible AI, such as fairness, explainability, security, and privacy. Consequently, it is ill-equipped to address the conflicting goals and priorities that arise from these varied interpretations. In contrast, our proposed approach is designed to be versatile and adaptable, accommodating multiple lenses and perspectives to suit any specific context within Responsible AI cases.
GR4ML is another related framework that employs a goal-oriented approach to link analytics and business goals [18]. However, GR4ML falls short in addressing the interrelationships and trade-offs between these goals, particularly within the scope of Responsible AI. To our knowledge, our approach represents the first goal-oriented conceptual modeling framework specifically tailored for Responsible AI.
Existing goal-oriented modeling languages exhibit limited capabilities in integrating Sensors, Actuators, and nested Iterative Loops. Although awareness requirements and adaptive systems in goal modeling address some aspects of sensing, they remain inadequate. For instance, Morandini et al. (2008) present a goal-oriented approach for designing self-adaptive systems, emphasizing the engineering of self-adaptive software.
Awareness Requirements [25] are defined as requirements that reference other requirements or domain assumptions, monitoring their success or failure at run-time. This type of reasoning facilitates adaptability by supporting the monitoring, diagnosis, planning, and execution of requirements. Our proposed Sensor modeling construct extends this concept by enabling inputs from the causal world to inform decisions based on sensed variable values, thereby facilitating interaction with the non-intentional world.
The design of Responsible AI solutions necessitates a systematic approach to accommodate the dynamically evolving decision points inherent in ML processes. This involves the alignment of both business and Responsible AI objectives, alongside the meticulous analysis of conflicts and trade-offs that emerge throughout the nested stages of the ML lifecycle. While contemporary goal modeling approaches offer potential value for designing Responsible AI solutions, they fall short in effectively supporting the analysis of nested iterative cycles in ML development specific to Responsible AI. This paper introduces three novel modeling constructs as part of an innovative goal modeling methodology aimed at systematically designing Responsible AI solutions. Given the benefits of the approach presented, we acknowledge the added complexity. Modelers would have to weigh whether the problem context warrants the added expressiveness and analytical capabilities from using the proposed approach.
In future work, we will augment our proposed goal modeling framework by integrating Agent-Oriented (AO) modeling. Specifically, we will explore how conflicting stakeholder goals might impact the modeling process or the resulting AI solutions. The various stages of the ML life-cycle often involve distinct individuals, where conflicts arising at each stage can be more localized than the current goal models suggest, requiring acknowledgment of the interests and cultural contexts of these individuals. Understanding how humans and AI co-evolve as a hybrid learning system within organizations is a critical area of exploration. Academic discourse has advocated for viewing human-AI systems as collaborative and co-creating rather than merely co-existing systems [32]. In this context, we propose the application of Agent-Oriented conceptual modeling to dissect and analyze conflicts and trade-offs among stakeholders during Responsible AI projects, thereby guiding the design of Responsible AI solutions in a manner that systematically balances diverse values, goals, and interests.
To demonstrate the utility of our modeling approach, we will focus on enhancing the initial analysis and results of the study by identifying the following:
• Strategic interests (i.e., values) of actors involved and the conflicts arising (1) between the interests of each actor and (2) among the subsequent goals in which they are involved.
• Specific points in the ML process where these actors are engaged.
• Extension of the goal modeling to examine how conflicts within nested iterative cycles in the ML lifecycle interact with the interests and priorities of actors. Subsequent development of our conceptual modeling framework will involve its application and validation through an empirical case study to assess its practical relevance in real-world settings. The framework will incorporate knowledge catalogs to aid in the design of Responsible AI solutions, and this research will identify the necessary catalogs. A comprehensive methodology and detailed guidelines will be formulated for the use of the new framework, encompassing phases such as Modeling, Evaluation, Exploration, and Implementation. We will also explore options for tool development to support our proposed approach.