This paper addresses the challenge of managing decisions in machine learning (ML) development, where choices in one iterative cycle affect subsequent cycles, each with varying evaluation results. The research objective is to evaluate how well our proposed modeling constructs-Sensors, Actuators, and Iterative Loops-enhance existing goal-oriented conceptual modeling to better analyze decisions in Responsible AI, particularly within nested iterative cycles. We evaluate the efficacy of our proposed goal modeling constructs in analyzing trade-offs among business, technical, and Responsible AI goals using these constructs. Our findings suggest that these constructs improve upon current goal modeling methods, offering more effective decision-making support for Responsible AI outcomes.
Bias and other social responsibility challenges in AI arise from both the underlying Machine
Learning (ML) model and the context in which it is used. AI systems, due to inherent model biases,
can propagate these issues at scale, affecting numerous user applications [
ML system design involves multiple stages, each with multiple decision points, and iterative cycles, including data gathering, feature engineering, ML model training, deployment, and user output. Responsible AI is an approach within ML-based that integrates fairness, transparency, and ethical considerations at each stage, ensuring that decisions are evaluated not only for technical effectiveness but also for their societal and ethical impact.
Supporting decision-making in iterative ML and Responsible AI cycles is crucial for refining
models and improving accuracy by quickly identifying and addressing issues like overfitting or data
drift. It ensures that each iteration adds value, ultimately leading to more reliable and robust
outcomes. Goal-oriented conceptual modeling is a well-established technique to support systematic
decision-making processes [
As goals are operationalized in terms of tasks, current goal modeling approaches do not consider
how each task alternative may contribute differently to various softgoals, across iterative cycles. To
deal with this problem, this paper proposes 3 new goal modeling constructs and examines how they
aid in analyzing tradeoffs and conflicts in Responsible AI that are distributed throughout the ML
lifecycle. Design decisions at each stage interact and contribute to goals at different stages, with
issues like concept drift causing ML processes to evolve over time. Recognizing where tradeoffs occur
is crucial, as focusing solely on technical decision points can lead to oversimplified solutions that do
not meet other objectives. We explore how tradeoffs at the operationalization level can eventually
impact those at the business level, emphasizing the importance of modeling processes and decisions.
To illustrate relevant aspects of Responsible AI and their tradeoffs, we draw upon the literature [
In Section 2, we first consider tradeoffs between business goals and technical ML goals. Then in Section 3, we introduce the proposed goal modeling notation. In Section 4, we apply the modeling notation to deal with tradeoffs between responsible AI and technical ML goals in various stages of ML development. We discuss related work in Section 5 and conclude with future work in Section 6.
In the design of ML-based systems, technical ML objectives can often conflict with business objectives due to competing priorities, leading to tradeoffs [27]. For instance, in the development of a customer feedback system, a technical ML objective might be to prioritize the accuracy of sentiment analysis, which could be achieved by using a Support Vector Machine (SVM) algorithm that maximizes the margin between positive and negative feedback [28]. However, this could conflict with the business objective of improving the ease for customers to provide feedback, as the SVM algorithm might require a large amount of labeled training data to achieve high accuracy, which could be timeconsuming and costly to obtain. Additionally, the SVM algorithm might be sensitive to noise and outliers in the data, which could lead to a poorer user experience if customers are required to provide precise and detailed feedback to be understood. In this case, the team may need to make tradeoffs, such as using a simpler ML algorithm that balances accuracy with ease of use, or implementing additional features that help customers provide more effective feedback, such as natural language processing or sentiment analysis tools. This tradeoff would allow the team to meet the business objective of improving the customer experience while still achieving a reasonable level of performance in the ML model.
While one can describe tensions and conflicts between various aspects of ML using narrative text, goal modeling supports decision-making, to help solve problems systematically, through incremental steps. For instance, there are known conflicts between explainability and accuracy [22]. Specific techniques, such as ad-hoc methods for explainability, can impede accuracy. But why? By utilizing goal modeling, we can see that specific techniques contribute to one or more softgoals, elucidating why the conflict occurs and at which point in the ML process.
The Goal model in Figure 1 demonstrates how we analyze the above example of conflicting priorities between technical ML and business objectives. The Goal “Customers are satisfied” is refined into two goals: “Ongoing customer satisfaction” (a business goal) and “Customer sentiment be predicted” (an ML goal).
Thus far, these goals have been business goals. To achieve the latter goal, ML goals are now required: "ML model be deployed" and "ML model be trained to predict customer sentiment". We can see how these goals are refined in Figure 1. As we conduct goal and task refinement on the ML goal, we analyze options for ML model techniques (SVM, Naive Bayes, and Decision Trees).
As we attach tasks to achieve the refined goals in Figure 1, the modeling techniques of SVM, Naive Bayes, and Decision Trees are represented as alternative Tasks. The tradeoff explained above is illustrated in this goal model using the positive and negative softgoal contributions among these Tasks. By conducting goal refinement, then identifying and analyzing conflicting contributions of task alternatives to softgoals, we can identify tradeoffs which occur during design decisions when choosing techniques, by deciding to prioritize between specific softgoals (e.g. interpretability of the model and improving the ease for customers to provide feedback).
Though the examples of goal modeling presented in this section is useful and allow us to identify simple tradeoffs, it does not allow us to identify tradeoffs that occur at different stages of the ML lifecycle. Specifically, the tradeoffs and decisions in these examples occur at the same stage of model training. In Figure 1, this can be seen as the tradeoff occurs at the same task refinement level, as we refine the goal of "ML model be trained to predict customer sentiment". This can lead to wrong or poor decisions because, through the goal modeling, we cannot trace the positive and negative effects that the softgoal contributions have to other stages in the ML lifecycle, such as data preparation, feature engineering, or business decisions such as cost effectiveness.
Decisions get made at different iterative cycles in the ML lifecycle. For example, in Figure 1, the primary goal of “ML model be trained to predict customer sentiment” would involve iterations where model training is continuously done until the stopping criteria is met. To achieve the success of “ongoing customer satisfaction”, this goal will involve iterations of continuous monitoring. When we drill down and expand into these goals, further tradeoffs appear with respect to how computational, business, and Responsible AI goals must be simultaneously achieved. Traditional goal modeling does not allow us to analyze the distribution of goals and tradeoffs at different iterative cycles, as well as how they interact with each other. In the remainder of this paper, we take the goal modeling a step further by analyzing conflicts between technical ML and Responsible AI goals at decision points distributed through various stages of the ML lifecycle.
We propose three modeling constructs to help us improve our design of ML processes concerning appropriate decisions at each iterative cycle: Sensors, Actuators, and Iterative Loops (Figure 2). A metamodel is shown in Figure 3. In each iteration, based on the most recent information, decisions are made and actions taken, to incrementally get closer to meeting the objectives. Since there are multiple decisions aiming to meet multiple interacting objectives, it is important to position the decision points and their associated information collection and actions appropriately in the nested iterative structures. Together, the three modeling constructs aim to facilitate systematic reasoning that ensures that decisions at each iteration are purposeful and aligned with the overarching goals of ML model development.
The concepts of goals and tasks are drawn from i* [
Actuators are used to manipulate the environment through Tasks. Actuator variables are settings for parameters in tasks. They are outputs of decisions, and can be thought of as levers or knobs for adjusting values. A Task may manipulate multiple Actuators.
Iterative Loops are associated with goals. They repeat until a condition, the stopping criteria, is reached. When a goal that has an Iterative Loop is refined into a subgoal that also has an iterative loop, the latter loop is said to be nested inside the former loop. The latter loop is the inner loop and the former the outer loop. The inner loop is iterated multiple times for each iteration of the outer loop.
Consider the following example. In the initial goal model below (Figure 4), we provide a detailed illustration of the various conflicts that can arise between different aspects of Responsible AI, specifically focusing on privacy and fairness at a high level. Below, we break down the initial set of useful features in this goal model.
In Figure 4, Fairness and Privacy are conveyed as separate goals with their own set of alternative tasks which can achieve the respective goal. For each of the goals of Fairness and Privacy, there is an Indicator which is used as a gauge to determine the success of the goal. To achieve the “Privacy Methods be Set” goal, the Indicator “Composite Privacy Score” is calculated to meet its desired threshold. To gauge whether this Indicator can be met, each of these alternative techniques, have a Sensor which senses a specific value to determine if the Indicator threshold has been met. For example, if the task T-closeness is chosen, the Sensor of “T-closeness” collects the T-closeness data value, which is used to gauge the success of the Indicator “Composite Privacy Score”. Task refinement allows for representing conditional softgoal contributions to Responsible AI goals based on the choice of alternative techniques. For instance, K-value (a technique to achieve privacy) can negatively impact the softgoal "Balanced Accuracy."
Let us consider the following challenge of Principal Component Analysis (PCA), which is a technique option that can help with feature dimensionality reduction but affects fairness. Traditional Principal Component Analysis (PCA) is not designed with fairness in mind and may perpetuate biases, leading to unequal reconstruction errors across different demographic groups, resulting in potentially harmful and unfair outcomes [20].
Let us consider the goals involved. In Figure 5, we present the following parent goals: Model algorithms be set and features be transformed. Together, these goals would eventually support the producing the prediction. Upon identifying the parent goals, refine the parent goals into further subgoals until we can identify alternative techniques (tasks) for accomplishing those goals. When refining the goals, we ensure that the topic is consistent. We refine the goal "Features be transformed" into the following sub-goals: "Features be normalized" "Features be encoded", and "Feature Dimensionality Reduction". We do not yet refine the goal Model algorithm(s) be set because we can identify alternative techniques for this goal.
Next, we identify the alternative tasks that can accomplish each of the sub-goals. Upon identifying these tasks, we attach softgoal contributions (help and hurt) to each softgoal. By identifying the softgoal contributions, we can visualize tradeoffs that arise as a result of choosing one alternative technique over another.
Finally, we attach the Actuators and Sensors to each Goal where they apply in Figure 5 to identify specifically where the tradeoff occurs and why. Toward the right of the model, we can see that we can see that the tradeoff between PCA and LDA can negatively affect the success of Balanced Accuracy, which in turn eventually affects Fairness, through softgoal contributions. This is helpful because it gives us a visual breakdown of why choosing PCA can eventually hurt fairness while providing technical benefits in feature generalization and noise reduction. However, how does this conflict then affect the larger ML development process? At what point in the iterative loops involved does this conflict occur and how does it affect other stages?
The conflict occurs at the "feature transformation" stage, where the PCA technique can be chosen as a dimensionality reduction technique for achieving feature transformation. This technique then negatively affects the group fairness softgoal "mortality prediction output be fair across groups". The Sensors "Explained Variance Ratio" and "Discriminant Power" are used as inputs to determine when Dimensionality Reduction has successfully converged (the Iterative loop stopping criteria for this goal). These Sensors serve as inputs for consideration to adjust the "number of components" Actuator for each of the PCA and LDA techniques.
In this section, we introduce the context of a recent case study [
Bias and perpetuation of bias (Bias) Conflicting values and perspectives on death and end-of-life care (Fairness) Transparency and evaluation of efficacy (Transparency) Determining the recipients of ML output (Explainability)
Patient consent and involvement (Privacy)
The ML model evaluated in the case study aims to identify what individuals might benefit from advance care planning by addressing a proxy problem: predicting the probability of a given patient passing away within the next 12 months, to aid in palliative care consults. Based on the outcome of the mortality prediction, patients will have the option of being notified if they ought to be considered for advance care planning based on the mortality prediction.
In the following subsections, for the purpose of our goal modeling in this paper, we will focus on
the prediction of mortality rates, and build upon the initial goal model (Figure 4). This model will
use the case study [
Conflicts often arise not only between technical ML priorities and Responsible AI objectives but also among different aspects of Responsible AI itself [22]. For instance, a model prioritizing fairness may compromise transparency, as fairness metrics might involve complex calculations that are challenging to interpret. Regarding explainability and interpretability, a model emphasizing explainability might sacrifice interpretability, as explanations could necessitate simplifications or approximations that obscure the original model’s nuances.
In the preceding sections, we explored (1) conflicts between Responsible AI and technical ML objectives (e.g., feature generalization vs. fairness) and (2) conflicts between distinct Responsible AI goals (e.g., explainability vs. fairness). In Figure 6, we illustrate the broader ML lifecycle as it relates to the case study. In this goal model, the nested iterative stages are represented in Loop 1.1 and Loop 1.2 nested within Outer Loop 1. This initial goal model provides us with a breakdown of where each functional goal and set of tasks exist with respect to the nested iterative loops they are a part of.
Building on this model, Figure 6 below introduces softgoal contributions and illustrates a goal model encompassing the broader context of technical ML, business, and Responsible AI goals throughout the ML lifecycle. This figure provides a comprehensive view, analyzing the interactions among three critical aspects of Responsible AI: interpretability, explainability, and fairness.
In this example, we identify two primary conflicts: (1) between fairness and interpretability and (2) between fairness and explainability. Regarding the first conflict, depicted on the left side of the goal model, fairness and interpretability are at odds because interpretability can enhance "Tolerance to outliers," which adversely affects the goal of "mortality prediction output being fair across groups," thereby undermining group fairness. Through softgoal refinement, it becomes evident that interpretability indirectly impacts fairness by enhancing tolerance to outliers. This figure allows us to visualize conflicts that can occur across different stages in the ML development process.
The second conflict, between fairness and explainability, arises because the Fairness softgoal of "mortality prediction output being fair across groups" may compromise decision trustworthiness and, consequently, the softgoal of "explainable prediction result." When considering these tradeoffs among different Responsible AI aspects represented as softgoals, the model designer must evaluate and prioritize these objectives accordingly.
Principle-based approaches are utilized within specific guidelines and ethical frameworks for Responsible AI. These approaches are often prescriptive to specific contexts and issues, rather than being universally applicable to the broader spectrum of Responsible AI. Current methodologies are constrained to addressing a finite set of ethical concerns, such as explainability, fairness, privacy, and accountability. They lack inclusivity regarding the various sub-concepts, perspectives, and interpretations of Responsible AI. Translating a list of ethical objectives into actionable steps poses significant challenges, including determining the most appropriate metric or technique for each use case.
Principle-based approaches, standards, and guidelines (e.g., [13]) are designed to be universal, aiming to apply to all projects. However, requirements are inherently project-specific. Often, principles may conflict with one another, and some may not be relevant or meaningful within the project’s specific context. Through Goal Modeling, principles (represented as softgoals) can be refined according to specific Responsible AI contexts, rather than adhering to a finite set of principles applied uniformly across all contexts.
Goal modeling facilitates the clarification and operationalization of vague or ambiguous requirements through goal refinement. Our approach extends the advantages of goal modeling by offering a reasoned and systematic methodology for making design decisions at various stages of the ML lifecycle.
Initial research on fairness predominantly concentrated on formulating quantitative definitions
of fairness (see, e.g., [
As the application of computational techniques proves valuable in addressing challenges within this domain, the notion of Responsible AI is increasingly recognized as contextual. This necessitates greater attention to the varying definitions and needs of Responsible AI, alongside the specific practices and requirements of practitioners. The inherent complexities and contextual nuances of fairness make it impractical to fully de-bias an AI system or guarantee its fairness [14], [21]. The primary objective, therefore, is to mitigate fairness-related harms and other adverse outcomes to the greatest extent possible ([16], [19]).
It is crucial to approach ML as a holistic process, actively considering the diverse social
perspectives, stakeholders, and interactions involved. For example, Srivastava et al. [30] discovered
that competing definitions of fairness often do not align with established mathematical definitions.
Current computational techniques (e.g. [
Kuwajima and Ishikawa [13] proposed a goal-oriented conceptual modeling approach that adheres to the Ethics guidelines for trustworthy AI set forth by the European Commission. While this approach is methodical, it is constrained by its narrow focus on a singular dimension of Responsible AI. It does not encompass the diverse interpretations of Responsible AI, such as fairness, explainability, security, and privacy. Consequently, it is ill-equipped to address the conflicting goals and priorities that arise from these varied interpretations. In contrast, our proposed approach is designed to be versatile and adaptable, accommodating multiple lenses and perspectives to suit any specific context within Responsible AI cases.
GR4ML is another related framework that employs a goal-oriented approach to link analytics and business goals [18]. However, GR4ML falls short in addressing the interrelationships and trade-offs between these goals, particularly within the scope of Responsible AI. To our knowledge, our approach represents the first goal-oriented conceptual modeling framework specifically tailored for Responsible AI.
Existing goal-oriented modeling languages exhibit limited capabilities in integrating Sensors, Actuators, and nested Iterative Loops. Although awareness requirements and adaptive systems in goal modeling address some aspects of sensing, they remain inadequate. For instance, Morandini et al. (2008) present a goal-oriented approach for designing self-adaptive systems, emphasizing the engineering of self-adaptive software.
Awareness Requirements [25] are defined as requirements that reference other requirements or domain assumptions, monitoring their success or failure at run-time. This type of reasoning facilitates adaptability by supporting the monitoring, diagnosis, planning, and execution of requirements. Our proposed Sensor modeling construct extends this concept by enabling inputs from the causal world to inform decisions based on sensed variable values, thereby facilitating interaction with the non-intentional world.
The design of Responsible AI solutions necessitates a systematic approach to accommodate the dynamically evolving decision points inherent in ML processes. This involves the alignment of both business and Responsible AI objectives, alongside the meticulous analysis of conflicts and trade-offs that emerge throughout the nested stages of the ML lifecycle. While contemporary goal modeling approaches offer potential value for designing Responsible AI solutions, they fall short in effectively supporting the analysis of nested iterative cycles in ML development specific to Responsible AI. This paper introduces three novel modeling constructs as part of an innovative goal modeling methodology aimed at systematically designing Responsible AI solutions. Given the benefits of the approach presented, we acknowledge the added complexity. Modelers would have to weigh whether the problem context warrants the added expressiveness and analytical capabilities from using the proposed approach.
In future work, we will augment our proposed goal modeling framework by integrating AgentOriented (AO) modeling. Specifically, we will explore how conflicting stakeholder goals might impact the modeling process or the resulting AI solutions. The various stages of the ML life- cycle often involve distinct individuals, where conflicts arising at each stage can be more localized than the current goal models suggest, requiring acknowledgment of the interests and cultural contexts of these individuals. Understanding how humans and AI co-evolve as a hybrid learning system within organizations is a critical area of exploration. Academic discourse has advocated for viewing humanAI systems as collaborative and co-creating rather than merely co-existing systems [32]. In this context, we propose the application of Agent- Oriented conceptual modeling to dissect and analyze conflicts and trade-offs among stakeholders during Responsible AI projects, thereby guiding the design of Responsible AI solutions in a manner that systematically balances diverse values, goals, and interests.
To demonstrate the utility of our modeling approach, we will focus on enhancing the initial analysis and results of the study by identifying the following: • • •
Strategic interests (i.e., values) of actors involved and the conflicts arising (1) between the interests of each actor and (2) among the subsequent goals in which they are involved. Specific points in the ML process where these actors are engaged.
Extension of the goal modeling to examine how conflicts within nested iterative cycles in the ML lifecycle interact with the interests and priorities of actors.
Subsequent development of our conceptual modeling framework will involve its application and validation through an empirical case study to assess its practical relevance in real-world settings. The framework will incorporate knowledge catalogs to aid in the design of Responsible AI solutions, and this research will identify the necessary catalogs. A comprehensive methodology and detailed guidelines will be formulated for the use of the new framework, encompassing phases such as Modeling, Evaluation, Exploration, and Implementation. We will also explore options for tool development to support our proposed approach. [11] Hardt, M., Price, E., Srebro, N. (2016). Equality of opportunity in supervised learning. Advances in neural information processing systems, 29. [12] Horkoff, J., Barone, D., Jiang, L., Yu, E., Amyot, D., Borgida, A., Mylopoulos, J. (2014). business modeling: representation and reasoning. Software Systems Modeling, 13, 1015-1041. [13] Kuwajima, H., Ishikawa, F. (2019). Adapting square for quality assessment of artificial intelligence systems. In 2019 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) (pp. 13-18). IEEE. [14] Kleinberg, J., Mullainathan, S., Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807. [15] Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., ... Goel, S. (2020). Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14), 7684-7689. [16] Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A. (2021). A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6), 1-35. [17] Morandini, M., Penserini, L., Perini, A. (2008). Towards goal-oriented development of selfadaptive systems. In Proceedings of the 2008 international workshop on Software engineering for adaptive and self-managing systems (pp. 9-16). [18] Nalchigar, S., Yu, E. (2020). Designing business analytics solutions: a model-driven approach.
Business Information Systems Engineering, 62(1), 61-75. [19] Norval, C., Cornelius, K., Cobbe, J., Singh, J. (2022). Disclosure by Design: Designing information disclosures to support meaningful transparency and accountability. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (pp. 679-690). [20] Pelegrina, G. D., Duarte, L. T. (2023). A novel approach for Fair Principal Component Analysis based on eigendecomposition. IEEE Transactions on Artificial Intelligence. [21] Pleiss, G., Raghavan, M., Wu, F., Kleinberg, J., Weinberger, K. Q. (2017). On fairness and calibration. Advances in neural information processing systems, 30. [22] Sanderson, C., Douglas, D., Lu, Q. (2023). Implementing responsible AI: Tensions and trade- offs between ethics aspects. In 2023 International Joint Conference on Neural Networks (IJCNN) (pp. 1-7). IEEE. [23] Scheuerman, M. K., Paul, J. M., Brubaker, J. R. (2019). How computers see gender: An evaluation of gender classification in commercial facial analysis services. Proceedings of the ACM on Human-Computer Interaction, 3(CSCW), 1-33. [24] Singh, J., Cobbe, J., Norval, C. (2018). Decision provenance: Harnessing data flow for accountable systems. IEEE Access, 7, 6562-6574. [25] Silva Souza, V. E., Lapouchnian, A., Robinson, W. N., Mylopoulos, J. (2011). Awareness requirements for adaptive systems. In Proceedings of the 6th international symposium on Software engineering for adaptive and self-managing systems (pp. 60-69). [26] Sothilingam & Yu, “A A Goal-Oriented Approach for Modeling Decisions in ML Processes”, 2024 IEEE 32nd International Requirements Engineering Conference Workshops (REW), Reykjavik, Iceland, 2024, pp.321-325, doi: 10.1109/REW61692.2024.00048. [27] Sothilingam, R.,, "A Requirements-Driven Conceptual Modeling Framework for Responsible AI," 2023 IEEE 31st International Requirements Engineering Conference (RE), Hannover, Germany, 2023, pp. 391-395, doi: 10.1109/RE57278.2023.00061. [28] Sothilingam, R., Pant, V., and Yu, E., "Using i* to Analyze Collaboration Challenges in MLOps
Project Teams", Proceedings of the 15th International i* Workshop 2022, 2022. [29] Sothilingam, R., and Yu, E., "Modeling Agents Roles and Positions in Machine Learning Project
Organizations", Proceedings of the 13th International i* Workshop 2020, vol. 2641, pp. 61-66, 2020. [30] Srivastava, M., Heidari, H., Krause, A. (2019). Mathematical notions vs. human perception of fairness: A descriptive approach to fairness for machine learning. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery data mining (pp. 2459- 2468). [31] Verma, S., Rubin, J. (2018). Fairness definitions explained. In Proceedings of the international workshop on software fairness (pp. 1-7). [32] Waardenburg, L., Huysman, M. (2022). From coexistence to co-creation: Blurring boundaries in the age of AI. Information and Organisation, 32(4). [33] Yu, E., Giorgini, P., Maiden, N., Mylopoulos, J. (2011). Social Modeling for Requirements
Engineering (Cooperative Information Systems Series). MIT Press. [34] Zafar, M. B., Valera, I., Gomez Rodriguez, M., Gummadi, K. P. (2017). Fairness beyond disparate treatment disparate impact: Learning classification without disparate mistreatment. In Proceedings of the 26th international conference on world wide web (pp. 1171-1180).