Recent massive successes of Large Language Models (LLMs) brought the AI Alignment problem to the forefront. This long-standing topic encompasses a range of concerns. For example, to ensure that super-learners are prevented from potentially harmful outputs. And, more broadly, whether AI systems can be imbued with ethics and values that align with users' specific values. Drawing on early works by Brent Flyvbjerg on values and power in human decision-making, we propose viewing alignment as a process where powerful stakeholders strive to actively shape the discourse through rationalizing, thereby obtaining a "license to operate" in the market. To foster a more balanced and equitable discourse we propose employing i* to facilitate a more systematic mapping out and analyzing of stakeholders' rationalizing to revealing unarticulated motivations and goals and alternatives, that were intentionally left hidden in the discourse, but that are crucial to identify supporting or conflicting stakeholders' goals and concerns. By adopting i* we believe that diverse stakeholders in asymmetric, yet interdependent, power relations could be better equipped to analyze power discourses, and thereby achieve more balanced and fair outcomes in the adoption, diffusion and use of powerful AI-based systems within their ecosystem.
The recent tremendous advancements of Large Language Models (LLMs) have demonstrated unparalleled AI capabilities in understanding, responding to, reasoning about, and fulfilling human queries and demands. Presently, virtually every major company is actively integrating LLMs into their technological infrastructure.
With such breathtaking developments accelerating at an unprecedented pace across industries, key
questions of AI safety and Alignment [
Concerns of AI safety, and the related topic of AI alignment, is already more than a decade old and
encompasses a range of concerns. From technical concerns: whether super-learners such as LLMs that
traverse vast state spaces can be prevented from arriving at unintended and even harmful outputs [
We posit that such concerns of technical and value alignments are indeed critical to the successful and safe operation of AI system within human society. However, we recognize another significant aspect of alignment: the alignment of interests among stakeholders involved in the creation, deployment, use of, and that are impacted by AI-based solutions introduced to the market.
With the emergence of today’s powerful AI technologies, that are about to change everything, there are vocal voices of powerful stakeholders, who have a lot to gain from adoption of AI, and that seek broad legitimacy in operating such systems in markets.
2023 Copyright for this paper by its authors.
Following the early works of Flyvbjerg [
Flyvbjerg presents ten propositions that capture key aspects of power and knowledge, and related rationalizing processes. The ten propositions are derived from studies of power in the field and include propositions such as “Power defines reality”, “Rationality is context-dependent, the context of rationality is power, and power blurs the dividing line between rationality and rationalization”, and the like.
Based on these principles Flyvbjerg posits that powerful stakeholders exert their power by defining reality, which includes deciding what knowledge to make available, and what rationalizations to thereby offer to influence outcomes favorably to their goals.
Flyvbjerg further emphasizes that rationality, the establishing of the normative, “what should be
done”, does not survive when it meets “what is actually done”, which is what power determines in
practice [
We see the i* modeling approach in particular well suited to offer support for Realrationalität. Unlike
mechanistic modeling methods common in the software requirements and engineering domain, the i*
modeling approach looks to bring a social understanding to rationalizing processes by adopting a social
ontology for its main modeling constructs [
Furthermore, the paradigm underpinning the i* modeling approach, and the social semantics
afforded to its constructs, in particular, looks to deal with inherent inability to know and control the
social world [
The i* modeling method therefore looks to provide analysis tools to externally characterize actors using rationalized intentionality and interdependencies. External characterizations are then a means for external observers to more systematically explore hidden (semi-opaque) motivations, knowledge and intents of actors pursuing their interest.
Adopting Flyvbjerg, we would say that knowledge that could be shared is however left unarticulated by actors as a key part of exerting power. We see overcoming such attempted obfuscation by using i* modeling and analysis capabilities as part of a Realrationalität alignment effort of stakeholder interests, to ensure equitable and fair outcomes.
The next section presents approaches of how power has been described and analyzed in relation to technology, and contrasts these with rationalizing as explained by Flyvbjerg, and proposed here. Section three illustrates the application of power through constructed rationalizing taken from a study performed by Flyvbjerg. Section four suggests how i* could be used as a tool to overcome such constructed rationalizing by looking to systematically broadening the discourse space.
Finally, section five concludes by pointing to key challenges in the use of a modeling technique in power discourse settings, and future work.
Milne and Maiden [
Milne and Maiden then propose a modeling technique based on social networks whereby directed links between stakeholder nodes are labeled to indicate what kind of power a source party can exert on a target party. Types of power include legitimacy through formal authority, reward, coerce, expertise, and power through personal characteristics.
Milne and Maiden further discuss the limitation of the i*’s strategic dependency model to capture power relationships and suggest augmenting strategic dependencies links with their labeling.
Alves, Valença and Franch [
Approaches such as above focus on a relational characterization of power, that allows labeling and then analyzing power relationships in terms of graph properties.
Offering an important step towards alignment, Schlichter [
Based on the work by Flyvbjerg [
We therefore see as a key need to systematically and critically explore rationalizing arguments, looking to identify unarticulated knowledge and intents.
Seen from this vantage point, we posit that key value can be derived from applying the i* modeling approach to systematically explore the open landscape of stakeholders, and alternatives, including scopes of decision-making, while putting a strong methodological emphasis on the underpinning social ontological modeling assumptions of i* -- the need to externally characterize unknowable, and uncontrollable stakeholders – and to further augmenting, as needed, i*’s social constructs to deal with power relationships between actors.
rationalizing of power through controlling knowledge and
Flyvbjerg [
The mayors, and his planners, argument centered around the objective need of reducing transfer time for travelers, presenting a seemingly compelling rationale to relevant stakeholders involved in this discourse, such as environmentalists, businesses, and the city architect's office.
However, concealed within this reasoning was the fact that the transportation system itself had been intentionally planned to justify the location of the bus terminal, in alignment with the mayor and his planners’ desires.
While providing stakeholders with a seemingly plausible justification for the chosen bus terminal location, the circularity of the decision-making process, the broader planning process and real intents, remained hidden.
Flyvbjerg explains that “[i]n real terms, by playing games of power covered up as technical reasoning, the mayor and the planners got what they wanted – a monument to themselves – despite rampant protest …”
Consequently, real concerns, which eventually materialized, regarding the bus terminal's location, such as increased traffic accidents, elevated levels of air and noise pollution, and physical blight, were rationally traded-off for speed of transfer, while also keeping the discourse scope focused on the terminal’s location only, and not the broader planning process.
To expose such rationalizing in the service of power, Flyvbjerg calls for a study of realrationalität (real-world rationalizing) “to depict what the lived, as opposed to the ideal, world of modern democracy and planning looks like and how it operates …”.
It’s a method that goes after the details, and that looks to unearth hidden motivations and choices.
We believe that i* [
Strategic rationale models support the exploration of possible hidden internal motivations and intents that are linked to, and explain, the external dependencies. The hierarchical structure of rationale models enables considering the provided rationales as starting points, to explore higher-level intents (the 'why') and possible alternatives (the 'how else'), which could then lead to alternative dependency configurations.
The open nature of the model enables a systematic broadening the scope of discourse, including, to alternative that have not been included in the discourse.
To illustrate, let us consider again the bus terminal location being justified by the need for fast transfers. In this case, one could use the strategic rational structures to start question the underlying reasons for the chosen locations of the transportation system, thereby uncovering potential hidden rationales, or lack thereof, that necessitate placing the transportation system at its location, in the first place. By moving up the rational hierarchy structures the topic “bus station” could be expanded to transportation system, and beyond, and alternatives and rationales for the overall design explored, leading to questions that expand the scope of discourse and thereby making it more difficult to obfuscate the overall design of the transport system including its locations.
An i* modeler would need to be critically aware that what may be presented as a constraint, the transport system locations, modeled as a claim softgoal, which then would not be part of the alternative exploration, may in fact be a hidden choice, better modeled as a task, or even an actor, and open for further exploration, evaluation and negotiation.
By engaging in such inquiry, it becomes possible to bring additional goals and choices of relevant stakeholders in more systematic manner into the conversation, such as reducing traffic and accidents, and pollution, and the rationales for the locations of the transport system as a whole in general.
Given the conceptual fit of i* with an exploration of rationalizing in social settings, we belief that additional empirical research that examine the tactics employed by powerful entities to rationalize and construct reality would lead to fruitful insights into further expressivity needs of i*, as well as the development of further key analysis methods specifically tailored for analyzing power discourses and alignment.
The recent advancements in AI, particularly LLMs, are attracting influential stakeholders who have a substantial stake in the adoption of these AI technologies. These stakeholders are actively engaged in advocating for the remarkable value of emerging AI solutions, while also justifying the transformative impact AI will have on the workforce, and society at large. As a result, there has been a remarkable increase in the creation and deployment of AI-based solutions across various aspects of our lives, which will see further tremendous acceleration in the future.
Given the magnitude of these developments, there is a critical need for the development of robust and easy to use analysis tools of power-driven rationalizing. The i* modeling approach is well positioned to offer along with additional tools such as Schlichter’s ethical maturity model, could play a pivotal role in facilitating the process of alignment of interests between stakeholders, addressing a key requirement in the field of AI.
Clearly, the challenge of intentionally obfuscating knowledge is a significant one, in particular in a power setting, and the mere use of a modeling technique may not make a difference. Yet, there is significant value in making verbal and written, argumentation, visible in models, and thereby enabling a more systematic analysis of the problem domain, and to provide tools for communicating knowledge and decision-making, which is a key value of modeling tools in general.
An i* based modeling and analysis at the intentional level could also make a significant contribution to establishing AI systems alignment with human values, by making visible and amendable for machine learning stakeholder motivation and goals through strategic dependencies and rationales.
The use of the i* modeling method could therefore also provide significant value for other aspects of alignment when deploying AI based systems in social settings.
We therefore advocate for further advancements in, and diffusion of, i* to support its adoption and use in the context of AI alignment, and the analysis of power and rationalizing. The requirements engineering (RE) community is uniquely positioned to make significant contributions to more equitable aligned AI based systems during this significant inflection point in time.