=Paper=
{{Paper
|id=Vol-2584/RE4AI-paper5
|storemode=property
|title=Trustworthy AI: Towards the Golden Age of RE?
|pdfUrl=https://ceur-ws.org/Vol-2584/RE4AI-paper5.pdf
|volume=Vol-2584
|authors=Matthieu Vergne
|dblpUrl=https://dblp.org/rec/conf/refsq/Vergne20
}}
==Trustworthy AI: Towards the Golden Age of RE?==
https://ceur-ws.org/Vol-2584/RE4AI-paper5.pdf
Trustworthy AI:
Towards the Golden Age of RE?
Matthieu Vergne
Consultant Engineer
matthieu.vergne@meritis.fr
Meritis PACA
Les Algorithmes Aristote B
2000 Route des Lucioles
06901 Sophia Antipolis Cedex
FRANCE
Abstract
In April, 2018, the European Commission established its vision of Arti-
ficial Intelligence (AI), leading to the production of guidelines to achieve
trustworthy AI one year later. These guidelines, although not men-
tioning it explicitly, overflow with issues well known in Requirements
Engineering (RE). By relating recent RE works to these guidelines, this
position paper attempts to show that RE is one of the core components
for achieving trustworthy AI, and thus can have a critical impact on
the evolution of AI systems and the AI field as a whole for the next
few years in Europe.
1 Introduction
In April, 2018, the European Commission established its vision of Artificial Intelligence (AI) [Eur18]. Similarly
to “the steam engine or electricity in the past”, AI is considered as “one of the most strategic technologies of
the 21st century”. It may help to “solve some of the world’s biggest challenges” and transform “our world, our
society and our industry”. Since the way we approach AI “will define the world we live in”, the Commission
considers that “a solid European framework is needed”. Pushed by this strong incentive, the Commission formed
a high-level expert group in AI (AI HLEG) which, in April, 2019, has produced guidelines to foster trustworthy
AI [AI 19].
In this position paper, we claim that these guidelines offer a tremendous amount of opportunities for the field
of Requirements Engineering (RE). Far to be a mere field of application of RE techniques, we try to show that
RE is a core element in achieving these guidelines. To show that, we describe in Section 2 the core content of the
AI HLEG guidelines, especially the seven key requirements they deem as imperative for achieving trustworthy
AI. In Section 3, we then relate recent RE works to these guidelines to put in light the deep bonds between the
two, before to highlight remaining challenges in Section 4.
Copyright c 2020 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International
(CC BY 4.0).
In: M. Sabetzadeh, A. Vogelsang, S. Abualhaija, M. Borg, F. Dalpiaz, M. Daneva, N. Fernández, X. Franch, D. Fucci, V.
Gervasi, E. Groen, R. Guizzardi, A. Herrmann, J. Horkoff, L. Mich, A. Perini, A. Susi (eds.): Joint Proceedings of REFSQ-2020
Workshops, Doctoral Symposium, Live Studies Track, and Poster Track, Pisa, Italy, 24-03-2020, published at http://ceur-ws.org
2 Trustworthy AI
The AI HLEG guidelines [AI 19] highlight three components which should be met throughout the entire life cycle
of an AI system, from its conception to its disposal. It must be lawful, thus complying with applicable laws and
regulations, ethical, thus ensuring adherence to ethical principles and values, and robust (both from a technical
and social perspective), thus avoiding unintentional harm. While the lawful aspect, despite being complex to
achieve, is a rather straightforward objective (i.e. comply to national, European, and international laws), the
ethical and robust aspects bring more interpretations, and thus are further investigated in these guidelines.
This investigation has lead the AI HLEG to establish seven key requirements to be met, through technical and
non-technical means, for the development, deployment, and use of AI systems.
• Human agency and oversight, thus allowing humans to assess or challenge the system, as well as governance
mechanisms of humans over the system.
• Technical robustness and safety, including resilience to attack and security, fall back plan and general safety,
accuracy, reliability and reproducibility.
• Privacy and data governance, especially privacy of acquired and generated user data, the quality and integrity
of the data used in learning processes, and a controlled access to user data.
• Transparency by supporting the traceability and explainability of the processes and decisions of the AI
system, as well as communicating about its abilities and limitations.
• Diversity, non-discrimination and fairness, especially the avoidance of unfair bias, allowing all people to
access and use the AI system through accessibility and universal design, and the participation of stakeholders
who might be directly or indirectly affected.
• Environmental and societal well-being by monitoring the impact on society and democracy and consider-
ing the broader society, other sentient beings and the environment, as stakeholders for sustainability and
environmental friendliness.
• Accountability through auditability, without impairing business models and intellectual property, minimi-
sation and reporting of negative impacts, including whistle-blowers and NGOs, ensuring trade-offs when
conflicts arise between these requirements, and supporting redress when adverse impacts occur despite the
care provided to meet these requirements.
To implement the above requirements, both technical and non-technical methods must be considered. By
technical methods, we could mention specific AI system architectures, “by design” conceptions (e.g. privacy-
by-design, security-by-design), explanation methods (e.g. field of Explainable AI (XAI)), testing and validating,
and quality of service (QoS) indicators. Non-technical methods could be regulations (e.g. product safety laws),
codes of conduct and key performance indicators (KPIs), standardisation (e.g. ISO, IEEE P7000), certification,
accountability via governance frameworks (e.g. person in charge of AI ethics), education, and social dialogue.
These methods, and more globally the requirements to satisfy, should encompass all stages of an AI systems life
cycle, as shown in Figure 1, reproduced from [AI 19].
Figure 1: Realising Trustworthy AI throughout the system’s entire life cycle
3 Opportunities for RE
If the reader is familiar with RE literature, we might dare to claim that reading the previous section was enough
to remind him or her of several RE works. Indeed, since the guidelines should be operationalized throughout
the whole life time of AI systems, it also covers all the phases of RE, from eliciting the requirements of the AI
system to verifying their satisfaction. Actually, each requirement can be related to recent works in the field:
• Human agency and oversight can be supported at development time, by supporting the decision making
about quality requirements at strategic and operational levels [OW18], as well as run time, by monitoring
the satifaction of the requirements and diagnosing their violations [VRCH17].
• Technical robustness and safety is also of interest, whether we speak about methods to follow official stan-
dards or the impacts of these methods on the people implementing them [PH17]. The AI HLEG also includes
in this requirement the security aspect, which can be covered by various sources of security requirements,
thus helping to reach a comprehensive set of security goals [GBO18].
• Privacy and data governance can be subject to interpretation misalignments, which can be mitigated through
shared ontologies [BHBN18], as well as incomplete privacy policies, which can increase the perceived privacy
risks of the users [BEB19].
• Transparency regarding how the AI system works can be achieved by showing how the components of the
produced AI system relate to its requirements, in other words having a performant traceability [HP18],
but transparency can also bring issues like information overload, information starvation, and transparency
leading to biased decisions [HSPA18].
• Diversity, non-discrimination and fairness can be achieved by ensuring that the relevant stakeholder pro-
files are explicitly considered, for example through a comprehensive list of personas and persona-based
modelling [NRJTW+ 18] or by analysing the massive feedback of actual users of an AI system [LZW18].
• Environmental and societal well-being is also an aspect that some RE researchers investigate, like integrating
green strategies in quality requirements for optimizing energy and other resources consumption [CFL18].
• Accountability is also of interest in RE, whether by supporting auditors of AI systems by ensuring that
traceability links reach a satisfying level of quality [HP18] or by supporting the audited companies in
evolving their acceptance tests based on the evolution of the requirements they should meet [HBCG18].
We can also go further by considering RE frameworks, like OpenReq [PFF18], which help requirements ana-
lysts through automated recommendations of various kinds, like new requirements, quality tips, stakeholders
to consider, or requirements prioritization. Such kind of framework could be strengthened and extended to
help requirements analysts to cope with the various dimensions covered by the AI HLEG requirements list. Al-
though not always related to AI, and without assuming they are the most relevant, these works deal with these
requirements in some ways, and thus provide illustrative examples of how RE is deeply linked to these guidelines.
At this point, we only related existing RE works to the high level requirements of the guidelines. But we can
further look in the details, especially by taking the dichotomy operated by the AI HLEG with technical and
non-technical methods to satisfy these requirements.
The technical methods mainly relate to what the AI scientific community may produce, but works in RE
can also help on some regards. For instance, we work on translating human-written mind map diagrams into
machine-readable graphs to produce knowledge bases to be queried [BGOK18]. The structured graphs and the
models used for translating the mind maps into the graphs offer a support to explain the translation. An AI
system based on such a knowledge base would be then at least partially explainable, as opposed to a knowledge
base directly built from unstructured data. We can also mention the writing of acceptance tests: automatically
translating requirements into acceptance tests is not a recent idea [GH99] and we are also working on how
to evolve tests with their requirements [HBCG18]. Quality of service (QoS) indicators can also build on the
satisfaction of quality requirements if we structure and formalize them well enough to build concrete quality
measures [OW18].
At the opposite, non-technical methods can have a rather comprehensive coverage of RE works. Regulations
are far to be ignored in RE, with for example the Nòmos framework to deal with them [Sie10, IJS+ 14]. Of
course, they are only one source of norms, and other approaches consider more sources of requirements to be
more comprehensive [GBO18], including standards and certifications. [BOJRG18] also highlights the lack of
standards to ensure the compatibility of the Internet of Things (IoT) ecosystems, an aspect also applicable to
ethical concerns since each component of an IoT ecosystem might fulfil them to different degrees and in different
ways, thus making hard to grasp the fulfilment of the whole. RE can also help in the elicitation of user preferences
from massive amounts of users [LZW18] and thus help in establishing lists of concrete requirements, based on
social dialogue, that governance frameworks could build on.
In brief, it is easy to find recent RE works to illustrate the multiple focuses of the AI HLEG guidelines. In
fact, to do so, we were prepared to look at papers published in several occurrences of RE-related venues like
REFSQ, EmpiRE, CAiSE, or iStar. In the end, the attentive reader may have noticed that, among the 18 RE
references we cite in this section, 11 are from REFSQ 2018 (almost half of the 23 papers of the venue), with 9 of
them used to illustrate all the key requirements presented. In other words, a single occurrence of a RE-related
conference can suffice to broadly illustrate the key requirements of the AI HLEG guidelines, an evidence that
further strenghten the correlation between these guidelines and the field of RE.
4 Challenges for RE
Of course, relating existing works to the AI HLEG requirements does not mean that we have all we need to
satisfy them. Rather, there is still a lot of work, starting from communicating better what we do. Indeed, despite
the 59 occurrences of “requirements” (without counting pictures) in the 41 pages of the AI HLEG document,
the total absence of “requirements engineering” should ring a bell.
Beside making people aware of the RE methods and tools, there is also space for improvements to support
the AI HLEG key requirements. For instance, although we have a growing interest in the challenges of AI-
related topics, like the Internet of Things (IoT) [BOJRG18], it is still young on the specificities of Big Data
issues [AM18], a core aspect of AI. There is also non AI-specific aspects which are still open issues in RE, like
how to deal with ambiguous and incomplete requirements [DvdSL18]. [CHD18] also shows that human analysts
can have various degrees of reliability in recovering traces, and thus further support is required. Requirements
analysts may exploit tools to help them in various RE tasks, but some are still far from a reliable use in
production [HP18]. Dealing with requirements in complex organizations is by itself challenging, with the difficulty
to detect infeasible requirements, the variance in assumptions and definitions between teams, or the overlook of
sources of requirements [ADW18, WPKG18].
We can also take a step further: after considering how to use RE for AI, we can also consider how to use AI
for RE. Like other fields, RE is gradually relying on more AI techniques to help the requirements analysts doing
their job [WV18, DvdSL18, PFF18, LZW18]. But if we claim that RE can help in designing ethical AI systems,
can we still say so if the AI systems RE relies on are not ethical? Can we ensure that the requirements we help
to discover allow to reach ethical AI if the AI used to identify those requirements is not ethical? In other words,
RE4AI is not the end of the loop: it is but a mere step towards RE4RE.
5 Conclusion
In this position paper, we attempted to show that RE has a tremendous opportunity in the current European
trends in the field of AI. We first explained how AI is considered as a key component for the future by the
European Comission, with the production of guidelines to produce trustworthy AI systems (i.e. lawful, ethical,
and robust). We listed the seven key requirements of these guidelines and tried to show how recent works in RE
relate to them, from both technical and non-technical perspectives. We hope that these relations show how the
field of RE, despite not being explicitly mentioned, is at the core of these guidelines, and thus the great amount
of opportunities that RE researchers may found in working on this topic. We also relied on recent works in RE
to remind some current challenges that may increase the difficulty of this task. Nevertheless, we have no doubt
that the RE community is strong and dynamic enough to tackle them. We are aware that the guidelines are
already two years old, and that our presentation is far from the high standards of the scientific rigour, but we
hope that this position paper brings a relevant and interesting point to the workshop discussions, maybe helping
at drawing a future research agenda.
6 Acknowledgements
This publication reflects the view only of the author, and Meritis PACA cannot be held responsible for any use
which may be made of the information contained therein.
References
[ADW18] Wasim Alsaqaf, Maya Daneva, and Roel Wieringa. Quality Requirements Challenges in the Con-
text of Large-Scale Distributed Agile: An Empirical Study. In Erik Kamsties, Jennifer Horkoff,
and Fabiano Dalpiaz, editors, REFSQ, volume 10753, pages 139–154. Springer International Pub-
lishing, Cham, 2018.
[AI 19] AI HLEG. Ethics Guidelines for Trustworthy AI. Report, European Commission, April 2019.
[AM18] Darlan Arruda and Nazim H. Madhavji. State of Requirements Engineering Research in the
Context of Big Data Applications. In Erik Kamsties, Jennifer Horkoff, and Fabiano Dalpiaz,
editors, REFSQ, volume 10753, pages 307–323. Springer International Publishing, Cham, 2018.
[BEB19] Jaspreet Bhatia, Morgan C. Evans, and Travis D. Breaux. Identifying incompleteness in privacy
policy goals using semantic frames. Requirements Engineering, 24(3):291–313, September 2019.
[BGOK18] Robert Andrei Buchmann, Ana-Maria Ghiran, Cristina-Claudia Osman, and Dimitris Karagian-
nis. Streamlining Semantics from Requirements to Implementation Through Agile Mind Mapping
Methods. In Erik Kamsties, Jennifer Horkoff, and Fabiano Dalpiaz, editors, REFSQ, volume
10753, pages 335–351. Springer International Publishing, Cham, 2018.
[BHBN18] Mitra Bokaei Hosseini, Travis D. Breaux, and Jianwei Niu. Inferring Ontology Fragments from
Semantic Role Typing of Lexical Variants. In Erik Kamsties, Jennifer Horkoff, and Fabiano
Dalpiaz, editors, REFSQ, volume 10753, pages 39–56. Springer International Publishing, Cham,
2018.
[BOJRG18] Johanna Bergman, Thomas Olsson, Isabelle Johansson, and Kirsten Rassmus-Grhn. An Ex-
ploratory Study on How Internet of Things Developing Companies Handle User Experience Re-
quirements. In Erik Kamsties, Jennifer Horkoff, and Fabiano Dalpiaz, editors, REFSQ, volume
10753, pages 20–36. Springer International Publishing, Cham, 2018.
[CFL18] Nelly Condori Fernandez and Patricia Lago. The Influence of Green Strategies Design onto Quality
Requirements Prioritization. In Erik Kamsties, Jennifer Horkoff, and Fabiano Dalpiaz, editors,
REFSQ, volume 10753, pages 189–205. Springer International Publishing, Cham, 2018.
[CHD18] Bhushan Chitre, Jane Huffman Hayes, and Alexander Dekhtyar. Second-Guessing in Tracing
Tasks Considered Harmful? In Erik Kamsties, Jennifer Horkoff, and Fabiano Dalpiaz, editors,
REFSQ, volume 10753, pages 92–98. Springer International Publishing, Cham, 2018.
[DvdSL18] Fabiano Dalpiaz, Ivor van der Schalk, and Garm Lucassen. Pinpointing Ambiguity and Incom-
pleteness in Requirements Engineering via Information Visualization and NLP. In Erik Kamsties,
Jennifer Horkoff, and Fabiano Dalpiaz, editors, REFSQ, volume 10753, pages 119–135. Springer
International Publishing, Cham, 2018.
[Eur18] European Comission. Artificial Intelligence for Europe. Communication from the Comission,
European Commission, April 2018. COM (2018) 237 final.
[GBO18] Ana-Maria Ghiran, Robert Andrei Buchmann, and Cristina-Claudia Osman. Security Require-
ments Elicitation from Engineering Governance, Risk Management and Compliance. In Erik
Kamsties, Jennifer Horkoff, and Fabiano Dalpiaz, editors, REFSQ, volume 10753, pages 283–289.
Springer International Publishing, Cham, 2018.
[GH99] Angelo Gargantini and Constance Heitmeyer. Using model checking to generate tests from re-
quirements specifications. ACM SIGSOFT Software Engineering Notes, 24(6):146–162, November
1999.
[HBCG18] Sofija Hotomski, Eya Ben Charrada, and Martin Glinz. Keeping Evolving Requirements and
Acceptance Tests Aligned with Automatically Generated Guidance. In Erik Kamsties, Jennifer
Horkoff, and Fabiano Dalpiaz, editors, REFSQ, volume 10753, pages 247–264. Springer Interna-
tional Publishing, Cham, 2018.
[HP18] Paul Hbner and Barbara Paech. Evaluation of Techniques to Detect Wrong Interaction Based
Trace Links. In Erik Kamsties, Jennifer Horkoff, and Fabiano Dalpiaz, editors, REFSQ, volume
10753, pages 75–91. Springer International Publishing, Cham, 2018.
[HSPA18] Mahmood Hosseini, Alimohammad Shahri, Keith Phalp, and Raian Ali. Four reference models
for transparency requirements in information systems. Requirements Engineering, 23(2):251–275,
June 2018.
[IJS+ 14] Silvia Ingolfo, Ivan Jureta, Alberto Siena, Anna Perini, and Angelo Susi. Nmos 3: Legal Compli-
ance of Roles and Requirements. In Eric Yu, Gillian Dobbie, Matthias Jarke, and Sandeep Purao,
editors, Conceptual Modeling, volume 8824, pages 275–288. Springer International Publishing,
Cham, 2014.
[LZW18] Tong Li, Fan Zhang, and Dan Wang. Automatic User Preferences Elicitation: A Data-Driven
Approach. In Erik Kamsties, Jennifer Horkoff, and Fabiano Dalpiaz, editors, REFSQ, volume
10753, pages 324–331. Springer International Publishing, Cham, 2018.
[NRJTW+ 18] Genana Nunes Rodrigues, Carlos Joel Tavares, Naiara Watanabe, Carina Alves, and Raian Ali. A
Persona-Based Modelling for Contextual Requirements. In Erik Kamsties, Jennifer Horkoff, and
Fabiano Dalpiaz, editors, REFSQ, volume 10753, pages 352–368. Springer International Publish-
ing, Cham, 2018.
[OW18] Thomas Olsson and Krzysztof Wnuk. QREME Quality Requirements Management Model for
Supporting Decision-Making. In Erik Kamsties, Jennifer Horkoff, and Fabiano Dalpiaz, editors,
REFSQ, volume 10753, pages 173–188. Springer International Publishing, Cham, 2018.
[PFF18] Cristina Palomares, Xavier Franch, and Davide Fucci. Personal Recommendations in Require-
ments Engineering: The OpenReq Approach. In Erik Kamsties, Jennifer Horkoff, and Fabiano
Dalpiaz, editors, REFSQ, volume 10753, pages 297–304. Springer International Publishing, Cham,
2018.
[PH17] Luciana Provenzano and Kaj Hnninen. Specifying Software Requirements for Safety-Critical
Railway Systems: An Experience Report. In Paul Grnbacher and Anna Perini, editors, REFSQ,
volume 10153, pages 363–369. Springer International Publishing, Cham, 2017.
[Sie10] Alberto Siena. Engineering Law-Compliant Requirements: the Nomos Framework. phd, University
of Trento, January 2010.
[VRCH17] Michael Vierhauser, Rick Rabiser, and Jane Cleland-Huang. From Requirements Monitoring to
Diagnosis Support in System of Systems. In Paul Grnbacher and Anna Perini, editors, REFSQ,
volume 10153, pages 181–187. Springer International Publishing, Cham, 2017.
[WPKG18] Rebekka Wohlrab, Patrizio Pelliccione, Eric Knauss, and Sarah C. Gregory. The Problem of Con-
solidating RE Practices at Scale: An Ethnographic Study. In Erik Kamsties, Jennifer Horkoff,
and Fabiano Dalpiaz, editors, REFSQ, volume 10753, pages 155–170. Springer International Pub-
lishing, Cham, 2018.
[WV18] Jonas Paul Winkler and Andreas Vogelsang. Using Tools to Assist Identification of Non-
requirements in Requirements Specifications A Controlled Experiment. In Erik Kamsties, Jennifer
Horkoff, and Fabiano Dalpiaz, editors, REFSQ, volume 10753, pages 57–71. Springer International
Publishing, Cham, 2018.