Possible extension of ISO/IEC 25000 quality models to Artificial Intelligence in the context of an international Governance Domenico Natale UNINFO UNI CT 504 Technical Committee System and Software Engineering Italy dnatale51@gmail.com Abstract— This paper examines the possibility of JTC1/SC7 [7] Working Group 6 "Software product and extending the principles of ISO/IEC 25000 (SQuaRE) to a system quality ". In the field of Artificial Intelligence (AI), quality model for Artificial Intelligence. Some results are the specific sub-committee ISO/IEC JTC1/SC42 [8] is described by comparing ISO/IEC 25000 product quality models to unstructured documents and guidelines concerning developing an extension of SQuaRE to a “Quality model quality aspects of Artificial intelligence. The analysis shows for AI-based systems”, with which the sub-committee SC7 some possible reuse of the models examined in the context of WG 6 is collaborating. An exemplary attempt to use the AI. The emerging important need of a general Governance of ISO/IEC 25000 quality characteristics for AI has already AI, including management, processes and organizational been studied for the conference IWESQ 2019 [9]. aspects, are outside the scope of SQuaRE. Keywords: quality model, quality characteristics of product, II. QUALITY CHARACTERISTICS CONSIDERED software quality, data quality, service quality, quality in use, The methods adopted consists in the logic comparison governance, standard, artificial intelligence. between SQuaRE quality characteristics and characteristics mentioned in the documents analysed [1]. I. INTRODUCTION The number of characteristics defined in SQuaRE for In this paper we examine the results of the analysis each model are: produced in six documents and guidelines concerning AI recently published by the European Commission, CEN - Software product: 8 CENELEC, ETSI, IEEE, the Italian authorities AGID and - Data: 15 MISE [1]. The aspects of quality related to products and - IT Services: 8 governance derive from shared considerations arisen by - Quality in use: 5 various kinds of experts and organizations. The goal is to The number of quality characteristics mentioned in the encourage further reflections and research works. documents analysed [1] are: - Software product: 11 Some of these documents have been presented at the - Data: 7 Conference “Artificial Intelligence: for human governance. - IT Services: 2 Educational and social perspectives” [2] held online in Italy - Quality in use: 3 on September 2020 25-26th. During the conference, the speakers introduced the issue of the need of including technological aspects into a governance schema for the various application fields. The purpose of this analysis is to promote the collection of new quality characteristics that could be considered for an extension of the ISO / IEC 25000 quality models [3], hoping for the harmonization of products with processes, management, organization and Governance. In this document we will focus mainly on the quality aspects of the products, while being aware of how much these aspects are influenced by the above factors. We will focus on the importance of the quality aspects of the product defined by SQuaRE, with particular reference to software [4], data [5], services [6] and quality in use [4]. It intends also to focus on aspects around quality management, quality measurement, quality requirements, quality evaluation. This Figure 1. Number of SQuaRE characteristics and number of further approach was used to classify the numerous quality quality aspects detected by Guidelines examined characteristics mentioned in the analyzed documents. The different numbers (SQuaRE and Aspect of AI) suggest taking more into account the data characteristics and The aspect of product quality is emerging in the series services in future documents concerning AI and in ongoing of ISO/IEC 25000 series developed by ISO/IEC projects. In fact, during the mentioned conference, the Copyright © 2020. Use permitted under Creative Commons License Attribution 4.0 (CC BY 4.0). majority of the speakers (in total about 100) pointed out the Data quality importance of data management and data quality, particularly within machine learning and specific ISO/IEC 25012: algorithms. It is also true that not all SQuaRE quality - Accuracy characteristics for software, services and quality in use - Completeness have been considered in the documents and that new - Consistency features have been added. - Credibility The Conference was born as a cultural and interdisciplinary - Currentness proposal inspired by a polyphony of knowledge and skills, - Accessibility with the aim of stimulating the broadest possible debate on - Compliance AI. For these reasons have been involved sociologists, - Confidentiality philosophers, educators, psychologists, students, - Efficiency programmers, technicians, managers, exponents of private - Precision companies and public institutions. - Traceability - Understandability The topics covered by the speakers (the most part by Italy) - Availability have been: - Portability - Recoverability - Ethics, communication, culture, education (28%) - Technology, quality aspects, security (22%) Guidelines: - Governance (17%) - Health, therapy, pharmacology application (17%) - Analyzability - Legislation, laws, economy, social aspects (16%) - Accessibility - Confidentiality The diversity of themes suggests new perspectives of - Data strategy standardization with multiple group and experts. - Role of data - Acquisibility - Provenance III. ELICITATION OF NEW ASPECTS OF QUALITY FOR PRODUCTS AND GOVERNANCE ACTIVITY Aspects in common between ISO/IEC 25012 and Guidelines: In the following list are reported the SQuaRE quality characteristics and the aspects of quality detected by - Accessibility experts and organizations mentioned in the Guidelines - Confidentiality for software, data, services, quality in use and Table 2. Comparaison between SQuaRE and Guidelines about data government. Service quality: Software and systems quality ISO/IEC TS 25011: ISO/IEC 25010 (par. 4.2): - Suitability - Functional suitability - Usability - Performance efficiency - Security - Compatibility - IT service reliability - Usability - Tangibility - Reliability (Maturity) - Responsiveness - Security (Accountability) - IT service adaptability - Maintainability (testability) - IT service maintenability - Portability Guidelines: Guidelines: - Velocity of the network - Sustainability - 5G - Equity - Use of IoT - Accountability - Transparency Aspects in common between ISO/IEC TS 25011 - Reliability and Guidelines: - Safety - none - Robustness - Compliance Table 3. Comparaison between SQuaRE and Guidelines about Service - Testing Quality in use Aspects in common between ISO/IEC 25010 and ISO/IEC 25010 (par. 4.1): Guidelines: - Effectiveness - Reliability - Efficiency Table 1. Comparaison between SQuaRE and Guidelines about - Satisfaction (Trust) software - Freedom for risk (Economic, Health, Copyright © 2020. Use permitted under Creative Commons License Attribution 4.0 (CC BY 4.0). Environment) models comparisons and researches. To clarify the - Context coverage relationships between the various element involved in AI, it could be useful to distinguish all the items described in in Guidelines the Glossary prepared for the mentioned Conference, and - Trustworthy published in the section “Shared knowledge” [11], - Health and well-being distinguishing: - Environment - the constraints that influence the development of an - Economic impacts AI system such as: Ethics, Human governance, Aspects in common between ISO/IEC 25000 and standards, laws, quality models; Guidelines: - the technologies and platforms, used in AI systems: Big data, Cloud computing, Quantum computer, - Freedom for risk (Economic, Health, Neural Network, Machine Learning, Robot. Environment) Table 4. Comparaison between SQuaRE and Guidelines about Quality Using for example the SADT (Structural Analisys and in use Design Technique) method [12], it is possible to put constraints on the top, and technology mechanism to the In addition to the quality aspects of the product counted bottom of the following Figure 2 simulating a simplified in previous Tables, have been highlighted in the documents representation of an “AI Ontology” schema. analyzed many elements useful for a definition of the activity of Governance, management and processes, issues that are outside the scope of ISO/IEC 25000 SQuaRE series. Governance, management, process ISO/25000: Governance and process are out of the scope, except some aspects related to quality management. Guidelines: - evaluation of the impact of algorithms - law and regulations - software engineering processes - human vigilance Figure 2. AI Ontology schema - data governance - legality It should be noted in the figure how the optimization - ethics activities in the real environment of the running system - non-discrimination and fairness could generate autonomous changes in algorithms to be - responsibility kept under control with governance activities. - human and machine roles - decision making - digital sovereignty V. CONCLUSION The components of AI are multiple, sometimes Aspects in common between ISO/IEC 25000 and considered independently of each other, sometimes seen as Guidelines: a whole. In many sectors, efforts are being made to - laws and regulations (limited to data quality incorporate further quality characteristics by experts that characteristic about compliance) could be included in the quality models being prepared by the ISO commissions, particularly by ISO/IEC JTC1 SC42 Working Group 3 with the support of SC7 liaison. The software/systems life cycle processes are the main Many organizations are active for a systemic accepted working area which SC7/WG7 actively is working worldwide view of AI (European Commission, including integration and acquisition processes. In International and national Authorities, Industries, addition, the related tools and methodologies are run by Universities and Associations). SC7/WG26 and WG4, thus activities for AI related quality have to be considered by work products developed by other SCs and SC7/WGs including SC42 also. IV. FURTHER STUDIES It may be interesting in the future to extend SQuaRE quality models to new technologies and to a deepen relationships and definitions between models by examining synonyms as well of the new specific quality characteristics for AI. Works under development [10] could benefit from Figure 3. Stakeholders involved in AI standardization activities Copyright © 2020. Use permitted under Creative Commons License Attribution 4.0 (CC BY 4.0). Many experts and institutions are seeking to build a foundation of trust for this new technology, aiming at improving IT services in the area of the economy, health and environment. REFERENCES [1] D. Natale “Guidelines on the quality model of Artificial Intelligence” https://intelligenzartificiale.unisal.it/orientamenti-sul- modello-di-qualita-dellintelligenza-artificiale/ [2] https://intelligenzartificiale.unisal.it (all traslations available are obtained automatically with Google translator) [3] ISO/IEC 25000 https://www.iso.org/standard/64764.html [4] ISO/IEC 25010:2011 Systems and Software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models [5] ISO/IEC 25012:2008 Systems and Software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - Data quality model [6] ISO/IEC TS 25011:2017, Systems and Software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) – Service quality model. [7] Brochure ISO/IEC JTC1/SC7 https://www.iso.org/files/live/sites/isoorg/files/developing_standard s/who_develops_standards/docs/ISO_IEC_JTC%201_SC%207%20 Brochure.pdf [8] ISO/IEC JTC1/SC42 https://www.iso.org/committee/6794475.html [9] D. Natale, A. Trenta “Examples of practical use of ISO/IEC 25000”, APSEC IWESQ 2019 (CEUR-WS.org, ISSN 1613- 0073) [10] ISO/IEC WD 5059 (it is just started and under development https://www.iso.org/standard/80655.html?browse=tc) In the last plenary of SC42 held in October 2020 the standard was renamed ISO/IEC 25059 as part of SQuaRE [11] D. Natale, M.C. Lorenzelli “Glossary and Video Glossary” https://intelligenzartificiale.unisal.it/glossario-e-videoglossario-ia/ [12] SADT technique was used since 1980 by the Italian DAFNE methodology (DAta and Function NEtworking) for analysis and design of traditional systems, D. Natale, “Quality and quantity in software systems”, FrancoAngeli, Italy, 1995 Copyright © 2020. Use permitted under Creative Commons License Attribution 4.0 (CC BY 4.0).