The integration of digital technology and AI systems has prompted extensive inquiries into their ethical design and implementation. Trustworthy AI, essential for fairness, robustness, safety, and transparency, is recognized as fundamental, transcending pure technological interaction. This is particularly pivotal in contexts involving minors, and especially within technologies fostering inclusion for students with disabilities and special educational needs (SEND). Inclusive education hinges on trustworthiness due to inherent asymmetries in learning structures and the relational aspect of educational environments. This paper starting from a case studies, i.e. an AI interface tailored to children with text comprehension dificulties, it introduces a co-creation strategy with diferent stakeholders. It develops an ecological framework for trustworthiness, rooted in value-sensitive design. Our approach emphasizes ethical and trustworthy AI development, prioritizing responsibility, reliability, and inclusivity. It addresses the concept of trustworthiness as a systemic relationship between multiple contexts (such as clinical and school environments) in the development of children's proximal processes and scafolding, as outlined by Bronfenbrenner's system ecological theory.
most multifaceted and the most interconnected with all the others domains.
This is especially true and even more crucial if we think about the use of AI with and by minors and, particularly, within technologies aimed at fostering inclusion in contexts of students with disability and special educational needs (SEND). Within the framework of inclusive education and learning, the aspect of trustworthiness is pivotal for several reasons: • The inherent asymmetry in the structure of learning, educational, and rehabilitative interactions. • The relational aspect of educational and rehabilitative environments, grounded in trust and communicative exchanges. • The developmental trajectory of the child, which encompasses cognitive and psychological dimensions.
Considering the intricate nature of the educational and learning dynamics, the conscientious and trustworthy integration of AI-based technologies within this domain, particularly when involving vulnerable subjects, necessitates a thorough evaluative analysis. Such an analysis should encompass an examination of the interplay among the diferent systems and players involved.
In this paper, we start by a case study to illustrate the example of a technology built for therapeutic and educational purposes, with a focus on inclusivity, ethical
In recent years, the increasingly technologically mediated nature of our societies, fueled by the systematic integration of digital technology and the escalating ubiquity of AI systems have prompted increasingly profound inquiries into the ethical, responsible, and trustworthy design and implementation of AI. This trend is further underscored by the ongoing legislative endeavors at an European and international level.
While there is no universally accepted definition of
trustworthy AI, it is widely acknowledged as a
fundamental principle [
Secondly, we propose a co-creation approach grounded tions and adopt a perspective that encompasses diverse
in value-sensitive design, involving key stakeholders to stakeholders, design aspects, and values promoted in the
ensure the ethical development of the AI interface. Draw- development and implementation process. Despite the
ing from ecological frameworks, such as Bronfenbren- recognized importance of stakeholder involvement,
rener’s ecological systems theory [
The issue of AI trustworthiness is therefore positioned framework proposed by [9], which emphasizes inclusive
at the crossroads of various interaction systems known as AI education grounded in Universal Design for Learning
proximal processes. These processes are pivotal because principles.
they serve as the conduit through which individuals learn The integration of Artificial Intelligence holds promise
and adapt over time within their interactional environ- for children with special learning needs, including those
ments [
In the context of European and international guide- and handicap (the condition of disadvantage resulting lines, as outlined by UNESCO specifically for AI in educa- from the objectification of the impairment). According to this conception, disability is seen as something exclu- renders it a critical resource for enhancing people’s lives sively pertaining to the individual and limits her ability to and expanding their opportunities. However, it is essenlive a full and satisfying life and to flourish in society [ 20]. tial to recognize its situated nature, shaped by social and Disability movement activists in the late 20th century value determinants. Technology thus becomes a matter posed a significant challenge to this prevailing paradigm. of justice. Consider, within the realm of education, the They critically examined the connection between impair- extent to which knowledge can be made accessible to ment and disability, thus afirming the social model of diverse needs and characteristics through design—both disability [21]. This conception emphasises the role of the technological and otherwise—that is attentive to difermaterial organisation of society in creating conditions ences, as articulated in Universal Design for Learning of disability from a deficit. Therefore, in this view an (UDL). inclusive society is one that can eliminate the disabling These considerations are particularly pertinent regardcontexts and barriers that society possesses in relation ing AI-based systems. Fourth-order technologies [28], to the diversity of human functioning. One of the most capable of replicating human actions in the absence of husignificant steps in this debate was the development of man intelligence, bring with them potentials, challenges, the ICF model [22]. At the centre of the bio-psycho-social and risks well-documented in literature and institutional approach, there is the individual-environment interac- discourse. For inclusive learning objectives, AI-based tion. Disabilities are considered beyond an exceptionalist technologies must be designed and implemented with logic and are placed at the centre of the human condition adherence to various criteria to ensure accountability itself. What links disability and health, according to this and reliability within an inclusive learning environment. model, depends on the functionings or, in other words, Beginning with an assessment of the ethical treatment the capabilities that each human being in relation to the of data (bias, privacy) and algorithms (fairness, transsocial context of reference is capable of having and being. parency, explainability), an inclusive perspective on techThus, the ICF model, and the Capability Approach (CA), nology prompts a critical examination of its embedding [23],[24] although not totally overlapping, emphasise context and its interaction with and involvement of key how well-being (individual and collective) arises from stakeholders. Moreover, from a technical perspective, the dynamic between internal dispositions/opportunities technologies for inclusion should be designed so that the and external dispositions/powers, a dynamic that is to be users can personalize their interaction with the AI system understood both as the absence of external impediments through adjusting parameters and controls according to (negative freedom) and as an analysis of the conditions their preferences or specific needs. and quality of life that people are able to lead (positive freedom).
In this context, the role of technology acquires a cen- 3. ARTIS: system description tral function, in fact, the use of technology in relation to disability could play a crucial role in the expansion In designing AI-based technologies for inclusive learnof individuals’ capabilities [25]. Technology finds a spe- ing objectives, such as the interface ARTIS developed by cific place within the Environmental Factors of the bio- Sony Computer Science Laboratories Paris and Centro psycho-social perspective of the ICF. Starting from the Ricerca e Cura, ensuring accountability and reliability assumption that people can function in diferent ways within an inclusive learning environment is paramount. depending on their environments and that disability is This necessitates adherence to various criteria, beginning therefore the result of a mode of interaction between with an ethical assessment of data and algorithms, encomthe individual and the environment, the ICF considers passing considerations of bias, privacy, fairness, transtechnologies as tools that mediate this interaction. Tech- parency, and explainability. An inclusive perspective on nologies thus, depending on the way they are designed, technology calls for a critical examination of its embedimplemented, and used, can act as facilitators or, on the ding context and its interaction with key stakeholders. contrary, as barriers in performing normal activities and ARTIS, as an interface powered by artificial intelligence creating an inclusive society. It would be naive to con- to support text comprehension, exemplifies this approach sider technology’s relationship with disability solely in by drawing on neuro-psycholinguistic models of reading instrumental terms. As extensively debated in the lit- comprehension, thus integrating linguistic components erature ([26], [27]), technologies always exist within a into its design to enhance accessibility and inclusivity in specific context and invariably embody certain underly- learning contexts. ARTIS [29] is an interface, powered ing values and orientations. In the context of disability, by artificial intelligence, designed to support text comtechnology assumes a pivotal role, often becoming an prehension. Born as a collaboration between Sony Computer Science Laboratories Paris 1 and Centro Ricerca integral part of an individual’s interaction with their environment. Technology’s capacity to mediate this relationship between the individual and the environment 1https://csl.sony.fr/ e Cura 2 in Rome, Italy. The interface was developed 4. Ecological framework for a from neuro-psycholinguistic models of reading compre- trustworthy AI with children hension, focusing on the linguistic components of text processing. The project description underlines how the objectives of
In particular, subjects with poor text comprehension this project are multifaceted. It emphasize rigorous represent dificulties related to the processing of syntactic search to ensure computational functionality, facilitating and semantic sentence components [30], the analysis of integration into rehabilitation and educational activities. lexical components of words [31] and deficits in the syn- Additionally, specialized research is essential to examine tactic representation of words and oral comprehension the impact, acceptance and feasibility within inclusive skills [32]. Moreover, [33] stated that the same subjects educational and rehabilitative contexts. To address this, report significant deficits in receptive vocabulary and se- we framed future research directions in a conceptual mantic processing. Finally, [34] and [35] addressed the framework considering impacts on child development issue of grammar, claiming that children and adolescents and the interplay between inter-subjective and objectual with problems in text comprehension show dificulties dimensions of proximal development. in understanding the role of pronouns within sentences, The literature on AI trustworthiness is vast and mulespecially if these are in clitic form. Considering this ap- tidisciplinary. With regard to HCI (human computer proach, ARTIS allows for personalized practice on texts interaction), the issues that are most focused on concern at diferent levels. Using AI algorithms, the interface can the design and perception of the user and the psychoautomatically extract keywords, associate pictograms, logical mechanisms that impact the perception of trustidentify more complex vocabulary and generate semantic worthiness and thus the subsequent usage behaviour networks, and practice grammatical components. ARTIS [36],[37],[38]. Therefore, one of the main focuses is on is aimed at primary and secondary school children with predictability, transparency, explainability, robustness of dificulties in text comprehension, but it can also be used the system. Trust is generally interpreted as a psychologas a support for L2. ical mechanism that occurs in social or intersubjective
We had three main goals in mind when designing interaction to reduce the uncertainty of the other’s beour prototype. Firstly, we aimed to develop a system haviour. specifically for children from second grade to adoles- According to the OECD, Organisation for Economic cents diagnosed with reading comprehension dificulties. Co-operation and Development : “AI might be considered Secondly, we wanted an interface suitable for therapeu- trustworthy when it does properly what it is supposed to do, tic sessions, always under professional supervision for but also when one can trust that human beings will use it younger age groups. Lastly, we aimed to involve end- in a fair and appropriate way”[39]. The trustworthiness users and stakeholders from the outset of the design of AI is also considered a crucial element within the eduprocess, collaborating with skilled speech and language cational literature, and involves the aspects of reliability, therapists for evaluation and feedback. Drawing inspi- fairness, transparency, explainability, data protection and ration from Kitsch and Van Dijk’s model, we integrated bias. functionalities to enhance the superficial representation However, we believe that the context of the ARTIS of language, focusing on lexical and morphosyntactic project poses an even greater challenge to the concept understanding. Our interface consists of three modules: of trustworthy AI, because it must be ensured in such a the first aids in understanding words and sentences, the way that this characteristic ’collapses’ within diferent second focuses on coherent sequence representations, settings and interactions: rehabilitation, school, family and the third aims at creating a broader mental model of and individual use. The interface, in fact, not only hylanguage. We also implemented a Synset Networks fea- bridises the educational and rehabilitative relationship ture to link words encountered in the text with previous in a consistent manner, but also assumes a scafolding knowledge. This approach fosters vocabulary expansion function for the child, crucial for proximal development and deeper understanding of word meanings. Overall, processes: that is, the diference between an individual’s our design reflects a comprehensive approach to address actual level of development and the level that can be reading comprehension challenges reached, through the help of a tutor or through social interaction [40]. Therefore, the ARTIS project shows how it is necessary to take a holistic view in order to correctly interpret the issue of trustworthiness within an inclusive learning context. In particular, the ecological system theory elaborated by Bronfenbrenner turns out to be a strategic framework.
According to the author of the ecological theory of human development, there are two forms of proximal to trustworthy in inclusive education and speech and development processes: a) those with other persons and language therapy through the cooperation with a broad b) those with symbols and objects; and these processes number of stakeholders. Moreover, we outlined the occur in a dynamic of reciprocal influence and determi- general framework to carry on future research on the nation of several environmental levels: the microsystem ARTIS project rooted in an developmental ecological (in which interpersonal relationships occur directly and theory and cooperation strategies. Future work in this in defined contexts, such as family, school, peers), the area should focus on developing an integrated approach mesosystem (relationships between microsystems, for to assess the impact and acceptance of technology in example), the macrosystem (cultural and social norms), educational settings, with a particular emphasis on the exosystem (environments that influence indirectly, engaging a more diverse range stakeholders such as such as parental work) and the chronosystem (temporal teachers, children and families. Building upon existing changes, such as historical events and life transitions). research, future studies could explore innovative
However, the nature of digital and AI technologies methodologies that combine quantitative metrics with is precisely to hybridise these systems and their inter- qualitative insights to gain a holistic understanding of actions [41]. The proximal process therefore occurs in the complex dynamics involved. Additionally, there is a a hybridised manner, involving both interpersonal re- need to investigate the long-term efects of technology lations and object relations, and collapsing the interac- integration on learning outcomes and socio-emotional tions between diferent levels even more directly, There- development across various age groups. Collaborative fore, it is crucial to ask, for example, in ARTIS how does eforts involving researchers, educators, policymakers, the macrosystem and exosystem (the decisions made by and technology developers will be essential to address the developers) afect the interactions in the microsys- the multifaceted concerns raised by teachers, families, tem? How does the hybridisation of the microsystem therapists, and children, thereby fostering a more (class, family, rehabilitation) reshape the concepts and inclusive and supportive educational environment. processes of inclusion? It’s crucial to consider not only the impact of technology failures, such as malfunctioning We acknowledge partial financial support from algorithms or discriminatory biases, but also its intended PNRR MUR project PE0000013-FAIR outcomes. To address these questions comprehensively, a collaborative strategy engaging all educational stakeholders—teachers, clinicians, parents, and students—is References essential for qualitative and quantitative insights.
As the project interfaces blend educational and rehabilitative dynamics, it becomes clear that a holistic approach is indispensable. In ARTIS, a collaborative strategy is essential, involving stakeholders across education—teachers, clinicians, parents, and students—to ensure AI trustworthiness in inclusive learning environments. This multifaceted cooperation strategy integrates academic and industry channels, with early involvement of speech and language therapists ensuring interdisciplinary engagement. After the initial proof of concept, the interface underwent testing by children, therapists, and the public, promoting User-Centered Design principles and enabling ongoing monitoring. Focus groups involving developers, therapists, and ethics experts ensured ethical oversight, fostering critical thinking and responsible technology usage.
In this paper, we presented the ARTIS project as a significant case study illustrating a systemic approach Sánchez, Uncovering blind spots in education [25] N. Coombs, Disability and technology: A historical ethics: Insights from a systematic literature review and social perspective. (1990). on artificial intelligence in education, International [26] B. Friedman, Value-sensitive design, interactions 3 Journal of Artificial Intelligence in Education (2023) (1996) 16–23.
1–40. [27] D. J. Gunkel, The machine question: Critical per[9] Y. Song, L. R. Weisberg, S. Zhang, X. Tian, K. E. spectives on AI, robots, and ethics, mit Press, 2012.
Boyer, M. Israel, A framework for inclusive ai [28] L. Floridi, The fourth revolution: How the infolearning design for diverse learners, Computers sphere is reshaping human reality, OUP Oxford, and Education: Artificial Intelligence (2024) 100212. 2014. [10] C. CAST, Universal design for learning guidelines [29] M. Galletti, E. Pasqua, F. Bianchi, M. Calanca, version 2.0, Author Wakefield, MA (2011). F. Padovani, D. Nardi, D. Tomaiuoli, A reading [11] W. Holmes, F. Miao, et al., Guidance for generative comprehension interface for students with learning AI in education and research, UNESCO Publishing, disorders, in: Companion Publication of the 25th In2023. ternational Conference on Multimodal Interaction, [12] M. Rowe, Shaping our algorithms before they shape 2023, pp. 282–287.
us, Artificial intelligence and inclusive education: [30] J. Oakhill, Instantiation in skilled and less skilled Speculative futures and emerging practices (2019) comprehenders, The Quarterly Journal of Experi151–163. mental Psychology Section A 35 (1983) 441–450. [13] P. Kousa, H. Niemi, Artificial intelligence ethics [31] K. Nation, M. J. Snowling, Factors influencing from the perspective of educational technology syntactic awareness skills in normal readers and companies and schools, Learning: Designing the poor comprehenders, Applied psycholinguistics 21 Future (2023) 283. (2000) 229–241. [14] W. Holmes, I. Tuomi, State of the art and practice [32] R. Padovani, La comprensione del testo scritto in in ai in education, European Journal of Education età scolare. una rassegna sullo sviluppo normale e 57 (2022) 542–570. atipico, Psicologia clinica dello sviluppo 10 (2006) [15] L. Cottini, et al., Didattica speciale e inclusione sco- 369–398.
lastica, Carocci, 2017. [33] K. Nation, M. J. Snowling, Beyond phonological [16] A. Lascioli, et al., Verso l’inclusive education, Edi- skills: Broader language skills contribute to the zioni del Rosone, 2014. development of reading, Journal of research in [17] S. Vehmas, Ethical analysis of the concept of dis- reading 27 (2004) 342–356.
ability, Mental retardation 42 (2004) 209–222. [34] F. Arosio, C. Branchini, L. Barbieri, M. T. Guasti, [18] L. Terzi, A capability perspective on impairment, Failure to produce direct object clitic pronouns as a disability and special needs: Towards social justice clinical marker of sli in school-aged italian speaking in education, Theory and research in education 3 children, Clinical linguistics & phonetics 28 (2014) (2005) 197–223. 639–663. [19] W. H. Organization, N. C. for Health Statistics (US), [35] A. Zachou, E. Partesana, E. Tenca, M. T. Guasti, The International Classification of Diseases, 9th et al., Production and comprehension of direct Revision, Clinical Modification: Procedures: tabular object clitics and definite articles by italian children list and alphabetic index, volume 3, Commission on with developmental dyslexia, Advances in language Professional and Hospital Activities., 1980. acquisition (2013) 464–471. [20] H. Kuhse, P. Singer, P. Singer, Should the baby live?: [36] R. Lukyanenko, W. Maass, V. C. Storey, Trust in artiThe problem of handicapped infants, volume 228, ifcial intelligence: From a foundational trust frameOxford University Press Oxford, 1985. work to emerging research opportunities, Elec[21] M. Oliver, M. Oliver, The politics of disable- tronic Markets 32 (2022) 1993–2020. ment—new social movements, The politics of dis- [37] L. P. Robert Jr, G. Bansal, N. Melville, T. Staford, ablement (1990) 112–131. Introduction to the special issue on ai fairness, trust, [22] W. H. Organization, International Classification of and ethics, AIS Transactions on Human-Computer Functioning, Disability, and Health: Children & Interaction 12 (2020) 172–178.
Youth Version: ICF-CY., World Health Organization, [38] H. Choung, P. David, A. Ross, Trust in ai and its role 2007. in the acceptance of ai technologies, International [23] S. M. McMurrin, The Tanner lectures on human Journal of Human–Computer Interaction 39 (2023) values, volume 8, Cambridge University Press, 2011. 1727–1739. [24] M. C. Nussbaum, Frontiers of justice: Disability, [39] S. Vincent-Lancrin, R. Van der Vlies, Trustworthy nationality, species membership, in: Frontiers of artificial intelligence (ai) in education: Promises Justice, Harvard University Press, 2007. and challenges (2020). [40] N. Yelland, J. Masters, Rethinking scafolding in the information age, Computers & Education 48 (2007) 362–382. [41] J. L. Navarro, J. R. Tudge, Technologizing bronfenbrenner: neo-ecological theory, Current Psychology 42 (2023) 19338–19354.