Artificial Intelligence (AI) is pervading our lives in numerous ways today. It is important to apply ethical principles to guide the development and usage of AI systems to prevent harms or discrimination through AI algorithms. This has led to various ethical regulations and guidelines being formed at the corporate, national and supra-national level. The EU AI Act classifies the usage of AI in education as 'high-risk' as “such systems may violate the right to education and training as well as the right not to be discriminated against and perpetuate historical patterns of discrimination” [1, p. 26]. However, there has been little attention paid to ethics in AI in Education (AIED) in literature and there is only one existing framework to ethically guide AIED. AIED ethics is complex as it has to combine both general AI ethics and the ethics of educational technology. We aim to create a theoretical framework for AIED, comprising implementation guidelines for developers and organizational users of AI in education. In this paper, an existing draft framework by Holmes et al. is adapted by using insights from literature in the ethics of AI, ethics of educational technology and ethics of AIED.
Artificial Intelligence (AI), once a buzzword, is now a reality. It is being used in many aspects of our lives including healthcare, transport, communication, agriculture, ifnance and education. The usage of AI in classrooms and in education is promising and provides opportunities to improve the education process with technological innovations. AI has been applied in educational contexts riculum and content development, instruction, and students’ learning processes [3]. AI systems have enabled early detection and redress of learning shortcoming by analyzing student data - thereby providing a more customized learning experience for students [3]. Over the past decade, the use of AI tools to support or enhance learning has grown exponentially [4]. In a recent literature review, Chen et al. looked at 20 years of AI in Education (AIED) from 2000 to 2019 and shared several relevant findings: (a) the domain of AIED has received increased interest in the last few years, owing to the positive efect of AI on learning performance; (b) there is an creased scientific output; (c) AIED research is especially found in interdisciplinary journals with a dual focus on education and technology [5].
AI in our lives. As defined by Potter Stewart, “Ethics is knowing the diference between what you have a right to do and what is right to do” [6]. It is important to ethically guide the development and usage of AI for several reasons. The primary reason is that AI is being increasingly integrated into our lives and therefore has the potential for widespread influence and direct control over people’s pact numerous lives with far-reaching consequences. AI technologies are being developed at a high speed to automate tasks that are traditionally done by humans. The parties implementing the automation of tasks are at risk of not fully considering the ethical consequences in an effort to improve eficiency and save costs [ 7]. When such automated tasks involve any sort of decision-making by
individuals and have a potential for dangerous consequences.
The EU AI Act [1] classifies the usage of AI in education as ‘high-risk’ as “such systems may violate the right discriminated against and perpetuate historical patterns of discrimination” [1, p. 26]. In addition, ethics for AIED have not been discussed at the forefront of national AI policy strategies [8]. Schif examined 24 national AI policy strategies from G-7 and OECD countries and other important global actors such as India, China, Russia, Singapore and Malta [8]. The author found that remarkable attention has been paid to AI ethics in general, but this
increase in AIED literature over the years, showing an in- to education and training as well as the right not to be of education as a sector is an anomaly because many of a single framework, considering the overlap between these national AI policy documents “discuss the use of these two domains. Thereby, 2.1 looks into the ethics AI not only for healthcare, but also for transportation, of AI and 2.2 looks into the ethics of educational techagriculture, finance, and many other sectors” [ 8]. In ad- nology individually. 2.3 examines the overlap between dition, of the 4-5 countries that discussed AIED as a tool the above two domains and looks at existing AIED ethics for teaching, learning and educational administration, frameworks. none of them commented on or discussed AIED ethics.
This is a cause of concern as there is no consideration 2.1. Ethics of AI about the ethical approach to AIED among policymakers [8]. Until now, there exists only one framework for The ethics of AI in general have been studied extensively ethical AIED developed by ‘The Institute for Ethical AI and numerous frameworks and policies have been develin Education’ aimed at those making procurement and oped for AI ethics. The inventory of AI Ethics guidelines application decisions regarding AIED [9]. by the Algorithm Watch [10] comprises 167 diferent
AIED ethics is complicated as it has to consider both guidelines on a corporate, national and supra-national general AI ethics and the ethics of educational technol- level. Among these, some frameworks are notable. The ogy. On the one hand, there is an overlap between the Asilomar AI principles developed by the Future of Life ethics of AI, ethics of educational technology and ethics Institute [11] has been adopted by 1797 AI and Robotics of AIED - suggesting that they should draw inspiration researchers and 3923 others. Furthermore, the ‘Ethics from each other [2]. On the other hand, the usage of Guidelines for trustworthy AI’ have been proposed by the AIED systems raises concerns such as the autonomy of European Union [12]. The guidelines are encompassed teachers, responsibility and accountability for decisions in the ‘AI Act’, which is a proposed European law to made by AIED systems, impact of potential discrimina- regulate the usage of AI [1]. tion by AIED systems through historical biases, explain- Floridi et al. encouraged an ethical approach to AI ability of AIED systems, etc [2]. Owing to these concerns to incorporate the benefits of AI and mitigate the poraised by AIED systems, AIED ethics deserves attention tential harms caused by AI in a balanced way. The auand there is a need to develop an ethical framework for thors proposed AI4People – a framework formed by the guiding AIED ethics that is targeted at developers and synthesis of existing sets of principles produced by variorganizational users of AIED. ous reputable, multi-stakeholder organisations and initia
Keeping the limited attention to AIED ethics in mind, tives [13]. Their framework comprised of five principleswe aim to create an ethical framework for AIED using beneficence, non-maleficence, autonomy, justice and exthe Ethical FRAPPE - a set of high-level ethical principles plicability [13]. These 5 principles have a major overlap for AIED that are derived in this paper. This paper aims with the principles found by Jobin et al. in their scoping to answer the following research question: “Can Ethi- review of AI ethics guidelines comprising 84 documents cal FRAPPE be used to construct an exhaustive ethical [14]. framework for AIED?” Multiple steps are necessary to While there is a growing body of AI ethics guidelines answer this research question: (1) Define the properties and frameworks that can be found in literature [14, 10], and aspects of an exhaustive ethical framework from lit- these initiatives have primarily produced high-level ethierature; (2) Identify the ethical principles that can be used cal principles, tenets, values and abstract requirements to form an exhaustive ethical framework for AIED; (3) for AI development and deployment [15]. This principleIdentify current and possible future use-case scenarios based approach towards AI is criticised due to its inability that an ethical framework for AIED can be applied to, to deal with the complexity of issues raised by AI [15, 16]. such that the framework can be future-proof and evolve More specifically, the high-level ethical principles do as AI evolves. However, several of these steps are out not translate into practice automatically with the tools of scope for this paper. In this paper, we focus on the presently available to developers [17]. With the high second step. As part of the second step, we build upon number of abstract guidelines proposed, ‘ethics washan existing draft framework for ethical AIED by Holmes ing’ is on the rise by technology companies [18]. ‘Ethics et al. using insights from literature. The other two steps washing’ occurs when technical companies define ethical are planned as part of the future work, as described in policies to maintain outward appearances without folsection 5. lowing the principles in practice [18]. A second reason for criticism stems from the principle-based approached being aimed at a range of stakeholders and are thereby 2. Background often dificult to understand for specific groups of users [16].
A framework for AIED should aim to combine both the Although the principle-based approach is criticized ethics of AI and the ethics of educational technology into to be inefective due to issues such as ethics washing, it forms a good first step towards defining an ethical student data. There is a clear overlap between ethics of framework. Thereby, we begin by defining the high-level educational technology and ethics of AIED - suggesting ethical principles in this paper. As part of future work, that ethics of AIED should draw inspiration from the we adopt a similar approach as [17], in which we plan ethics of educational technology and should build on top to define requirements from ethical principles for AIED of frameworks for ethics of educational technology. and map them to design-based research (DBR) process instead, as elaborated in section 5. Armstrong et al. define 2.3. Ethics of AIED DBR in an educational setting as “a research approach that engages in iterative designs to develop knowledge This section looks at existing frameworks and guidelines that improves educational practices” [19]. As DBR brings for AIED ethics. educational research closer to everyday practice, this The conversation revolving around ethics for AIED methodology is increasingly being used in designing ed- was started over 20 years ago by Aiken and Epstein with ucational research [20]. an aim to raise awareness of researchers while designing educational systems [27]. The authors set down 10 prin2.2. Ethics of educational technology ciples for AIED systems based on “The Golden Rule for Computers in Education: Teach others as you would like As AIED ethics needs to consider the ethics of educational to be taught” [27]. technology, ethical policies for educational technology The first ethical framework for AIED was developed are reviewed here. by The Institute for Ethical AI in Education that involves
Pardo and Siemens identified four principles to catego- designers and developers for AIED and sets down guiderize the issues derived from privacy in educational data: lines for them [9]. However, this framework is aimed at transparency, student control over the data, security, and the decision makers during the process of procurement accountability and assessment [21]. As Learning Analyt- and the application of AIED. This framework focuses ics (LA) is a sub-field of AIED that uses educational data on defining high-level ethical principles without any imto optimize learning, the ethics of AIED should consider plementation guidelines that are relatable to developers the ethics of LA. LA is defined in the proceedings of the during the design of AIED systems. It contains the down1st International Conference on Learning Analytics and sides of the principle-based approach to AI ethics in the Knowledge as “the measurement, collection, analysis and form of a lack of translation into practice for developers. reporting of data about learners and their contexts, for Holmes et al. conducted a survey with 17 domain expurposes of understanding and optimising learning and perts comprising 10 open questions to gauge expert opinthe environments in which it occurs” [22]. Sclater devel- ion about ethics of AIED [2]. They examined the various oped a code of practice for LA that advises educational aspects of ethics of AIED and concluded that “the ethics institutions on how to use LA ethically. The authors of AIED cannot be reduced to questions about data or considered eight focus areas - ownership and control, computational approaches alone” [2] and needs to acconsent, transparency, privacy, validity, access, action, count for the ethics of education – including, but not adverse impact, stewardship [23]. In a recent literature limited to – the purpose of learning, choice of pedagogy, review on the ethics of LA in higher education, Pargman role of technology with respect to teachers and access and McGrath found that the top three ethical areas most to education [2]. The authors created a ‘strawman draft’ in LA articles are transparency, privacy, and informed framework, shown in Figure 1, that identified three areas consent [24]. In the context of Dutch higher education, of focus: “the ethics of data, computational approaches Engelfriet et al. developed a guide to LA that focuses and education” and emphasized the overlaps between on the protection of personal student data. Drachsler these foci. The authors identified 3 levels of overlap in and Greller developed an eight point checklist named their ‘strawman draft’ framework . The first level comDELICATE that can serve as a reflection aid for ethi- prised of three foci: “the ethics of data, computational cal and privacy-supported LA. The DELICATE checklist approaches and education” while the second level comcomprises 8 checkpoints- “Determination, Explain, Le- prised of the overlap between each pair of foci. These gitimate, Involve, Consent, Anonymise, Technical and 2 layers form the ‘known unknowns’ while the overlap External” as a quality checklist to make stakeholders between these 3 foci formed the ‘unknown unknowns’ aware and guide them through the process. [2].
The ethics of educational technology contains issues that are relevant to the domain of education. Issues relating to student autonomy and control over their data can 3. Methods have long-term efects on the future of students. There needs to be regulations regarding informed consent and This paper aims to answer the following research quesprivacy of students, interpretation and management of tion: “Can Ethical FRAPPE be used to construct an exhaustive ethical framework for AIED?”
In order to answer this research question, the draft framework by Holmes et al. was selected as a foundational framework. This is because this ‘strawman draft’ Figure 2: Revised version for the ‘strawman’ draft framework framework is well-informed by experts in the domain of for the ethics of AIED adopted from [2] AIED and considers a template model for the essential aspects of ethical AIED. However, it only forms a skeleton model and does not contain the ethical principles involved in these domains. After making a few modifications, we fill in this gap in the ‘strawman draft’ framework by Holmes et al. by examining existing literature in the domains of both AI ethics, ethics of educational technology and ethics of AIED. High-level ethical principles are identified from literature and incorporated into this framework.
We proposed two modifications to the ‘strawman draft’ framework by Holmes et al.. Firstly, we elaborated on and defined the aspects in the intersection of these foci with an aim to throw light upon the ‘known unknowns’ Figure 3: Adapted version for the ‘strawman’ draft framework and the ‘unknown unknowns’ stated by Holmes et al.. for the ethics of AIED adopted from [2] The ‘strawman draft’ framework defines the domains involved in ethics of AIED but does not elaborate on the ethical aspects of these domains. Thereby, we identified the ethical aspects involved in each of these foci based on literature, as shown in Figure 2.
Secondly, a huge overlap was noticed in the ethical aspects mapped to the foci of ‘ethics of data’ and ‘ethics of computational approaches’, as can be seen in Figure 2. Data and computational approaches were seen to be tightly coupled as any changes in one of them leads to changes in the other. For example, bias in data can lead to bias in the computational algorithm. Similarly, the interpretation and management of the data can have a direct efect on the privacy of the computational approach in the form of exposing sensitive attributes. Due to this tight coupling between the ethics of data and the ethics of computational approaches, they cannot be separated into 2 separate foci. Hence, we decided to combine them into a single focus. The revised and adapted version of our framework draft can be seen in Figure 3. It contains 2 focal areas: ethics of AI algorithms and ethics of educational technology, each containing corresponding ethical principles. The intersection of these 2 foci contains the ethical principles that form our theoretical framework.
Following these modifications, literature in the domains of AI ethics, ethics of educational technology and ethics of AIED were reviewed. This was then used to obtain the ethical principles relevant to an ethical framework for AIED, abbreviated as the Ethical FRAPPE. The list of articles reviewed is grouped using the adapted draft framework as shown in Figure 3 into the ethics of AI, ethics of educational technology and ethics of AIED.
Table 1 contains the list of selected articles that were reviewed to form the Ethical FRAPPE in order of year of publication. Following the adapted draft framework, these articles are grouped into the domains of ethics of AI, ethics of educational technology (EdTech) and ethics of AIED.
The high-level ethical principles seen in the literature were compared to each other. The ethical principles seen in a majority of the articles in each domain are identified and consolidated to create the Ethical FRAPPE. The 6 ethical principles identified as part of the Ethical FRAPPE are: Fairness, or Freedom from Bias is defined as - ”Systematic unfairness perpetrated on individuals or groups, including pre-existing social bias, technical bias, and emergent social bias” [38]. AIED systems should not be designed 1. Fairness such that the algorithms develop historically unfair prej2. Responsibility udices by ensuring fair data that is inclusive, represen3. Autonomy tative of the target population and without inaccuracies 4. Privacy [16]. Any conscious or unconscious biases that are in5. Purpose of learning corporated into AI algorithms through the data analysis 6. Explainability can have a negative impact on the rights of individual students [4]. AIED should strive towards equitable ac
Despite the presence of a large body of ethical guide- cess to AI technologies for all, keeping in line with SDG lines, these guidelines rely on context-specific keywords 4 set down by the UNESCO - “ensure inclusive and eqand there exist multiple definitions of the ethical prin- uitable quality education and promote lifelong learning ciples and technical terms involved [28]. This makes it opportunities for all” [4]. challenging to interpret and operationalize these ethical values [28]. Keeping in mind the need for a common 3.2. Responsibility vocabulary to avoid misinterpretation of the ethical principles [28], we define the ethical aspects and explain them in the light of AIED ethics as below. “Responsible AI is concerned with the fact that decisions and actions taken by intelligent autonomous systems have consequences that can be seen as being of an ethical nature” [31]. In [31], the author states that Responsible
3. Transparency: refers to the capability of AI sys- plication use-case of AI instead of being motivated by tems to ”describe, inspect and reproduce the learning goals. AIED is partly taking the form of data scimechanisms through which AI systems make de- entists looking for a context where predictive modelling cisions” [31] and other AI techniques can be applied [39]. There is a In the light of AIED, responsibility is required to ensure need to criticially examine the purpose of learning and accountability of decisions, responsibility of the devel- the performance measures that this purpose of learning opers and maintainers of AI towards its users and trans- is being reduced to. It is important to keep in mind that parency of data and purpose of the system. “theories of learning cannot, after all, be ‘discovered’ by algorithms” [39] and the usage of student data for commercial purposes [35].
Explainibility is “understanding how an AI model makes its decision” [40]. AIED systems should be actively monitored to ensure accurate and reproducible results that can be explained with the data and algorithmic functionality [16]. AIED models should be built to be explainable by design (using partially or fully explainable models) or post-hoc explainability methods should be used in the case of black-box models that are not inherently explainable [40]. In the case of AIED models, it is necessary to ensure that the decisions taken by the algorithm are explainable to humans in order to avoid negative harms to students.
Human autonomy is defined as “Refers to people’s ability to decide, plan, and act in ways that they believe will help them to achieve their goals” [38]. Autonomy, also called ‘agency’ in some ethical guidelines, ensures that the users of the systems are informed actors and have full control over their own decisions when they interact with the AIED system [16]. In AIED systems, student autonomy is important to ensure that students understand the purpose of the system and have complete control over their personal data, including the right to opt out of such systems without negative consequences. Students should be informed about the data being collected about them and should be involved in any decisions made using such data. Teacher autonomy is equally important to ensure that the role of teachers is highlighted in the form of the human-in-the-loop in the AIED system. By allowing teachers to review and act upon the decisions made by autonomous AIED systems, teacher autonomy can be ensured and unfair decisions by the AIED system can be reduced. It is also important that the data collected about the teachers should not have an adverse efect on their role in the classroom. This can be ensured by ensuring both teacher and student autonomy in AIED systems.
gling students in a university, online, distance education setting. The main goal of this sample use case would The construction of our framework and implementation be to improve teaching and learning processes on the guidelines will be conducted in 3 phases: ‘theoretical whole and support teachers. The theoretical framework framework’, ‘evaluation framework’ and ‘instantiation’. will be used to guide the design of this use case and the
In the first phase, a theoretical ethical framework will evaluation framework will be integrated into this AIED be developed for AIED. In order to define an exhaus- application to evaluate the ethics of this application. Adtive ethical framework for AIED, it is first essential to ditionally, the evaluation framework will be applied to look at what comprises a good ethical framework. To some selected AIED models for evaluation such that they answer this, a literature review will be conducted. This can cover various use cases. Finally, recommendations paper describes the first part of the first phase where and guidelines will be provided for application of the an existing draft model for AIED ethics was adapted by theoretical and evaluation frameworks into other AIED identifying high-level ethical principles from literature. applications. These recommendations will be developed In the future work, these high-level ethical principles will for common challenges (such as biases or issues) seen in be converted to requirements and then be used to create diferent classes/applications of reviewed AI algorithms the theoretical framework in the form of a checklist that from the literature. The applications of AI seen from contains practical guidelines for developers of AIED. The literature will be grouped based on parameters such as expected theoretical framework will be a checklist com- the class of algorithms, coding language used and data prising definitions, requirements, formula and guidelines type used. for ethical principles. This first draft of the framework will be evaluated by experts in the domain of AIED for face validity and content validity. References
In the second phase, a methodology will be developed to quantify the ethics of AIED applications based on the [1] E. Commission, C. Directorate-General for Comtheoretical framework. We refer to this methodology as munications Networks, Technology, Proposal for the ‘evaluation framework’. This evaluation framework a regulation of the european parliament and the will provide quantification tools for the ethical princi- council laying down harmonised rules on artificial ples integrated in the form of a pipeline that can check intelligence (artificial intelligence act) and amendexisting AI systems for ethical soundness and provide ing certain union legislative acts (2021). URL: https: recommendations for improvement. First, a subset of the //eur-lex.europa.eu/legal-content/EN/TXT/?uri=C ethical principles from the theoretical framework will be ELEX%3A52021PC0206. identified as ‘focus’ principles based on their prominence [2] W. Holmes, K. Porayska-Pomsta, K. Holstein, and relevance. Following this, various tools will be exam- E. Sutherland, T. Baker, S. B. Shum, O. C. Santos, ined to identify suitable quantification tools for the focus M. T. Rodrigo, M. Cukurova, I. I. Bittencourt, K. R. ethical principles. Lastly, there will be an evaluation of Koedinger, Ethics of AI in Education: Towards a diferent technologies for the architecture, followed by Community-Wide Framework, International Jourdesign and implementation of the evaluation pipeline. nal of Artificial Intelligence in Education (2021). This evaluation pipeline will receive the trained AI al- doi:1 0 . 1 0 0 7 / S 4 0 5 9 3 - 0 2 1 - 0 0 2 3 9 - 1 . gorithm, input data and output data as inputs and will [3] L. Chen, P. Chen, Z. Lin, Artificial intelligence give an ethical score as an output. This ethical score will in education: A review, IEEE Access 8 (2020) be calculated as the sum of individual scores for each 75264–75278. doi:1 0 . 1 1 0 9 / A C C E S S . 2 0 2 0 . 2 9 8 8 5 1 0 . ethical principle. The individual score for each ethical [4] F. Miao, W. Holmes, R. Huang, H. Zhang, et al., principle will be based on the implementation of the AI and education: A guidance for policymakers, guidelines from the theoretical framework and will also UNESCO Publishing, 2021. contain recommendations for improvements. If the eth- [5] X. Chen, D. Zou, H. Xie, G. Cheng, C. Liu, Two ical score for a majority of the ethical principles (exact decades of artificial intelligence in education, Eduthreshold to be decided based on the number of ethical cational Technology & Society 25 (2022) 28–47. principles) is above 80%, the ethical evaluation will be [6] P. Stewart, Ethics and problem solving 11 (2021). passed. Such an ethical score allows for some trade-ofs [7] M. Whittaker, K. Crawford, R. Dobbe, G. Fried, between principles in the event of conflicts between them, E. Kaziunas, V. Mathur, S. M. West, R. Richardson, while ensuring that the system is ethical as a whole. J. Schultz, O. Schwartz, AI now report 2018, AI Now
In the third phase, called ‘instantiation’, a proof of con- Institute at New York University New York, 2018. cept or instantiation of the evaluation framework will be [8] D. Schif, Education for AI, not AI for Education: developed. For this purpose, an AIED application will The Role of Education and Ethics in National AI be developed which enables the identification of strug- Policy Strategies, International Journal of Artificial [35] S. Vincent-Lancrin, R. van der Vlies, Trustworthy artificial intelligence (ai) in education: Promises and challenges (2020). [36] B. Li, P. Qi, B. Liu, S. Di, J. Liu, J. Pei, J. Yi, B. Zhou,
Trustworthy ai: From principles to practices, arXiv preprint arXiv:2110.01167 (2021). [37] R. S. Baker, A. Hawn, Algorithmic bias in education,
International Journal of Artificial Intelligence in
Education (2021) 1–41. [38] B. Friedman, P. H. Kahn, A. Borning, A. Huldtgren,
Value Sensitive Design and Information Systems, Springer Netherlands, Dordrecht, 2013, pp. 55–95.
URL: https://doi.org/10.1007/978-94-007-7844-3_4.
doi:1 0 . 1 0 0 7 / 9 7 8 - 9 4 - 0 0 7 - 7 8 4 4 - 3 _ 4 . [39] C. Perrotta, N. Selwyn, Deep learning goes to school: toward a relational understanding of ai in education, Learning, Media and Technology 45 (2020) 251–269. doi:1 0 . 1 0 8 0 / 1 7 4 3 9 8 8 4 . 2 0 2 0 . 1 6 8 6 0 1 7 . [40] B. Li, P. Qi, B. Liu, S. Di, J. Liu, J. Pei, J. Yi, B. Zhou,
Trustworthy ai: From principles to practices, arXiv preprint arXiv:2110.01167 (2021). URL: https://doi. org/10.48550/arXiv.2110.01167.