=Paper=
{{Paper
|id=Vol-3927/paper5
|storemode=property
|title=Ethical framework for AI in education
|pdfUrl=https://ceur-ws.org/Vol-3927/paper5.pdf
|volume=Vol-3927
|authors=Bhoomika Agarwal
|dblpUrl=https://dblp.org/rec/conf/ectel/Agarwal24
}}
==Ethical framework for AI in education==
https://ceur-ws.org/Vol-3927/paper5.pdf
Ethical framework for AI in education
Bhoomika Agarwal
Open Universiteit, Valkenburgerweg 177, 6419 AT Heerlen, The Netherlands
Abstract
As Artificial Intelligence (AI) becomes increasingly integrated into various facets of our lives, including the educational
domain, it is important to apply ethical principles to guide the development and deployment of AI systems. This ethically
guided approach aims to mitigate potential harms or discriminatory outcomes resulting from AI algorithms. As a result,
various ethical regulations and guidelines for AI ethics have emerged at the corporate, national, and supranational levels.
However, the literature has paid relatively scant attention to the specific ethical considerations within the domain of AI in
Education (AIED). AIED ethics represents a complex intersection, necessitating the combination of general AI ethics and the
ethics of educational technology. This research aims to find the key constituents of an ethical framework for educational
stakeholders of AIED that can be used to identify ethical issues in an AIED system. In this paper, we outline the methodology
employed in this research to create an ethical framework for AIED. A systematic literature review will first look into the ethics
of AI, the ethics of education and the ethics of AIED to consolidate the Ethical Values (EVs) and Ethical norms (ENs) for AIED
ethics. Building on this knowledge from the literature, additional ENs will be collected through stakeholder consultation.
These ENs will then be ranked by experts and used to form an ethical framework.
Keywords
Artificial Intelligence, Education, Ethics
1. Introduction An ethical guidance for AIED is necessary to reduce the
negative impacts caused due to propagation of historical
Artificial Intelligence (AI) is reshaping the world in pro- biases and discrimination that can result from the usage
found ways and has a widespread impact on our lives, of AIED. At the same time, it is important to protect the
including education. The usage of AI in classrooms and privacy and autonomy of students and teachers so that
in education is promising and provides opportunities the data collected by educational institutes cannot be
to improve the education process. AI has been applied used for other purposes. Hence, there is a need to use an
in educational contexts in a range of contexts varying ethical framework to regulate AIED.
from automation of administrative processes and tasks to In addition to considering general AI ethics, AIED
curriculum and content development, instruction to un- ethics has to also consider the ethics of education. The
derstanding and improving students’ learning processes overlap between the ethics of AI, ethics of education and
through analysis of student data [1]. ethics of AIED suggests that they should draw inspira-
Over the past decade, the use of AI tools to support tion from each other [5]. The usage of AI technologies in
and enhance learning has grown exponentially [2]. In a education raises questions linked to ethical issues such
recent literature review, Chen et al. looked at 20 years as data ownership and control, privacy, biases in algo-
of AIED from 2000 to 2019 and shared several relevant rithms, data management, transparency, and a need for
findings: (a) AIED has seen an increased interest due educational context [5]. Despite these concerns raised
of the positive effect of AI on learning; (b) there is an by AIED systems, limited attention has been paid to the
increase in AIED literature over the years; (c) AIED re- ethics of AIED [6, 5, 7, 8, 9, 10]. AIED ethics could bor-
search is especially found in interdisciplinary journals row from both domains and add additional ethical values
with a dual focus on education and technology [3]. With as required by the domain specifically, while also con-
the increased interest in AIED, there is a need to ethi- sidering the applicability of these values to the domain
cally guide the usage of AIED systems. The EU AI Act of AIED [5]. Due to these complexities, AIED ethics
classifies the usage of AIED as ‘high-risk’ as “such sys- deserves attention.
tems may violate the right to education and training as The main research question guiding this research is:
well as the right not to be discriminated against and per- “What are the key constituents of an ethical framework
petuate historical patterns of discrimination” [4, p. 26]. for educational stakeholders of AIED that should be used
to identify ethical issues in an AIED system?”. This can
Proceedings of the Doctoral Consortium of the Nineteenth European
Conference on Technology Enhanced Learning, September 16–20, 2024, be divided into three sub-questions for the three groups
Krems, Austria of educational stakeholders in AIED - students, educators
∗
Corresponding author. and educational institutes. Thus, the three sub research
Envelope-Open bhoomika.agarwal@ou.nl (B. Agarwal) questions are:
Orcid 0000-0002-7347-8465 (B. Agarwal)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0). 1. RQ1: What are the key constituents of an ethical
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
framework for educational institutes of AIED that
should be used to identify ethical issues in an
AIED system?
2. RQ2: What are the key constituents of an ethical
framework for educators of AIED that should be
used to identify ethical issues in an AIED system?
3. RQ3: What are the key constituents of an ethical
framework for students of AIED that should be
used to identify ethical issues in an AIED system?
The next section explains the methodology followed
to answer these research questions.
2. Methodology
In order to identify the key constituents of an ethical
framework for AIED, this research follows an adapted
version of the generalizable model for developing Codes
of Practice (CoP) developed by Sclater. The ‘CoP for
Learning Analytics’ developed by the authors contains a
set of guidelines and some ethical principles [11]. Due
to the similarity of the format of the developed CoP to
an ethical framework and the systematic nature of the
generalizable methodology, we chose to use this approach
to develop our ethical framework for AIED.
The author presents the activities followed as a basis Figure 1: Research methodology
for a generalizable model that can be used for developing
CoP in other professions or areas of education [11]. The
generalizable model involves developing five products: for AIED. Following this, the gaps in literature will be
1) a literature review identifying ethical, legal, and logis- filled in through stakeholder consultation. The output
tical concerns, 2) a taxonomy of issues refined through of the stakeholder consultation will then be ranked by
expert consultation, 3) a draft CoP, 4) a final publicly experts and used to form an ethical framework. The
released CoP incorporating feedback from public consul- stakeholder consultation and expert consultation will
tation, and 5) a supporting website with guidance and ensure that the viewpoints of all the stakeholders of AIED
case studies [11]. An advisory group of experts and stake- can be incorporated into the framework.
holders provides input throughout. As the CoP is piloted,
feedback informs updates to subsequent versions.
2.1. Systematic literature review
This research uses the generalizable model developed
by Sclater but makes three modifications. Firstly, we start The existing literature on AIED ethics is lacking in three
with consulting stakeholders and incorporating their in- aspects: (a) a theoretical definition of the essence of AIED
put from the beginning. This is because the ethics of ethics, (b) a hierarchical classification of AIED ethics,
AIED is a relatively new domain and has a direct impact and (c) reflection on the regulations [12]. As the first
on the lives of the stakeholders. Secondly, the experts step of this research, a systematic literature review (SLR)
are involved in a later stage of the model to review the was conducted with an aim to address these gaps by
input collected from the stakeholders. Additionaly, we identifying and defining the key constituents of AIED
gather input about the design of the framework from the ethics.
experts. Lastly, due to time limitations, we do not follow In this SLR, the normative approach to ethics - which
the last two steps described in the generalizable model. proposes “how to act, how to live and what kind of person
Figure 1 shows adapted the methodology followed in this to be” [13, p. 2] - was followed. Ethics is defined in terms
research. of values and norms. Values are abstract ideas that are
To answer the main research question, a systematic strived for via certain types of behavior while norms are
literature review (SLR) that looks into the ethics of AI, rules that specify actions to achieve certain values [14].
the ethics of education and the ethics of AIED will first Using this definition of ethics, the key constituents of an
be conducted. This SLR will serve as the foundation by ethical framework are identified as ethical values (EVs)
identifying the key constituents for an ethical framework and ethical norms (ENs). The SLR was guided by two
central research questions: work for AIED. This SLR distinguished the key con-
stituents that would comprise such an ethical framework,
1. What are the main ethical values (EVs) for AIED?
thereby setting the stage for further elaboration and flesh-
2. What are the ethical norms (ENs) for AIED?
ing out of these key constituents.
This SLR aims to identify the main EVs and ENs for
AIED by analyzing relevant literature published between 2.2. Stakeholder consultation
2010 and 2022 that is available in the English language.
Through a comprehensive database search and back- This research study aims to explore the ethical perspec-
ward snowballing technique, a total of 25 articles were tives of key stakeholders in the field of AIED. The SLR
identified and included for analysis. The Preferred Re- highlighted a lack of ENs for end users and educational
porting Items for Systematic reviews and Meta-Analyses institutes. Building on this result, the target population
(PRISMA) method [15] was used to ensure transparency for the stakeholder consultation comprised three distinct
and reproducibility of the SLR. roups: students, teachers, and educational administrators
To identify the EVs, the definitions found within the who act as representatives from educational institutions.
literature were collected and reported. To analyse the To facilitate an in-depth exploration of AIED ethics, the
EVs, they were grouped into common themes or topics. study employs a qualitative research methodology involv-
The grouping criterion was based on the identification of ing focus group discussions. For this study, participants
common key terms that appear consistently across mul- were recruited based on the match of their profile with the
tiple definitions of the EVs. Subsequently, the emergent stakeholder groups identified, i.e. educational administra-
themes were assigned descriptive labels that encapsu- tors, students and educators active in higher education.
lated the overarching meaning and conceptual essence The participants were approached during workshops,
conveyed by the set of definitions within each respec- presentations at conferences, summer schools, and other
tive cluster. This thematic analysis process allowed for academic events related to artificial intelligence, educa-
a systematic consolidation and organization of the EVs tion, or both fields. After initially showing interest in
identified in the literature. The analysis of the EVs re- participating, potential participants received an email in-
vealed six main EVs of AIED: non-discrimination, data vitation along with a letter providing information about
stewardship, human oversight, goodwill, explicability, the details of the study. This study was performed in the
and educational aptness. Netherlands, and all participants were active in higher
ENs were identified by looking for keywords such as education in said country.
‘norms’, ‘guidelines’, ‘regulations’, ‘recommendations’ or Five separate focus groups were conducted in the pe-
another synonym of these terms in the selected articles. riod from March 2024 to May 2024 (inclusive). Two focus
Four main stakeholder groups were identified from the groups each were conducted for students and teachers,
literature: end users, developers, regulators and educa- and one was conducted for educational administrators.
Each focus group was conducted with the group size
tional institutes [16]. The ENs identified in the literature
were categorized according to the stakeholder groups of five to eight participants to optimize participant en-
they are relevant for and the corresponding main EV gagement and interaction [17]. The discussions revolved
they uphold. Subsequently, these two categorizations around three dilemmas and ethical considerations sur-
were combined into a matrix, mapping ENs for specific rounding the integration of AI in educational settings.
stakeholder groups to the implementation of particular Furthermore, pre-prepared questions guided the discus-
main EVs. This result could be used to provide the ENs sions, enabling the researchers to gather qualitative data
for specific stakeholder sets to implement a given main on the ENs and perspectives held by each stakeholder
EV. group. Through these questions, particular emphasis
In addition to answering the two research questions, was placed on addressing gaps identified through the
the following points of discussion were highlighted by SLR conducted prior to this study.
the SLR: The qualitative data collected through the focus groups
will be coded deductively with a-priori coding [18] using
1. Ethics should be included in the design of AIED
thematic analysis. The six steps of thematic analysis
2. Ethics of AIED should focus on the educational described by Braun and Clarke will be followed with the
aptness of AIED solutions software tool Dedoose [20]. The starting coding tree for
3. More ENs should be established for end users of the thematic analysis will be based on the results of the
AIED SLR. If any topics are identified through the thematic
4. There exists a tight coupling between EVs, leading analysis that do not fit within this coding tree, they will
to possible ethical dilemmas be added inductively to the coding tree. The ENs and
The main EVs and ENs identified in this SLR could viewpoints elicited from the focus group discussions will
serve as a foundation for developing an ethical frame- serve as valuable inputs for the subsequent stages of the
research project. orous and multifaceted approach, drawing upon various
The stakeholder consultation aims to gather input from data sources and stakeholder perspectives.
the various stakeholder groups involved in AIED regard- The initial phase of the study will involve a systematic
ing the ethical considerations and challenges associated literature review (SLR), which will serve as the founda-
with using AIED. This input will then be integrated into tion for identifying the main ethical values (EVs) and
the development of an ethical framework for AIED. More- ethical norms (ENs) pertinent to the domain of AIED.
over, the data analysis process will involve mapping the This extensive review will consolidate and synthesize the
ENs identified by stakeholders onto the main EVs for existing knowledge base, providing a theoretical ground-
AIED. This process will also verify the comprehensive- ing for the subsequent stages of the research.
ness of the main EVs and supplement the ethical frame- Building upon this foundation, the second study en-
work with any missing EVs and ENs pertaining to AIED. gage will key stakeholder groups through a series of focus
group discussions. This qualitative approach will allow
2.3. Expert consultation for an in-depth exploration of the perspectives, concerns,
and expectations of students, teachers, and educational
This research study will engage an expert panel compris- administrators regarding the ethical implications of AIED.
ing individuals with extensive knowledge and expertise The insights gathered from these discussions will enrich
in the domain of AIED. The panel will consist of five to the understanding of the ethical landscape of AIED.
ten experts. Furthermore, in a third study, we seek the expertise
The primary objective of this study is to leverage the of a panel of domain specialists to validate, rank and
collective expertise of the panel in evaluating and pri- refine the proposed ethical framework. The culmination
oritizing ENs for an ethical framework for AIED. Based of these efforts would result in a comprehensive ethical
on the methodology described by Sclater, the experts framework that addresses the ethical challenges posed
will be tasked with rating the identified ENs on a three- by the usage of AIED.
point scale, assessing their criticality to the proposed This ethical framework could be used by students,
framework. The ratings will be categorized as follows: 1) teachers and educational administrators to identify ethi-
Critical, 2) Important, and 3) Less important. This eval- cal issues in the usage of AIED systems. It could ensure
uation will facilitate the selection of the most pertinent that the usage of AIED does not have any intentional or
ENs, which will subsequently be incorporated into the unintentional adverse effects on their lives. Additionally,
ethical framework. it underscores the importance of having a continuous
Furthermore, the study will solicit input from the ex- dialogue about the potential ethical issues caused by the
pert panel regarding the optimal format and presentation rapid evolution of AIED systems. By prioritizing ethi-
of the final ethical framework. This consultation will en- cal considerations, the educational sector can harness
sure that the framework is not only theoretically robust the transformative potential of AI while safeguarding
but also practical and accessible, facilitating its effective the well-being, privacy, and fundamental rights of all
dissemination and adoption within the AIED commu- stakeholders involved.
nity and related stakeholders. It is to be noted that the
detailed design of this study is still under progress.
The expert consultation will serve to refine and distill Acknowledgements
the ENs identified through the preceding studies, con-
centrating on the most critical and essential ones. This I would like to thank my PhD supervisors - Dr. Corrie
refinement process aims to prevent the resulting ethical Urlings, Dr. Giel van Lankveld and Prof. Dr. Roland
framework from becoming overly intricate or overwhelm- Klemke - for their unwavering support and guidance, and
ing for stakeholders to use effectively. Concurrently, the for always having my back through my PhD trajectory.
insights and input garnered from the experts will guide
the shaping of the framework into a format that is both References
usable and comprehensive, ensuring its practical applica-
bility and completeness. [1] L. Chen, P. Chen, Z. Lin, Artificial Intelli-
gence in Education: A Review, IEEE Access
8 (2020) 75264–75278. doi:10.1109/ACCESS.2020.
3. Conclusion 2988510 .
In conclusion, this research aims to make a contribution [2] F. Miao, W. Holmes, R. Huang, H. Zhang, et al.,
to the field of Artificial Intelligence in Education (AIED) AI and education: A guidance for policymakers,
by developing a comprehensive ethical framework to UNESCO Publishing, 2021.
guide ethical usage of AIED. The research follows a rig- [3] X. Chen, D. Zou, H. Xie, G. Cheng, C. Liu, Two
Decades of Artificial Intelligence in Education, Ed- T. C. Hoffmann, C. D. Mulrow, L. Shamseer, J. M.
ucational Technology & Society 25 (2022) 28–47. Tetzlaff, E. A. Akl, S. E. Brennan, et al., The PRISMA
[4] European Commission and Directorate-General 2020 statement: an updated guideline for reporting
for Communications Networks, Content and Tech- systematic reviews, International journal of surgery
nology, Proposal for a Regulation of the Euro- 88 (2021).
pean Parliament and the Council laying down [16] N. A. Smuha, Pitfalls and pathways for Trustworthy
harmonised rules on Artificial Intelligence (Ar- Artificial Intelligence in education, in: The Ethics
tificial Intelligence Act) and amending certain of Artificial Intelligence in Education, Routledge,
Union legislative acts, EC, COM 206 (2021). 2022, pp. 113–145.
URL: https://eur-lex.europa.eu/legal-content/EN/ [17] R. A. Krueger, Focus groups: A practical guide for
ALL/?uri=CELEX:52021PC0206. applied research, Sage publications, 2014.
[5] W. Holmes, K. Porayska-Pomsta, K. Holstein, [18] M. B. Miles, A. M. Huberman, J. Saldana, Qualitative
E. Sutherland, T. Baker, S. B. Shum, O. C. Santos, data analysis - international student edition, 4 ed.,
M. T. Rodrigo, M. Cukurova, I. I. Bittencourt, K. R. SAGE Publications, Thousand Oaks, CA, 2019.
Koedinger, Ethics of AI in Education: Towards a [19] V. Braun, V. Clarke, Using thematic analysis in
Community-Wide Framework, International Jour- psychology, Qualitative research in psychology 3
nal of Artificial Intelligence in Education (2021). (2006) 77–101.
doi:10.1007/S40593- 021- 00239- 1 . [20] M. Salmona, E. Lieber, D. Kaczynski, Qualitative
[6] D. Schiff, Education for AI, not AI for Educa- and mixed methods data analysis using Dedoose:
tion: The Role of Education and Ethics in Na- A practical approach for research across the social
tional AI Policy Strategies, International Jour- sciences, Sage Publications, 2019.
nal of Artificial Intelligence in Education (2021).
doi:10.1007/S40593- 021- 00270- 2/ .
[7] P. Blikstein, Y. Zheng, K. Z. Zhou, Ceci n’est pas
une école: Discourses of artificial intelligence in
education through the lens of semiotic analytics,
European Journal of Education 57 (2022) 571–583.
[8] A. Bozkurt, A. Karadeniz, D. Baneres, A. E.
Guerrero-Roldán, M. E. Rodríguez, Artificial intelli-
gence and reflections from educational landscape: a
review of AI studies in half a century, Sustainability
13 (2021) 800.
[9] P. Lameras, S. Arnab, Power to the teachers: an
exploratory review on artificial intelligence in edu-
cation, Information 13 (2021) 14.
[10] M. Huiling, Research Hotspots and Trends of ‘Arti-
ficial Intelligence + Education’ Abroad in the Last
Decade —- Knowledge graph analysis based on 876
articles in the WOS database from 2011 to 2020, in:
2020 The 4th International Conference on E-Society,
E-Education and E-Technology, 2020, pp. 132–134.
[11] N. Sclater, Developing a code of practice for learn-
ing analytics, Journal of Learning Analytics 3
(2016) 16–42. URL: https://learning-analytics.info/
index.php/JLA/article/view/4512. doi:10.18608/
jla.2016.31.3 .
[12] L. Zhang, K. Fu, X. Liu, Artificial Intelligence in
Education: Ethical Issues and its Regulations, in:
Proceedings of the 5th International Conference on
Big Data and Education, 2022, pp. 1–6.
[13] S. Kagan, Normative ethics, Routledge, 2018.
[14] I. Van de Poel, L. Royakkers, Ethics, technology,
and engineering: An introduction, John Wiley &
Sons, 2011.
[15] M. J. Page, J. E. McKenzie, P. M. Bossuyt, I. Boutron,