Artificial Intelligence (AI) continues to transform industries and everyday life. It is increasingly important to foster AI literacy, particularly in secondary education. AI literacy equips students with the knowledge and skills to understand, interact with, and critically evaluate AI technologies. It empowers them to become informed citizens and active contributors in an AI-driven future. Many universities, organizations, and initiatives have created AI literacy learning resources and made them publicly available. However, to the best of our knowledge, there is no existing efort to categorize the learning objectives of these resources.
AI literacy has emerged as a critical competency in the educational landscape, reflecting the pervasive
integration of AI technologies into various aspects of daily life. The concept of AI literacy encompasses
a range of skills, including understanding AI principles, ethical considerations, and the ability to engage
with AI systems critically [
Touretzky’s AI4K12 initiative is a foundational reference in this domain, supporting the inclusion of AI
education in K-12 curricula [
Moreover, recent literature reveals that efective AI education must integrate psychological competencies alongside traditional knowledge frameworks. This integration is crucial for enhancing students’ self-eficacy in problem-solving and emotional regulation when interacting with AI systems [ 3, 8]. Research also indicates that fostering AI literacy can significantly influence students’ attitudes towards technology, encouraging them to engage with AI in a responsible and informed manner [9, 4].
AI literacy equips individuals with the tools to critically assess and utilize AI technologies in their personal and professional lives. As AI continues to evolve and influence various sectors, educational initiatives like AI4K12 are essential for ensuring that future generations are not only consumers of AI but 1st Workshop on Education for Artificial Intelligence (edu4AI 2024, https:// edu4ai.di.unito.it/ ), co-located with the 23rd International
CEUR
ceur-ws.org also informed citizens capable of contributing to discussions about its ethical and societal implications [3, 7, 4].
The landscape of AI literacy resources is growing rapidly. The type of providers difer from commercial providers, universities, and educational research initiatives up to governmental interest in fostering AI literacy amongst citizens. The individual resources are, therefore, often not accompanied by scientific publications. Even if that is the case, the contents might have changed since the publications. For this reason, we decided on an unconventional approach to identify, collect, and screen current online resources on AI literacy.
The global increase in available AI literacy resources leads us to the following research questions: RQ1 How diverse are the individual learning resources? How are the competence dimensions (Attitudes,
Behavior, Cognition) distributed over all learning objectives? RQ2 How well is AI ethics literacy covered?
First, preliminary research is conducted to find related work and popular AI literacy frameworks and identify relevant keywords for further steps. Then, a systematic approach is used to identify resources, which are screened and analyzed. Selected resources are then analyzed in-depth, i.e. individual learning objectives are identified. Finally, we categorize these learning objectives into a competence dimension (Cognition, Behavior, Attitudes) and an AI literacy type (Generic, Ethic). This allows a comparison based on the holistic AI literacy assessment matrix [10].
We search for current AI literacy frameworks or curated materials, which are sometimes but not always accompanied by scientific literature. Therefore, we identify potentially relevant resources through a Google search. The incognito mode helps to avoid search history bias through the browser cache. We determine relevant search terms through preliminary research. Then, we combine those search terms as boolean logic search requests to optimize the search results and make it as reproducible as possible.
("term1" OR "term2" OR "term3") AND ("term4" OR "term5") All results on the first page, except sponsored results, are added to the list of identified targets.
The identified resources are then selected in a two-step screening process. In the 1st screening step, the type of target is analyzed. Since we are searching for high-quality learning resources appropriate for secondary school students, we exclude anything else. In the second screening, we look at diverse criteria: 1. Who published the resource? 2. Accessibility 3. Provided language(s) 4. Type of resource (mooc, videos, textual, plugged, unplugged...) 5. Appropriate for secondary school students 6. Quality of content
Eforts to conceptualize AI literacy and assessment seem very specific and isolated. The holistic AI literacy assessment matrix [10] provides a structured and comprehensive way to assess AI literacy, ensuring that technical knowledge, practical application, and ethical considerations are accounted for. It addresses three core areas of AI literacy: Generic AI Literacy, Domain-Specific AI Literacy, AI Ethics Literacy. The matrix assesses AI literacy along three dimensions: Cognition, Behavior (skills), and Attitudes (values) following the “concept of competence” [11].
We use the matrix to categorize individual learning objectives of a selected resource along these dimensions and literacy types. Since, in our case, only educational AI literacy resources are selected, the domain-specific area becomes less relevant. We augment the original authors’ concept to inspect the holistic coverage of AI literacy resources.
The individual classification depends significantly on the context, how the learning objective is defined, and how it is meant to be achieved. A practical exercise to achieve a learning objective is likely classified as Behavior competence. On the other hand, a purely theoretical component, where facts must be understood or memorized, is likely classified as Cognition competence. Attitude learning objectives are components that prompt self-reflection or require expressing one’s attitude toward a specific topic.
We identified currently available online learning resources on AI literacy, selected high-quality representatives, and inspected the diversity of the learning objectives.
Based on insights from the preliminary research, we formulated key search terms. These were combined into the nine search queries (see Table 1) to reach a representative number of results.
The incognito search on these queries resulted in 85 targets (see Figure 1). From these 85 targets, 20 exact duplicates were removed immediately. In the first screening, we selected high-quality learning resources with a dedicated focus on AI literacy. During the process, 51 targets were excluded; 5 were excluded as duplicates since they refer to already identified targets or are simply in the same domain. 14 targets led to scientific publications, possibly relevant for further literature research, but didn’t contain actual material. 22 were blog-like articles on AI literacy and/or lists of external AI literacy-related resources. 8 sources were excluded because they were restricted to a paying audience or enrolled students at a specific institution. Two didn’t fit into our topic.
n o it a c iift n e d I g n i n e e r c S t s 1 g n i n e e r c S d n 2 s i s y l a n A
web search (n = 85 )
Resources screened (n = 14)
Resources classified (n = 3)
The second screening investigated the remaining 14 targets (see Table 2) in more detail. We examined who published the resource and their motivations for doing so. Additionally, we verified if registration and or a fee is required to access or use the material. Further, we looked into the individual materials and investigated usability, optional languages, and age-appropriateness. Finally, we selected three targets that fulfilled all our criteria, ofered good usability, and clearly stated the learning objectives.
The three selected resources from the second screening were analyzed in-depth. In the first step, individual learning objectives were identified. These learning objectives were assigned to the holistic AI literacy assessment matrix [10]. The learning objectives distribution is shown in Table 3 and the competence distribution is shown in Figure 2. 3.2.1. Elements of AI Elements of AI is a free online course series that aims to make artificial intelligence more accessible to a wide audience. It was created by the University of Helsinki in collaboration with MinnaLearn, and was first launched in 2018. In 2021, the course was made publicly available in all oficial EU languages. The course is designed for people with no prior knowledge of AI or programming, although there is an advanced course available for those looking to deepen their technical skills.
The primary objective of Elements of AI is to educate people about the fundamentals of AI and encourage informed discussions about the potential of AI in society. It seeks to demystify AI and make it approachable by combining theory with practical exercises. The course is part of a broader initiative to increase AI literacy and equip individuals with the skills and knowledge needed to understand the impact of AI on various industries and everyday life. The course ofers a structured process and maps personal progress through exercises after each section. Upon completion, a certificate is available for purchase.
With a total of 27 learning objectives, two-thirds were classified as Cognition. One-third of the learning objectives were classified as Behavior, and none as Attitudes. The number of learning objectives classified as Ethic in the two Elements of AI courses is surprisingly low (n=3). However, investigation during the 2nd screening revealed that the University of Helsinki also created a separate MOOC Ethics of AI. This might explain the lack of ethics-related learning objectives in Elements of AI. 3.2.2. ENARIS The international AI education and awareness project ENARIS [12] aims to spark interest in Artificial Intelligence in young students aged 10 to 14 and to foster their technical knowledge in a playful way. The material of ENARIS is created with a classroom environment in mind and can be used by interested teachers without limitations. The website ofers 10 ready-to-use workshop-sized modules on diferent AI literacy topics in English, German, and Hungarian languages. Enaris has a total of 67 identified learning objectives, with 39 classified as Cognition, 20 as Behavior, and eight as Attitudes. In total 12 learning objectives were classified as ethic literacy, evenly distributed along the competence dimensions. 3.2.3. CRAFT The third selected target, CRAFT AI literacy resources, is a collection of free, high-quality AI literacy resources by Stanford University. These resources are “informed by learning sciences research for non-profit use by classroom educators”. The materials provided in the individual lessons are ready-touse and specifically designed for high-school teachers. From 23 learning objectives in total, 13 were classified as Cognition, eight as Behavior, and three as Attitudes. Six learning objectives were classified as ethic literacy, evenly distributed along the three competencies.
This study highlights the critical role of high-quality AI literacy resources in secondary education, emphasizing their importance in equipping students with both technical knowledge and ethical awareness. As AI technologies increasingly influence daily life, preparing students to engage with AI critically is essential. The analysis of available AI literacy resources demonstrates a growing variety of tools and platforms, each aimed at diferent aspects of AI education, from basic concepts to more advanced applications. However, gaps in accessibility, depth of content, and integration of ethical considerations remain challenges that must be addressed.
The analysis of the three selected resources has shown a similar distribution of the learning objectives on the holistic assessment matrix. The classification of these high-quality resources could form the base for a benchmark for other AI literacy resources. Further research is necessary to identify meaningful metrics for a better comparison.
Nevertheless, this study has certain limitations. While it provides insights into currently available resources, more research is needed to evaluate the efectiveness of these resources in diverse educational contexts. The proposed process to identify learning objectives and classify them into the holistic AI literacy matrix is yet to be improved. A semiautomatic process (e.g., through NLP) could support a consistent, measurable way to compare AI literacy learning resources. Another interesting metric to explore would be the number of learning objectives along the investigated competence dimensions and literacy types. Future studies could further explore how AI literacy programs impact student learning, particularly in underrepresented or underserved communities, and investigate the best methods for integrating ethical discussions into technical AI education. every child know about ai?, Proceedings of the AAAI Conference on Artificial Intelligence 33 (2019) 9795–9799. URL: https://ojs.aaai.org/index.php/AAAI/article/view/5053. doi:10.1609/AAAI.
V33I01.33019795. [3] G. Biagini, Assessing the assessments: Toward a multidimensional approach to ai literacy, Media
Education (2024). doi:10.36253/me- 15831. [4] D. D. Figaredo, Responsible ai literacy: A stakeholder-first approach, Big Data & Society (2023).
doi:10.1177/20539517231219958. [5] L. Casal-Otero, A. Catala, C. Fernández-Morante, M. Taboada, B. Cebreiro, S. Barro, Ai literacy in k-12: a systematic literature review, International Journal of STEM Education 2023 10:1 10 (2023) 1–17. URL: https://stemeducationjournal.springeropen.com/articles/10.1186/s40594-023-00418-7. doi:10.1186/S40594- 023- 00418- 7. [6] D. T. K. Ng, J. K. L. Leung, S. K. W. Chu, M. S. Qiao, Conceptualizing ai literacy: An exploratory review, Computers and Education: Artificial Intelligence 2 (2021) 100041. doi: 10.1016/J.CAEAI. 2021.100041. [7] K. Kahn, N. Winters, Constructionism and ai: A history and possible futures, British Journal of
Educational Technology (2021). doi:10.1111/bjet.13088. [8] Y. Shen, Perceived support and ai literacy: The mediating role of psychological needs satisfaction,
Frontiers in Psychology (2024). doi:10.3389/fpsyg.2024.1415248. [9] H. A. Modran, Using the theoretical-experiential binomial for educating ai-literate students,
Sustainability (2024). doi:10.3390/su16104068. [10] N. Knoth, M. Decker, M. C. Laupichler, M. Pinski, N. Buchholtz, K. Bata, B. Schultz, Developing a holistic ai literacy assessment matrix – bridging generic, domain-specific, and ethical competencies, Computers and Education Open 6 (2024) 100177. doi:10.1016/J.CAEO.2024.100177. [11] F. E. Weinert, Concept of competence: A conceptual clarification, in: D. S. Rychen, L. H. Salganik (Eds.), Defining and selecting key competencies, Hogrefe & Huber, Seattle, WA, 2001, pp. 45–65. [12] M. Kandlhofer, P. Weixelbraun, M. Menzinger, G. Steinbauer-Wagner, Ágoston Kemenesi, Education and awareness for artificial intelligence, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 14296 LNCS (2023) 3–12. doi:10.1007/978- 3- 031- 44900- 0_1.