Artificial Intelligence (AI) literacy is emerging as a key competency for young learners, yet many AI concepts remain dificult to teach in lower secondary classrooms. In particular, classification-how AI systems label new data based on features-is a foundational but underrepresented topic. This paper presents a study design comparing two pedagogical approaches for teaching classification to 10-14-year-old students in Austria's digital literacy curriculum. One group uses an unplugged decision-tree game (AI Unplugged), while the other uses a plugged digital tool (Teachable Machine). We outline research questions, study methodology, assessment strategy (AI Concept Inventory), and anticipated challenges. We further integrate findings from computing education literature comparing unplugged and plugged instruction. Our goal is to evaluate how modality influences conceptual understanding, engagement, and misconceptions, providing insights for future AI literacy interventions.
2nd Workshop on Education for Artificial Intelligence (edu4AI 2025, https:// edu4ai.di.unito.it/ ), Co-located with ECAI 2025, the 28th European Conference on Artificial Intelligence which will take place on October 26, 2025 in Bologna, Italy * Corresponding author.
© 2025 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Research on AI literacy has expanded rapidly in recent years [
Unplugged activities—like sorting games, kinesthetic simulations, and logic puzzles—can make abstract
computing concepts accessible, especially for young or novice learners. Brackmann et al. [
Meta-analyses confirm that unplugged methods play a positive role in K–12 CT development [
Plugged tools—such as Scratch, programmable robots, and ML platforms—ofer interactivity, immediate
feedback, and data-rich environments. Sigayret et al. [
Block-based tools like Machine Learning for Kids, Teachable Machine, and open online platforms
have lowered the barrier for engaging with real AI models in schools [
Teachable machine [
Open-source resources such as AI Unplugged [
The study will involve two classes ( ≈ 20 each) of 10–14-year-old students in Austrian Digitale Grundbildung courses. One class will use the unplugged method; the other the plugged method (as shown in Figure 2). Both lessons are 45 minutes long.
& Label
& Test
(a) First example of a play card from the aiunplugged classification game (b) Second example with diferent characteristics
Unplugged: A paper-based decision-tree classification game from https://www.aiunplugged.org/. Students collaboratively build and use a decision tree to classify image cards (as shown in Figure 3 ) based on observable features to tell if the monkey bites or doesn’t bite.
Plugged: The GenAI Teachable Machine activity at https://tm.gen-ai.fi/, where students train a binary image classifier using uploaded (adapted) variations of the aiunplugged image cards. The GenAI teachable machine was chosen to ensure GDPR compliance. A worksheet guides them through data collection, labeling, training, and testing.
1. Pre-test: 6 AI-CI items focused on classification. 2. Lesson: Method A (unplugged) or B (plugged). 3. Post-test: 6 parallel AI-CI items. 4. Survey: Likert-scale questions on confidence, enjoyment, and perceived dificulty.
5. Observation: Teacher logs of engagement, technical issues, and timing.
We use the AI Concept Inventory (AI-CI) [
In addition to the AI-CI multiple choice items, we include: • Transfer Task: A set of novel fruit or object examples that students classify using the same decision logic they learned, administered as a short worksheet. • Open-Response Question: “How does the AI know which label to choose for a new item?” This invites students to describe in their own words how data features and decision rules result in classification. • Misconception Probe: “Which of these statements about decision-tree classification is incorrect? Explain why.” This item surfaces common misunderstandings such as treating AI decisions as guesses or attributing human-like reasoning to the system.
When comparing methods, several confounding variables must be controlled or at least monitored: 1. Teacher/Instructor Diferences: If diferent teachers conduct each method, their familiarity with AI and teaching style could afect results. Ideally, the same instructor (or equally trained instructors) should teach both methods. Provide training or detailed guides to ensure consistent delivery. 2. Student Prior Knowledge and Interest: Students may vary in their familiarity or attitudes toward AI or computers. Random assignment to conditions helps, and pre-testing for baseline knowledge can control for these diferences.
No Is it round?
Yes
Is it small? No
Yes
No
Yes Banana
Orange
Apple
Cherries 3. Group vs. Individual Work: Whether an activity is completed in teams or individually can change engagement and learning outcomes. Group structure should be kept consistent across conditions. 4. Time and Resources: Ensure each method receives the same amount of instruction and practice time. If technology is required for one method, technical issues (e.g., internet outages, slow devices) are a risk. Pilot-testing can help identify such practical issues. 5. Novelty and Engagement: A brand-new game or flashy software may temporarily increase engagement (novelty efect). Observers should note if one method appears more exciting solely due to its novelty or diference. 6. Assessment Interaction: The act of assessment (tests or quizzes) should not favor one method.
For example, if a test is administered on computers, students from the unplugged condition might be disadvantaged simply due to format familiarity. A balanced assessment design is essential.
Documenting these factors (e.g., through teacher logs, technical checklists, and possibly questionnaires about student experience) will help interpret any observed diferences between methods.
The comparative study will be piloted in Winter Semester 2025. Based on outcomes, we will refine materials, assessment tools, and teacher support. A larger-scale rollout with additional classrooms is planned for 2026.
Our goal is to build an evidence base on modality efects in AI literacy and to co-develop curriculumaligned resources for Austrian educators.
This research was done as part of the "FutureDEAL - Future of Digital Education and Learning" initiative within the doctoral program "Bildungsinnovation braucht Bildungsforschung", which is supported and partially funded by the Austrian Federal Ministry of Education, Science, and Research.
During the preparation of this work, the author(s) used Chat-GPT-4 and Perplexity.ai in order to: Draft content, Generate Latex Code, Paraphrase, Improve writing style and Plagiarism detection. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content.