<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Gaining Insight into Effective Teaching of AI Problem-Solving Through CSEDM: A Case Study</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Spencer Yoder</string-name>
          <email>smyoder@ncsu.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Sankalp Boorugu</string-name>
          <email>sboorug@ncsu.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Cansu Tatar</string-name>
          <email>ctatar@ncsu.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Shiyan Jiang</string-name>
          <email>sjiang24@ncsu.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ifeoluwa Aderemi</string-name>
          <email>iwaderem@ncsu.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Bita Akram</string-name>
          <email>bakram@ncsu.edu</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>North Carolina State, University</institution>
          ,
          <country country="US">USA</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>CS education for all has been established as an important area of focus for both researchers and practitioners in recent years. At the same time, due to the increasing prevalence of AI technology in humans' everyday lives, arti cial intelligence (AI) education for K-12 is gaining special attention among CS educators. AI literacy, even more than general CS competencies, requires evidence-based research to be e ectively integrated in our schools. The common learning environments utilized for CS education enable us to go beyond conventional educational research approaches by providing a platform where detailed data can be collected from students' interaction with CS-education-related activities. Thus, conventional educational research approaches coupled with insights gained from pattern recognition and student modeling approaches enable us to e ectively improve our instruction and to provide students with adaptive sca olding. In this work, we present our rst AI curriculum module that is designed to teach a fundamental AI search algorithm, Breadth-First Search (BFS), through a series of progressively sca olded activities. Data is collected from a preliminary pilot of this activity with a high-school student in the form of a think-aloud protocol, screen capture, submitted block-based programming artifacts, and interview questions. Our results demonstrate that our activities have been successful in increasing the student 's knowledge about the BFS algorithm and more importantly, how this particular AI algorithm can be utilized to solve real world problems. Based on the results of this pilot study, we propose designing a comprehensive AI curriculum contextualized within a learning environment that collects detailed data from students' progress to inform instructional design and facilitate adaptive sca olding for students.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>
        Copyright ©2021 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0)
Integrating AI education into the K-12 curriculum poses
signi cant challenges for educators since teachers often lack
prior education in CS related elds [
        <xref ref-type="bibr" rid="ref9">11</xref>
        ]. Furthermore, the
skills, knowledge, and abilities required for getting engaged
in AI-related activities are novel to students and do not align
with their conventional ways of learning [cite]. Thus, e
ective instructional approaches should be identi ed through
evidence-based research. This is especially important since
strong stereotypes and biases about who can learn this eld
prevent some of the students from getting engaged in the
devised curriculum in the rst place [
        <xref ref-type="bibr" rid="ref10">12</xref>
        ]. This can be
facilitated by taking advantage of one of the main a ordances of
CS learning environments that is their inherent capability
to collect ne-grained data from students' interactions with
the learning activities. The collected log data can then be
utilized in data-driven analysis that can inform instructors
about the curriculum e ectiveness and students' learning.
In this work we present the preliminary design of an AI
curricular module that teaches one of the most fundamental AI
search algorithms, breadth- rst search (BFS). The module
consists of a series of activities that aims to teach learners
about the BFS algorithm and ends with a coding activity
where students have to incorporate the BFS algorithm in a
real-world problem scenario to build a contact tracing
application for COVID-19. While many devised AI curricula
focus on the machine learning (ML) aspect of AI, we aim
to provide K-12 students with a comprehensive view of AI
that includes ML as an integrated aspect of AI. We present
a case study [1] where this curricular module is piloted with
a high-school student and data is collected through think
aloud protocol [
        <xref ref-type="bibr" rid="ref1">3</xref>
        ], screen recording, and clinical interviews
[
        <xref ref-type="bibr" rid="ref6">8</xref>
        ]. This data is then analyzed to make inferences about the
e ectiveness of the curriculum, required improvements and
implications for design for a comprehensive AI curriculum
integrated within a learning environment that collects
learners' interaction data to provide both educators and students
with adaptive assessment and feedback.
      </p>
      <p>Section 2 discusses related work. The details of the devised
curriculum are presented in section 3. The case study is
presented in section 4, and the results of the pilot study are
discussed in section 5. Section 6 concludes the work, and
section 7 discusses implications for design and future work.</p>
    </sec>
    <sec id="sec-2">
      <title>2. RELATED WORKS</title>
      <p>
        Emerging research is exploring the design of learning
experiences to foster youths' Arti cial Intelligence (AI)
literacy so that they are prepared to enter and engage with an
AI- lled future [
        <xref ref-type="bibr" rid="ref8">10</xref>
        ]. In particular, e orts have been made
to engage youth in understanding AI through the
development of machine learning (ML) models. For instance, a work
by Zimmermann-Nie eld and colleagues (2019) provided an
embodied learning experience for youth to create ML
models for recognizing their own physical activities [
        <xref ref-type="bibr" rid="ref17">19</xref>
        ]. They
found that youth developed an understanding of how ML
models learned patterns of body movements and this could
contribute to the understanding of the iterative process of
ML. In addition, Google developed web-based tools (e.g.,
Teachable Machine) to make ML accessible to the public,
including youth [
        <xref ref-type="bibr" rid="ref1">3</xref>
        ]. These studies stressed the cultivation
of data literacy among youth as modeling data as a core
concept in ML. While K-12 AI education is gaining momentum,
the curricula being devised for K-12 students group tend to
focus on basic ML approaches. However, these curricula fail
to demonstrate the connection between ML techniques and
the broader schema of automated problem-solving [
        <xref ref-type="bibr" rid="ref3 ref4 ref7">5, 9, 6</xref>
        ].
As AI technology is being more thoroughly integrated into
our everyday lives, it is imperative for students to become
familiar with a holistic view of problem solving with AI.
In this paper, we present a preliminary study that
investigates how students learn in a series of carefully-designed
activities that introduce them to automated problem-solving
approaches and their ability to leverage AI techniques to
improve their decision-making process.
      </p>
    </sec>
    <sec id="sec-3">
      <title>3. ACTIVITY DESIGN</title>
      <p>We developed 5 exercises in UC Berkeley's Snap block-based
programming language that introduce and use a simple
graphical interface for teaching the breadth- rst search algorithm
(BFS). The rst four exercises serve as an introduction to
BFS, and feature a character, West, pictured in Figure 1,
who guides student through the steps of the algorithm by
telling them how to act as a GPS. The last activity is
formatted as a programming exercise, where students are given
an incomplete implementation of BFS, and are required to
ll in the gaps. The algorithm is used for contact tracing,
similar to the COVID contact tracing apps, to showcase a
real-world application of BFS.</p>
    </sec>
    <sec id="sec-4">
      <title>3.1 Exercise 1</title>
      <p>Exercise 1 is designed to introduce graph search as a problem
in the context of a GPS navigating between cities, as well as
to show how the methodologies for performing this search
di er between computers and humans. West shows the
student a city map as a graph, with nodes representing cities
and transitions representing roads between them (Figure 2).
The student is then instructed to click through the cities to
nd the shortest path from a starting city to a goal. The
student does this rst with the simple map shown in gure
2, then with a more complicated map, and then nally with
a map which has cities hidden from the student, which can
only be shown once they click an adjacent city. This last
example is meant to describe the perspective of the algorithm
to the students, to contrast it with the human perspective
of seeing the whole state space at once.</p>
    </sec>
    <sec id="sec-5">
      <title>3.2 Exercise 2</title>
      <p>Exercise 2 is designed to introduce BFS as an algorithm for
performing graph search. BFS is introduced to the students
via a series of animations and they are instructed to perform
each of the steps of BFS by interacting with our graphical
interface, shown in Figure 3. This interface provides two ways
to perform the actions of the algorithm. The rst is to add
nodes to the fringe by clicking them on the displayed graph.
The second is to remove nodes from the fringe. The
legend in Figure 3, which shows the graphical representations
of each state for the nodes and transitions, is displayed and
the interface updates accordingly as students interact with
it.</p>
    </sec>
    <sec id="sec-6">
      <title>3.3 Exercise 3</title>
      <p>Exercise 3 consists of a completely guided run-through of
BFS for a medium-sized graph with eight nodes and ten
transitions. At every step of the algorithm, students are
told what to do next and upon making a mistake, they are
informed about what the mistake is and re-prompted until
choosing the correct option.</p>
    </sec>
    <sec id="sec-7">
      <title>3.4 Exercise 4</title>
      <p>Exercise 4 is composed of a partially guided run-through
of BFS. To connect this exercise with a real-world
application, we show the graph superimposed over a map of the
east coast of the U.S., with the nodes named for their
corresponding cities. For the duration of the exercise, students
complete the steps of the algorithm without feedback unless
they make a mistake. Upon making a mistake, they are
informed about the nature of their mistake and re-prompted
until they perform the correct next step.</p>
    </sec>
    <sec id="sec-8">
      <title>3.5 Exercise 5</title>
      <p>Exercise 5 challenges students to implement their
understanding of BFS as Snap code. For this exercise, we
recontextualized the state space for BFS from a map of cities
to a social network of people. When the exercise is
initialized, one to three random people are marked as infected
with COVID-19. The objective for BFS was changed from
nding the shortest path from a starting city to a goal city
to nding the shortest number of social connections from
a user-selected person to an infected person. This scenario
was inspired by the COVID contact tracing apps that have
recently been developed and implemented in order to show
students a real-world application of BFS. Figure 4, right
shows the graph of people with the infected people in red.
Students were given a partial implementation of the
algorithm along with missing blocks, with some instructions on
how to snap the blocks into their proper place (Figure 4,
left). Upon running the algorithm, if it is correctly
implemented, students see the path from the user-selected person
to the nearest infected person.</p>
    </sec>
    <sec id="sec-9">
      <title>4. PRELIMINARY CASE STUDY</title>
    </sec>
    <sec id="sec-10">
      <title>4.1 Research Design</title>
      <p>
        This study employed the case study method [
        <xref ref-type="bibr" rid="ref14">16</xref>
        ]. This
method has been seen as a framework to determine the
problems that need to be studied in a bounded system [
        <xref ref-type="bibr" rid="ref14">16</xref>
        ].
Furthermore, Creswell and Poth [
        <xref ref-type="bibr" rid="ref2">4</xref>
        ] elaborate that the case
study approach is a research methodology that aims to
identify speci c cases and to explore in-depth content related to
these cases. Since this study aims to understand problems,
challenges, and opportunities while piloting the AI
curriculum module within a bounded system, this methodology was
followed for the research design.
      </p>
    </sec>
    <sec id="sec-11">
      <title>4.2 Participant and Context</title>
      <p>The case study participant was Selim (pseudonym). He was
a student in the 11th grade at a public high school in the
United States. He had an interest in Arti cial Intelligence
and was actively looking for opportunities to learn about
how AI worked and how AI could be used in the eld of
his interest, Biomedical Engineering. He had some general
knowledge about AI and Python programming. The study
took place in the context of two informal virtual sessions
that were designed to introduce Arti cial Intelligence with
the BFS search algorithm. The sessions were conducted
remotely, a week apart, via Zoom, a video conferencing tool.
The rst session took approximately one hour. In this
session, we rst described the goal of the session and
emphasized that the student was expected to experience the rst
four learning activities as a user, complete a post-activity
questionnaire, and participate in an interview. The second
session was mainly focused on the fth activity and took 1.5
hours. In this session, Selim answered questions related to
the previous session, followed a written Snap tutorial,
completed the activity, and participated in the interview.</p>
    </sec>
    <sec id="sec-12">
      <title>4.3 Data Collection Process</title>
      <p>In this study, we collected data in the form of a
thinkaloud protocol with approximately thirty minutes of a
semistructured interview, post-questionnaire, screen capture, and
submitted block-based programming artifacts. In the rst
session, we asked the participant to answer warm-up
questions (e.g., Have you ever taken any class on AI?) and
complete an activity that aimed to extract his interests and
general knowledge about AI. In addition to the warm-up
activity, at the end of each learning activity, we also asked
\What do you think you have learned? What else would you
like to explore?". With these questions, the participant
reected upon his own learning. Also, after he completed the
activities, we asked him to complete the post-questionnaire
that included seven multiple-choice questions related to the
BFS learning activities. At the end of the rst session, we
conducted semi-structured interviews for approximately 10
minutes to learn more about his experience. In the second
session, we followed a similar structure. We started the
session with the warm-up questions to recall the student's
understanding from the previous session and to correct
misconceptions. Before the activity, we guided the student through
the creation of pseudocode which re ected his understanding
of the algorithm. We then provided a written Snap tutorial
to allow the student to get familiar with the learning
environment. After he skimmed through the tutorial document,
he moved on to the fth activity. At the end of the
activity, we conducted an interview to learn more about his
experiences.</p>
    </sec>
    <sec id="sec-13">
      <title>4.4 Data Analysis</title>
      <p>To analyze the transcripts of interviews, we followed the
thematic coding method [2], which is a widely used method
to analyze qualitative data for identifying, organizing, and
describing themes. In the rst round of the analysis, one
researcher analyzed the data to explore learning challenges
and opportunities. After the rst-round coding, a
peerdebrie ng meeting was held with three other researchers
to ensure the trustworthiness of the ndings.
Additionally, we analyzed the student's process of building the
blockbased programming artifacts by following the thematic
coding strategies. We open-coded the students' actions in the
activities to identify learning challenges and opportunities.
After that, we discussed these codes and came up with the
general themes discussed in section 5.3.</p>
    </sec>
    <sec id="sec-14">
      <title>5. RESULTS AND DISCUSSION</title>
      <p>Results are organized around three overarching themes:
teaching AI as an interdisciplinary eld, creating opportunities
for students to discuss who bene ts from AI and is left out,
and promoting e ective AI learning through developing a
humanized curriculum.</p>
    </sec>
    <sec id="sec-15">
      <title>5.1 Teaching AI as an interdisciplinary field</title>
      <p>We should provide opportunities for students to understand
the integration of AI in application areas and teach AI as
an interdisciplinary eld. Selim was interested in the eld
of biomedical engineering and believed that AI would
contribute to innovations in every eld, including biomedical
engineering. Thus, he was motivated to learn more about
the application of AI in the eld of his interest. During
the rst session, he shared, \If you are trying to determine,
for tissue engineering, maybe you could use a model to
determine what's the best material to use based on di erent
situations." At the end of the rst session, Selim asked the
research team about learning experiences in biomedical
engineering and computer science in higher education since
he was at the critical stage of choosing future careers. His
question highlighted the need of supporting students to
develop interdisciplinary learning and collaboration skills and
understand the integration of AI in elds of their interest.</p>
    </sec>
    <sec id="sec-16">
      <title>5.2 Creating opportunities for students to discuss who benefits from AI and is left out</title>
      <p>Our analysis indicates that topics related to AI ethics might
serve as a catalyst for students to have in-depth discussions
about who bene ts from AI and who is left out. For instance,
in the excerpt below (Excerpt 1), guided by the research
team, Selim emphasized that AI ethics could be an engaging
topic of discussion. Excerpt 1.</p>
      <p>1. Research team: Have you talked about AI with your
family and friends?
2. Selim: Not really my friends. With family, yeah. We've
had a few discussions. Um, it's probably a really
common discussion. I think one of my parents heard it
on a podcast or something like the thing about a
selfdriving car having an unavoidable accident.
3. Research team: Yeah, I remember this one.
4. Selim: Like, who it (referring to the self-driving car)
picked to crash into or something. Just like ethics of AI
is probably the most interesting conversation to have
with other people.</p>
      <p>In Turn 4, he highlighted that a model would fall short in
addressing the moral question of who a self-driving car should
kill in an unavoidable accident. However, he did not
recognize that essentially, it's the car maker or designer who
would make the decision.</p>
      <p>1. Research team:Do you nd AI ethics interesting?
2. Selim: Oh, yeah.
3. Research team: In what ways? Can you give an
example?
4. Selim: So another thing that I found interesting is the
hiring process. I think it's Amazon. It has implicit
bias against hiring applicants that had women in their
application. That's obviously an important issue that
needs to be xed. So there are really complex issues
when you talk about AI ethics.</p>
      <p>In turn 8, he raised the concern about gender bias using AI
recruiting or resume screening tools. Overall, we can see that
AI ethics and AI decision making that would have an impact
on who bene ts and is left is a promising topic for students
to think critically about the impacts of AI technologies.</p>
    </sec>
    <sec id="sec-17">
      <title>5.3 Promoting effective AI learning through developing a humanized curriculum</title>
      <p>As described in section 3, the student is presented with a
series of ve activities. The rst four activities are designed
to introduce the BFS algorithm to the student while the fth
requires him to implement the BFS algorithm in a
blockbased programming environment in the context of a contact
tracing app. Selim progressed though the rst four activities
rather smoothly. After each activity he indicated a deeper
understanding of the BFS algorithm. He then successfully
answered content knowledge questions presented to him at
the end of the rst session
In the beginning of the second session, the student started by
re ecting on what he recalled from the previous session and
was then asked to devise pseudocode for the BFS algorithm.
Through this process he demonstrated a good understanding
of the BFS algorithm but a lack of familiarity with writing
pseudocode. With the guidance of researchers he managed
to turn his knowledge of the BFS algorithm into a
pseudocode format. He was then presented with an incomplete
contact tracing app that utilizes BFS to nd the shortest
path between an arbitrary person and an infected person
in his social circle.The BFS algorithm was partially
implemented as a Parson's problem, and the student snapped the
readily available blocks on the screen to complete the
algorithm. Here again, the student showed an adequate
understanding of the BFS algorithm. However, he faced some
challenges when trying to implement it within the
blockbased programming environment. We identi ed three main
themes of challenges faced by the student while conducting
the programming. First, he was not familiar with some of
the blocks, list-related blocks in particular, and was
confused about what they represent in the incomplete program.
Secondly, he had trouble understanding the functionality of
some of the custom blocks provided for him. Finally, the
student was unclear about how to map from steps of his
pseudocode onto the di erent sections of the partial code.
While the rst challenge can be addressed by providing
students with a more extensive block-based programming
tutorial beforehand, the second two challenges require
improvements in the design of the activity. For example, a better
description of the functionality of custom blocks and a clearer
distinction between custom blocks and contextually named
variables can alleviate some of the confusions that students
might encounter when analyzing the partial code presented
to them. Additionally, having students implement a simple
case of BFS following the same context as the previous
activities might facilitate the transition to implementing the
BFS algorithm in an arbitrary real-world context.</p>
    </sec>
    <sec id="sec-18">
      <title>6. CONCLUSION</title>
      <p>With the advances of AI technology and its rapid
integration within our society, it is imperative that AI education
is included as part of the K-12 standard curriculum. In
this study, we presented the preliminary design of a
curricular module that is designed to teach a foundational AI
algorithm, breadth- rst search (BFS), to high school
students. We presented the results of a pilot implementation of
this curriculum with a high-school student as a case study.
Our results demonstrated that this intervention helped the
student gain a better perception about the interdisciplinary
nature of AI and how it can be incorporated to enrich other
elds. Furthermore, the discussions fostered curiosity around
AI ethics which is an important aspect of AI education.
Finally, our analysis of the student's interactions with the
designed activities and his re ections showed that the activities
were successful in improving the student's understanding of
the BFS algorithm while posing some challenges when the
student tried to map from the algorithm to a block-based
programming implementation.</p>
    </sec>
    <sec id="sec-19">
      <title>7. FUTURE WORK</title>
      <p>In the future, we aim to utilize insights obtained from this
case study to not only improve the design of the current
module but also to inform design of the future AI-related
curricular modules. While this study provided a qualitative
analysis of observed interactions between the student and
activities, we plan to automate this process in the future
through data-driven approaches to obtain more insights into
the e ectiveness of our curriculum and students' learning.
We propose creating a series of curricular modules that teach
students a holistic view of AI and equip them with
knowledge, skills, and abilities that prepare them for conducting
problem-solving with AI in humanized real-world scenarios.
We further propose situating our curriculum within a
learning environment where data can be collected and analyzed
for informing instruction and providing students with
adaptive sca olding.
8. REFERENCES
[1] M. Bassey. Case study research. Research methods in
educational leadership and management, pages
108{121, 2002.
[2] V. Braun and V. Clarke. Using thematic analysis in
psychology. Qualitative research in psychology,
3(2):77{101, 2006.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [3]
          <string-name>
            <surname>E. Charters.</surname>
          </string-name>
          <article-title>The use of think-aloud methods in qualitative research an introduction to think-aloud methods</article-title>
          .
          <source>Brock Education: A Journal of Educational Research and Practice</source>
          ,
          <volume>12</volume>
          (
          <issue>2</issue>
          ),
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>J. W.</given-names>
            <surname>Creswell</surname>
          </string-name>
          and
          <string-name>
            <given-names>C. N.</given-names>
            <surname>Poth</surname>
          </string-name>
          .
          <article-title>Qualitative inquiry and research method: Choosing among ve approaches</article-title>
          ,
          <year>2007</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>J.</given-names>
            <surname>Estevez</surname>
          </string-name>
          , G. Garate, and
          <string-name>
            <surname>M.</surname>
          </string-name>
          <article-title>Gran~a. Gentle introduction to arti cial intelligence for high-school students using scratch</article-title>
          .
          <source>IEEE Access</source>
          ,
          <volume>7</volume>
          :
          <fpage>179027</fpage>
          {
          <fpage>179036</fpage>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>T.</given-names>
            <surname>Hitron</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Orlev</surname>
          </string-name>
          ,
          <string-name>
            <surname>I. Wald</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Shamir</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Erel</surname>
          </string-name>
          , and
          <string-name>
            <given-names>O.</given-names>
            <surname>Zuckerman</surname>
          </string-name>
          .
          <article-title>Can children understand machine learning concepts? the e ect of uncovering black boxes</article-title>
          .
          <source>In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems</source>
          , pages
          <fpage>1</fpage>
          {
          <fpage>11</fpage>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>S.</given-names>
            <surname>Isotani</surname>
          </string-name>
          ,
          <string-name>
            <given-names>E.</given-names>
            <surname>Millan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ogan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Hastings</surname>
          </string-name>
          ,
          <string-name>
            <surname>B. McLaren,</surname>
          </string-name>
          and
          <string-name>
            <given-names>R.</given-names>
            <surname>Luckin</surname>
          </string-name>
          .
          <source>Arti cial Intelligence in Education</source>
          . Springer,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>V. R.</given-names>
            <surname>Lee</surname>
          </string-name>
          and
          <string-name>
            <given-names>D. A.</given-names>
            <surname>Fields</surname>
          </string-name>
          .
          <article-title>A clinical interview for assessing student learning in a university-level craft technology course</article-title>
          .
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>P.</given-names>
            <surname>Lin</surname>
          </string-name>
          ,
          <string-name>
            <surname>J. Van Brummelen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Lukin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Williams</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Breazeal</surname>
          </string-name>
          . Zhorai:
          <article-title>Designing a conversational agent for children to explore machine learning concepts</article-title>
          .
          <source>In Proceedings of the AAAI Conference on Arti cial Intelligence</source>
          , volume
          <volume>34</volume>
          , pages
          <fpage>13381</fpage>
          {
          <fpage>13388</fpage>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>D.</given-names>
            <surname>Long</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Magerko</surname>
          </string-name>
          .
          <article-title>What is ai literacy? competencies and design considerations</article-title>
          .
          <source>In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems</source>
          , pages
          <fpage>1</fpage>
          {
          <fpage>16</fpage>
          ,
          <year>2020</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>P. S.</given-names>
            <surname>Mohammed</surname>
          </string-name>
          et al.
          <article-title>Towards inclusive education in the age of arti cial intelligence: Perspectives, challenges, and opportunities</article-title>
          .
          <source>In Arti cial intelligence and inclusive education</source>
          , pages
          <volume>17</volume>
          {
          <fpage>37</fpage>
          . Springer,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>D. J.</given-names>
            <surname>Moran</surname>
          </string-name>
          .
          <article-title>The need for evidence-based educational methods</article-title>
          .
          <source>In Evidence-based educational methods</source>
          , pages
          <volume>3</volume>
          {
          <fpage>7</fpage>
          .
          <string-name>
            <surname>Elsevier</surname>
          </string-name>
          ,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>S.</given-names>
            <surname>Nikiforos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Kontomaris</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Chorianopoulos</surname>
          </string-name>
          .
          <article-title>Mit scratch: A powerful tool for improving teaching of programming</article-title>
          .
          <source>Conferrence on Informatics in Education, pages 1{5</source>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>S.</given-names>
            <surname>Papert</surname>
          </string-name>
          .
          <article-title>The children's machine: Rethinking school in the age of the computer</article-title>
          .
          <source>ERIC</source>
          ,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>D.</given-names>
            <surname>Passey</surname>
          </string-name>
          .
          <article-title>Computer science (cs) in the compulsory education curriculum: Implications for future research</article-title>
          .
          <source>Education and Information Technologies</source>
          ,
          <volume>22</volume>
          (
          <issue>2</issue>
          ):
          <volume>421</volume>
          {
          <fpage>443</fpage>
          ,
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>R. E.</given-names>
            <surname>Stake</surname>
          </string-name>
          .
          <article-title>Qualitative case studies</article-title>
          .
          <year>2005</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>C.</given-names>
            <surname>Tatar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Yoder</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Coven</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Wiedemann</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Chao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Finzer</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Jiang</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Rose</surname>
          </string-name>
          .
          <article-title>Modeling unstructured data: Teachers as learners and designers of technology-enhanced arti cial intelligence curriculum</article-title>
          .
          <source>In International Society of the Learning Sciences Annual Meeting</source>
          <year>2021</year>
          ,
          <year>2021</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>A.</given-names>
            <surname>Tucker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Deek</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Jones</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>McCowan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Stephenson</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Verno</surname>
          </string-name>
          .
          <article-title>A Model Curriculum for K{12 Computer Science</article-title>
          . ACM/Association for Computing Machinery,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [19]
          <string-name>
            <given-names>A.</given-names>
            <surname>Zimmermann-Nie eld</surname>
          </string-name>
          , M. Turner,
          <string-name>
            <given-names>B.</given-names>
            <surname>Murphy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. K.</given-names>
            <surname>Kane</surname>
          </string-name>
          , and
          <string-name>
            <given-names>R. B.</given-names>
            <surname>Shapiro</surname>
          </string-name>
          .
          <article-title>Youth learning machine learning through building models of athletic moves</article-title>
          .
          <source>In Proceedings of the 18th ACM International Conference on Interaction Design and Children</source>
          , pages
          <volume>121</volume>
          {
          <fpage>132</fpage>
          ,
          <year>2019</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>