=Paper=
{{Paper
|id=Vol-3051/CSEDM_11
|storemode=property
|title=Gaining Insight into Effective Teaching of AI Problem-Solving Through CSEDM: A Case Study (Work in Progress)
|pdfUrl=https://ceur-ws.org/Vol-3051/CSEDM_11.pdf
|volume=Vol-3051
|authors=Spencer Yoder,Cansu Tatar,Ifeoluwa Aderemi,Sankalp Boorugu,Shiyan Jiang,Bita Akram
|dblpUrl=https://dblp.org/rec/conf/edm/YoderTABJA21
}}
==Gaining Insight into Effective Teaching of AI Problem-Solving Through CSEDM: A Case Study (Work in Progress)==
https://ceur-ws.org/Vol-3051/CSEDM_11.pdf
Gaining Insight into Effective Teaching of AI
Problem-Solving Through CSEDM: A Case Study
Spencer Yoder Cansu Tatar Ifeoluwa Aderemi
North Carolina State North Carolina State North Carolina State
University University University
smyoder@ncsu.edu ctatar@ncsu.edu iwaderem@ncsu.edu
Sankalp Boorugu Shiyan Jiang Bita Akram
North Carolina State North Carolina State North Carolina State
University University University
sboorug@ncsu.edu sjiang24@ncsu.edu bakram@ncsu.edu
ABSTRACT Keywords
CS education for all has been established as an important K-12 AI Education, Breadth-first-search AI algorithm CSEDM
area of focus for both researchers and practitioners in re- for AI Education
cent years. At the same time, due to the increasing preva- 1. INTRODUCTION
lence of AI technology in humans’ everyday lives, artificial It has been almost half a decade since Simon Papert intro-
intelligence (AI) education for K-12 is gaining special at- duced the idea of computer science (CS) for all [14]. Since
tention among CS educators. AI literacy, even more than then, many researchers[15], educational institutes (e.g. [13])
general CS competencies, requires evidence-based research and organizations [18] have picked up the effort to create ac-
to be effectively integrated in our schools. The common cessible CS curricula for K-12 classrooms. Meanwhile, artifi-
learning environments utilized for CS education enable us cial intelligence, a sub-field of computer science, is becoming
to go beyond conventional educational research approaches a domineering aspect of people’s everyday lives [17]. Daily
by providing a platform where detailed data can be collected technologies and decisions are becoming more and more de-
from students’ interaction with CS-education-related activ- pendent on AI. Under these circumstances, it is imperative
ities. Thus, conventional educational research approaches for our new generation to gain a fundamental understand-
coupled with insights gained from pattern recognition and ing of AI mechanisms and also its potential to introduce
student modeling approaches enable us to effectively im- biases and unfairness through automated decision-making.
prove our instruction and to provide students with adaptive Furthermore, the advances in AI systems are substituting
scaffolding. In this work, we present our first AI curriculum many of the old jobs with ones that require the ability to do
module that is designed to teach a fundamental AI search problem-solving with AI. For the aforementioned reasons,
algorithm, Breadth-First Search (BFS), through a series of efforts for integrating AI into K-12 curriculum as an impor-
progressively scaffolded activities. Data is collected from tant part of CS education is gaining momentum [11], [7].
a preliminary pilot of this activity with a high-school stu-
dent in the form of a think-aloud protocol, screen capture, Integrating AI education into the K-12 curriculum poses sig-
submitted block-based programming artifacts, and interview nificant challenges for educators since teachers often lack
questions. Our results demonstrate that our activities have prior education in CS related fields [11]. Furthermore, the
been successful in increasing the student ’s knowledge about skills, knowledge, and abilities required for getting engaged
the BFS algorithm and more importantly, how this particu- in AI-related activities are novel to students and do not align
lar AI algorithm can be utilized to solve real world problems. with their conventional ways of learning [cite]. Thus, effec-
Based on the results of this pilot study, we propose design- tive instructional approaches should be identified through
ing a comprehensive AI curriculum contextualized within a evidence-based research. This is especially important since
learning environment that collects detailed data from stu- strong stereotypes and biases about who can learn this field
dents’ progress to inform instructional design and facilitate prevent some of the students from getting engaged in the
adaptive scaffolding for students. devised curriculum in the first place [12]. This can be facili-
tated by taking advantage of one of the main affordances of
CS learning environments that is their inherent capability
to collect fine-grained data from students’ interactions with
the learning activities. The collected log data can then be
utilized in data-driven analysis that can inform instructors
about the curriculum effectiveness and students’ learning.
In this work we present the preliminary design of an AI cur-
Copyright ©2021 for this paper by its authors. Use permitted under Cre- ricular module that teaches one of the most fundamental AI
ative Commons License Attribution 4.0 International (CC BY 4.0) search algorithms, breadth-first search (BFS). The module
Figure 1: West, the character to guide students through the
exercises.
consists of a series of activities that aims to teach learners
about the BFS algorithm and ends with a coding activity
where students have to incorporate the BFS algorithm in a
real-world problem scenario to build a contact tracing ap-
plication for COVID-19. While many devised AI curricula
focus on the machine learning (ML) aspect of AI, we aim Figure 2: The graph representation of a city map.
to provide K-12 students with a comprehensive view of AI
that includes ML as an integrated aspect of AI. We present
In this paper, we present a preliminary study that investi-
a case study [1] where this curricular module is piloted with
gates how students learn in a series of carefully-designed ac-
a high-school student and data is collected through think
tivities that introduce them to automated problem-solving
aloud protocol [3], screen recording, and clinical interviews
approaches and their ability to leverage AI techniques to
[8]. This data is then analyzed to make inferences about the
improve their decision-making process.
effectiveness of the curriculum, required improvements and
implications for design for a comprehensive AI curriculum
integrated within a learning environment that collects learn- 3. ACTIVITY DESIGN
ers’ interaction data to provide both educators and students We developed 5 exercises in UC Berkeley’s Snap block-based
with adaptive assessment and feedback. programming language that introduce and use a simple graph-
ical interface for teaching the breadth-first search algorithm
Section 2 discusses related work. The details of the devised (BFS). The first four exercises serve as an introduction to
curriculum are presented in section 3. The case study is BFS, and feature a character, West, pictured in Figure 1,
presented in section 4, and the results of the pilot study are who guides student through the steps of the algorithm by
discussed in section 5. Section 6 concludes the work, and telling them how to act as a GPS. The last activity is for-
section 7 discusses implications for design and future work. matted as a programming exercise, where students are given
an incomplete implementation of BFS, and are required to
fill in the gaps. The algorithm is used for contact tracing,
2. RELATED WORKS similar to the COVID contact tracing apps, to showcase a
Emerging research is exploring the design of learning ex- real-world application of BFS.
periences to foster youths’ Artificial Intelligence (AI) liter-
acy so that they are prepared to enter and engage with an
AI-filled future [10]. In particular, efforts have been made 3.1 Exercise 1
to engage youth in understanding AI through the develop- Exercise 1 is designed to introduce graph search as a problem
ment of machine learning (ML) models. For instance, a work in the context of a GPS navigating between cities, as well as
by Zimmermann-Niefield and colleagues (2019) provided an to show how the methodologies for performing this search
embodied learning experience for youth to create ML mod- differ between computers and humans. West shows the stu-
els for recognizing their own physical activities [19]. They dent a city map as a graph, with nodes representing cities
found that youth developed an understanding of how ML and transitions representing roads between them (Figure 2).
models learned patterns of body movements and this could The student is then instructed to click through the cities to
contribute to the understanding of the iterative process of find the shortest path from a starting city to a goal. The
ML. In addition, Google developed web-based tools (e.g., student does this first with the simple map shown in figure
Teachable Machine) to make ML accessible to the public, 2, then with a more complicated map, and then finally with
including youth [3]. These studies stressed the cultivation a map which has cities hidden from the student, which can
of data literacy among youth as modeling data as a core con- only be shown once they click an adjacent city. This last ex-
cept in ML. While K-12 AI education is gaining momentum, ample is meant to describe the perspective of the algorithm
the curricula being devised for K-12 students group tend to to the students, to contrast it with the human perspective
focus on basic ML approaches. However, these curricula fail of seeing the whole state space at once.
to demonstrate the connection between ML techniques and
the broader schema of automated problem-solving [5, 9, 6]. 3.2 Exercise 2
As AI technology is being more thoroughly integrated into Exercise 2 is designed to introduce BFS as an algorithm for
our everyday lives, it is imperative for students to become performing graph search. BFS is introduced to the students
familiar with a holistic view of problem solving with AI. via a series of animations and they are instructed to perform
was inspired by the COVID contact tracing apps that have
recently been developed and implemented in order to show
students a real-world application of BFS. Figure 4, right
shows the graph of people with the infected people in red.
Students were given a partial implementation of the algo-
rithm along with missing blocks, with some instructions on
how to snap the blocks into their proper place (Figure 4,
left). Upon running the algorithm, if it is correctly imple-
mented, students see the path from the user-selected person
to the nearest infected person.
4. PRELIMINARY CASE STUDY
4.1 Research Design
This study employed the case study method [16]. This
method has been seen as a framework to determine the
Figure 3: The graphical interface students interact with to problems that need to be studied in a bounded system [16].
perform BFS. (Bottom-left) The legend, which explains to Furthermore, Creswell and Poth [4] elaborate that the case
students how the different states for nodes and transitions study approach is a research methodology that aims to iden-
are displayed. (Middle) The graph representing a city map. tify specific cases and to explore in-depth content related to
(Right) The fringe. these cases. Since this study aims to understand problems,
challenges, and opportunities while piloting the AI curricu-
lum module within a bounded system, this methodology was
each of the steps of BFS by interacting with our graphical in-
followed for the research design.
terface, shown in Figure 3. This interface provides two ways
to perform the actions of the algorithm. The first is to add
nodes to the fringe by clicking them on the displayed graph. 4.2 Participant and Context
The second is to remove nodes from the fringe. The leg- The case study participant was Selim (pseudonym). He was
end in Figure 3, which shows the graphical representations a student in the 11th grade at a public high school in the
of each state for the nodes and transitions, is displayed and United States. He had an interest in Artificial Intelligence
the interface updates accordingly as students interact with and was actively looking for opportunities to learn about
it. how AI worked and how AI could be used in the field of
his interest, Biomedical Engineering. He had some general
knowledge about AI and Python programming. The study
3.3 Exercise 3 took place in the context of two informal virtual sessions
Exercise 3 consists of a completely guided run-through of that were designed to introduce Artificial Intelligence with
BFS for a medium-sized graph with eight nodes and ten the BFS search algorithm. The sessions were conducted re-
transitions. At every step of the algorithm, students are motely, a week apart, via Zoom, a video conferencing tool.
told what to do next and upon making a mistake, they are The first session took approximately one hour. In this ses-
informed about what the mistake is and re-prompted until sion, we first described the goal of the session and empha-
choosing the correct option. sized that the student was expected to experience the first
four learning activities as a user, complete a post-activity
3.4 Exercise 4 questionnaire, and participate in an interview. The second
Exercise 4 is composed of a partially guided run-through session was mainly focused on the fifth activity and took 1.5
of BFS. To connect this exercise with a real-world applica- hours. In this session, Selim answered questions related to
tion, we show the graph superimposed over a map of the the previous session, followed a written Snap tutorial, com-
east coast of the U.S., with the nodes named for their cor- pleted the activity, and participated in the interview.
responding cities. For the duration of the exercise, students
complete the steps of the algorithm without feedback unless 4.3 Data Collection Process
they make a mistake. Upon making a mistake, they are in- In this study, we collected data in the form of a think-
formed about the nature of their mistake and re-prompted aloud protocol with approximately thirty minutes of a semi-
until they perform the correct next step. structured interview, post-questionnaire, screen capture, and
submitted block-based programming artifacts. In the first
3.5 Exercise 5 session, we asked the participant to answer warm-up ques-
Exercise 5 challenges students to implement their under- tions (e.g., Have you ever taken any class on AI?) and com-
standing of BFS as Snap code. For this exercise, we re- plete an activity that aimed to extract his interests and
contextualized the state space for BFS from a map of cities general knowledge about AI. In addition to the warm-up
to a social network of people. When the exercise is initial- activity, at the end of each learning activity, we also asked
ized, one to three random people are marked as infected “What do you think you have learned? What else would you
with COVID-19. The objective for BFS was changed from like to explore?”. With these questions, the participant re-
finding the shortest path from a starting city to a goal city flected upon his own learning. Also, after he completed the
to finding the shortest number of social connections from activities, we asked him to complete the post-questionnaire
a user-selected person to an infected person. This scenario that included seven multiple-choice questions related to the
Figure 4: Exercise 5. (Left) The incomplete BFS implementation with the missing blocks located under the code. (Right) the
graph representing the social network from which the student selects an arbitrary person to trace to the nearest infected person.
BFS learning activities. At the end of the first session, we an interdisciplinary field. Selim was interested in the field
conducted semi-structured interviews for approximately 10 of biomedical engineering and believed that AI would con-
minutes to learn more about his experience. In the second tribute to innovations in every field, including biomedical
session, we followed a similar structure. We started the ses- engineering. Thus, he was motivated to learn more about
sion with the warm-up questions to recall the student’s un- the application of AI in the field of his interest. During
derstanding from the previous session and to correct miscon- the first session, he shared, “If you are trying to determine,
ceptions. Before the activity, we guided the student through for tissue engineering, maybe you could use a model to de-
the creation of pseudocode which reflected his understanding termine what’s the best material to use based on different
of the algorithm. We then provided a written Snap tutorial situations.” At the end of the first session, Selim asked the
to allow the student to get familiar with the learning envi- research team about learning experiences in biomedical en-
ronment. After he skimmed through the tutorial document, gineering and computer science in higher education since
he moved on to the fifth activity. At the end of the ac- he was at the critical stage of choosing future careers. His
tivity, we conducted an interview to learn more about his question highlighted the need of supporting students to de-
experiences. velop interdisciplinary learning and collaboration skills and
understand the integration of AI in fields of their interest.
4.4 Data Analysis
To analyze the transcripts of interviews, we followed the 5.2 Creating opportunities for students to dis-
thematic coding method [2], which is a widely used method
to analyze qualitative data for identifying, organizing, and cuss who benefits from AI and is left out
describing themes. In the first round of the analysis, one re- Our analysis indicates that topics related to AI ethics might
searcher analyzed the data to explore learning challenges serve as a catalyst for students to have in-depth discussions
and opportunities. After the first-round coding, a peer- about who benefits from AI and who is left out. For instance,
debriefing meeting was held with three other researchers in the excerpt below (Excerpt 1), guided by the research
to ensure the trustworthiness of the findings. Addition- team, Selim emphasized that AI ethics could be an engaging
ally, we analyzed the student’s process of building the block- topic of discussion. Excerpt 1.
based programming artifacts by following the thematic cod-
ing strategies. We open-coded the students’ actions in the
1. Research team: Have you talked about AI with your
activities to identify learning challenges and opportunities.
family and friends?
After that, we discussed these codes and came up with the
general themes discussed in section 5.3.
2. Selim: Not really my friends. With family, yeah. We’ve
had a few discussions. Um, it’s probably a really com-
5. RESULTS AND DISCUSSION mon discussion. I think one of my parents heard it
Results are organized around three overarching themes: teach- on a podcast or something like the thing about a self-
ing AI as an interdisciplinary field, creating opportunities driving car having an unavoidable accident.
for students to discuss who benefits from AI and is left out,
and promoting effective AI learning through developing a 3. Research team: Yeah, I remember this one.
humanized curriculum.
4. Selim: Like, who it (referring to the self-driving car)
5.1 Teaching AI as an interdisciplinary field picked to crash into or something. Just like ethics of AI
We should provide opportunities for students to understand is probably the most interesting conversation to have
the integration of AI in application areas and teach AI as with other people.
In Turn 4, he highlighted that a model would fall short in ad- student was unclear about how to map from steps of his
dressing the moral question of who a self-driving car should pseudocode onto the different sections of the partial code.
kill in an unavoidable accident. However, he did not rec-
ognize that essentially, it’s the car maker or designer who While the first challenge can be addressed by providing stu-
would make the decision. dents with a more extensive block-based programming tuto-
rial beforehand, the second two challenges require improve-
ments in the design of the activity. For example, a better de-
1. Research team:Do you find AI ethics interesting? scription of the functionality of custom blocks and a clearer
distinction between custom blocks and contextually named
2. Selim: Oh, yeah.
variables can alleviate some of the confusions that students
3. Research team: In what ways? Can you give an exam- might encounter when analyzing the partial code presented
ple? to them. Additionally, having students implement a simple
case of BFS following the same context as the previous ac-
4. Selim: So another thing that I found interesting is the tivities might facilitate the transition to implementing the
hiring process. I think it’s Amazon. It has implicit BFS algorithm in an arbitrary real-world context.
bias against hiring applicants that had women in their
application. That’s obviously an important issue that 6. CONCLUSION
needs to be fixed. So there are really complex issues
With the advances of AI technology and its rapid integra-
when you talk about AI ethics.
tion within our society, it is imperative that AI education
is included as part of the K-12 standard curriculum. In
In turn 8, he raised the concern about gender bias using AI this study, we presented the preliminary design of a cur-
recruiting or resume screening tools. Overall, we can see that ricular module that is designed to teach a foundational AI
AI ethics and AI decision making that would have an impact algorithm, breadth-first search (BFS), to high school stu-
on who benefits and is left is a promising topic for students dents. We presented the results of a pilot implementation of
to think critically about the impacts of AI technologies. this curriculum with a high-school student as a case study.
Our results demonstrated that this intervention helped the
student gain a better perception about the interdisciplinary
5.3 Promoting effective AI learning through nature of AI and how it can be incorporated to enrich other
developing a humanized curriculum fields. Furthermore, the discussions fostered curiosity around
As described in section 3, the student is presented with a AI ethics which is an important aspect of AI education. Fi-
series of five activities. The first four activities are designed nally, our analysis of the student’s interactions with the de-
to introduce the BFS algorithm to the student while the fifth signed activities and his reflections showed that the activities
requires him to implement the BFS algorithm in a block- were successful in improving the student’s understanding of
based programming environment in the context of a contact the BFS algorithm while posing some challenges when the
tracing app. Selim progressed though the first four activities student tried to map from the algorithm to a block-based
rather smoothly. After each activity he indicated a deeper programming implementation.
understanding of the BFS algorithm. He then successfully
answered content knowledge questions presented to him at 7. FUTURE WORK
the end of the first session
In the future, we aim to utilize insights obtained from this
case study to not only improve the design of the current
In the beginning of the second session, the student started by
module but also to inform design of the future AI-related
reflecting on what he recalled from the previous session and
curricular modules. While this study provided a qualitative
was then asked to devise pseudocode for the BFS algorithm.
analysis of observed interactions between the student and
Through this process he demonstrated a good understanding
activities, we plan to automate this process in the future
of the BFS algorithm but a lack of familiarity with writing
through data-driven approaches to obtain more insights into
pseudocode. With the guidance of researchers he managed
the effectiveness of our curriculum and students’ learning.
to turn his knowledge of the BFS algorithm into a pseu-
We propose creating a series of curricular modules that teach
docode format. He was then presented with an incomplete
students a holistic view of AI and equip them with knowl-
contact tracing app that utilizes BFS to find the shortest
edge, skills, and abilities that prepare them for conducting
path between an arbitrary person and an infected person
problem-solving with AI in humanized real-world scenarios.
in his social circle.The BFS algorithm was partially imple-
We further propose situating our curriculum within a learn-
mented as a Parson’s problem, and the student snapped the
ing environment where data can be collected and analyzed
readily available blocks on the screen to complete the al-
for informing instruction and providing students with adap-
gorithm. Here again, the student showed an adequate un-
tive scaffolding.
derstanding of the BFS algorithm. However, he faced some
challenges when trying to implement it within the block-
based programming environment. We identified three main 8. REFERENCES
themes of challenges faced by the student while conducting [1] M. Bassey. Case study research. Research methods in
the programming. First, he was not familiar with some of educational leadership and management, pages
the blocks, list-related blocks in particular, and was con- 108–121, 2002.
fused about what they represent in the incomplete program. [2] V. Braun and V. Clarke. Using thematic analysis in
Secondly, he had trouble understanding the functionality of psychology. Qualitative research in psychology,
some of the custom blocks provided for him. Finally, the 3(2):77–101, 2006.
[3] E. Charters. The use of think-aloud methods in S. K. Kane, and R. B. Shapiro. Youth learning
qualitative research an introduction to think-aloud machine learning through building models of athletic
methods. Brock Education: A Journal of Educational moves. In Proceedings of the 18th ACM International
Research and Practice, 12(2), 2003. Conference on Interaction Design and Children, pages
[4] J. W. Creswell and C. N. Poth. Qualitative inquiry 121–132, 2019.
and research method: Choosing among five
approaches, 2007.
[5] J. Estevez, G. Garate, and M. Graña. Gentle
introduction to artificial intelligence for high-school
students using scratch. IEEE Access, 7:179027–179036,
2019.
[6] T. Hitron, Y. Orlev, I. Wald, A. Shamir, H. Erel, and
O. Zuckerman. Can children understand machine
learning concepts? the effect of uncovering black
boxes. In Proceedings of the 2019 CHI Conference on
Human Factors in Computing Systems, pages 1–11,
2019.
[7] S. Isotani, E. Millán, A. Ogan, P. Hastings,
B. McLaren, and R. Luckin. Artificial Intelligence in
Education. Springer, 2019.
[8] V. R. Lee and D. A. Fields. A clinical interview for
assessing student learning in a university-level craft
technology course. 2013.
[9] P. Lin, J. Van Brummelen, G. Lukin, R. Williams, and
C. Breazeal. Zhorai: Designing a conversational agent
for children to explore machine learning concepts. In
Proceedings of the AAAI Conference on Artificial
Intelligence, volume 34, pages 13381–13388, 2020.
[10] D. Long and B. Magerko. What is ai literacy?
competencies and design considerations. In
Proceedings of the 2020 CHI Conference on Human
Factors in Computing Systems, pages 1–16, 2020.
[11] P. S. Mohammed et al. Towards inclusive education in
the age of artificial intelligence: Perspectives,
challenges, and opportunities. In Artificial intelligence
and inclusive education, pages 17–37. Springer, 2019.
[12] D. J. Moran. The need for evidence-based educational
methods. In Evidence-based educational methods,
pages 3–7. Elsevier, 2004.
[13] S. Nikiforos, C. Kontomaris, and K. Chorianopoulos.
Mit scratch: A powerful tool for improving teaching of
programming. Conferrence on Informatics in
Education, pages 1–5, 2013.
[14] S. Papert. The children’s machine: Rethinking school
in the age of the computer. ERIC, 1993.
[15] D. Passey. Computer science (cs) in the compulsory
education curriculum: Implications for future
research. Education and Information Technologies,
22(2):421–443, 2017.
[16] R. E. Stake. Qualitative case studies. 2005.
[17] C. Tatar, M. Yoder, M. Coven, K. Wiedemann,
J. Chao, W. Finzer, S. Jiang, and C. Rosé. Modeling
unstructured data: Teachers as learners and designers
of technology-enhanced artificial intelligence
curriculum. In International Society of the Learning
Sciences Annual Meeting 2021, 2021.
[18] A. Tucker, F. Deek, J. Jones, D. McCowan,
C. Stephenson, and A. Verno. A Model Curriculum for
K–12 Computer Science. ACM/Association for
Computing Machinery, 2003.
[19] A. Zimmermann-Niefield, M. Turner, B. Murphy,