This paper presents the design and implementation of a chatbot developed to support history teaching in primary schools. The project was inspired by two main goals: on the one side, we wanted to involve teachers and educators in the creation of the chatbot, asking them to co-design both the character and the content to be delivered; on the other side, we wanted to test the use of dramatization techniques for the design of the chatbot. The validation of the chatbot was conducted through an experimental phase involving five primary school classes. The classes experience with the chatbot received positive ratings, thus confirming its capability to engage the users with its behavior and appearance. The study also demonstrates the importance of teacher collaboration in the development of educational technologies: the design principles behind chatbots illustrate the broader mechanics of how generative AI systems operate in tasks like content creation, problem-solving, and personalized interactions. This allowed us to introduce AI-based tools to the teachers in a conscious and participatory manner.
In the last few years, thanks to the advent of AI-powered speech and text-based chats, the conversational
mode for acquiring information from and engaging with artificial agents has become an everyday
experience for many children. The architectures of today’s end user applications, then, make chatbots
potentially available in all areas reached by mobile services, since computation tends to occur on
centralized remote severs, with only a major obstacle still represented by linguistic minorities. In this
sense, chatbots represent an opportunity for teaching, as witnessed by several research projects that
have explored their use in schools [
In this paper, we report about a project in which we experimented the use of an educational chatbot in primary school inspired by two main goals: on the one side, we wanted to involve teachers and educators in the creation of the chatbot, asking them to co-design both the character and the content to be delivered; on the other side, we wanted to test the use of dramatization techniques for the design of the chatbot. To do so, we relied on artificial intelligence technologies that allow the chatbot to be entirely scripted by the developers, so as to avoid the pitfalls of generative models in factual knowledge and to control closely the character creation. 1st Workshop on Education for Artificial Intelligence (edu4AI 2024, https:// edu4ai.di.unito.it/ ), co-located with the 23rd International
CEUR
ceur-ws.org
The chatbot design and the experiments were carried out as part of the project “Chi decide per noi? Percorsi tra Storia ed Educazione Civica dall’università alla scuola primaria” (“Who decides for us? Pathways between History and Civic Education from university to primary school”) at the University of Turin. This project aimed to integrate the Civic Education programme of Italian primary school with education about political representation and participation from the perspective of their historical development. The project involved the University of Turin and a group of primary schools (grades IV and V) and relied on the use of traditional, media and digital teaching tools to convey the educational contents.
The involvement of teachers in the design of the chatbot had a twofold objective. Firstly, the
participatory design methodology chosen for this project required teachers as the main users of the
ifnal product. Secondly, the design process allowed us to give the teachers a role as active creators of
the interaction script between the chatbot and the users. The design of a chatbot serves as an excellent
way to understand how generative AI systems work. Chatbots, especially those powered by AI like GPT
models, simulate human-like conversation by generating text based on input. The design principles
behind chatbots illustrate the broader mechanics of how generative AI systems operate in tasks like
content creation, problem-solving, and personalized interactions. This allowed us to introduce ai-based
tools to the teachers in a conscious and participative manner. This aspect is very important especially in
the context of the Italian school. Indeed despite many contributions made by various organisations, the
level of digital preparation of the Italian teachers is still limited[
The paper is organized as follows: after reviewing the related work in Section 2, we describe the
design and creation of the chatbot in Section 3. Evaluation is illustrated and discussed in Section 4.
Conclusion and future work end the paper.
2. Related work
[
The project “Chi decide per noi?” included diferent phases: the first was the lesson planning by the group of teachers, which included primary school teachers and university staf; the second consisted of participatory lessons in class on the historical evolution of the concepts and practices of participation and representation; the third consisted of the elaboration of laboratory experiences on the topics covered in the lessons, which included the production of textual products (text and images), videos (stop-motion documentary), and of the chatbot. Funded by Fondazione CRT, the project was carried out from September 2022 to August 2024.
In order to avoid the well-know pitfalls of state-of-the-art chatbots (see [
Although the AIML programmer seeks to cover all possible user’s intents in the given domain, using regular expressions to improve the patterns’ capability to generalize, the coverage of an AIML script in most cases will still be limited, with the bot providing a generic reply when the user’s input cannot be matched against any patterns. For this reason, for the chatbot to be able to interact in a fluent and natural way with the users, some techniques can be adopted to minimize unpredictability. In general, the more constraining are the chatbot’s turns in terms of possible users answers, the more predictable and easier to handle will be the sentences produced by the user. If it is coherent with the context of the interaction, the chatbot’s turns may also include multimedia elements such as images and animations to clarify better the chatbot’s intended meaning and input elements for collecting the user’s input (buttons, lists, etc.), thus enforcing the chatbot’s capability to deal with it. For example, the following template creates the menu displayed in Figure 1 1 <template> 2 Preferisci un nome femminile o maschile? 3 <button> 4 <text>Femminile</text> 5 <postback>femminile</postback> 6 </button> 7 <button> 8 <text>Maschile</text>
In the listing above, the <button> tags (lines 3 and 7) create two buttons with text “Femminile” (“Female”) and “Maschile” (“Male”) respectively. Each button, when selected, generates the text value set by the <postback> button that follows it; this text, not visible by the user, is fed to the rule engine, and will eventually match the pattern contained in one or more categories, making the conversation advance.
Secondly, the conceptual framework of dramatization can ofer a way to deal with the rigidity required
in the creation of the chatbot by the aforementioned need to constrain the user input. A well-structured,
consistent character, in fact, is not only more efective in bringing the user to behave in a certain way,
but is at the same time engaging at the social and emotional level. Dramatization, tested among others
by [
The design of the character was carried out by a multidisciplinary team formed by primary school teachers, Roman history experts, HCI and digital education experts and developers. The character of Roman child, younger brother of the famous Roman orator Cicero, was selected due to its capability to mediate between the students and the historical contents, being a fictional young person students can identify with. After introducing himself and greeting the user, the character expresses his desire to tell about himself and the life in the ancient Rome, teaching basic notions (e.g., Latin greetings) with a peer-to-peer attitude, “sending” images and short animations from time to time. For example, he greets the user with the Latin expression “Ave” and immediately after briefly explains the meaning of the word “ave” in very simple terms (“It is a word that comes from the imperative of the verb to have, in the sense of being well. When I say “Ave” to you, I am wishing you to be well”). To describe himself, the character discloses information about himself in a casual way, for example by mentioning his well known elder brother Cicero (“You may know my brother Marco, a very famous politician and writer”). While ofering to talk about the expansion of Rome across the centuries and the evolution of its political system (“I would like to talk you about politicians in the ancient Rome”), the chatbot leaves the choice of the specific argument to the user by showing a list of buttons. The latter choice, apart from the need to constrain user input to manageable alternatives, is also functional to the interaction with groups of children, since it drastically simplifies the negotiation between the children about the choice by providing a limited, easily identified set of keywords. Finally, the character, like every child, wishes to play with his peers, so he engages in various games, making riddles that give him the opportunity to convey notions about ancient Rome and other small games. For example, after talking about Roman proper names, he proposed the user to invent their fantasy Roman name by choosing a personal name, a surname, and the name of a gens (a Roman clan) (see Figure 1, right).
After discussing the content to be conveyed as part of the educational goals of the project, the character creation methodology was shared with the team in a seminar delivered by the HCI and AI experts. Then, one of the teachers was commissioned to write the interaction between the chatbot and a sample user a script. The task explicitly included the request to break down the interaction into diferent phases, each characterized by a set of alternatives that would become user choice in the final program, and to insert graphic elements in the chatbot’s turns. The script, revised by the historians, was handed to the developer for implementation. In the end, the chatbot consisted of 75 categories that covered four main topics: getting to know each other, Latin language, the expansion and form of governement of Rome, political life and career.
The chatbot, shared with the team on Github, was debugged and finally tested in an experimental session with a primary school class (5th grade) in April 2024 as part of a program run by the University of Turin (“A day at the university”) in which primary and secondary school classes visit the university departments and attend educational labs organized by the staf. 2 The test was aimed at investigating two main aspects: on the one side, we wanted to observe how students reacted to the chatbot, and collect information about how to improve it from their point of view; on the other side, we wanted to collect the opinion of the teachers about the possible use of the chatbot in the classroom and its possible function as a teaching tool.
The presentation of the chatbot was done with the experimenter interacting with the chatbot (Quinto Tullio Cicerone) on the screen, reading the answers out loud. We used the simple test interface of the Pandorabots development environment that has a smartphone-like chat box on one side of the screen. After a while, it was evident that the simplest way to manage the interaction was to leave to the class the decision about what to write in the chat box or what option to select in case of multiple choices.
At the end of the presentation, we conducted a focus group, asking students and teachers for their opinion with some guiding questions: what do you think about the chatbot, how would you like it, what would you like it to talk about. Two main requests emerged from the discussion with the students and teachers. They took part in the activity with enthusiasm and generally liked the chatbot, but suggested that it should talk more about interesting topics for children, such as habits, daily life (clothing, transport, houses), and geography. Also, they suggested that the chatbot should be characterized and have an appearance of somebody they could relate to, namely a boy or a girl.
Concerning the use of the chatbot in the classroom, teachers were positive about this possibility, justifying their opinion on the basis of the fact that they already used multimedia material during lessons. In particular, we asked them if they thought it was dificult to manage interactivity, but they gave us a positive opinion on this aspect too. Furthermore, they indicated a series of elements that they believe are important for engaging children based on their experience with educational videos (which they often find on online platforms Italian accompanying textbooks). According to them, a major factor of engagement is the use of characters who speak in first person (versus a third person narrator); humorous elements in the narration and dialogues with other characters (e.g., when characters interact 2https://www.unito.it/ateneo/gli-speciali/bambine-e-bambini-un-giorno-alluniversita in a video) also emerged as capable of engaging the students. The preference for a character who speaks in the first person was also confirmed by the children in reference to a video from the same project about Marco Polo (“a positive aspect is that he speaks”). Finally, we discussed the interaction mode with the teachers: in their opinion it would be appropriate for the chatbot to speak (rather than respond only in writing) but they believe that the current input mode (text and buttons) is preferable to spoken input in the classroom context, because it allowed for a more ordered interaction. Finally, they were not favorable to individual interactions with the chatbot on personal devices in the classroom, considered the group mode more manageable.
Consequently to the observations made during the first test, a few redesign elements were put into practice. Texts were shortened, especially given the limited space allowed for by the phone (or phone-like) interface. Although it was possible to keep the original texts by using a full screen interface, we observed that the chat provided by the environment employed for the test (visible in Figure 1), which is similar to that of a standard smartphone, introduced a very powerful element of credibility to the interaction with the chatbot, instilling the feeling of chatting with a person from the past, distant in space and time but co-present in the chat space. Also, to balance the weight of the text, we decided to insert more images and animations. Finally, in order to make the activity more compelling, we decided to introduce in the script the “name game” mentioned in the previous section.
We carried out a series of educational workshops in primary schools with the aim of validating the chatbot. In particular, we were interested in i) understanding whether the users (i.e., primary school students) had a positive experience while playing our serious game in the context of an in-class educational workshop, since this is a prerequisite for its adoption and ii) assessing the learning outcomes, iii) to obtain a general assessment of our approach and web platform, together with some suggestions from the teachers involved.
The classes involved in the experiment were selected as part of the ‘Chi decide per me’ project through the University of Turin’s educational activities.
We established a formal agreement with all those schools; we distributed the informed consent to the parents of all children possibly involved in our experiment and asked them for the signed privacy consent form. Only those children whose parents had signed the consent form were allowed to participate in our experiment. Overall, 5 V grade classes were involved in our study.
All classes involved had some classroom lectures on history and politics in ancient Rome in April 2024 and worked for the same number of hours and on the same topics.
Afterwards, in May 2024 the classes were divided into 2 groups: control classes and experimental classes. The activities were organized as follows. Experimental classes: 3 classes did a revision activity on the diferent habits, daily life and geography in the ancient Rome using the chatbot. The process begun with the researcher introducing the chatbot and explaining its role as a source of information. Each time the chatbot prompted a response or question, the students paused to decide collaboratively on the next question they would like to ask. With the assistance of a class teacher, the researcher encouraged active participation by ensuring that every child has the chance to contribute to the conversation, promoting teamwork and consensus-building. After discussing the possible questions, the students voted on a final choice. The selected question was then submitted to the chatbot, and the class waited for its response. Once the chatbot provided an answer, the students analyzed the information together, and the cycle repeated, with the class deciding the next step based on the chatbot’s answers.
Statement 2 Statement 3 Statement 4 Statement 5 Statement 6 Statement 7 Statement 8 Statement 9 Statement 10 Statement 11 Statement 12 Statement 13 Statement 14
I was confused many times when I was playing
I thought the chatbot was easy to use
I would need help from an adult to continue to use the chatbot
I always felt like I knew what to do next when I played with the chatbot Some of the things I had to do when using the chatbot did not make sense
I think most of my friends could learn to use the chatbot very quickly Some of the things I had to do to use the chatbot were kind of weird
I was confident when I was using the chatbot I had to learn a lot of things before using the chatbot well
I really enjoyed using the chatbot
I liked the images I saw If we had more time, I would keep using the chatbot
I plan on telling my friends about the chatbot
Control classes: children reviewed history topics by reading fictional interviews with characters from Ancient Rome. The students were divided into small groups, and each group was assigned two or three interviews featuring a diferent Roman figures. The interviews covered various historical themes and matched the chatbot interaction scripts.
At the end of the activity we administered questionnaires to each participant. In this process, the
informed consent form was presented to the children in a clear and age-appropriate manner. The
researcher began by explaining what informed consent means, emphasizing that it is a way for them
and their parents to understand the study and decide whether they want to participate. The explanation
included a discussion of the children’s rights, such as the option to withdraw from the research at
any time without any consequences. The implications of their participation were outlined, including
what the study would involve, how their information would be used, and the confidentiality of their
responses. The children were encouraged to ask questions to ensure they fully understood everything.
After the explanation, the informed consent forms were distributed. Finally, two questionnaires were
handed out to those who had provided consent, and they were asked to fill it out:
• To measure the chatbot usability, as well as users’ experience and engagement with the chatbot
we used that adaptation of System Usability Scales (SUS) for children between ages of 7-11 [
The study was approved by the University of Turin Ethics Committee.
In this section, we illustrate the data collected about the use of digital media by the students involved in the workshops, the results of the adapted SUS administered to the test group to evaluate the usability of the chatbot, and the results of the multiple choice test that all students completed at the end of each activity (both test and control groups). 3Linktothetesthttps://acesse.one/historyquestionnaire the chatbot) the ratings are reported in Table 4.4 (b). All statements received positive ratings, thus confirming the capability of the chatbot to engage the users with its behavior and appearance. Ratings, in fact, ranged from 3.57 (statement 14) and 4.19 (statement 12), showing the students’ willingness to repeat the experience.
Finally, in order to test the role of the chatbot in teaching, namely its capability to help the students to learn the notions provided by the chatbot during the interaction with the students, we asked the participants from both groups to take a quiz about Roman history (structured as a multiple choice test), as illustrated in the previous section. We collected 50 valid copies of the quiz (3 participants dropped of) for the test groups and 30 copied for the control group. The average score for the test group was 55.72% of correct answers; the average score for the control group was 52.65% of correct answers. The two groups were compared with a non-parametric test (Mann Whitney U test) but no statistically significant diference was found.
We computed the overall score obtained by each participant from the questionnaire on the use of digital media by summing the declared number of devices, accounts and chat systems, and we computed the correlation between the obtained score and the SUS score using Spearman’s Rho. However, no statistically significant correlation was found. This result is not surprising given two factors: on the one side, the SUS scores do not have a large standard deviation, showing small diferences in the individual reception of the chatbot; on the other side, the large majority of the participants appeared to be very familiar with digital media, so we didn’t expect this aspect to afect the reception of the chatbot.
The impact on the chatbot activity with children was really efective: the collaborative approach in deciding the question to prompt fosters critical thinking, active listening, and decision-making skills. It also helps students understanding how to gather and evaluate information in a group setting through an engaging experience that was positively evaluated by the participants. Throughout the activity, the teacher serves as a facilitator, guiding discussions and ensuring that all students are engaged in the process. The impact on teachers was really positive: • When teachers are involved in the design process, they feel a sense of ownership over the technology. This engagement helps them overcome potential fears or resistance to adopting new digital tools. By contributing their insights and ideas, they build confidence in their ability to use these technologies efectively, even if they start with limited digital skills. • Teachers know their classroom environments, students’ needs, and curricular goals better than anyone else. By involving them in the design, the tools can be tailored to fit real-life teaching scenarios. This reduces the risk of creating technology that is impractical or dificult for teachers to integrate into their daily lessons, leading to higher adoption rates and more efective use of the tools. • Participation in the design process naturally encourages teachers to learn more about the digital tools they will use. Through hands-on involvement, teachers can build their technical skills in a supportive, low-pressure environment. This gradual learning helps them feel more comfortable and competent with AI based technology. • When tools are co-designed with input from teachers, they are more likely to be sustainable and efective in the long term. Teachers are more inclined to continue using tools that they helped shape, ensuring that the investment in technology leads to meaningful, ongoing improvements in the classroom.
As future work, we intend to replicate the methodological framework and technology employed to create the chatbot, extending it to the other historical and political contexts explored in the overarching project. To do so, we intend to co-create with the teachers similar characters that are able to engage positively with the students.