Research on pedagogical agents (PAs) focuses on personalizing and adapting content and instruction to students' diverse needs to support learning. Teachers can use this technology to support individual students' work. However, it is not clear what could be the impact of a PA that helps teachers to orchestrate collaborative learning activities on the classroom level. Our work explores two dimensions. Firstly, the effects of employing a PA in a technology-enhanced learning setting to promote students' motivation and learning outcomes. To that end, we will conduct a series of studies employing various methods and data (tests, questionnaires, observations, and students' performance data). Secondly, this work aims to develop a design framework based on teachers' expectations and needs when using a PA to orchestrate collaborative learning tasks. To build the PA design framework, we will conduct a study to categorize teachers' PA expectations and needs, accompanied with findings from the literature. Our hypothesis is that classrooms, where the PA is used to support teachers in the learning activity, will demonstrate high learning gains and students' perceived motivation.
Pedagogical agents (PAs) are lifelike
virtual characters playing an educational role,
aiming to facilitate learning in digital learning
environments (DLE) [
Integrating PAs in learning digital
systems goes in line with social learning theory
[
Literature shows that PAs can contribute to the motivation of students at an individual level [8]. Our research interest is whether a PA that targets the classroom as a whole, would have a similar impact on students’ motivation, thus, better learning outcomes. 1.2.
Dillenbourg [11] defines collaborative
learning as the situation in which two or more
people learn or attempt to learn something
together. The research field that explores how
technology impacts and can promote
collaboration is computer supported
collaborative learning. Stahl et al. [
According to Tobias [14], teaching and
learning experiences that are not based on a
traditional mathematics curriculum can bridge
the achievement gap and reduce mathematical
anxiety. To that end, we argue that combining
mathematics with music may potentially help to
bridge the achievement gap and reduce anxiety.
However, it is not evident what pedagogical
strategies need to be considered to ensure this
successful combination. For instance, Vaughn
[
PAs are typically used to support students on the individual level, while it is not clear how a PA can be used in the classroom as a teacher’s support in classroom orchestration. Dillenbourg [19] defined classroom orchestration as a teacher's ability to manage, in real-time, the activities and contextual constraints inherent to the learning session. This managerial instance encompasses the nature of the activity (for example, individual, teamwork, class-wide), the pedagogical tools (such as simulations, wikis, quizzes), and the distribution channels (for example laptops, tablets, smartPhones). Conversely – and complementary – to instructional design and adaptive learning, classroom orchestration deals with extrinsic activities (moving chairs, collecting papers, checking on students’ activity status, student log-in problems) and extrinsic constraints (discipline, limited lesson time, energy management, classroom physical space) [19]. Regarding the technological aspects, related work has explored teachers’ needs for educational technologies. For example, Holstein et al. [20] showed that teachers expressed their wish to be able to see students thinking process and being able to adopt system-like features like monitoring all students at the same time. Furthermore, another case study by Chounta et al. [21] showed that teachers would like to receive support to be more efficient and effective in their practice. The authors convey the message that systems including artificial intelligent techniques, could address such teachers’ needs. Amarasinghe et al. [22], presented the notion of orchestration agents, which can help teachers by suggesting orchestration actions, thus offloading decisionmaking responsibilities whilst respecting their agency. They referred to the latter scenario as a hybrid human-machine approach. Our work expands on what teachers expect from a PA (in the form of a 2D character) helping them at a classroom level and exploring the impact of a hybrid system solution for K-12 education. For our PA system, we envision the agent helping teachers with the activities as well as orchestration decisions. One example of an activity employed at a classroom level can be found in Chin et al. [23]. In this case, the feature allowed the teacher to show on a projected screen students’ teachable agents with the aim to discuss on agents’ different answers, hence, students understanding. Additionally, we are taking inspiration from existing PA systems targeting mathematics [2] [8]. However, our approach is different from the aforementioned studies in that the learning activities combine music and mathematics as means to motivate and support students’ conceptual and procedural knowledge understanding. 1.5.
To understand how PA technology could support social dimensions on the classroom level rather than the individual level, we further investigate when and how a PA can help teachers in collaborative learning activity while motivating students to learn and be engaged in the task. To that end, the PA will be used in the classroom by integrating a virtual character to assist the teacher. We are interested to see whether employing a PA in the classroom makes a difference in terms of learning outcomes and contribute to students’ motivation in the collaborative activity. In this study, we examine the following research questions (RQs): [RQ1] Which kind of interventions and affordances do the current PA systems in K12 education have for teachers at a classroom level? [RQ2] What do teachers expect and need from a PA helping them to instruct and orchestrate collaborative learning activities? [RQ3] What is the impact, in terms of learning gains and motivation, when employing a PA hybrid system at a classroom level? [RQ4] What benefits, challenges, and constraints can be seen when employing a PA at a classroom level?
In this study, we have selected a mixedmethod approach. In terms of qualitative research, we will conduct a literature review and a case study to develop the design framework for our PA. In terms of quantitative research, we will conduct a series of studies to report on the learning outcomes and perceived students’ motivation via tests and questionnaires. We elaborate more on the planned studies in the next sections. 2.1.
For the case study, we plan to carry out focus groups with teachers (n = 5 to 7) to understand their expectations and needs when using a PA as orchestration support at a classroom level. The target population are mathematics teachers from primary education. For the pilot study, we will test the PA in one elementary classroom (n = 15-30 students). This will allow us to modify and adjust the PA system as well as our planned measuring instruments. Finally for the main study, we will employ the PA system in elementary classrooms (n = (4 to 8) including experimental and control groups) to evaluate the PA design framework based on teachers’ insights, and to evaluate students’ (100-200) learning gains and perceived motivation. 2.2.
Pedagogical Agent and learning activities. For the PA system, we are using a face tracking solution, the agent can emulate gestures, eye blinking, lip-synching to a sound source, and head swing. Additionally, by key commands, the agent can walk, run, wave, point out, and trigger special moves (i.e., thinking pose, wearing glasses, eating a banana). On the other hand, for the learning activities, we want to build on prior research done with an educational game that combines music and mathematics in the format of a board game and a digital version. In the case of the board-game format, two elementary schools in Belgrade, Serbia, played the game for two sessions. Students were randomly assigned to play in small groups and to answer questionnaires targeting their learning experience. The results showed that the educational game supported their cognitive development while boosting their motivation and desire to have success on the learning tasks [24]. Building on the latter study, we used the digital version of the game and focused on group formation strategies and learning outcomes [13]. Using students’ prior knowledge (as assessed by pre-knowledge tests), authors formed homogeneous (high or low performers only) and heterogeneous (high and low performers mixed together) groups and explored whether their game performance would be reflected on students’ learning gains. Conversely to related research [25], the aforementioned study reported students belonging to heterogeneous condition to benefit less than homogeneous groups in terms of learning gains. However, this was not the case for the game score, where HE groups outperformed HO groups playing the game [13]. This article is particularly relevant because we are considering the authors’ suggestions to better align the educational game with the learning goals and to enhance the collaboration activities.
Instruments. Furtermore, we will create an interview protocol and we will carry out teachers’ focus groups. The aim is to develop a design framework based on their expectations and needs when collaborating and employing a pedagogical agent in the classroom. Regarding the students, they will be asked to answer a questionnaire (still to be defined) targeting motivation, from which we will analyze and report on PA effects and design challenges. Finally, we will use students’ data to find a possible correlation between their perceived motivation and their learning outcomes (prepost tests evaluation). 2.3.
In the following figure (Figure 1) we present and describe the aim of the planned studies and timeline. We link each study to our research questions.
We will adapt collaborative learning activities with and without the PA (independent variable), as they will serve as the educational context of the experiments. We will divide half of the participating school groups of students into experimental condition (with PA) and control group (without PA). Our dependent variables are, on one hand, students’ performance (as assessed by pre and post knowledge tests) along with their perceived motivation when working with or without the agent (questionnaires); and on the other hand, teachers’ evaluation of the PA when facilitating the orchestration of the collaborative tasks.
We hypothesize that the experimental group will benefit more from having a PA helping the teacher to orchestrate the collaborative learning activity. The benefits will be reflected in terms of students’ learning outcomes and motivation. On the other hand, we expect teachers to evaluate the PA system in the expected dimensions we will be able to find after concluding the case study with them.
We are currently in the process of conducting a systematic literature review (SLR) considering all types of PAs addressing mathematics education in K-12 education. The aim is to identify all pedagogical affordances these systems offer teachers when PAs are used on a classroom level. We envision the findings of the latter will support us in designing and proposing a theoretical framework for teachers’ interventions in PA systems, and to classify PA systems affordances when using them on the classroom level for supporting instruction and orchestration activities. In parallel, we are in the process of conducting focus groups to understand teachers’ expectations and needs when having a PA helping them to orchestrate collaborative activities in the classroom. From both, the SLR and the case study, we aim to have the pedagogical and orchestral design requirements to be met by the PA.
Technology wise, currently, we are performing tests with the PA system in realtime in remote teaching settings (video conferencing platforms) and classrooms settings by using projection. The aim is to test and adjust PA social cues, signals, reactions [26], and social fidelity contributors (i.e., personalization, slang, politeness, enthusiasm, interactivity) [27]. and practical
We envision that this research will contribute to the field of PA by bringing teachers’ perspectives and needs when having the agent in a classroom collaborative setting. Moreover, by using PA systems technology at a classroom level, it could create a bond between the agent and student that could impact learning outcomes. Finally, this study will report on an innovative technology-enhanced learning scenario, and future research could potentially expand on the psychological, collaborative, and social effects this technology may bring for both teachers and students.
[