=Paper=
{{Paper
|id=None
|storemode=property
|title=A Teachable Agent Game for Elementary School Mathematics promoting Causal Reasoning and Choice
|pdfUrl=https://ceur-ws.org/Vol-587/paper3.pdf
|volume=Vol-587
}}
==A Teachable Agent Game for Elementary School Mathematics promoting Causal Reasoning and Choice==
https://ceur-ws.org/Vol-587/paper3.pdf
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
A Teachable Agent Game for Elementary School
Mathematics promoting Causal Reasoning and Choice
Lena Pareto
Media Produktion Department, University West, Sweden.
lena.pareto@hv.se
Abstract: We describe a mathematics computer game for children designed to
promote causal reasoning, choice-making, and other higher-order cognitive
activities. The game consists of a choice-based board game, enhanced with a
conversational, teachable agent, taught to play the game, by the child, through
demonstrations and questions. Game design is motivated by causal reasoning
theory and educational psychology. The game is currently evaluated in an
ongoing large-scale study that seeks to investigate the game’s effects on the
players’ abilities to reason and make productive choices. The study involves 20
elementary-school classes at different levels.
Keywords: Teachable agent, intelligent game, mathematics, elementary school,
causal reasoning, choice, metacognition
1 Introduction
Educational games have documented potential effects on learning and motivation
[1,2,3], but their delimitations regarding developed skills and competencies, attitudes
towards a subject, and understanding of symbolic content are less understood [4].
The purpose of our research is to show that educational games are effective for the
development of higher-order cognitive and metacognitive skills. The paper presents
an educational game designed to develop such skills in the context of elementary
mathematics, e.g., ability to reason over, reflect over, and invent strategies for solving
mathematical problems. In the game, players take turns by choosing a card
(representing a number) and placing it on a game board (also representing a number).
The game challenge is to make as good as possible choices with respect to cards at
hand and the game goal in question. Each card may yield points and its strategic value
depends on the situation, so the choices give opportunity to reason. We have found
the game to give substantial training of causal reasoning and choice which are basic
cognitive processes that underpin all higher-order activities [5], and which are
regarded as essential to train by educational psychologists [3]. Empirical research on
instructional methods for supporting causal reasoning is scarce [5].
The game relies on two threads of research: the “Squares Family" microworld for
understanding arithmetic concepts of the author [6]; the teachable agent of Biswas,
Schwarz et al. [7,8]. The first version of the game was developed in 1998 and field-
Y 13 Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
tested in schools in 1999. The game presented in this paper is the result of a decade of
evaluation and evolution of the initial game using an iterative, user-centric approach
to development. The most recent addition is a teachable agent that starts out with no
knowledge about mathematics but which has a built-in ability to learn it from the
child, using the teach-by-guidance model [9]: the agent learns by observing the
child’s game playing behavior and by posing reflective questions about the choices. In
this way, the teachable agent paradigm provides structural guidance and reflection
techniques [10] known to help learners achieve deep understanding [11,12,13,4].
We are currently conducting large-scale studies of learning and motivational
effects of the game using experimentation, observation, and inquiry in situations
where students play the game as part of their regular education. Experiments involve
playing and control conditions with pre-post tests, and game log analysis.
Observations are concerned with behavior and social interaction in-class. Inquiries are
concerned with end-user perceptions and attitudes towards the game. The game was
evaluated in 9 classes in 2009, and is being evaluated in 20 classes during 2010.
The paper’s focus is on the design of a conversational, teachable agent as a mean
to stimulate causal reasoning and productive choice strategies. The agent presented
has undergone several iterations of field tests to become “smart enough” to learn
game playing strategies discovered by children. The contribution over past research
[9] is a knowledge model that also involves choice-strategy knowledge and a
reasoning-oriented dialogue based on that model.
2 The Teachable Agent Math Game
The game environment consists of combined card and board games, with a variety of
levels and goals. We illustrate it by a few steps from a simple game (see Figure 1):
Fig. 1. Game play scenario: start (1a), during 2nd turn (1b), after 2nd turn (1c)
Two players on each side of a common game board receive 10 cards each: 4 face-up;
6 face-down. In Fig 1a the left player has received the cards 7, 2, 2 and 6 and the right
player 5, 7, 3 and 4; the game board is empty and represents 0. The left player starts
by choosing card 6 (bottom-right), causing 6 squares to be added to the board (not
shown). The right player then chooses the card 5 (top-left of her cards), which causes
a packing operation: 10 (i.e. 6+4) squares are packed into a square-box which is then
placed in the board’s leftmost compartment; the right player is awarded a point
indicated by a flashing star (Fig 1b); in the same turn, the remaining square is placed
on the board (Fig. 1c). This has illustrated that 6+5=11.
Y 14 Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
How well did the players do in this scenario? A better first choice for the left
player would have been the card 2, since the right player’s largest card is 7, and thus
not enough for a carry-over: choosing 2 would have blocked the right player from
scoring; further, choosing 2, neither of the right player’s cards would have prevented
the first player from scoring. Could the right player have made a better choice than 5,
when the game board was 6? Three of the original cards would score (4, 5 and 7)
whereas 3 would not so that 3 would have been a bad choice. However, card 4 is
slightly better than 5, since 6+4 is 10, and no 1-digit number would yield a carry-over
when added to 10. In this particular scenario the left player cannot score in either case
with a board of 10 or 11, but keeping the 5 instead of the 4 might make a difference in
later turns.
Already in this simple game, making good choices involve reasoning on several
levels. Other games are more challenging: 3-digit cards can generate between 0-3
points per turn, and each digit in the result may allow or block the opponent to score
in that position. There are games that include negative numbers, other operations than
addition and have goals that are more difficult to fulfill than carry-overs. Players can
choose to either compete or collaborate, and the strategies for playing well differ.
2.1 The Conversional Teachable Agent
Besides playing self, children can teach an agent how to play and watch the agent
play. The agent performs according to its current knowledge level, which depends on
how well it is taught. There are two ways to teach: by showing the agent how to play
(show-mode); by letting the agent try and then accept or reject the agent’s choice (try-
mode). In both modes the agent will ask multiple-choice questions to its teacher,
concerning the choice(s) just made (see Figure 2a).
The questions asked depend on several factors: the game’s state (the chosen card,
the board, and the players’ hands), the teaching mode, whether players compete or
collaborate, and the agent’s current knowledge level.
The agent’s knowledge level is estimated from a trace of the child’s actions and
from a record of her responses, as is explained in [9]. The general idea is that we keep
track of positive indications (scoring rules and strategic values of chosen cards or
correctly answered questions) as well as negative indications (missed rules in better
choices or incorrect answers) and calculate a knowledge level from these indications.
Our current model includes knowledge in five categories 1) the game idea, 2) how
the graphical model behaves, 3) how to score, 4) how to choose the best card
considering own cards, and 5) how to choose the best card considering the other
player’s cards too. Categories 1-3 represent mathematical knowledge; while 4-5
represent strategic and choice-making knowledge.
During the game, the agent’s knowledge is shown using 5 knowledge meters (see
Figures 2b, 2c, 3d and 3e). Taken together, the knowledge meters reflect a
progression in sophistication of choice: from knowing what the game is about, to
consideration of all possible paths 2 steps ahead.
Y 15 Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
Fig. 2. Game UI (2a), game idea question (2b) and graphical model question (2c)
Questions difficulty level advance with the agent’s knowledge. The simple
question in Figure 2b is asked when the agent knows very little (the meters are low)
and the more difficult question in 2c, later in the game, when the agent knows more.
Questions are chosen to be slightly above the agent’s knowledge level to allow
progression towards the child’s level; when the child’s level is reached, the child is
challenged by reflective questions. If progression stops, so will the advancement of
questions. This follows the idea of Vygotskij’s zone of proximal development [14].
Fig. 2 gives examples of advanced questions. Question 3a is from show-mode,
where the child has chosen a card and the agent has made a hypothetical choice
according to its knowledge, which is reflected in the question: “I also thought of card
4...” (the agent’s choice was the same as the child’s). Question 3b is from try-mode,
where the child accepted the agent’s choice: “So I made a good choice, right?”
Question 3c occurs in either mode, and compares the child’s and the agent’s choice:
“Why is card 2 better than card 4?”. Alternative responses reflect the choice
sophistication level: being able to distinguish between 1) different scores, 2) general
strategic value of same-score cards and 3) situation-specific strategic value of same-
score cards. Question 3d is raised when the agent’s scoring knowledge is high (the
middle meter is almost full), so the question considers both scoring and the strategy to
leave few squares (general strategy). Question 3e is the most challenging type of
questions which involves both scoring and blocking the opponent in the next turn
(situation-specific strategy). To be sure of the correct response: “It’s obviously the
best one! It gives 1 point and it’s the only card that blocks the opponent”, the child
must predict and distinguish between 16 alternative paths: each own card composed
with any of the opponent’s cards.
•
Fig. 2. Five examples (3a-3e) of questions from scoring and strategic categories
Y 16 Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
Our approach is related to the programming by demonstration principle [15,16] in
the sense that the user demonstrates examples of desired behavior and the system
generalizes the examples to rules. However, the agent-teaching extends the principle
with reflective dialogue, it targets mathematical and strategic knowledge rather than
programming, and the agent can perform (i.e., play the game) at any knowledge level.
3 Promoting Causal Reasoning and Choice
Our game is designed to promote causal reasoning and choice. In particular, it fosters
the following sub-forms of causal reasoning identified by Jonassen and Ionas [5]:
prediction, implication, inference, and explanatory explanations.
Prediction is defined as reasoning about possible future states on the basis of a
given set of states and possible effects. Players of our game need to predict the effects
of cards regarding point generation and strategic value (to play well).
Implication is defined as hypothesizing state-effect relationships. Players of our
game do not know how cards score a priori, but successively discover this through
hypothesizing the cards’ effects on the score (while making the choice), and by
observing the played card’s actual effect (once the choice has been made).
Inference is defined as backwards reasoning from effect to cause. This form of
reasoning occurs when a player starts from a game goal (e.g., producing a carry-over
in a compartment), then decides what is needed (e.g., a card greater than or equal to
7), and finally checks for a matching of such card at hand.
Explanation in this context is defined as being able not only to induce causal
relations, but also to explain them. Players of our game are prompted with reflective,
explanatory questions of the agent, and thereby encouraged to reason about and
verbalize why a choice is good.
Our game fulfills Jonassen and Ionas’s recommendations on using explorations in
microworlds and explanatory questions as means of achieving such reasoning skills
[5]. The game relies on a microworld of arithmetic; the agent’s interaction with the
user relies on questions in which the child explains her choices in relation to
alternatives.
The ability to make good choices is fostered by the game as a whole: the incentive
of the causal reasoning, illustrated in the examples above, is to make good choices;
good choice is the only way to perform well in the game. Our notion of good is
captured by a context dependent goodness value for each possible card. The goodness
value reflects the card’s score and its strategic value, and is used in the assessment of
the player’s level of knowledge.
4 Discussions and Future Work
Schwartz and Arena argue that choice-making will be an important skill in the 21st
century that should be practiced and assessed in education [3]. Our game gives
opportunity to practice choice-making in a playful way, within a well-defined domain
(arithmetic) with immediate feedback and progress, and with few choices (four cards).
Y 17 Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
Guided learning has repeatedly been shown to be superior to unguided learning
[13]. The teachable agent provides implicit guidance through reflective questions that
direct the player’s attention to discriminating properties of the choices: each choice is
assessed to give a goodness value; this allows us to study progression of choice-
making, and players to value their performance (irrespective of their luck with cards).
Observations and previous studies [17] show that children quickly learn to play
well, but that the degrees to which they challenge themselves vary. This is a matter of
motivation; and teachable agents have increased motivation in other contexts [18].
With our new agent and with extrinsic incentives (medals, meters, high-scores,
and statistics) we hope to motivate students to further engage in learning productive
choice strategies. This is investigated in our current study: i.e., players’ abilities to
reason and make productive choices, progression paths, choice-patterns, and learning-
and motivational effects compared to traditional mathematical instruction.
Finally, we think that our game mediates that mathematics is not merely a matter
of right and wrong – computation is – but mathematics is much more than
computation.
Acknowledgments. This work is financed by Wallenberg Global Learning Network.
References
1. Ke, F: Alternative goal structures for computer game-based learning. International Journal
of Computer-Supported Collaborative Learning vol 3:429--445 (2008)
2. Vogel, J.F. Vogel, D.S. Cannon-Bowers, J. Bowers, C.A. Muse, K. & Wright, M.:
Computer gaming and interactive simulations for learning: A meta-analysis. Journal of
Educational Computing Research, 34(3) 229--243 (2006)
3. Schwartz, D. L., & Arena, D. A.: Choice-based assessments for the digital age. [White
paper]. Retrieved from http://aaalab.stanford.edu. (2009)
4. Moreno, R. & Mayer, R. E.: Role of guidance, reflection and interactivity in an agent-based
multimedia game. Journal of Educational Psychology, 97(1), 117--128 (2005)
5. Jonassen, D., & Ionas, I.: Designing effective supports for causal reasoning. Educational
Technology Research and Development, 56(3), 287--308 (2008)
6. Pareto, L: The Squares Family: A Game and Story based Microworld for Understanding
Arithmetic Concepts designed to attract girls. In World Conference on Educational
Multimedia, Hypermedia and Telecommunications, Issue 1, pp. 1567--1574 (2004)
7. Biswas, G,T. Katzlberger, J. Brandford, D. Schwartz & TAG-V: Extending intelligent
learning environments with teachable agents to enhance learning. In Artificial Intelligence in
Education, J.D. Moore et al. (Eds.) IOS Press, 389--397 (2001)
8. Schwartz, D.L. Blair, K.P., Biswas, G. Leelawong, K. & Davis, J.: Animations of thought:
Interactivity in the teachable agents paradigm. In R. Lowe & W. Schnotz (Eds). Learning
with Animation: Research and Implications for Design. UK: Cambridge Univ. Press (2007)
9. Pareto, L: Teachable Agents that Learn by Observing Game Playing Behavior. In
proceedings of the Workshop on Intelligent Educational Games at the 14th International
Conference on Artificial Intelligence in Education (AIED), pp 31--40 (2009).
10 Schwartz, D. L Chase, C. Wagster, J. Okita, S. Roscoe, R. Chin, D. & Biswas, G.:
Interactive metacognition: Monitoring and regulating a teachable agent. In D. J. Hacker, J.
Dunlosky, and A. C. Graesser (Eds.), Handbook of Metacognition in Education. (in press)
Y 18 Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
11 Kirschner, P. A, Sweller J. & Clark, R. E.: Why minimal guidance during instruction does
not work: an analysis of the failure of constructivist, discovery, problem-based, experiential,
and inquiry-based teaching. Educational Psychologist, 41(2), 75--86 (2006)
12 Manske, M & Conati, C.: Modelling Learning in Educational Games. Proceedings of the
International Conference on Artificial Intelligence in Education, 411—418 (2005)
13 Mayer, R. E.: Should there be a three-strikes rule against pure discovery learning? The case
for guided methods of instruction. Educational Psychologist, 59, 14--19 (2004)
14 Vygotskij, L. Tänkande och språk (eng. Thought and Language), Göteborg: Daidlos (2001)
15 Cypher, A.: Watch What I Do: Programming by Demonstration. MIT Press (1993)
16 Dey, A.K., Hamid, R, Beckmann, C, Li, I, & Hsu, D: a CAPpella: Programming by
Demonstration of Context-Aware Applications. CHI Letters 6(1): pp. 33--40 (2004)
17 Pareto, L., Schwartz, D.L. & Svensson, L.: Learning by Guiding a Teachable Agent to Play
an Educational Game. In Proceedings of the International Conference on Artificial
Intelligence in Education,. IOS press, pp 662--664 (2009).
18 Chase, C; Chin, D.B; Oppezzo, M, & Schwartz, D.L. Teachable agents and the protégé
effect: Increasing the effort towards learning. Journal of Science Education and Technology,
18(4), 334--352 (2009)
Y 19 Z