=Paper=
{{Paper
|id=None
|storemode=property
|title=Characters that Help You Learn: Individualized Practice with Virtual Human Role Players
|pdfUrl=https://ceur-ws.org/Vol-587/paper2.pdf
|volume=Vol-587
}}
==Characters that Help You Learn: Individualized Practice with Virtual Human Role Players==
https://ceur-ws.org/Vol-587/paper2.pdf
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
Characters that Help You Learn:
Individualized Practice with Virtual Human Role Players
H. Chad Lane
Institute for Creative Technologies
University of Southern California
lane@ict.usc.edu
Abstract. This paper describes how virtual humans can be used as role players
in for communicative tasks that require modification of one’s social skills.
Examples are discussed, including systems for intercultural communication and
doctor-patient interviewing, and we conclude with a discussion of the
challenges of providing individualized practice by dynamically adjusting the
behaviors of virtual humans to meet specific learner needs.
Keywords: virtual humans; social skills; pedagogical experience manipulation
1 Introduction
Pedagogical agents are most often designed to play the role of tutor or peer in virtual
learning environments [1]. In these roles, the agent works alongside the learner to
solve problems, ask questions, hold conversations, and provide guidance. Over the
last decade or so, a new breed of pedagogical agents has emerged that do not play the
role of expert or peer, but rather act as the object of practice. That is, instead of
helping on the side, it is the interaction itself (with the agent) that is intended to have
educational value. Here, the agent is usually a virtual human that is playing some
defined social role in an interaction. To “succeed”, the learner must apply specific
communicative skills. For example, to prepare for an international business trip, one
might meet with a virtual foreign business partner to negotiate a contract agreement.
The technological goal is to simulate an authentic social context for the practice
and learning of new communicative skills. In describing the challenges of modeling
human reasoning and emotion related to building virtual humans, Gratch and Marsella
[2] state that “The design of these systems is essentially a compromise, with little
theoretical or empirical guidance on the impact of these compromises on pedagogy”
(p.215). What are the implications of the pedagogical demands on virtual human
design? How could virtual humans facilitate learning? In this paper, we explore some
methods for providing guidance through the virtual human role players. Inspired by
anecdotal statements from expert human role players who reported adjusting their
behaviors based on observations of learners, we outline the dimensions of what is
adjustable in virtual and discuss some examples of how virtual human role players
might similarly adapt to meet specific learner needs.
Y7Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
2 Virtual human role players
Live role playing has a long history in education [3] and because it is interactive and
situated, is a common strategy for teaching social interaction skills [4]. There are
problems, however, with the approach. Role playing in classrooms or offices is not
situated in a realistic context, and when done with peers, raises authenticity concerns.
Expert human role players are generally the best option, but are not cost-effective and
can be prone to inconsistency (between different role players and due to fatigue).
Virtual humans that exist in authentic, virtual environments, are beginning to emerge
that address some of these problems.
Cultural learning, interpersonal communication, and language learning are popular
targets for virtual human-based training systems. For example, BiLAT [5] is a serious
game that situates the learner in a narrative context to prepare and meet with a series
of virtual humans to solve problems. A similar structure is used in the Tactical
Language family of serious games where the focus is on conversational language,
communicative, and intercultural competence [6]. Another prominent domain for
virtual humans is clinical training. Virtual “standardized” patients have been used to
train psychiatric students in the classification of post-traumatic stress disorder (PTSD)
cases [7] as well for the practice of positive non-verbal behaviors during clinical
interviewing, such as body positioning and eye gaze [8]. Virtual humans have been
used in countless other social contexts, including for police officer training [9],
teaching coping behaviors for bullying in schools [10], and demonstrating healthy
play for children with autism [11]. Across the wide spectrum of these applications,
most of the individualization that occurs is (1) at the learner’s discretion, and (2) at
the scenario level (e.g., to select appropriate characters to meet with). In the sections
that follow, we discuss how the level of individualization might be pushed down into
the dynamic behaviors of the characters themselves.
3 What can be tailored in a virtual human?
The efficacy of virtual humans to support intercultural and social skill learning has
been shown in numerous studies [12-14]. In each case, character models were
developed based on analysis of human-human data and input from experts with
realism taking highest priority. What counts as “realistic” is therefore based primarily
on expert opinion and subject to great amount of variance given the often inconsistent
nature of human behavior. People with the same cultural background may possess
very different opinions about a certain cultural value because of regional or
personality differences, for example. Stories for characters can be easily constructed
that lead to different outcomes (e.g., “the character is having a bad day”). Thus,
different reactions to the same action – either between characters, or even from the
same character at a different time or place – are entirely plausible. It seems there is a
vast (and to date, unarticulated) space of communicative experiences that we might
consider “realistic”. This section describes a few of the more prominent dimensions in
which current virtual humans communicate.
Y8Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
Fig.1. Expressions of anger, skepticism, appreciation, and umbrage by ICT virtual humans [15]
Nonverbal behaviors. Observable, nonverbal behaviors during interactions with
virtual humans are often a primary focus in studies of their communicative
competency and fluidity. For example, the role of eye gaze, nodding, and gestures
play a significant role in generating feelings of rapport in users [16]. When no attempt
is made to align nonverbal behaviors with the utterances of users (“non-contingent”
responses), feelings of distraction and disfluency in speech follow. The implication
for learning with virtual humans is that if their nonverbal behaviors are unnatural to
the point of being a distraction, learning may be hindered.
Nonverbal behaviors play a large part in the expression of emotion and it is
possible to convey a great deal of implicit feedback through them. There is staggering
complexity that emerges from facial expressions alone, but also through gaze, body
positioning and movement, and gesturing (examples are shown Figure1). Such signals
also come in varying levels of intensity, as measured by onset, duration, and length
[17], and so these all represent adjustable parameters that would enable the system to
dampen or magnify nonverbal backchannel feedback from the virtual human.
Content. The information conveyed and the words used to encode a message
represent another critical dimension in the space of configurability. A message may
have more or less content, more or less meaning, more or fewer emotive words, more
or less explanatory content, and so on. The “best” choice of content depends heavily
on many factors, including the context of the simulated social situation (e.g., business
vs. casual), the culture and personality of the virtual human (e.g., reticent vs.
talkative), the familiarity of the character with the user, and more.
Cognitive, communicative, and emotional models. The most sophisticated virtual
humans are able to do complex, task-based reasoning and behave based on underlying
representations of the dialogue, their intentions, desires, the task domain, and their
emotions [15]. Nonverbal and verbal behaviors follow from these basic underlying
representations and they are naturally influenced by the incoming utterances of a
human user. For example, a threatening utterance might trigger a withdraw intention,
which in turn increases terseness and the likelihood of compliance. Speaker intentions
may vary greatly from how the message is received. Misunderstandings between a
learner and a virtual human role player can have a profound effect on the learner’s
evolving understanding of the skills being practiced.
Y9Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
4 Towards adaptive virtual human role players
Dynamic tailoring can be understood as influencing or overriding the standard
behaviors of a simulation, as it is running, for pedagogical reasons [18]. In domains
like human behavior, where there is significantly more freedom in what may be
considered realistic than in many other domains (like physics), the idea is to select
actions within this range of acceptability that will have the most pedagogical benefit.
Given the dimensions of adjustability discussed in the previous section, some
pedagogical goals dynamic tailoring could be used to achieve are:
1. support recognition when errors are committed or ideal actions taken
2. provide an explanation for observed reactions and emotional state changes
3. suggest a repair for how a learner might revise their beliefs
These are the same broad goals typically addressed by explicit feedback from a
human or computer tutor [19]. The difference is that these goals are achieved through
the character, by modifying utterances, beliefs, or behaviors, while maintaining the
narrative context and not detracting from the perceived realism of the experience.
Achieving these pedagogical goals is more complicated than it is with explicit
feedback. To alter behavior, it is necessary to both select what dimension to tailor
(e.g., nonverbal, content, model) and how to do it. Further, a method for ensuring
fidelity (acceptability, believability, etc.) should be included in the form of
preconditions on modification rules or as a separate filter. Some examples of how a
character might achieve the goals of recognition, explanation, or repair include:
1. amplification of virtual human response behavior, such as the intensity of
facial expressions or use of emotionally charged vocabulary (recognition)
2. description of a causal link between a user action and a negative (or positive)
result via additional content (e.g., “By suggesting X you are essentially
blaming me for the problem.”; explanation)
3. clarification of a relevant domain concept by including it in the content of an
utterance (“In my culture we believe X…”; explanation; repair).
4. suggestion of an alternative communicative action that would have produced
a better outcome (e.g., “If I were you, I’d …”; repair)
The central idea behind all of these strategies is to build on the existing feedback
already coming from the virtual human, but alter it to address a specific need of the
learner. The changes can be generated from shallow modification rules, such as
“increase the intensity of facial expressions to enhance feedback” or through deeper,
model-based adjustments like “increase the cultural pride of the character, which will
produce longer utterances that explain beliefs and/or values.”
We have completed a prototype system that modifies the content of character
utterances to both amplify feedback and provide explanations [18]. The system, built
as a supplemental component to BiLAT [5], tracks meetings with characters and
augments character utterances when errors are made and when a specific knowledge
component (cultural knowledge, in this case) is first encountered. For example, if an
error is made by a beginner, the character might bring up the underlying cultural
difference in their response (a content adjustment). Other learners would get the
standard simulation response. Currently, the system uses a rudimentary student model
to track learner’s progress and studies of the system effectiveness are being planned.
Y 10 Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
For virtual human role players to adapt based on pedagogical aims, it is likely that
more sophisticated learner models will be necessary. Building learner models for
domains such as cultural learning and interpersonal skills is no simple task, but even
crude distinctions can be helpful. Of course, a key question is whether such
adaptations threaten fidelity and the implications of that. If learners figure out the
characters are secretly “helping”, does it ruin the fantasy? How does this affect
learner affect and motivation to engage? Also, what if realism is breached – does this
necessarily hinder learning? Future studies will need to address these questions as
well as determining if support from pedagogical experience manipulation can be as
effective (or complementary to) explicit help from a tutoring system.
Acknowledgments. The project or effort described here has been sponsored by the
U.S. Army Research, Development, and Engineering Command (RDECOM).
Statements and opinions expressed do not necessarily reflect the position or the policy
of the United States Government, and no official endorsement should be inferred.
Thanks to Bob Wray at SoarTech and Mark Core at ICT for many fruitful discussions
in formulating the ideas presented in this paper.
References
1. Johnson, W.L., Rickel, J., Lester, J.C.: Animated Pedagogical Agents: Face-to-Face
Interaction in Interactive Learning Environments. International Journal of Artificial
Intelligence in Education 11 (2000) 47 - 48
2. Gratch, J., Marsella, S.: Lessons from emotion psychology for the design of lifelike
characters. Applied Artificial Intelligence 19 (2005) 215-233
3. Kane, P.E.: Role Playing for Educational Use. Speech Teacher 13 (1964) 320-323
4. Segrin, C., Givertz, M.: Methods of social skills training and development. In: Greene, J.O.,
Burleson, B.R. (eds.): Handbook of Communication and Social Interaction Skills.
Routledge, New York, NY (2003) 135--176
5. Kim, J.M., Hill, R.W., Durlach, P.J., Lane, H.C., Forbell, E., Core, M., Marsella, S.,
Pynadath, D.V., Hart, J.: BiLAT: A Game-based Environment for Practicing Negotiation in
a Cultural Context. International Journal of Artificial Intelligence in Education (in press)
6. Johnson, W.L., Valente, A.: Tactical language and culture training systems: using artificial
intelligence to teach foreign languages and cultures. Proceedings of the 20th national
conference on Innovative applications of artificial intelligence - Volume 3. AAAI Press,
Chicago, Illinois (2008) 1632-1639
7. Kenny, P., Parsons, T., Gratch, J., Rizzo, A.: Virtual humans for assisted health care.
Proceedings of the 1st international conference on PErvasive Technologies Related to
Assistive Environments. ACM, Athens, Greece (2008) 1-4
8. Johnsen, K., Raij, A., Stevens, A., Lind, D.S., Lok, B.: The validity of a virtual human
experience for interpersonal skills education. Proceedings of the SIGCHI conference on
Human factors in computing systems. ACM, San Jose, California, USA (2007) 1049-1058
9. Hubal, R.C., Frank, G.A., Guinn, C.I.: Lessons learned in modeling schizophrenic and
depressed responsive virtual humans for training. Proceedings of the 8th international
conference on Intelligent user interfaces. ACM, Miami, Florida, USA (2003) 85-92
10.Sapouna, M., Wolke, D., Vannini, N., Watson, S., Woods, S., Schneider, W., Enz, S., Hall,
L., Paiva, A., Andre, E., Dautenhahn, K., Aylett, R.: Virtual learning intervention to reduce
Y 11 Z
Diana Pérez-Marín, Ismael Pascual-Nieto, Susan Bull (Eds): 1st APLEC Workshop Proceedings, 2010
bullying victimization in primary school: a controlled trial. Journal of Child Psychology and
Psychiatry 51 (2010) 104-112
11.Tartaro, A., Cassell, J.: Playing with virtual peers: bootstrapping contingent discourse in
children with autism. Proceedings of the 8th international conference on International
conference for the learning sciences - Volume 2. International Society of the Learning
Sciences, Utrecht, The Netherlands (2008) 382-389
12.Surface, E.A., Dierdorff, E.C., Watson, A.M.: Special operations language training software
measurement of effectiveness study: Tactical Iraqi study final report. Special Operations
Forces Language Office (2007)
13.Lane, H.C., Hays, M.J., Auerbach, D., Core, M.: Investigating the relationship between
presence and learning in a serious game. In: Kay, J., Aleven, V. (eds.): 10th International
Conference on Intelligent Tutoring Systems. Springer-Verlag (in press)
14.Johnsen, K., Raij, A., Stevens, A., Lind, D.S., Lok, B.: The validity of a virtual human
experience for interpersonal skills education. Proceedings of Computer Human Interaction
2007 (2007) 1049-1058
15.Swartout, W., Gratch, J., Hill, R.W., Hovy, E., Marsella, S., Rickel, J., Traum, D.: Toward
virtual humans. AI Magazine 27 (2006) 96-108
16.Gratch, J., Wang, N., Gerten, J., Fast, E., Duffy, R.: Creating Rapport with Virtual Agents.
Proceedings of the 7th international conference on Intelligent Virtual Agents. Springer-
Verlag, Paris, France (2007) 125-138
17.Ekman, P.: Facial expression and emotion. American Psychologist 48 (1993) 384-392
18.Wray, R., Lane, H.C., Stensrud, B., Core, M., Hamel, L., Forbell, E.: Pedagogical
experience manipulation for cultural learning. In: Blanchard, E. (ed.): Proceedings of the
2nd Workshop on Culturally Aware Tutoring Systems at the 14th International Conference
on Artificial Intelligence in Education (2009) 35-44
19.Shute, V.: Focus on Formative Feedback. Review of Educational Research 78 (2008) 153-
189
Y 12 Z