Introduction

Animated pedagogical agents have shown significant potential for promoting learning [5]. A number of recent studies have identified benefits from using them (e.g., [1]). However other studies have produced mixed results [9], or have suggested that agent features such as voice [6], language style [7], and adherence to politeness norms [8] are more important than having an animated persona. For the domain of language learning, however, animated agents offer obvious benefits, if they are designed and utilized properly. Animated agents that can engage in face-to-face conversation can give learners rich opportunities to develop and practice their language skills. This paper gives an overview of a framework for utilizing animated pedagogical agents to promote intercultural skills, implemented in a deployed suite of learning products. The characteristics of the domain (second language learning) and the teaching method (game-based learning) necessitate an approach centering on the use of virtual conversational partners and virtual coaches, in contrast to the tutor-centric approach which is common in intelligent tutoring systems. We then discuss the general issue of adaptation in our courses, and the specific issues involved in tracking the learner's application of communication skills and adapting agent responses accordingly. Finally, we discuss current work aimed at incorporating explicit cultural models into conversational agents, which will make it possible to create agents with varying degrees of intercultural sensitivity, affecting the difficulty of the scenario.

Y 27 Z 2 Background: Intercultural Skill Learning Environments Fig. 1. Operational Indonesian language and culture training system. Figure 1 shows an example learning environment, Operational Indonesian. In this course learners can learn the basic skills necessary to engage in overseas operations such as humanitarian assistance. They practice their skills in interactive game scenarios. In this scenario the learner's character (center left) is engaged in a conversation with the local military commander (center right) about providing aid. The learner communicates with the non-player characters by speaking in Indonesian into a microphone, and selecting accompanying nonverbal gestures as appropriate. The goal is to get learners to the point where they can engage in conversation without hints or assistance, but until they get to that point they can refer to a list of hints of what to say, either in English (top left), or in Indonesian.

The courses cover the language and cultural knowledge and skills necessary to be effective in the target missions and situations. Curricula employ a stepwise process of knowledge acquisition and skill development. Lessons introduce the relevant phrases, vocabulary, cultural knowledge, and linguistic knowledge, and then give learners opportunities to practice applying this knowledge. Learners practice individual conversational turns, and then progress to more extended conversations, as in Fig. 1.

Approximately 100,000 people around the world have used these courses to date to learn about foreign languages and cultures [3]. We have developed a major language learning Web site that has over 10,000 registered users around the world, and many more guest users. Feedback from this user base has contributed to the development of the ideas presented here.

3 Conversational Partners, Coaches, and Scaffolds Conversational agents in these courses fall into two main categories: conversational partners and virtual coaches. Conversational partners respond to the learner's spoken utterance and nonverbal actions, in a manner that is appropriate for the culture, the partner's social role, and the social context of the conversation. For example in Fig. 1 the conversational partners represent officers in the Indonesian Army, and the learner should address them in a manner appropriate each officer's social standing. The manner in which the agents respond provides learners with cues as to how well they are performing. For example, conversational partners may express approval then learners speak in a courteous and culturally appropriate manner, or may express offence when they commit a faux pas and say something inappropriate. This helps to make feedback become an intrinsic part of the interaction of the practice scenarios. We find that such intrinsic feedback is generally more salient and memorable than extrinsic feedback such as critiques and commentary of the learner's performance. For example, if a learner says something that is culturally offensive and inappropriate, learners will be more likely to remember and learn from their mistake if they can see the conversational partner display offence at the learner's actions.

For scenarios designed as final learning assessments, the feedback from the learner comes only from the conversational partners. In such cases the learner should be able to decide to say and do based only on what the non-player characters say, and if they require help beyond that they will receive deductions in their performance score. For practice scenarios however learners typically require more feedback than what the conversational partners provide. The agent's reaction to the learner may be subtle or ambiguous, just like in real intercultural situations, where people often avoid showing offence, out of politeness. Reactions to faux pas may be subtle and easily overlooked by someone who is not familiar with the culture. And even when learners recognize that they have made a mistake, they may not understand what exactly they did wrong or understand why it is a mistake. We therefore often find it useful to scaffold practice dialogs with hints and additional feedback and explanations.

Virtual coaches play an important role in providing this scaffolding. They help present and explain the cultural and linguistic knowledge that they will require, providing voiceover narrations of learning materials. They introduce conversational exercises, preparing learners cognitively for the exercise (by reminding them of communication skills that they will need to employ during the exercise) and preparing them affectively as well (by encouraging attitudes and affective states conducive to successful conversation). After the exercise is complete, the coach provides the learner with feedback on how they performed, so they understand what they did wrong and why. It may also give advice on which skills the learner ought to practice to perform better in the future. However we deliberately avoid developing coaches that engage in extended tutorial dialogs, so that the learners can focus attention on culturally appropriate interactions with conversational partners.

Figure 2 illustrates how a conversational partner and a virtual coach are combined in a single exercise. The learner is requested to ask his friend Matt (on the left) whether he wants to stop for a burger. The learner has attempted to make the request, but got it wrong, and so the coach has come in and explained what the learner should have said. One disadvantage of using a virtual coach or tutor is that the coach's intervention can disrupt the flow of the scenario and distract the learner from the conversation. Therefore during ongoing scenarios we use subtler scaffolding cues instead. We employ simple auditory signals (earcons), as graphical symbols (green plusses and red minuses) to signal when the learner has done something particularly good or bad. These alone are usually sufficient to make the learner aware of what they have done and help them adjust their behavior. Then when the scenario is done the virtual coach can come in and explain what exactly the learner did wrong and why.

Adaptation

It is useful to adapt the level of difficulty of practice scenarios according to the skill level of the learner. This is currently accomplished by adjusting the amount of additional scaffolding that is provided in the scenario. Depending upon the level of difficulty selected the symbols and earcons that signal a change in the agent's attitude and reaction can be disabled, and subtitles and translations can be removed.

The most important type of adaptation is in making the behavior of the conversational partners adapt in real time to the level of communicative skill of the learners, in the course of the conversations between the learners and the agents. Each agent has a level of rapport with the learner, which increases when the learner says culturally appropriate things and decreases when the learner says culturally inappropriate things. In more complex scenarios agents may include additional dynamic social variables, such as the agent's level of trust of and fondness toward the learner. The agent's response to the learner is dependent in part upon the levels of rapport and other social variables that have been established to that point. This is particularly important when modeling relationship-oriented cultures, where it is important to establish a personal relationship with one's counterpart before getting down to business.

Agent processing is organized in a pipeline. The agent first interprets the meaning of the learner's speech and gestural inputs as a communicative act, i.e., a generalization of the concept of speech act. The agent then selects a communicative act to perform in response. Finally, it generates a combination of speech and body movements to realize the communicative act. In our currently deployed learning environments, such as those illustrated in Figures 1 and 2, agent communicative act selection is implemented using finite state machines, where state transitions may be conditioned by predicates over the social variables. We have recently developed a new architecture, called VRP (Virtual Role Player) [4], which incorporates explicit representations of the physical and social environment, and rules governing agent behavior.

We have also been experimenting with dynamic learner models that track the learner's ability to use words and phrases in conversation. The learner model tracks and records each attempt on the part of the learner to say a particular phrase. We intend to use this information to filter the curriculum, to focus on learning activities that require learners to practice the phrases that they are having difficulty with.

Explicit Models of Culture and Cultural Sensitivity

In our current work we are extending our VRP agent architecture to increase the level of flexibility and adaptability that is supported. This provides additional opportunities for adapting agent behavior to adjust to the skill level of the learner. By making these representations part of a shared state across multiple dialog instances, we can create agents whose behavior adapts over a series of episodes to the learner's communicative competence, creating practice experiences that are both more realistic and provide learners with an appropriate level of challenge.

A new project named CultureCom is developing formal models of the cultural influences underlying dialog and utilizing them to increase the flexibility and realism of the behavior of non-player characters in training simulations. The work is being conducted in collaboration with Dr. Michael Agar of Ethnoworks and Prof. Jerry Hobbs of the University of Southern California. Cultural and linguistic anthropologists are developing validated sociocultural data sets for Afghanistan and other cultures of interest, consisting of annotated dialogs of cross-cultural interactions. Experts in artificial intelligence then use these data to develop logical models of sociocultural behavior in different cultures, based upon a formal ontology of microsocial concepts underlying interpersonal communication. This in turn is being used to create an enhanced version of the VRP architecture in which agent intent planning utilizes explicit validated models of sociocultural reasoning for different cultures, which can swapped in and out to enable agents to model a variety of different cultural characteristics.

The following example illustrates how CultureCom cultural models will be developed and used. American culture and Afghan culture differ in the way they express promises and commitments. Afghans sometimes agree to a request as a way of being socially agreeable, without making a firm commitment. In CultureCom we explicitly model for communicative acts what sociocultural inferences can be made from them, such as whether a statement of agreement constitutes a firm promise and commitment. This in turn can be used to ensure that the non-player character's actions consistent with the culture throughout, and can also provide helpful feedback to the learner. For example it can help learners to recognize when intercultural misunderstandings can arise due to different views of what has been promised and agreed to.