Do Birds of a Feather Work Better Together? The Impact of Virtual Agent Personality on a Shared Mental Model with Humans during Collaboration

Abstract

The development of a Shared Mental Model (SMM) between team members and effective communication of the shared knowledge have been found to improve teamwork performance. In human-IVA heterogeneous teams, the communication required to develop a SMM is difficult as each party belongs to different worlds (i.e. real and virtual). Moreover, humans may differ in how they produce and perceive communication acts according to their personality traits. The influence of IVA personality, exhibited via verbal and non-verbal communication, on collaboration and development of a SMM within a human-IVA team has not been previously studied. In this paper, we explore the impact of IVA's with two different combinations of personality traits, i.e. extraversion and agreeableness, on the development of a SMM with human teammates. Additionally, this study investigated the influence of the match in the two personality traits between IVAs and humans on the development of a SMM. The results showed that agreeable IVAs positively impacted on the development of taskwork and teamwork SMMs; whereas extraversion did not influence development of the SMM. Moreover, when collaborating humans and IVAs had matching agreeableness personality traits there was a positive influence on the SMM between them and better performance outcomes.

1. INTRODUCTION

Teamwork refers to a group of entities that use their knowledge and capabilities in an organized way towards achieving a shared goal that would not be carried out effectively with the effort of a single entity [ 28 ]. Other definitions for ‘team’ exist [ 15 ]. A number of studies have targeted human teams to understand the processes that enhance collaboration within teams. Studies have indicated that effective teamwork often relies on the acquisition of a shared mental model [ 2 ]. While most research on teamwork concerns human teams, some early research studying human interactions with computers has provided evidence that people treat computers like they would treat real people [ 43 ]. In later studies, researchers began to transfer these findings to human interaction with Intelligent Virtual Agent (IVAs).

Due to the increasing interest in heterogeneous teams and the challenges in human-agent teamwork coordination [ 10 ] [ 48 ], researchers have explored several factors that may influence these teams. Cohen et al. [ 14 ] stressed the importance of having shared objectives and mental state or mental model between team members. A Shared Mental Model (SMM) is the state among team members where the members have overlapping knowledge and beliefs. SMM was introduced by Cannon-bowers et al. [ 12 ] in the context of teamwork amongst humans. Most research into SMMs concern human-human teamwork and communication (e.g., [ 18 ]). Some research exists that considers a SMM in the context of agent-agent teamwork. Later, it became apparent that SMM is not only important in human teams, but also in humanagent teams [ 31 ]. Many researchers who have been studying SMM classified the shared knowledge into two categories: knowledge about the team and knowledge about the task [ 12 ]. SMM concepts resemble Traum’s use of grounding models [ 53 ] or mutual beliefs between humans and an IVA. Traum’s work focused on studying a human’s dialogue and creating a conversation system that mimics human verbal communication to establish mutual understanding with a conversational virtual human [ 50 ]. However, collaborative activities need more than grounding based only on verbal conversation.

Many aspects relating to the development of a human-IVA SMM are understudied. In particular, we note the lack of human-agent studies that explore whether the personalities of the human and/or IVA have an impact on their teamwork and the establishment of a SMM. Integrating personality into agents is not a new research topic. Twenty years ago, Loyall and Bates introduced an agent with personality that communicated through bubble text [ 35 ]. Later, several studies have been carried out to explore the influence of IVAs with personality traits on the interaction with humans. This interaction was either social, behavioural [ 13 ], emotional or cognitive. For instance, Parada and Paiva [ 47 ] developed an agent model to support group dynamics of autonomous synthetic characters (Synthetic Group Dynamics mode or SGD mode) based on two personality traits, extraversion and agreeableness.

Given the importance of a SMM for human teams and influence of personality in human teams, we address this current gap by conducting a study to investigate the effect of the combined human and IVA personalities on the development of a SMM (i.e. shared understanding of the task and the team).

This paper proceeds as follows. Section 2 provides an overview of related research. In Section 3, we briefly describe the measurement and communication of personality traits, followed by our research questions in Section 4. A description of our experimental methodology is given in Section 5. The results are presented in Sections followed by discussion in Section 7. Finally, the conclusion and future work appear in Section 8.

2. RELATED RESEARCH

A review of the literature has identified two classes of related work. First, studies that considered the establishment of a SMM between humans and IVAs. Second, studies that explored the influence of humans’ or IVAs’ personality traits on human-IVA interaction. No related work was found that combined both classes.

In the first class of studies, Yen and Fan’s (e.g., [ 56 ], [ 20 ]) agents were designed to use SMM knowledge of the task to communicate information with other agents in a team. However, this work focused on a team of agents. Fan and Yen also reported a survey [ 19 ] of research that studied SMM between humans and agents. A noteworthy study in this survey was R-CAST agents [ 57 ] that share with their human team members the decision-making process and their dynamic progress. Hanna and Richards [ 28 ] studied the impact of a proposed multimodal communication model (called HAT-CoM) between a human and her IVA teammate on the establishment of a SMM. This study investigated the influence of communication on different outcomes of SMM such as anticipating a teammate’s decisions, reduced explicit communication, match in cognitive perspective, and competence in decision-making. Nevertheless, this study did not explore the effect of personality of both humans and IVAs on the human’s perception of the proposed communication model.

The body of work in the second class of studies concerning IVAs and personality is more extensive. Luse et al. [ 36 ] found that the human’s personality influenced the humans’ preferences to work in teams. A number of studies have investigated team member personality as a predictor of both team processes and outcomes [ 6 ]. A number of researchers studied the influence of personality traits on human decision-making while achieving a task. For example, Schmitt et al. [60] asked the human subjects to play the ultimatum game. In this game, two players had to reach an agreement about how to divide money through proposing and responding. This work used Myer-Briggs Temperament Index (MBTI) test to get personality traits of players. The results showed that extravert players indicated a willingness to accept lower offers than introvert players did.

In a study to determine what combinations of personalities resulted in the best-performing teams, Gorla and Lam[ 25 ] surveyed 92 employees from 20 small software development teams. The results showed that heterogeneity among team members had no significant effect on team performance. In another study [ 4 ], the performance of sixty three (63) virtual human teams was studied with respect to extraversion personality traits. Extraversion was found to be an important trait to promote group interaction and teams with lower variance in extraversion levels did better.

Isbister and Nass [ 29 ] studied the effect of consistency in representing personality via an IVA’s verbal and non-verbal communication and human preferences. In addition, human preferences for IVAs with personalities that matched their own personality was investigated. The results showed that humans prefer the personality of an IVA to be consistent in both verbal and non-verbal communication. Moreover, the results indicated that participants tended to prefer a character whose personality was complementary, rather than similar, to their own. Kang et al. [ 30 ] explored associations between the Five Factor Model (FFM) (see next section) personality traits of human subjects and their feelings of rapport when they interacted with a virtual agent. The results showed that users’ personality traits affect users’ perception, regardless of the implementation of personality within the virtual agent. The results in [ 54 ] showed that participants’ personality traits influenced their subjective feelings after the interaction, as well as their evaluation of the virtual character and their actual behavior. Du and Huhns [ 17 ] studied whether human behaviour towards other humans and agents is related to their personality types. Although this study used a different personality test, the results showed that humans of different personality types behave differently towards other humans and agents.

3. MEASURING AND COMMUNICATING

PERSONALITY TRAITS

Among the measures of personality traits, the FFM of personality has proven to be a robust tool for understanding personality variations across individuals. FFM [ 23 ] claims that personality differs on five factors: Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism. Openness means being open to experience new things, being imaginative, and intelligent. Conscientiousness indicates responsibility, reliability and tidiness. An extravert is outgoing, sociable, assertive and energetic. Agreeableness means a person is trustworthy, kind and cooperative by considering others’ goals. A neurotic character is anxious, nervous, prone to depression and lacks emotional stability.

Studies that have explored personality traits and teamwork stress the role of both extraversion and agreeableness to foster interrelationships between team members. Extraversion and agreeableness were selected in our study because they have been shown to be predominant traits in collaboration and teamwork [ 8 ]. The extraversion trait affects interpersonal relations through the quality of social interactions [ 7 ] [ 40 ]. Extraverts are usually active members in teamwork interactions and often popular among their teammates [ 37 ].

Personality is communicated verbally as our personality is likely to influence how we speak [ 52 ]. Speaking style can reveal certain personality traits; some traits are easier to detect than others [ 51 ]. A number of studies have used verbal capabilities to represent different IVA personalities [ 32 ]. Neff et al. [ 45 ] determined a number of aspects that demonstrate the impact of the IVA’s extravert personality on the IVA’s verbal behaviour.

Additionally, IVA personality may be communicated nonverbally through the IVA’s physical position relative to the human’s view or their avatar. Argyle’s [ 3 ] status and affiliation model for animating non-verbal behavior of virtual agents identified two fundamental dimensions for non-verbal behavior: affiliation and status. Affiliation can be characterized as wanting a close relationship and it is associated with non-verbal clues such as close physical position. Other studies (e.g., [ 11 ]) suggest that agents approaching the subject’s avatar were judged as more extraverted than agents not approaching them, regardless of smile and the amount of gaze they gave.

4. RESEARCH QUESTIONS

To build upon and draw this literature together and potentially enhance human-IVA teamwork, the following research questions aim to investigate the relationship between the IVA’s personality and the development of a SMM between the human team member and the IVA: 1. Are the IVA’s personality traits, i.e. extraversion and agreeableness, as presented in its verbal and non-verbal communication, significantly differentiated by humans? 2. Do the IVA’s personality traits, i.e. extraversion and agreeableness, significantly influence the humans’ perception of the taskwork and/or the teamwork SMM? 3. Does a match in human-IVA personality traits, i.e. extraversion and agreeableness, influence the humans’ perception of the taskwork and/or the teamwork SMMs with the IVA? 4. Do taskwork or teamwork SMMs affect human-IVA team performance?

5. METHODOLOGY

An experiment was conducted to answer the four research questions. The participants, design, procedure and collaborative scenario (the materials) are described below.

5.1 Participants

Fifty-five (55) second-year undergraduate science students enrolled in a biology unit completed the collaborative task. Participants were aged between 18 and 51 years (mean=22.56; SD=6.95) Fifty-two participants were native English speakers; the remaining three participants had been speaking English on a daily basis on average for 13 years. On a scale with 6 levels (level 1 the least experienced and level 6 the highest experience), 13 had basic (level 2), 39 had proficient (level 5) and 3 had advanced (level 6) computer skills. Participants played computer games on average 2.73 times a week, with a standard deviation of 4.69.

5.2 Collaborative Scenario

The collaborative scenario was implemented using the Unity3D game engine (www.unity3d.com). The scenario included a task where both a human user and an IVA, named Charlie, have to collaborate to achieve a shared goal. The goal is to pass a sequence of four obstacles to reach their target (scientific laboratory). The four obstacles included a brick wall, wooden gate, bush and hill (see Figure 1). In order to get over each of these obstacles both the human and IVA have to select a pair of tools from a toolbox that contains 12 tools (pruning shears, bush hook, hammer, chisel, ladder, rope, matchsticks, matchbox, screwdriver, nipper, shovel and mattock). These tools were picked so that each pair of tools would be complementary, i.e. a single tool cannot work without the function of the complementary tool. For example, the chisel needs the hammer and matchstick needs the matchbox. In addition, each obstacle could be passed using a different method and the corresponding combination of tools. For example, the bush obstacle could be chopped, burnt or climbed. Hence, there should be agreement between the human and the IVA concerning the best way to overcome the obstacle and to select which pair of tools is most suitable for the task. HumanIVA interaction (i.e. communication) during the collaborative activity is described below as part of the experimental design.

5.3 Experimental Design

To answer the research questions, an experiment was conducted. The experiment consisted of five different treatments with the same virtual scenario but the IVA, i.e. Charlie, had different personalities. One treatment was a control with a neutral personality IVA. The aim of the control treatment was to measure whether inclusion of IVA personality made a difference and to allow comparison with the other experimental treatments. The other four experimental treatments had the four combinations of the two studied personality traits, i.e. extraversion and agreeableness. The four combinations were extraversionagreeableness, extraversion-antagonism, introversionagreeableness and introversion-antagonism.

The two studied personality traits were incorporated into the IVA’s verbal and non-verbal communication. To express the personality traits in the IVA’s verbal communication, the literature was reviewed to find out what verbal aspects were affected the most by personality. Among the list of aspects mentioned in the work of Neff et al. [ 45 ], we selected the dominant aspects as the basis of the design of the IVA in our study. Verbal messages were designed and reviewed by the authors according to the criteria in [ 45 ] and as shown in Table 1. A number of studies addressed how the extraversion personality trait can be represented in an IVA’s non-verbal signaling. As verbal behaviours have already been identified that show an IVA’s personality, Doce et al. [ 16 ] proposed several non-verbal features that could be used to show personality traits in IVA, these features include: spatial extent, temporal extent, fluidity, power and repetitiveness. To design the non-verbal communication of the IVA, we chose the dominant features, shown in Table 2.

5.4 Data Collection and Data Analysis

The following variables were measured to answer the research questions:     

Participant’s personality: participants completed a 7-item personality test to measure the two personality traits using International Personality Item Pool (IPIP) [ 24 ].

IVA’s personality: participants completed a test of the perceived personality of the IVA by answering four items of the Ten-Item Personality Inventory (TIPI) [ 26 ].

IVA’s verbal and non-verbal communication: participants

answered ten items. Five items measured the IVA’s verbal communication (e.g., “Charlie’s requests and replies were helpful to complete the task”) and five items measured the IVA’s non-verbal communication (e.g., “Charlie’s actions were suitable to the situation”).

Taskwork and teamwork SMM: Participants answered ten items in a survey, similar to other studies measuring SMM (e.g., [ 28 ]). Five items measured human perception of taskwork SMM (e.g., “Charlie and I have a shared understanding about how best to ensure we meet our goal”). Five items measured teamwork SMM (e.g., “Charlie and I Value collaborating with each other”).

Team performance: participants answered five items to measure their perception of team performance with the IVA.

Parameter

Verbosity

Restatements

Request confirmation

Negation

Filled pauses

Description

Control the number of propositions in the utterance

Paraphrase an existing proposition Begin the utterance with a confirmation of the propositions

Negate a verb by replacing its modifier by its antonym Insert syntactic elements expressing hesitancy

Emphasizer hedges Insert syntactic elements (really, basically, actually, just) to strengthen a proposition Both personality tests, i.e. IPIP and TIPI, and the communication and SMM questions used a 5-item Likert Scale, where 1 corresponded to “Strongly Disagree” and 5 to “Strongly Agree”.

In addition to these subjective measures, all inputs from the user were logged to allow recreation of navigation paths and record inputs such as responses and selected tools. These inputs included selected regions in the scenario. Analysis of interaction logs to find the most frequently triggered stimuli in the scenario was used before in other studies [ 32 ].

The statistical package IBM SPSS v.20 was used for the statistical analysis. A number of tests for normality distribution of the study variables were run to determine whether to use parametric or nonparametric tests. Shapiro-Wilk normality test as well as Skewness and Kurtosis were used to test normality distribution of the study variables. Spearman’s rho Correlation analysis was used to quantify the degree and the direction to which the study variables are related. To measure the difference between the different experimental treatments, one-way ANOVA test and Kruskal Wallis test were utilized. Regression analysis was utilized to learn more about the relationship between an independent or predictor variable and a dependent or criterion variable.

5.5 Procedure

Each participant was randomly assigned to one of the five treatments by a web-based system containing all five treatments. Participants used the virtual system individually so that the collaboration would be one-to-one between him/herself and the agent. We allocated twenty minutes for the study that consisted of five parts in one session requiring the participant to: Part 1: Sign consent form and complete biographical information. Part 2: Take a personality test to measure their own personality. Part 3: Participate in the scenario in the 3D virtual scene. In the beginning of the scenario, the participants were provided with online instructions about the goal of the virtual scenario, the name and the use of each tool in the toolbox and how to select/close the verbal messages.

Part 4: Complete a survey that measures the participant's perception of some study variables.

Part 5: Answer a short personality test about the assigned IVA.

6. RESULTS

First, the study variables were tested for the normality distribution in order to determine whether to use parametric or nonparametric tests. Tests for Skewness and Kurtosis showed that the z-value of the variables are in the span -1.96 and +1.96, and thus they do not differ significantly from normality. A Shapiro-Wilk normality test showed that, except for verbal communication variable, all the other variables had p-values less than 0.05. Based on the results of Skewness and Kurtosis as well as Shapiro-Wilk we concluded that the four variables (non-verbal communication, taskwork SMM, teamwork SMM, and team performance) are not normally distributed.

To measure the strength and direction of association between the five variables, Spearman’s rho correlation method was used. Spearman’s rho correlation was selected, as it is more appropriate for non-normally distributed responses. To estimate how well the set of items measure each variable, Cronbach’s Alpha (ɑ) was used to measure the internal consistency or reliability of these items. The value of Cronbach’s Alpha (ɑ) may lie between negative infinity and one. However, only positive values of α make sense. Generally, Cronbach’s alpha (ɑ) coefficient ranges in value from zero to one and may be used to describe the reliability of factors. Some statisticians insist on a reliability score of 0.70 or higher in order to assess the studied items are internally consistent. Table 3 shows that Cronbach’s Alpha (ɑ) for the five variables are over 0.70. We concluded that the survey items to measure each variable are reliable to measure this variable. Table 3 presents the means, standard deviations, and correlations for all the variables. Verbal and nonverbal communication were significantly positively related to taskwork SMM (r=0.461, p<0.01 and r=0.351, p<0.01 respectively) suggesting a positive association between both verbal and non-verbal communication during a collaborative task on developing common understanding of the taskwork. Moreover, team verbal and nonverbal communication were significantly positively related to teamwork SMM (r=0.465, p<0.01 and r=0.308, p<0.05, respectively) suggesting a positive association between both verbal and nonverbal communication during a collaborative task on developing common understanding of the teamwork. Taskwork SMM was significantly positively correlated to teamwork SMM (r= 0.704, p<0.01) suggesting that human-IVA teams whose members share similar taskwork mental models are likely to have shared

1. Verbal Communication

2. Non-Verbal Communication

3. Taskwork SMM

4. Teamwork SMM

5. Team Performance

Cronbach’s

Alpha

teamwork mental models as well. As expected, both taskwork and teamwork SMM were significantly positively correlated to human-IVA team performance (r=0.569, p<0.01 and r=0.489, p<0.01 respectively). This result suggests a positive association between taskwork and teamwork SMMs and overall team performance. The following subsections analyse the data related to the three research questions.

6.1 Can Humans Recognize the IVA’s

Personality?

The first research question inquired if there were significant differences between the five groups of participants in perceiving the IVA’s two implemented personality traits. This question was segmented into two sub-questions.

The first sub-question asked if the IVA’s introvert/extravert personality trait as presented in the IVA’s verbal and non-verbal communication is perceived differently by the human participants. The results of one-way ANOVA showed that there was a significant difference p<0.01 [F(2, 52) = 15.014, p < 0.01, η2=0.37] between the groups of participants in their perception of the personality of IVA, i.e. introvert, extravert or neutral IVA, because of the verbal messages of the IVA. In addition, the results of one-way ANOVA showed that there was a significant difference p<0.01 [F(2, 52) = 11.424, p < 0.01, η2=0.31] between the groups of participants in their perception of different personality of IVA, i.e. introvert, extravert or neutral IVA, because of the non-verbal messages of the IVA.

The second sub-question asked if the IVA’s agreeableness/antagonism personality trait as presented in the IVA’s verbal and non-verbal communication is perceived differently by the human participants. The results of one-way ANOVA showed that there was a significant difference p<0.01 [F(2, 52) = 6.086, p < 0.01, η2=0.19] between the groups of participants in their perception of the IVA’s personality, i.e. agreeableness, antagonism or neutral, because of the verbal messages of the IVA. In addition, the results of one-way ANOVA showed that there was a significant difference p<0.05 [F(2, 52) = 3.90, p <0.05, η2=0.13] between the groups of participants in their perception of different IVA personality, i.e. agreeableness, antagonism or neutral IVA, because of the IVAs non-verbal messages.

6.2 Does the IVA’s Personality Influence the

Development of a SMM?

The second research question inquired whether the participants’ perception of the IVA’s personality traits, i.e. extraversion and agreeableness, influenced their perception of the SMMs for taskwork and teamwork. The results did not show any significant difference between the perception of either taskwork or teamwork SMM according to the IVA’s extraversion personality. The results of ANOVA test showed that there was a significant difference p<0.01 [F(2, 52) = 4.312, p<0.01, η2=0.14] between the groups of participants in their perception to taskwork SMM according to the IVA’s agreeableness/antagonism personality trait. This result was supported by the outcome of the nonparametric Kruskal-Wallis H test (H=6.725, df=2, n=55, p<0.035). To understand the which condition/s accounted for the significant difference in taskwork SMM, post hoc comparisons using the Tukey HSD and Bonferroni tests indicated that the mean score for the antagonistic IVA condition (M = 3.36, SD = 0.64) was significantly different than the agreeable IVA condition (M = 3.88, SD = 0.51) at p < 0.05. The latter was significantly different from neutral IVA (M = 3.36, SD = 0.87) at p < 0.05. However, the antagonism condition did not significantly differ from the neutral condition. The results of ANOVA test showed that there was a significant difference p<0.01 [F(2, 52) = 6.942, p<0.01, η2=0.21] between the groups of participants in their perception of teamwork SMM according to the IVA’s agreeableness/antagonism personality trait. The significance was also identified by non-parametric Kruskal-Wallis test (H=10.634, df=2, n=55, p<0.005). The results of post hoc comparisons indicated that the mean score for the antagonistic IVA condition (M = 3.11, SD = 0.80) was significantly different from the agreeable IVA condition (M = 3.84, SD = 0.55) at p < 0.05. The later was significantly different from neutral IVA (M = 3.20, SD = 0.63) at p < 0.05. However, the antagonism condition did not significantly differ from the neutral condition.

6.3 Does a Match in Human-IVA Personality

Influence the Perception of a SMM?

The third research question inquired whether the match in personality traits, i.e. extraversion and agreeableness, between the participants and the IVAs significantly influenced their perception of the SMM. The results did not show any significant difference between in the perception of either the taskwork or teamwork SMM according to the match in extraversion personality between the human and the IVA teammate.

The results of ANOVA test showed that there was a significant difference p<0.05 [F(2, 52) = 5.224, p<0.05, η2=0.09] in the perception of a taskwork SMM between the participants who had a match in the agreeableness personality (M= 3.80, SD= 0.48) with the IVA and those who were in mismatch with the IVA (M= 3.40, SD= 0.75). Moreover, the results showed that there was a significant difference p<0.05 [F(2, 52) = 6.199, p<0.05, η2=0.105] in the perception of a teamwork SMM between the participants who had a match in the agreeableness personality (M= 3.70, SD= 0.56) with the IVA and those who were in mismatch with the IVA (M= 3.21, SD= 0.80). The results of ANOVA test was supported by the non-parametric Mann-Whitney test based on ranking. The results of Mann-Whitney U test indicated that the participants who matched with the IVA in agreeableness personality were significantly higher than mismatched group in perceiving taskwork SMM (U=260.5, n=55, p<0.05) and teamwork SMM (U= 232.5, n=55, p<0.05).

To evaluate whether the match in agreeableness personality trait between the humans and the IVAs in a collaboration context could be a predictor of the humans’ perception of both taskwork and teamwork SMM, a linear regression test was used. The results show, see Table 4, that the match in agreeableness personality between the human and the IVA was a significant predictor of taskwork SMM, R2= 0.073, F (2, 52) =5.224, p<0.05. Furthermore, we investigated if the match in agreeableness personality trait between the humans and the IVAs in a collaboration context could be a predictor of the humans’ perception of teamwork SMM. The results, as can be seen in Table 4, showed that the match in agreeableness personality was a significant predictor of teamwork SMM, R2= 0.088, F (2, 52) =6.199, p<0.05. This result suggests that matching human-IVA agreeable personalities is likely to be a significant predictor of the human’s perception of both taskwork and teamwork SMMs.

6.4 Do Taskwork and Teamwork SMMs

Affect Human-IVA Team Performance?

The last research question aimed to investigate the influence of taskwork and teamwork SMMs on human-IVA team performance. The result of multiple regression showed that both taskwork and teamwork SMM would predict the overall team performance to achieve the common goal. The results show that 30.9% of the variance in team performance can be accounted for by taskwork and teamwork SMM between the human and IVA. To assess the overall statistical significance of the model, the results shows that both predictors were significant R2= 0.309, F (2, 52) = 13.068, p<0.001. Thus, we can say that the existence of taskwork and teamwork SMMs do impact on human-IVA team performance in answer to the third research question.

Moreover, to evaluate which one of the two factors, i.e. taskwork or teamwork SMM, contributes more to team performance, the results, as shown in Table 4, indicated that standardized coefficient  of taskwork SMM (0.434) is greater than standardized coefficient  of teamwork SMM (0.170), showing a stronger effect for taskwork over teamwork SMM.

7. DISCUSSION

This study aimed to investigate the influence of an IVAs’ personality as represented in its multimodal communication, i.e. verbal and non-verbal, on the human’s perception of the SMM with the IVA. To reach this aim, four research questions were proposed. The first research question inquired if there was a significant difference between the five treatment groups of participants in their perception of the IVA’s two implemented personality traits, i.e. extraversion and agreeableness. Data analysis revealed that at a statistically significant level participants identified the multimodal communication, verbal and non-verbal communication, of the extravert IVA as more extravert than the introvert IVA. Moreover, the results showed that the participants recognized the multimodal communication of the agreeable IVA as more agreeable than the antagonist IVA.

Other researchers have also studied the influence of personality traits on human-agent interaction. Similar to our study, these studies have concentrated on the incorporation of personality traits in an agent [ 1 ] and/or whether the human could identify the agent’s personality [ 41 ]. For instance, Isbister and Nass [ 29 ] reported that their participants found extraverted IVAs

Model

Taskwork SMM

Teamwork SMM

Agreeableness Match

Agreeableness Match

Team performance

Taskwork SMM

Teamwork SMM

* Significance level p<0.05 significantly more extraverted than the introverted IVAs. Numerous studies have considered whether human participants are able to perceive an IVA’s personality through communication with the IVA. Doce et al. [ 16 ] presented a model to create an IVA with distinguishable FFM personality traits. Neff et al. exploited the extraversion [ 45 ] and neuroticism [ 44 ] traits of the FFM in multimodal characters evaluating the effects of verbal and nonverbal behavior in personality perception studies. Cafaro et al. [ 11 ] conducted a study to investigate how IVA’s non-verbal communication influence the first encounters between humans and virtual agents.

Our study sought to go beyond identification of personality to consider the impact of personality on aspects of human-IVA teamwork. The result of the second research question showed that the participants who had the agreeable IVA were significantly more likely to develop both a strong taskwork and teamwork SMM than those who had the antagonistic IVA. Post hoc tests showed the participants who had received the agreeable IVA developed significantly greater taskwork and teamwork SMMs than those who had either the antagonistic or the neutral IVA treatments. This finding indicated that IVAs with an agreeable personality trait tend to develop SMMs with human teammates. Meanwhile, the results showed that participants who had the extravert or introvert IVA treatment did not differ in their development of taskwork or teamwork SMMs. This finding indicated that an IVA with an extraversion personality is not likely to influence the development of a SMM.

Although the literature of human-agent interaction has not studied the influence of an IVA’s personality on the perception of SMMs with IVA, some researchers in human teams reported a significant interaction between the trust facet of agreeableness in predicting a shared mental model between team members [ 21 ]. Barrick et al. [ 5 ] suggested that an agreeable personality may predict working well in teams; although no direct relationship between agreeableness and team performance was found. Neuman and Wright [ 46 ] concluded that agreeableness between team members help a group come to a consensus on a SMM. An explanation for the findings of our study and other studies in human teams could be that agreeable characters do their best to avoid teamwork disruptions that might occur if there was interpersonal conflict. While our results did not report a significant impact of extraversion on the development of a SMM, some other studies found extraversion as a factor that impact teams. Givney et al. Givney et al. [ 22 ] investigated the influence of personality on human teams; extraversion was found to impact on tasks that did not enforce very short time constraints, while agreeableness was important for tasks where tight collaboration was required. A study of sixty-three (63) virtual teams found that extraversion was an important personality trait to promote group interaction and teams with lower variances in extraversion levels did better [ 6 ]. The results of the third research question indicated that there was a positive significant association between humans who had a match in agreeableness personality trait with IVAs and the development of both taskwork and teamwork SMM. Nevertheless, the results did not show a significant relationship between a human-IVA match in extraversion and the development of a SMM. Studies in human teams indicated that the composition of members’ personalities influence team interaction and performance [ 9 ]. However, these studies have not agreed on whether the variation or similarity in personality have a positive effect on teamwork. Some researchers claimed that variation in personality is likely to be associated with variant skills [ 33 ]. Other studies argued that homogeneity in personality traits among team members tends to improve team performance [ 2 ]. These contradictory results concerning the role of variation of personality in teams may be due to the nature of the task in which the team members are involved.

Studies have indicated that users’ own personality traits affect their behaviour in virtual worlds [ 55 ]. In the literature, matching human-IVA personalities have not been studied in association with SMM and so our results could not be compared with others. Nevertheless, our findings are in line with some previous humancomputer interaction literature [ 49 ] that indicated that humans were more likely to prefer IVAs with similar personality. This opinion was supported by Nass and Lee [ 42 ] who indicated that people prefer to interact with other individuals who have a similar personality to them; while other work showed that people preferred IVAs that were complementary to them [ 29 ]. In their study, Kang et al. [ 30 ] investigated the association between FFM personality traits of human subjects and their feelings when they interacted with an IVA. Their result indicated that agreeable personalities felt strong rapport with an agent that embodies an agreeable personality.

Many studies found a positive correlation between the development of a SMM between team members and their team performance [ 39 ]. Although some studies have found the strongest correlation is between teamwork SMM and team performance [ 34 ], other studies reported the strongest positive correlation is between taskwork SMM and team performance [ 38 ] [ 27 ]. Despite the fact that the influence of either taskwork or teamwork SMM is likely to rely on the nature of the collaborative situation, our results support previous findings that taskwork SMM has a stronger effect on team performance.

8. CONCLUSION AND FUTURE WORK

This paper described a study on whether an IVA’s personality traits influence the establishment of a SMM with a human teammate. Additionally, this study investigated whether the match between IVAs and humans in personality traits impacts on the establishment of taskwork and teamwork SMMs. Through an experiment, it was found that designing an IVA incorporating personality traits is likely to improve the performance of the human-IVA team. In addition, these findings indicated that, similar to human teams, the personality of both the human and the IVA teammate should to be taken into consideration to foster team productivity.

In future work, the other personality traits need to be studied for possible influences on human-IVA teamwork. Additionally, IVA and/or human emotion should be incorporated to investigate if emotions aid human perception of an IVA’s personality and the resultant effect on SMM and team performance.

References

[1] Allbeck , J. and Badler , N. , 2002 . Toward Representing Agent Behaviors Modified by Personality and Emotion . In Workshop on Embodied Conversational Agents - Let's specify and evaluate them ! AAMAS 2002 , Bologna, Italy

[2] Anderson , M.H. , 2009 . The Role of Group Personality Composition in the Emergence of Task And Relationship Conflict within Groups . Journal of Management and Organization 15 , 82 - 96 .

[3] Argyle , M. , 1988 . Bodily Communication . Routledge.

[4] Balthazard , P. , Potter , R.E. , and Warren , J. , 2004 . Expertise, extraversion and group interaction styles as performance indicators in virtual teams: how do perceptions of IT's performance get formed? SIGMIS Database 35 , 1 , 41 - 64 .

[5] Barrick , M.R. , Mount , M.K. , and Judge , T.A. , 2001 . Personality and Performance at the Beginning of the New Millennium: What Do we Know and Where Do we Go Next . International Journal of Selection and Assessment 9 , 9 - 30 .

[6] Barrick , M.R. , Stewart , G.L. , Neubert , M.J. , and Mount , M.K. , 1998 . Relating Member Ability and Personality to Work-Team Processes and Team Effectiveness . Journal of Applied Psychology 83 , 3 , 377 - 391 .

[7] Barry , B. and Stewart , G.L. , 1997 . Composition, Process and Performance in Self-Managed Groups: The Role of Personality . Journal of Applied Psychology 82 , 1 , 62 - 78 .

[8] Bosch , K. , Brandenburgh , A. , Muller , T. , and Heuvelink , A. , 2012 . Characters with Personality! In Intelligent Virtual Agents , Y. Nakano , M. Neff , A. Paiva and M. Walker Eds. Springer Berlin Heidelberg, 426 - 439 .

[9] Bradley , B.H. , Klotz , A.C. , Postlethwaite , B.E. , and Brown , K.G., 2013 . Ready to Rumble: How Team Personality Composition and Task Conflict Interact to Improve Performance . Journal of Applied Psychology 98 , 385 - 392 .

[10] Bradshaw , J.M. , Feltovich , P.J. , Johnson, M.J., Bunch , L. , Breedy , M.R. , Eskridge , T. , Hyuckchul , J. , Lott , J. , and Uszok , A. , 2008 . Coordination in Human-Agent-Robot Teamwork . In Collaborative Technologies and Systems , 2008 . CTS 2008. International Symposium on, 467 - 476 .

[11] Cafaro , A. , Vilhjálmsson , H. , Bickmore , T. , Heylen , D. , Jóhannsdóttir , K. , and Valgarðsson , G. , 2012 . First Impressions: Users' Judgments of Virtual Agents' Personality and Interpersonal Attitude in First Encounters . In Intelligent Virtual Agents, Y. Nakano , M. Neff , A. Paiva and M. Walker Eds. Springer Berlin Heidelberg, 67 - 80 .

[12] Cannon-Bowers , J.A. , Salas , E. , and Converse , S. , 1993 . Shared Mental Models in Expert Team Decision-Making . In Proceedings of the Individual and group decision making , 221 - 246 .

[13] Chittaro , L. and Serra , M. , 2004 . Behavioral programming of autonomous characters based on probabilistic automata and personality . Computer Animation and Virtual Worlds 15 , 3 - 4 , 319 - 326 .

[14] Cohen , P. , Levesque , H. , and Smith , I. , 1997 . On Team Formation . In Contemporary Action Theory, G. HolmstromHintikka and R . Tuomela Eds. Kluwer Academic.

[15] Cohen , P.R. and Levesque , H.J. , 1991 . Teamwork. Noûs 25 , 4 , 487 - 512 .

[16] Doce , T. , Dias , J. , Prada , R. , and Paiva , A. , 2010 . Creating Individual Agents through Personality Traits . In Intelligent Virtual Agents, J. Allbeck , N. Badler , T. Bickmore , C. Pelachaud and A . Safonova Eds. Springer Berlin Heidelberg, 257 - 264 .

[17] Du , H. and Huhns , M.N. , 2013 . Determining the Effect of Personality Types on Human-Agent Interactions . In Proceedings of the Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 02, IEEE Computer Society , 2569317 , 239 - 244 .

[18] Espevik , R. , Johnsen , B.H. , Eid , J. , and Thayer , J. , 2006 . Shared Mental Models and Operational Effectiveness: Effects on Performance and Team Processes in Submarine Attack Teams . Military Psychology 18 , 23 - 36 .

[19] Fan , X. and Yen , J. , 2004 . Modeling and simulating human teamwork behaviors using intelligent agents . Physics of Life Reviews 1 , 3 , 173 - 201 .

[20] Fan , X. and Yen , J. , 2011 . Modeling Cognitive Loads for Evolving Shared Mental Models in Human-Agent Collaboration . IEEE Transactions on Systems, Man, and Cybernetics , Part B: Cybernetics 41 , 2 , 354 - 367 .

[21] Fisher , D.M. , Bell , S.T. , Dierdorff , E.C. , and Belohlav , J.A. , 2012 . Facet Personality and Surface-Level Diversity as Team Mental Model Antecedents: Implications for Implicit Coordination . J Appl Psychol . 97 , 4 , 825 - 841 .

[22] Givney , S. , Smeaton , A. , and Lee , H. , 2009 . The Effect of Personality on Collaborative Task Performance and Interaction . In Collaborative Computing: Networking, Applications and Worksharing, E. Bertino and J.D. Joshi Eds. Springer Berlin Heidelberg, 499 - 511 .

[23] Goldberg , L.R. , 1990 . An Alternative Description of Personality: The Big-Five Factor Structure . Journal of Personality and Social Psychology 59 , 1216 - 1229 .

[24] Goldberg , L.R. , Johnson, J.A. , Eber , H.W. , Hogan , R. , Ashton , M.C. , Cloninger , C.R. , and Gough , H.G. , 2006 . The International Personality Item Pool and the Future of PublicDomain Personality Measures . Journal of Research in Personality 40 , 1 , 84 - 96 .

[25] Gorla , N. and Lam , Y.W. , 2004 . Who Should Work with Whom?: Building Effective Software Project Teams . Commun. ACM 47 , 6 , 79 - 82 .

[26] Gosling , S.D. , Rentfrow , P.J. , and Swann , W.B., 2003 . A Very Brief Measure of the Big-Five Personality Domains . Journal of Research in Personality 37 , 6 , 504 - 528 .

[27] Hanna , N. and Richards , D. , 2014 . The Impact of Communication on a Human-Agent Shared Mental Model and Team Performance . In Proceedings of the the 13th international conference on Autonomous agents and multiagent systems (AAMAS'14) (Paris, France), 1485 - 1486 .

[28] Hanna , N. , Richards , D. , and Hitchens , M. , 2013 . Evaluating the Impact of the Human-Agent Teamwork Communication Model (HAT-CoM) on the Development of a Shared Mental Model . In PRIMA 2013: Principles and Practice of MultiAgent Systems , G. Boella, E. Elkind , B. Savarimuthu , F. Dignum and M. Purvis Eds. Springer Berlin Heidelberg, 453 - 460 .

[29] Isbister , K. and Nass , C. , 2000 . Consistency of Personality in Interactive Characters: Verbal Cues, Non-Verbal Cues, and User Characteristics . International Journal of HumanComputer Studies 53 , 2 , 251 - 267 .

[30] Kang , S.-H. , Gratch , J. , Wang , N. , and Watt , J. , 2008 . Agreeable People Like Agreeable Virtual Humans . In Intelligent Virtual Agents, H. Prendinger , J. Lester and M. Ishizuka Eds. Springer Berlin Heidelberg, 253 - 261 .

[31] Kieft , I. , Jonker , C. , and Riemsdijk , M.B. , 2011 . Explaining Negotiation: Obtaining a Shared Mental Model of Preferences . In Modern Approaches in Applied Intelligence, K. Mehrotra , C. Mohan , J. Oh , P. Varshney and M. Ali Eds. Springer Berlin Heidelberg, 120 - 129 .

[32] Krishnan , V. , Foster , A. , Kopper , R. , and Lok , B. , 2012 . Virtual Human Personality Masks: A Human Computation Approach to Modeling Verbal Personalities in Virtual Humans . In Intelligent Virtual Agents, Y. Nakano , M. Neff , A. Paiva and M. Walker Eds. Springer Berlin Heidelberg, 146 - 152 .

[33] LePine , J.A. , Buckman , B.R. , Crawford , E.R. , and Methot , J.R. , 2011 . A review of research on personality in teams: Accounting for pathways spanning levels of theory and analysis . Human Resource Management Review 21 , 311 - 330 .

[34] Lim , B. and Klein , K. , 2006 . Team Mental Models and Team Performance: A Field Study of the Effects of Team Mental Model Similarity and Accuracy . Journal of Organizational Behavior 27 , 4 , 403 - 418 .

[35] Loyall , A.B. and Bates , J. , 1997 . Personality-rich Believable Agents that Use Language . In Proceedings of the Proceedings of the first international conference on Autonomous agents (Marina del Rey , California, USA), ACM, 267681 , 106 - 113 .

[36] Luse , A. , McElroy , J.C. , Townsend , A.M. , and DeMarie, S., 2013 . Personality and Cognitive Style as Predictors of Preference for Working in Virtual Teams . Computers in Human Behavior 29 , 4 , 1825 - 1832 .

[37] Mann , R.D. , 1959 . A Review of the Relationships Between Personality and Performance in Small Groups . Psychological Bulletin 56 , 241 - 270 .

[38] Mathieu , J.E. , Heffner , T.S. , Goodwin , G.F. , CannonBowers , J.A. , and Salas , E. , 2005 . Scaling the Quality of Teammates' Mental Models: Equifinality and Normative Comparisons . Journal of Organizational Behavior 26 , 37 - 56 .

[39] Mathieu , J.E. , Heffner , T.S. , Goodwin , G.F. , Salas , E. , and Cannon-Bowers , J.A. , 2000 . The Influence of Shared Mental Models on Team Process and Performance . Journal of Applied Psychology 85 , 2 , 273 - 283 .

[40] McCrae , R.R. and John, O.P. , 1992 . An Introduction to the Five-Factor Model and Its Applications . Journal of Personality 60 , 2 , 175 - 215 .

[41] McRorie , M. , Sneddon , I. , McKeown , G. , Bevacqua , E., de Sevin , E., and Pelachaud , C. , 2012 . Evaluation of Four Designed Virtual Agent Personalities . IEEE Transactions on Affective Computing 3 , 3 , 311 - 322 .

[42] Nass , C. and Lee , K.M. , 2000 . Does computer-generated speech manifest personality An experimental test of similarity-attraction . In CHI '00: Proceedings of the SIGCHI conference on Human factors in computing systems ACM , NY, USA, 329 - 336 .

[43] Nass , C. and Moon , Y. , 2000 . Machines and mindlessness: Social responses to computers . Journal of Social Issues 56 , 1 , 81 - 103 .

[44] Neff , M. , Toothman , N. , Bowmani , R. , Fox Tree , J. , and Walker , M. , 2011 . Don't Scratch! Self-adaptors Reflect Emotional Stability . In Intelligent Virtual Agents, H. Vilhjálmsson , S. Kopp , S. Marsella and K. Thórisson Eds. Springer Berlin / Heidelberg, 398- 411 .

[45] Neff , M. , Wang , Y. , Abbott , R. , and Walker , M. , 2010 . Evaluating the effect of gesture and language on personality perception in conversational agents . In Proceedings of the the 10th international conference on Intelligent virtual agents (IVA'10) (Philadelphia, PA), Springer-Verlag, 222 - 235 .

[46] Neuman , G.A. and Wright , J. , 1999 . Team Effectiveness: Beyond Skills and Cognitive Ability . Journal of Applied Psychology 84 , 3 , 376 - 389 .

[47] Prada , R. and Paiva , A. , 2009 . Teaming up Humans with Autonomous Synthetic Characters . Artificial Intelligence 173 , 1 , 80 - 103 .

[48] Prada , R. and Paiva , A. , 2014 . Human-Agent Interaction: Challenges for Bringing Humans and Agents Together . In Third International Workshop of Human-Agent Interaction Design and Models (HAIDM '14)@AAMAS2014 , Paris, France.

[49] Reeves , B. and Nass , C. , 1996 . The Media Equation: How People Treat Computers, Television, and New Media Like Real People and Places . Cambridge University Press, New York.

[50] Roque , A. and Traum , D. , 2009 . Improving a Virtual Human Using a Model of Degrees of Grounding . In Proceedings of the proceedings of International Joint Conerence on Artificial Intelligence (IJCAI-09) (Pasadena, CA).

[51] Scherer , K.R. , 1978 . Personality Inference from Voice Quality: The Loud Voice of Extroversion . European Journal of Social Psychology 8 , 467 - 487 .

[52] Scherer , K.R. , 1979 . Personality Cambridge University Press, London.

[53] Traum , D.R. and Allen , J.F. , 1992 . A Speech Acts Approach to Grounding in Conversation . In Proceedings 2nd International Conference on Spoken Language Processing (ICSLP '92) , 137 - 140 .

[54] von der Pütten , A. , Krämer , N. , and Gratch , J. , 2010 . How Our Personality Shapes Our Interactions with Virtual Characters - Implications for Research and Development . In Intelligent Virtual Agents, J. Allbeck , N. Badler , T.

Bickmore , C. Pelachaud and A . Safonova Eds. Springer Berlin Heidelberg, 208 - 221 .

Decision Makings . Decision support systems, special issue on intelligence and security informatics 41 , 3 , 634 - 653 .

[55] Yee , N. , Harris , H. , Jabon , M. , and Bailenson , J.N. , 2011 . The Expression of Personality in Virtual Worlds . Social Psychological and Personality Science 2 , 1 , 5 - 12 .

[56] Yen , J. , Fan , X. , Sun , S. , Hanratty , T. , and Dumer , J. , 2006 . Agents with Shared Mental Models for Enhancing Team

[57] Yen , J. , Xiaocong , F. , Shuang , S. , McNeese , M. , and Hall , D. , 2004 . Supporting Anti-Terrorist Analyst Teams Using Agents with Shared RPD Process . In Computational Intelligence for Homeland Security and Personal Safety , 2004 . CIHSPS 2004 . Proceedings of the 2004 IEEE International Conference on , 53 - 60 .