=Paper=
{{Paper
|id=Vol-1569/paper4
|storemode=property
|title=Do Birds of a Feather Work Better Together? The Impact
of Virtual Agent Personality on a Shared Mental Model
with Humans during Collaboration
|pdfUrl=https://ceur-ws.org/Vol-1569/paper4.pdf
|volume=Vol-1569
|authors=Nader Hanna,Deborah Richards
|dblpUrl=https://dblp.org/rec/conf/atal/HannaR15a
}}
==Do Birds of a Feather Work Better Together? The Impact
of Virtual Agent Personality on a Shared Mental Model
with Humans during Collaboration==
https://ceur-ws.org/Vol-1569/paper4.pdf
Do Birds of a Feather Work Better Together? The Impact
of Virtual Agent Personality on a Shared Mental Model
with Humans during Collaboration
Nader Hanna Deborah Richards
Computing Department, Macquarie University Computing Department, Macquarie University
NSW 2109, Australia NSW 2109, Australia
+61(0)2 9850 9092 +61(0)2 9850 9567
nader.hanna@mq.edu.au deborah.richards@mq.edu.au
ABSTRACT teams. Cohen et al. [14] stressed the importance of having shared
The development of a Shared Mental Model (SMM) between objectives and mental state or mental model between team
team members and effective communication of the shared members. A Shared Mental Model (SMM) is the state among
knowledge have been found to improve teamwork performance. team members where the members have overlapping knowledge
In human-IVA heterogeneous teams, the communication required and beliefs. SMM was introduced by Cannon-bowers et al. [12] in
to develop a SMM is difficult as each party belongs to different the context of teamwork amongst humans. Most research into
worlds (i.e. real and virtual). Moreover, humans may differ in how SMMs concern human-human teamwork and communication
they produce and perceive communication acts according to their (e.g., [18]). Some research exists that considers a SMM in the
personality traits. The influence of IVA personality, exhibited via context of agent-agent teamwork. Later, it became apparent that
verbal and non-verbal communication, on collaboration and SMM is not only important in human teams, but also in human-
development of a SMM within a human-IVA team has not been agent teams [31]. Many researchers who have been studying
previously studied. In this paper, we explore the impact of IVA’s SMM classified the shared knowledge into two categories:
with two different combinations of personality traits, i.e. knowledge about the team and knowledge about the task [12].
extraversion and agreeableness, on the development of a SMM SMM concepts resemble Traum’s use of grounding models [53]
with human teammates. Additionally, this study investigated the or mutual beliefs between humans and an IVA. Traum’s work
influence of the match in the two personality traits between IVAs focused on studying a human’s dialogue and creating a
and humans on the development of a SMM. The results showed conversation system that mimics human verbal communication to
that agreeable IVAs positively impacted on the development of establish mutual understanding with a conversational virtual
taskwork and teamwork SMMs; whereas extraversion did not human [50]. However, collaborative activities need more than
influence development of the SMM. Moreover, when grounding based only on verbal conversation.
collaborating humans and IVAs had matching agreeableness Many aspects relating to the development of a human-IVA SMM
personality traits there was a positive influence on the SMM are understudied. In particular, we note the lack of human-agent
between them and better performance outcomes. studies that explore whether the personalities of the human and/or
IVA have an impact on their teamwork and the establishment of a
Categories and Subject Descriptors SMM. Integrating personality into agents is not a new research
I.2.11 [Artificial Intelligence]: Distributed Artificial topic. Twenty years ago, Loyall and Bates introduced an agent
Intelligence– intelligent agents, multiagent systems. with personality that communicated through bubble text [35].
Later, several studies have been carried out to explore the
Keywords influence of IVAs with personality traits on the interaction with
Personality Traits, FFM, Extraversion, Agreeableness, Shared humans. This interaction was either social, behavioural [13],
Mental Modal, Human-Agent Teamwork. emotional or cognitive. For instance, Parada and Paiva [47]
developed an agent model to support group dynamics of
1. INTRODUCTION autonomous synthetic characters (Synthetic Group Dynamics
Teamwork refers to a group of entities that use their knowledge mode or SGD mode) based on two personality traits, extraversion
and capabilities in an organized way towards achieving a shared and agreeableness.
goal that would not be carried out effectively with the effort of a
single entity [28]. Other definitions for ‘team’ exist [15]. A Given the importance of a SMM for human teams and influence
number of studies have targeted human teams to understand the of personality in human teams, we address this current gap by
processes that enhance collaboration within teams. Studies have conducting a study to investigate the effect of the combined
indicated that effective teamwork often relies on the acquisition of human and IVA personalities on the development of a SMM (i.e.
a shared mental model [2]. While most research on teamwork shared understanding of the task and the team).
concerns human teams, some early research studying human
This paper proceeds as follows. Section 2 provides an overview of
interactions with computers has provided evidence that people
related research. In Section 3, we briefly describe the
treat computers like they would treat real people [43]. In later
measurement and communication of personality traits, followed
studies, researchers began to transfer these findings to human
by our research questions in Section 4. A description of our
interaction with Intelligent Virtual Agent (IVAs).
experimental methodology is given in Section 5. The results are
Due to the increasing interest in heterogeneous teams and the presented in Sections followed by discussion in Section 7. Finally,
challenges in human-agent teamwork coordination [10] [48], the conclusion and future work appear in Section 8.
researchers have explored several factors that may influence these
2. RELATED RESEARCH perception, regardless of the implementation of personality within
A review of the literature has identified two classes of related the virtual agent. The results in [54] showed that participants’
work. First, studies that considered the establishment of a SMM personality traits influenced their subjective feelings after the
between humans and IVAs. Second, studies that explored the interaction, as well as their evaluation of the virtual character and
influence of humans’ or IVAs’ personality traits on human-IVA their actual behavior. Du and Huhns [17] studied whether human
interaction. No related work was found that combined both behaviour towards other humans and agents is related to their
classes. personality types. Although this study used a different personality
test, the results showed that humans of different personality types
In the first class of studies, Yen and Fan’s (e.g., [56], [20]) agents behave differently towards other humans and agents.
were designed to use SMM knowledge of the task to communicate
information with other agents in a team. However, this work 3. MEASURING AND COMMUNICATING
focused on a team of agents. Fan and Yen also reported a survey PERSONALITY TRAITS
[19] of research that studied SMM between humans and agents. A Among the measures of personality traits, the FFM of personality
noteworthy study in this survey was R-CAST agents [57] that has proven to be a robust tool for understanding personality
share with their human team members the decision-making variations across individuals. FFM [23] claims that personality
process and their dynamic progress. Hanna and Richards [28] differs on five factors: Openness, Conscientiousness,
studied the impact of a proposed multimodal communication Extraversion, Agreeableness and Neuroticism. Openness means
model (called HAT-CoM) between a human and her IVA being open to experience new things, being imaginative, and
teammate on the establishment of a SMM. This study investigated intelligent. Conscientiousness indicates responsibility, reliability
the influence of communication on different outcomes of SMM and tidiness. An extravert is outgoing, sociable, assertive and
such as anticipating a teammate’s decisions, reduced explicit energetic. Agreeableness means a person is trustworthy, kind and
communication, match in cognitive perspective, and competence cooperative by considering others’ goals. A neurotic character is
in decision-making. Nevertheless, this study did not explore the anxious, nervous, prone to depression and lacks emotional
effect of personality of both humans and IVAs on the human’s stability.
perception of the proposed communication model.
Studies that have explored personality traits and teamwork stress
The body of work in the second class of studies concerning IVAs the role of both extraversion and agreeableness to foster inter-
and personality is more extensive. Luse et al. [36] found that the relationships between team members. Extraversion and
human’s personality influenced the humans’ preferences to work agreeableness were selected in our study because they have been
in teams. A number of studies have investigated team member shown to be predominant traits in collaboration and teamwork [8].
personality as a predictor of both team processes and outcomes The extraversion trait affects interpersonal relations through the
[6]. A number of researchers studied the influence of personality quality of social interactions [7] [40]. Extraverts are usually active
traits on human decision-making while achieving a task. For members in teamwork interactions and often popular among their
example, Schmitt et al. [60] asked the human subjects to play the teammates [37].
ultimatum game. In this game, two players had to reach an
agreement about how to divide money through proposing and Personality is communicated verbally as our personality is likely
responding. This work used Myer-Briggs Temperament Index to influence how we speak [52]. Speaking style can reveal certain
(MBTI) test to get personality traits of players. The results personality traits; some traits are easier to detect than others [51].
showed that extravert players indicated a willingness to accept A number of studies have used verbal capabilities to represent
lower offers than introvert players did. different IVA personalities [32]. Neff et al. [45] determined a
number of aspects that demonstrate the impact of the IVA’s
In a study to determine what combinations of personalities extravert personality on the IVA’s verbal behaviour.
resulted in the best-performing teams, Gorla and Lam[25]
surveyed 92 employees from 20 small software development Additionally, IVA personality may be communicated non-
teams. The results showed that heterogeneity among team verbally through the IVA’s physical position relative to the
members had no significant effect on team performance. In human’s view or their avatar. Argyle’s [3] status and affiliation
another study [4], the performance of sixty three (63) virtual model for animating non-verbal behavior of virtual agents
human teams was studied with respect to extraversion personality identified two fundamental dimensions for non-verbal behavior:
traits. Extraversion was found to be an important trait to promote affiliation and status. Affiliation can be characterized as wanting a
group interaction and teams with lower variance in extraversion close relationship and it is associated with non-verbal clues such
levels did better. as close physical position. Other studies (e.g., [11]) suggest that
agents approaching the subject’s avatar were judged as more
Isbister and Nass [29] studied the effect of consistency in extraverted than agents not approaching them, regardless of smile
representing personality via an IVA’s verbal and non-verbal and the amount of gaze they gave.
communication and human preferences. In addition, human
preferences for IVAs with personalities that matched their own 4. RESEARCH QUESTIONS
personality was investigated. The results showed that humans To build upon and draw this literature together and potentially
prefer the personality of an IVA to be consistent in both verbal enhance human-IVA teamwork, the following research questions
and non-verbal communication. Moreover, the results indicated aim to investigate the relationship between the IVA’s personality
that participants tended to prefer a character whose personality and the development of a SMM between the human team member
was complementary, rather than similar, to their own. Kang et al. and the IVA:
[30] explored associations between the Five Factor Model (FFM) 1. Are the IVA’s personality traits, i.e. extraversion and
(see next section) personality traits of human subjects and their agreeableness, as presented in its verbal and non-verbal
feelings of rapport when they interacted with a virtual agent. The communication, significantly differentiated by humans?
results showed that users’ personality traits affect users’
2. Do the IVA’s personality traits, i.e. extraversion and IVA concerning the best way to overcome the obstacle and to
agreeableness, significantly influence the humans’ perception select which pair of tools is most suitable for the task. Human-
of the taskwork and/or the teamwork SMM? IVA interaction (i.e. communication) during the collaborative
3. Does a match in human-IVA personality traits, i.e. activity is described below as part of the experimental design.
extraversion and agreeableness, influence the humans’
perception of the taskwork and/or the teamwork SMMs with 5.3 Experimental Design
the IVA? To answer the research questions, an experiment was conducted.
4. Do taskwork or teamwork SMMs affect human-IVA team The experiment consisted of five different treatments with the
performance? same virtual scenario but the IVA, i.e. Charlie, had different
personalities. One treatment was a control with a neutral
5. METHODOLOGY personality IVA. The aim of the control treatment was to measure
An experiment was conducted to answer the four research whether inclusion of IVA personality made a difference and to
questions. The participants, design, procedure and collaborative allow comparison with the other experimental treatments. The
scenario (the materials) are described below. other four experimental treatments had the four combinations of
the two studied personality traits, i.e. extraversion and
5.1 Participants agreeableness. The four combinations were extraversion-
Fifty-five (55) second-year undergraduate science students agreeableness, extraversion-antagonism, introversion-
enrolled in a biology unit completed the collaborative task. agreeableness and introversion-antagonism.
Participants were aged between 18 and 51 years (mean=22.56;
SD=6.95) Fifty-two participants were native English speakers; the The two studied personality traits were incorporated into the
remaining three participants had been speaking English on a daily IVA’s verbal and non-verbal communication. To express the
basis on average for 13 years. On a scale with 6 levels (level 1 the personality traits in the IVA’s verbal communication, the
least experienced and level 6 the highest experience), 13 had basic literature was reviewed to find out what verbal aspects were
(level 2), 39 had proficient (level 5) and 3 had advanced (level 6) affected the most by personality. Among the list of aspects
computer skills. Participants played computer games on average mentioned in the work of Neff et al. [45], we selected the
2.73 times a week, with a standard deviation of 4.69. dominant aspects as the basis of the design of the IVA in our
study. Verbal messages were designed and reviewed by the
5.2 Collaborative Scenario authors according to the criteria in [45] and as shown in Table 1.
The collaborative scenario was implemented using the Unity3D A number of studies addressed how the extraversion personality
game engine (www.unity3d.com). The scenario included a task trait can be represented in an IVA’s non-verbal signaling. As
where both a human user and an IVA, named Charlie, have to verbal behaviours have already been identified that show an
collaborate to achieve a shared goal. The goal is to pass a IVA’s personality, Doce et al. [16] proposed several non-verbal
sequence of four obstacles to reach their target (scientific features that could be used to show personality traits in IVA, these
laboratory). The four obstacles included a brick wall, wooden features include: spatial extent, temporal extent, fluidity, power
gate, bush and hill (see Figure 1). In order to get over each of and repetitiveness. To design the non-verbal communication of
these obstacles both the human and IVA have to select a pair of the IVA, we chose the dominant features, shown in Table 2.
tools from a toolbox that contains 12 tools (pruning shears, bush
hook, hammer, chisel, ladder, rope, matchsticks, matchbox, 5.4 Data Collection and Data Analysis
screwdriver, nipper, shovel and mattock). These tools were picked The following variables were measured to answer the research
so that each pair of tools would be complementary, i.e. a single questions:
tool cannot work without the function of the complementary tool.
For example, the chisel needs the hammer and matchstick needs Participant’s personality: participants completed a 7-item
the matchbox. In addition, each obstacle could be passed using a personality test to measure the two personality traits using
different method and the corresponding combination of tools. For International Personality Item Pool (IPIP) [24].
example, the bush obstacle could be chopped, burnt or climbed. IVA’s personality: participants completed a test of the
Hence, there should be agreement between the human and the perceived personality of the IVA by answering four items of
the Ten-Item Personality Inventory (TIPI) [26].
IVA’s verbal and non-verbal communication: participants
answered ten items. Five items measured the IVA’s verbal
communication (e.g., “Charlie’s requests and replies were
helpful to complete the task”) and five items measured the
IVA’s non-verbal communication (e.g., “Charlie’s actions
were suitable to the situation”).
Taskwork and teamwork SMM: Participants answered ten
items in a survey, similar to other studies measuring SMM
(e.g., [28]). Five items measured human perception of
taskwork SMM (e.g., “Charlie and I have a shared
understanding about how best to ensure we meet our goal”).
Five items measured teamwork SMM (e.g., “Charlie and I
Value collaborating with each other”).
Team performance: participants answered five items to
Figure 1. Snapshots from the scenario, the first obstacle and measure their perception of team performance with the IVA.
IVA personality is low extrovert and low agreeableness.
Table 1. Verbal aspects used to express introversion/extraversion in IVA’s behaviour
Parameter Description Introvert Extravert
Verbosity Control the number of propositions in the utterance low high
Restatements Paraphrase an existing proposition low high
Request confirmation Begin the utterance with a confirmation of the propositions low high
Emphasizer hedges Insert syntactic elements (really, basically, actually, just) to strengthen a proposition low high
Negation Negate a verb by replacing its modifier by its antonym high low
Filled pauses Insert syntactic elements expressing hesitancy high low
Table 2. Non-verbal aspects used to express introversion/extraversion in IVA’s behaviour
Parameter Description Introvert Extravert
Spatial extent amount of space required to perform an expression low high
Temporal extent amount of time spent to perform an expression long short
Repetitivity repetition of certain movements low high
Body position close physical postures far close
Both personality tests, i.e. IPIP and TIPI, and the communication 6. RESULTS
and SMM questions used a 5-item Likert Scale, where 1 First, the study variables were tested for the normality distribution
corresponded to “Strongly Disagree” and 5 to “Strongly Agree”. in order to determine whether to use parametric or nonparametric
In addition to these subjective measures, all inputs from the user tests. Tests for Skewness and Kurtosis showed that the z-value of
were logged to allow recreation of navigation paths and record the variables are in the span -1.96 and +1.96, and thus they do not
inputs such as responses and selected tools. These inputs included differ significantly from normality. A Shapiro-Wilk normality test
selected regions in the scenario. Analysis of interaction logs to showed that, except for verbal communication variable, all the
find the most frequently triggered stimuli in the scenario was used other variables had p-values less than 0.05. Based on the results of
before in other studies [32]. Skewness and Kurtosis as well as Shapiro-Wilk we concluded that
the four variables (non-verbal communication, taskwork SMM,
The statistical package IBM SPSS v.20 was used for the statistical teamwork SMM, and team performance) are not normally
analysis. A number of tests for normality distribution of the study distributed.
variables were run to determine whether to use parametric or
nonparametric tests. Shapiro-Wilk normality test as well as To measure the strength and direction of association between the
Skewness and Kurtosis were used to test normality distribution of five variables, Spearman’s rho correlation method was used.
the study variables. Spearman’s rho Correlation analysis was used Spearman’s rho correlation was selected, as it is more appropriate
to quantify the degree and the direction to which the study for non-normally distributed responses. To estimate how well the
variables are related. To measure the difference between the set of items measure each variable, Cronbach’s Alpha (ɑ) was
different experimental treatments, one-way ANOVA test and used to measure the internal consistency or reliability of these
Kruskal Wallis test were utilized. Regression analysis was utilized items. The value of Cronbach’s Alpha (ɑ) may lie between
to learn more about the relationship between an independent or negative infinity and one. However, only positive values of α
predictor variable and a dependent or criterion variable. make sense. Generally, Cronbach’s alpha (ɑ) coefficient ranges in
value from zero to one and may be used to describe the reliability
5.5 Procedure of factors. Some statisticians insist on a reliability score of 0.70 or
Each participant was randomly assigned to one of the five higher in order to assess the studied items are internally
treatments by a web-based system containing all five treatments. consistent. Table 3 shows that Cronbach’s Alpha (ɑ) for the five
Participants used the virtual system individually so that the variables are over 0.70. We concluded that the survey items to
collaboration would be one-to-one between him/herself and the measure each variable are reliable to measure this variable.
agent. We allocated twenty minutes for the study that consisted of Table 3 presents the means, standard deviations, and correlations
five parts in one session requiring the participant to: for all the variables. Verbal and nonverbal communication were
Part 1: Sign consent form and complete biographical information. significantly positively related to taskwork SMM (r=0.461,
Part 2: Take a personality test to measure their own personality. p<0.01 and r=0.351, p<0.01 respectively) suggesting a positive
Part 3: Participate in the scenario in the 3D virtual scene. In the association between both verbal and non-verbal communication
beginning of the scenario, the participants were provided during a collaborative task on developing common understanding
with online instructions about the goal of the virtual of the taskwork. Moreover, team verbal and nonverbal
scenario, the name and the use of each tool in the toolbox communication were significantly positively related to teamwork
and how to select/close the verbal messages. SMM (r=0.465, p<0.01 and r=0.308, p<0.05, respectively)
Part 4: Complete a survey that measures the participant's suggesting a positive association between both verbal and non-
perception of some study variables. verbal communication during a collaborative task on developing
Part 5: Answer a short personality test about the assigned IVA. common understanding of the teamwork. Taskwork SMM was
significantly positively correlated to teamwork SMM (r= 0.704,
p<0.01) suggesting that human-IVA teams whose members share
similar taskwork mental models are likely to have shared
Table 3. Spearman's rho inter-correlations among variables
Cronbach’s
M SD 1 2 3 4 5
Alpha
1.000
1. Verbal Communication 0.852 3.88 0.62
.
0.670** 1.000
2. Non-Verbal Communication 0.760 3.96 0.51
0.000 .
0.461** 0.351** 1.000
3. Taskwork SMM 0.793 3.57 0.68
0.000 0.009 .
0.465** 0.308* 0.704** 1.000
4. Teamwork SMM 0.849 3.42 0.75
0.000 0.022 0.000 .
0.368** 0.286* 0.569** 0.489** 1.000
5. Team Performance 0.730 3.70 0.51
0.006 0.034 0.000 0.000 .
** Correlation is significant at the 0.01 level (2-tailed). * Correlation is significant at the 0.05 level (2-tailed).
teamwork mental models as well. As expected, both taskwork and perception of different IVA personality, i.e. agreeableness,
teamwork SMM were significantly positively correlated to antagonism or neutral IVA, because of the IVAs non-verbal
human-IVA team performance (r=0.569, p<0.01 and r=0.489, messages.
p<0.01 respectively). This result suggests a positive association
between taskwork and teamwork SMMs and overall team 6.2 Does the IVA’s Personality Influence the
performance. The following subsections analyse the data related Development of a SMM?
to the three research questions. The second research question inquired whether the participants’
perception of the IVA’s personality traits, i.e. extraversion and
6.1 Can Humans Recognize the IVA’s agreeableness, influenced their perception of the SMMs for
taskwork and teamwork. The results did not show any significant
Personality? difference between the perception of either taskwork or teamwork
The first research question inquired if there were significant SMM according to the IVA’s extraversion personality.
differences between the five groups of participants in perceiving
the IVA’s two implemented personality traits. This question was The results of ANOVA test showed that there was a significant
segmented into two sub-questions. difference p<0.01 [F(2, 52) = 4.312, p<0.01, η2=0.14] between the
groups of participants in their perception to taskwork SMM
The first sub-question asked if the IVA’s introvert/extravert according to the IVA’s agreeableness/antagonism personality
personality trait as presented in the IVA’s verbal and non-verbal trait. This result was supported by the outcome of the non-
communication is perceived differently by the human participants. parametric Kruskal-Wallis H test (H=6.725, df=2, n=55,
The results of one-way ANOVA showed that there was a p<0.035). To understand the which condition/s accounted for the
significant difference p<0.01 [F(2, 52) = 15.014, p < 0.01, significant difference in taskwork SMM, post hoc comparisons
η2=0.37] between the groups of participants in their perception of using the Tukey HSD and Bonferroni tests indicated that the mean
the personality of IVA, i.e. introvert, extravert or neutral IVA, score for the antagonistic IVA condition (M = 3.36, SD = 0.64)
because of the verbal messages of the IVA. In addition, the results was significantly different than the agreeable IVA condition (M =
of one-way ANOVA showed that there was a significant 3.88, SD = 0.51) at p < 0.05. The latter was significantly different
difference p<0.01 [F(2, 52) = 11.424, p < 0.01, η2=0.31] between from neutral IVA (M = 3.36, SD = 0.87) at p < 0.05. However, the
the groups of participants in their perception of different antagonism condition did not significantly differ from the neutral
personality of IVA, i.e. introvert, extravert or neutral IVA, condition. The results of ANOVA test showed that there was a
because of the non-verbal messages of the IVA. significant difference p<0.01 [F(2, 52) = 6.942, p<0.01, η 2=0.21]
The second sub-question asked if the IVA’s between the groups of participants in their perception of
agreeableness/antagonism personality trait as presented in the teamwork SMM according to the IVA’s
IVA’s verbal and non-verbal communication is perceived agreeableness/antagonism personality trait. The significance was
differently by the human participants. The results of one-way also identified by non-parametric Kruskal-Wallis test (H=10.634,
ANOVA showed that there was a significant difference p<0.01 df=2, n=55, p<0.005). The results of post hoc comparisons
[F(2, 52) = 6.086, p < 0.01, η2=0.19] between the groups of indicated that the mean score for the antagonistic IVA condition
participants in their perception of the IVA’s personality, i.e. (M = 3.11, SD = 0.80) was significantly different from the
agreeableness, antagonism or neutral, because of the verbal agreeable IVA condition (M = 3.84, SD = 0.55) at p < 0.05. The
messages of the IVA. In addition, the results of one-way ANOVA later was significantly different from neutral IVA (M = 3.20, SD =
showed that there was a significant difference p<0.05 [F(2, 52) = 0.63) at p < 0.05. However, the antagonism condition did not
3.90, p <0.05, η2=0.13] between the groups of participants in their significantly differ from the neutral condition.
6.3 Does a Match in Human-IVA Personality 6.4 Do Taskwork and Teamwork SMMs
Influence the Perception of a SMM? Affect Human-IVA Team Performance?
The third research question inquired whether the match in The last research question aimed to investigate the influence of
personality traits, i.e. extraversion and agreeableness, between the taskwork and teamwork SMMs on human-IVA team performance.
participants and the IVAs significantly influenced their perception The result of multiple regression showed that both taskwork and
of the SMM. The results did not show any significant difference teamwork SMM would predict the overall team performance to
between in the perception of either the taskwork or teamwork achieve the common goal. The results show that 30.9% of the
SMM according to the match in extraversion personality between variance in team performance can be accounted for by taskwork
the human and the IVA teammate. and teamwork SMM between the human and IVA. To assess the
overall statistical significance of the model, the results shows that
The results of ANOVA test showed that there was a significant both predictors were significant R2= 0.309, F (2, 52) = 13.068,
difference p<0.05 [F(2, 52) = 5.224, p<0.05, η2=0.09] in the p<0.001. Thus, we can say that the existence of taskwork and
perception of a taskwork SMM between the participants who had teamwork SMMs do impact on human-IVA team performance in
a match in the agreeableness personality (M= 3.80, SD= 0.48) answer to the third research question.
with the IVA and those who were in mismatch with the IVA (M=
3.40, SD= 0.75). Moreover, the results showed that there was a Moreover, to evaluate which one of the two factors, i.e. taskwork
significant difference p<0.05 [F(2, 52) = 6.199, p<0.05, η2=0.105] or teamwork SMM, contributes more to team performance, the
in the perception of a teamwork SMM between the participants results, as shown in Table 4, indicated that standardized
who had a match in the agreeableness personality (M= 3.70, SD= coefficient of taskwork SMM (0.434) is greater than
0.56) with the IVA and those who were in mismatch with the IVA standardized coefficient of teamwork SMM (0.170), showing a
(M= 3.21, SD= 0.80). The results of ANOVA test was supported stronger effect for taskwork over teamwork SMM.
by the non-parametric Mann-Whitney test based on ranking. The
results of Mann-Whitney U test indicated that the participants 7. DISCUSSION
who matched with the IVA in agreeableness personality were This study aimed to investigate the influence of an IVAs’
significantly higher than mismatched group in perceiving personality as represented in its multimodal communication, i.e.
taskwork SMM (U=260.5, n=55, p<0.05) and teamwork SMM verbal and non-verbal, on the human’s perception of the SMM
(U= 232.5, n=55, p<0.05). with the IVA. To reach this aim, four research questions were
proposed. The first research question inquired if there was a
To evaluate whether the match in agreeableness personality trait
significant difference between the five treatment groups of
between the humans and the IVAs in a collaboration context could
participants in their perception of the IVA’s two implemented
be a predictor of the humans’ perception of both taskwork and
personality traits, i.e. extraversion and agreeableness. Data
teamwork SMM, a linear regression test was used. The results
analysis revealed that at a statistically significant level participants
show, see Table 4, that the match in agreeableness personality
identified the multimodal communication, verbal and non-verbal
between the human and the IVA was a significant predictor of
communication, of the extravert IVA as more extravert than the
taskwork SMM, R2= 0.073, F (2, 52) =5.224, p<0.05.
introvert IVA. Moreover, the results showed that the participants
Furthermore, we investigated if the match in agreeableness
recognized the multimodal communication of the agreeable IVA
personality trait between the humans and the IVAs in a
as more agreeable than the antagonist IVA.
collaboration context could be a predictor of the humans’
perception of teamwork SMM. The results, as can be seen in Other researchers have also studied the influence of personality
Table 4, showed that the match in agreeableness personality was a traits on human-agent interaction. Similar to our study, these
significant predictor of teamwork SMM, R2= 0.088, F (2, 52) studies have concentrated on the incorporation of personality traits
=6.199, p<0.05. This result suggests that matching human-IVA in an agent [1] and/or whether the human could identify the
agreeable personalities is likely to be a significant predictor of the agent’s personality [41]. For instance, Isbister and Nass [29]
human’s perception of both taskwork and teamwork SMMs. reported that their participants found extraverted IVAs
Table 4. Regression of taskwork and teamwork SMM on agreeableness match
Unstandardized Coefficients Standardized
Model Coefficients R Adjusted R2 F Sig.
Std. Error of the
Unstandardized B
Estimation
Taskwork SMM
Agreeableness Match 0.409 0.179 0.300 0.300 0.073 5.224 0.026*
Teamwork SMM
Agreeableness Match 0.486 0.195 0.324 0.324 0.088 6.199 0.016*
Team performance
Taskwork SMM 0.323 0.139 0.434
0.578 0.309 13.068 0.000*
Teamwork SMM 0.115 0.126 0.170
* Significance level p<0.05
significantly more extraverted than the introverted IVAs. studies argued that homogeneity in personality traits among team
Numerous studies have considered whether human participants members tends to improve team performance [2]. These
are able to perceive an IVA’s personality through communication contradictory results concerning the role of variation of
with the IVA. Doce et al. [16] presented a model to create an IVA personality in teams may be due to the nature of the task in which
with distinguishable FFM personality traits. Neff et al. exploited the team members are involved.
the extraversion [45] and neuroticism [44] traits of the FFM in
multimodal characters evaluating the effects of verbal and Studies have indicated that users’ own personality traits affect
nonverbal behavior in personality perception studies. Cafaro et al. their behaviour in virtual worlds [55]. In the literature, matching
[11] conducted a study to investigate how IVA’s non-verbal human-IVA personalities have not been studied in association
communication influence the first encounters between humans with SMM and so our results could not be compared with others.
and virtual agents. Nevertheless, our findings are in line with some previous human-
computer interaction literature [49] that indicated that humans
Our study sought to go beyond identification of personality to were more likely to prefer IVAs with similar personality. This
consider the impact of personality on aspects of human-IVA opinion was supported by Nass and Lee [42] who indicated that
teamwork. The result of the second research question showed that people prefer to interact with other individuals who have a similar
the participants who had the agreeable IVA were significantly personality to them; while other work showed that people
more likely to develop both a strong taskwork and teamwork preferred IVAs that were complementary to them [29]. In their
SMM than those who had the antagonistic IVA. Post hoc tests study, Kang et al. [30] investigated the association between FFM
showed the participants who had received the agreeable IVA personality traits of human subjects and their feelings when they
developed significantly greater taskwork and teamwork SMMs interacted with an IVA. Their result indicated that agreeable
than those who had either the antagonistic or the neutral IVA personalities felt strong rapport with an agent that embodies an
treatments. This finding indicated that IVAs with an agreeable agreeable personality.
personality trait tend to develop SMMs with human teammates.
Meanwhile, the results showed that participants who had the Many studies found a positive correlation between the
extravert or introvert IVA treatment did not differ in their development of a SMM between team members and their team
development of taskwork or teamwork SMMs. This finding performance [39]. Although some studies have found the
indicated that an IVA with an extraversion personality is not strongest correlation is between teamwork SMM and team
likely to influence the development of a SMM. performance [34], other studies reported the strongest positive
correlation is between taskwork SMM and team performance [38]
Although the literature of human-agent interaction has not studied [27]. Despite the fact that the influence of either taskwork or
the influence of an IVA’s personality on the perception of SMMs teamwork SMM is likely to rely on the nature of the collaborative
with IVA, some researchers in human teams reported a significant situation, our results support previous findings that taskwork
interaction between the trust facet of agreeableness in predicting a SMM has a stronger effect on team performance.
shared mental model between team members [21]. Barrick et al.
[5] suggested that an agreeable personality may predict working 8. CONCLUSION AND FUTURE WORK
well in teams; although no direct relationship between This paper described a study on whether an IVA’s personality
agreeableness and team performance was found. Neuman and traits influence the establishment of a SMM with a human
Wright [46] concluded that agreeableness between team members teammate. Additionally, this study investigated whether the match
help a group come to a consensus on a SMM. An explanation for between IVAs and humans in personality traits impacts on the
the findings of our study and other studies in human teams could establishment of taskwork and teamwork SMMs. Through an
be that agreeable characters do their best to avoid teamwork experiment, it was found that designing an IVA incorporating
disruptions that might occur if there was interpersonal conflict. personality traits is likely to improve the performance of the
human-IVA team. In addition, these findings indicated that,
While our results did not report a significant impact of similar to human teams, the personality of both the human and the
extraversion on the development of a SMM, some other studies IVA teammate should to be taken into consideration to foster team
found extraversion as a factor that impact teams. Givney et al. productivity.
Givney et al. [22] investigated the influence of personality on
human teams; extraversion was found to impact on tasks that did In future work, the other personality traits need to be studied for
not enforce very short time constraints, while agreeableness was possible influences on human-IVA teamwork. Additionally, IVA
important for tasks where tight collaboration was required. A and/or human emotion should be incorporated to investigate if
study of sixty-three (63) virtual teams found that extraversion was emotions aid human perception of an IVA’s personality and the
an important personality trait to promote group interaction and resultant effect on SMM and team performance.
teams with lower variances in extraversion levels did better [6].
9. REFERENCES
The results of the third research question indicated that there was [1] Allbeck, J. and Badler, N., 2002. Toward Representing
a positive significant association between humans who had a Agent Behaviors Modified by Personality and Emotion. In
match in agreeableness personality trait with IVAs and the Workshop on Embodied Conversational Agents – Let’s
development of both taskwork and teamwork SMM. Nevertheless, specify and evaluate them! AAMAS 2002, Bologna, Italy
the results did not show a significant relationship between a
human-IVA match in extraversion and the development of a [2] Anderson, M.H., 2009. The Role of Group Personality
SMM. Studies in human teams indicated that the composition of Composition in the Emergence of Task And Relationship
members’ personalities influence team interaction and Conflict within Groups. Journal of Management and
performance [9]. However, these studies have not agreed on Organization 15, 82-96.
whether the variation or similarity in personality have a positive
effect on teamwork. Some researchers claimed that variation in [3] Argyle, M., 1988. Bodily Communication. Routledge.
personality is likely to be associated with variant skills [33]. Other
[4] Balthazard, P., Potter, R.E., and Warren, J., 2004. Expertise, [17] Du, H. and Huhns, M.N., 2013. Determining the Effect of
extraversion and group interaction styles as performance Personality Types on Human-Agent Interactions. In
indicators in virtual teams: how do perceptions of IT's Proceedings of the Proceedings of the 2013 IEEE/WIC/ACM
performance get formed? SIGMIS Database 35, 1, 41-64. International Joint Conferences on Web Intelligence (WI)
and Intelligent Agent Technologies (IAT) - Volume 02, IEEE
[5] Barrick, M.R., Mount, M.K., and Judge, T.A., 2001. Computer Society, 2569317, 239-244.
Personality and Performance at the Beginning of the New
Millennium: What Do we Know and Where Do we Go Next. [18] Espevik, R., Johnsen, B.H., Eid, J., and Thayer, J., 2006.
International Journal of Selection and Assessment 9, 9-30. Shared Mental Models and Operational Effectiveness:
Effects on Performance and Team Processes in Submarine
[6] Barrick, M.R., Stewart, G.L., Neubert, M.J., and Mount, Attack Teams. Military Psychology 18, 23-36.
M.K., 1998. Relating Member Ability and Personality to
Work-Team Processes and Team Effectiveness. Journal of [19] Fan, X. and Yen, J., 2004. Modeling and simulating human
Applied Psychology 83, 3, 377-391. teamwork behaviors using intelligent agents. Physics of Life
Reviews 1, 3, 173-201.
[7] Barry, B. and Stewart, G.L., 1997. Composition, Process and
Performance in Self-Managed Groups: The Role of [20] Fan, X. and Yen, J., 2011. Modeling Cognitive Loads for
Personality. Journal of Applied Psychology 82, 1, 62-78. Evolving Shared Mental Models in Human-Agent
Collaboration. IEEE Transactions on Systems, Man, and
[8] Bosch, K., Brandenburgh, A., Muller, T., and Heuvelink, A., Cybernetics, Part B: Cybernetics 41, 2, 354-367.
2012. Characters with Personality! In Intelligent Virtual
Agents, Y. Nakano, M. Neff, A. Paiva and M. Walker Eds. [21] Fisher, D.M., Bell, S.T., Dierdorff, E.C., and Belohlav, J.A.,
Springer Berlin Heidelberg, 426-439. 2012. Facet Personality and Surface-Level Diversity as Team
Mental Model Antecedents: Implications for Implicit
[9] Bradley, B.H., Klotz, A.C., Postlethwaite, B.E., and Brown, Coordination. J Appl Psychol. 97, 4, 825-841.
K.G., 2013. Ready to Rumble: How Team Personality
Composition and Task Conflict Interact to Improve [22] Givney, S., Smeaton, A., and Lee, H., 2009. The Effect of
Performance. Journal of Applied Psychology 98, 385-392. Personality on Collaborative Task Performance and
Interaction. In Collaborative Computing: Networking,
[10] Bradshaw, J.M., Feltovich, P.J., Johnson, M.J., Bunch, L., Applications and Worksharing, E. Bertino and J.D. Joshi
Breedy, M.R., Eskridge, T., Hyuckchul, J., Lott, J., and Eds. Springer Berlin Heidelberg, 499-511.
Uszok, A., 2008. Coordination in Human-Agent-Robot
Teamwork. In Collaborative Technologies and Systems, [23] Goldberg, L.R., 1990. An Alternative Description of
2008. CTS 2008. International Symposium on, 467-476. Personality: The Big-Five Factor Structure. Journal of
Personality and Social Psychology 59, 1216–1229.
[11] Cafaro, A., Vilhjálmsson, H., Bickmore, T., Heylen, D.,
Jóhannsdóttir, K., and Valgarðsson, G., 2012. First [24] Goldberg, L.R., Johnson, J.A., Eber, H.W., Hogan, R.,
Impressions: Users’ Judgments of Virtual Agents’ Ashton, M.C., Cloninger, C.R., and Gough, H.G., 2006. The
Personality and Interpersonal Attitude in First Encounters. In International Personality Item Pool and the Future of Public-
Intelligent Virtual Agents, Y. Nakano, M. Neff, A. Paiva and Domain Personality Measures. Journal of Research in
M. Walker Eds. Springer Berlin Heidelberg, 67-80. Personality 40, 1, 84–96.
[12] Cannon-Bowers, J.A., Salas, E., and Converse, S., 1993. [25] Gorla, N. and Lam, Y.W., 2004. Who Should Work with
Shared Mental Models in Expert Team Decision-Making. In Whom?: Building Effective Software Project Teams.
Proceedings of the Individual and group decision making, Commun. ACM 47, 6, 79-82.
221-246.
[26] Gosling, S.D., Rentfrow, P.J., and Swann, W.B., 2003. A
[13] Chittaro, L. and Serra, M., 2004. Behavioral programming of Very Brief Measure of the Big-Five Personality Domains.
autonomous characters based on probabilistic automata and Journal of Research in Personality 37, 6, 504-528.
personality. Computer Animation and Virtual Worlds 15, 3-4,
319-326. [27] Hanna, N. and Richards, D., 2014. The Impact of
Communication on a Human-Agent Shared Mental Model
[14] Cohen, P., Levesque, H., and Smith, I., 1997. On Team and Team Performance. In Proceedings of the the 13th
Formation. In Contemporary Action Theory, G. Holmstrom- international conference on Autonomous agents and multi-
Hintikka and R. Tuomela Eds. Kluwer Academic. agent systems (AAMAS'14) (Paris, France), 1485-1486.
[15] Cohen, P.R. and Levesque, H.J., 1991. Teamwork. Noûs 25, [28] Hanna, N., Richards, D., and Hitchens, M., 2013. Evaluating
4, 487-512. the Impact of the Human-Agent Teamwork Communication
Model (HAT-CoM) on the Development of a Shared Mental
[16] Doce, T., Dias, J., Prada, R., and Paiva, A., 2010. Creating Model. In PRIMA 2013: Principles and Practice of Multi-
Individual Agents through Personality Traits. In Intelligent Agent Systems, G. Boella, E. Elkind, B. Savarimuthu, F.
Virtual Agents, J. Allbeck, N. Badler, T. Bickmore, C. Dignum and M. Purvis Eds. Springer Berlin Heidelberg, 453-
Pelachaud and A. Safonova Eds. Springer Berlin Heidelberg, 460.
257-264.
[29] Isbister, K. and Nass, C., 2000. Consistency of Personality in Designed Virtual Agent Personalities. IEEE Transactions on
Interactive Characters: Verbal Cues, Non-Verbal Cues, and Affective Computing 3, 3, 311-322.
User Characteristics. International Journal of Human-
Computer Studies 53, 2, 251-267. [42] Nass, C. and Lee, K.M., 2000. Does computer-generated
speech manifest personality An experimental test of
[30] Kang, S.-H., Gratch, J., Wang, N., and Watt, J., 2008. similarity-attraction. In CHI '00: Proceedings of the SIGCHI
Agreeable People Like Agreeable Virtual Humans. In conference on Human factors in computing systems ACM,
Intelligent Virtual Agents, H. Prendinger, J. Lester and M. NY, USA, 329-336.
Ishizuka Eds. Springer Berlin Heidelberg, 253-261.
[43] Nass, C. and Moon, Y., 2000. Machines and mindlessness:
[31] Kieft, I., Jonker, C., and Riemsdijk, M.B., 2011. Explaining Social responses to computers. Journal of Social Issues 56, 1,
Negotiation: Obtaining a Shared Mental Model of 81-103.
Preferences. In Modern Approaches in Applied Intelligence,
K. Mehrotra, C. Mohan, J. Oh, P. Varshney and M. Ali Eds. [44] Neff, M., Toothman, N., Bowmani, R., Fox Tree, J., and
Springer Berlin Heidelberg, 120-129. Walker, M., 2011. Don’t Scratch! Self-adaptors Reflect
Emotional Stability. In Intelligent Virtual Agents, H.
[32] Krishnan, V., Foster, A., Kopper, R., and Lok, B., 2012. Vilhjálmsson, S. Kopp, S. Marsella and K. Thórisson Eds.
Virtual Human Personality Masks: A Human Computation Springer Berlin / Heidelberg, 398-411.
Approach to Modeling Verbal Personalities in Virtual
Humans. In Intelligent Virtual Agents, Y. Nakano, M. Neff, [45] Neff, M., Wang, Y., Abbott, R., and Walker, M., 2010.
A. Paiva and M. Walker Eds. Springer Berlin Heidelberg, Evaluating the effect of gesture and language on personality
146-152. perception in conversational agents. In Proceedings of the the
10th international conference on Intelligent virtual agents
[33] LePine, J.A., Buckman, B.R., Crawford, E.R., and Methot, (IVA'10) (Philadelphia, PA), Springer-Verlag, 222-235.
J.R., 2011. A review of research on personality in teams:
Accounting for pathways spanning levels of theory and [46] Neuman, G.A. and Wright, J., 1999. Team Effectiveness:
analysis. Human Resource Management Review 21, 311-330. Beyond Skills and Cognitive Ability. Journal of Applied
Psychology 84, 3, 376-389.
[34] Lim, B. and Klein, K., 2006. Team Mental Models and Team
Performance: A Field Study of the Effects of Team Mental [47] Prada, R. and Paiva, A., 2009. Teaming up Humans with
Model Similarity and Accuracy. Journal of Organizational Autonomous Synthetic Characters. Artificial Intelligence
Behavior 27, 4, 403-418. 173, 1, 80-103.
[35] Loyall, A.B. and Bates, J., 1997. Personality-rich Believable [48] Prada, R. and Paiva, A., 2014. Human-Agent Interaction:
Agents that Use Language. In Proceedings of the Challenges for Bringing Humans and Agents Together. In
Proceedings of the first international conference on Third International Workshop of Human-Agent Interaction
Autonomous agents (Marina del Rey, California, USA), Design and Models (HAIDM '14)@AAMAS2014, Paris,
ACM, 267681, 106-113. France.
[36] Luse, A., McElroy, J.C., Townsend, A.M., and DeMarie, S., [49] Reeves, B. and Nass, C., 1996. The Media Equation: How
2013. Personality and Cognitive Style as Predictors of People Treat Computers, Television, and New Media Like
Preference for Working in Virtual Teams. Computers in Real People and Places. Cambridge University Press, New
Human Behavior 29, 4, 1825-1832. York.
[37] Mann, R.D., 1959. A Review of the Relationships Between [50] Roque, A. and Traum, D., 2009. Improving a Virtual Human
Personality and Performance in Small Groups. Psychological Using a Model of Degrees of Grounding. In Proceedings of
Bulletin 56, 241-270. the proceedings of International Joint Conerence on
Artificial Intelligence (IJCAI-09) (Pasadena, CA).
[38] Mathieu, J.E., Heffner, T.S., Goodwin, G.F., Cannon-
Bowers, J.A., and Salas, E., 2005. Scaling the Quality of [51] Scherer, K.R., 1978. Personality Inference from Voice
Teammates' Mental Models: Equifinality and Normative Quality: The Loud Voice of Extroversion. European Journal
Comparisons. Journal of Organizational Behavior 26, 37-56. of Social Psychology 8, 467-487.
[39] Mathieu, J.E., Heffner, T.S., Goodwin, G.F., Salas, E., and [52] Scherer, K.R., 1979. Personality Markers in Speech.
Cannon-Bowers, J.A., 2000. The Influence of Shared Mental Cambridge University Press, London.
Models on Team Process and Performance. Journal of
Applied Psychology 85, 2, 273-283. [53] Traum, D.R. and Allen, J.F., 1992. A Speech Acts Approach
to Grounding in Conversation. In Proceedings 2nd
[40] McCrae, R.R. and John, O.P., 1992. An Introduction to the International Conference on Spoken Language Processing
Five-Factor Model and Its Applications. Journal of (ICSLP '92), 137-140.
Personality 60, 2, 175-215.
[54] von der Pütten, A., Krämer, N., and Gratch, J., 2010. How
[41] McRorie, M., Sneddon, I., McKeown, G., Bevacqua, E., de Our Personality Shapes Our Interactions with Virtual
Sevin, E., and Pelachaud, C., 2012. Evaluation of Four Characters - Implications for Research and Development. In
Intelligent Virtual Agents, J. Allbeck, N. Badler, T.
Bickmore, C. Pelachaud and A. Safonova Eds. Springer Decision Makings. Decision support systems, special issue
Berlin Heidelberg, 208-221. on intelligence and security informatics 41, 3, 634-653.
[55] Yee, N., Harris, H., Jabon, M., and Bailenson, J.N., 2011. [57] Yen, J., Xiaocong, F., Shuang, S., McNeese, M., and Hall,
The Expression of Personality in Virtual Worlds. Social D., 2004. Supporting Anti-Terrorist Analyst Teams Using
Psychological and Personality Science 2, 1, 5-12. Agents with Shared RPD Process. In Computational
Intelligence for Homeland Security and Personal Safety,
[56] Yen, J., Fan, X., Sun, S., Hanratty, T., and Dumer, J., 2006. 2004. CIHSPS 2004. Proceedings of the 2004 IEEE
Agents with Shared Mental Models for Enhancing Team International Conference on, 53-60.