=Paper= {{Paper |id=Vol-2730/paper8 |storemode=property |title=The appearance of the avatar can enhance the sense of co-presence during virtual interactions with users |pdfUrl=https://ceur-ws.org/Vol-2730/paper8.pdf |volume=Vol-2730 |authors=Mariachiara Rapuano,Antonella Ferrara,Filomena Leonela Sbordone,Francesco Ruotolo,Gennaro Ruggiero,Tina Iachini |dblpUrl=https://dblp.org/rec/conf/psychobit/RapuanoFSRRI20 }} ==The appearance of the avatar can enhance the sense of co-presence during virtual interactions with users== https://ceur-ws.org/Vol-2730/paper8.pdf

The Appearance of the Avatar Can Enhance the Sense of
Co-Presence During Virtual Interactions with Users

Mariachiara Rapuano[0000-0002-3391-5421], Antonella Ferrara[0000-0001-5229-538X], Filomena
Leonela Sbordone[0000-0001-7496-9170], Francesco Ruotolo[0000-0002-1807-8282], Gennaro
Ruggiero[0000-0002-3940-6740]and Tina Iachini[0000-0001-8405-8768]

Laboratory of Cognitive Science and Immersive Virtual
Reality, CS-IVR, Department of Psychology, University of Campania Luigi Vanvitelli, Viale
Ellittico, 31, 81100 Caserta, Italy

mariachiara.rapuano@unicampania.it
antonella.ferrara@unicampania.it
filomenaleonela.sbordone@unicampania.it
francesco.ruotolo@unicampania.it
gennaro.ruggiero@unicampania.it
santa.iachini@unicampania.it

Abstract. Computer media, interactive interfaces and virtual environments are
becoming more social and increasingly used for interpersonal interactions. In-
deed, nowadays we can observe an increment in social interactions between users
and computer agents (i.e. avatars). The inclusion of avatars within virtual envi-
ronments is known to facilitate interaction and to improve users' sense of pres-
ence (being in there) and social presence (being with another) during these sim-
ulations. Therefore, understanding which characteristics of avatars’ appearance
increase users' involvement and social presence in a virtual world can be useful
for theoretical and applied purposes. We tried to address this issue by considering
individuals' reaction to an ancient evolutionary behavior: grinding their mouth
and showing their teeth to express anger. Participants determined their comfort-
distance from male and female avatars showing anger and happiness with open
and closed mouth. Results showed that comfort-distance was larger with angry
avatars with open mouth than all other conditions. Participants approached happy
avatars at shorter distances with both open and closed mouths. Therefore, if we
control the way avatars manifest emotions by paying attention to their adaptive
evolutionary roots, we can improve the capacity of virtual systems to induce a
sense of co-presence, the feeling of being in touch with another in the computer-
generated world.

Keywords: Emotions – Adaptive function– Avatars' appearance – Social pres-
ence – IVR

Copyright © 2020 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
2

1 Introduction

Technological advances shape our society and thus deeply influence human thinking
and behavior [1]. An example of such transformative technology is, for instance, the
fact that computer media, interactive interfaces and virtual environments are becoming
more social and increasingly used for interpersonal interactions [2, 3].
Indeed, nowadays we can observe an increment in social interactions not only among
users, but also between users and computer agents (i.e. avatars), and this is particularly
true for advanced interactive media, including Immersive Virtual Reality (IVR) [3, 4].
IVR is a medium that is able to induce the experience of “presence” in a computer-
generated world. Presence refers to the “sense of being there” or the “feeling of being
in a world that exists outside the self” [5-7]. Therefore, the sense of presence is rooted
on a paradoxical state of consciousness: people behave and feel as if they actually were
in the virtual world even though they know there is nothing real [5, 8]. The inclusion
of avatars (i.e. representations of human users) within IVR environments is known to
facilitate interaction and to improve users' sense of involvement and realism of this
simulations [3]. Indeed, if the anthropomorphism of avatars is sufficiently “natural”,
we tend to attribute them feelings and thoughts as if they had a mind similar to our own
mind (e.g. [9, 10, 5, 11]). In other word, we develop the so-called “social presence”,
the sense of “being together with another” [2, 12]. Therefore, beyond the “sense of
being in the virtual place” there is the experience of “co-presence”, or quite essentially,
the “sense of another through the virtual medium” [2, 12]. This phenomenon can be
explained by the notion of “derivative intentionality” proposed by Dennett [13, 14]: the
attribution of goals, desires, rationality to a system or an entity which hence referred to
as intentional. Therefore, understanding which characteristics of avatars’ appearance
increase the social presence can be useful for theoretical and applied purposes (e.g.,
[3]).
In this study we address this issue by considering a normal social behavior that is
rooted in an ancient evolutionary behavior: opening or closing the mouth when we ex-
press emotions [15-18]. Facial expressions are salient social cues that signal an indi-
vidual’s emotional state, behavioral intentions or action requests [17, 19, 20]. There-
fore, successful social exchanges require both the ability to detect the emotions of oth-
ers and to adapt our behavior to theirs [21].
According to an adaptive perspective, several emotional expressions may express a
direct or indirect evolutionary root [15, 17, 20]. For example, grinding mouth and show-
ing teeth is a typical behavior in the animal world to express aggression and attack
against someone [22, 15, 17, 20, 23; see also [24]. Here we wondered if this evolution-
ary characteristic of emotions is also reflected in the appearance of avatars, that is if we
are more afraid of avatars that seem angry with their mouths open or closed. And what
happens with avatars that look happy? In other words, we want to exploit the adaptive
functions of human facial expressions to explore the degree of social and physical in-
volvement during social interactions with avatars.
According to proxemics, individuals actively use the space and manipulate their en-
vironment in order to establish their preferred degree of closeness and involvement [25]
deploying approach and avoidance behaviors during interpersonal encounters [26].
3

In proxemics, this interpersonal space (IPS) represents the optimal social distance
from conspecifics thus, a good measure of the quality of social interactions [27, 28]. In
the social psychology literature, a typical task to assess the size of interpersonal space
is based on comfort-distance judgments provided through the ‘approach-distance’ par-
adigm: participants have to stop themselves at the point where they still feel comforta-
ble with the proximity of the interactant.
Typically, this distance increases in uncomfortable/threatening situations and de-
creases in comfortable/safe situations [27-30]. As signal of salient environmental
events, emotional facial expressions trigger approaching-avoiding reactions that reveal
evolutionary adaptations rooted in basic survival mechanisms [22, 15, 17, 22, 31] even
when interacting with virtual conspecifics in IVR environments (e.g., [32]).
However, to our knowledge, it is still unclear if whether the way the face expresses
emotions, i.e. with the mouth open or closed, can influence this basic survival response
and, through distance modulation, provide an index of involvement in social-presence.
To this end, we devised an IVR study in which participants were asked to determine
the comfort-distance while approached virtual avatars showing anger and happiness
with open and closed mouth. We hypothesize that if we are more afraid of avatars who
seem angry with their mouths open than closed, then we should expect a larger distance
in the former. Moreover, since previous studies have shown gender differences in in-
terpersonal distance [25, 28, 33, 34, 35] we also investigate this aspect.

2 Method

2.1 Participants

Thirty-two right-handed participants (16 males), aged 18-29 years (Mage= 23; SD= 3),
were recruited in exchange of course credit. Participants had normal/corrected-to-nor-
mal vision. Nobody claimed discomfort or vertigo during the IVR experience and re-
ported being aware of the experimental purpose. All participants gave their written con-
sent to take part in the study. The study was in conformity with the local Ethics Com-
mittee requirements and the Declaration of Helsinki (World Medical Association,
2013).

2.2 Setting and apparatus

The experimental setting and the virtual scenario were similar to those of previous stud-
ies [32-34]. The IVR equipment was installed in a 5x4x3 m room of the Laboratory of
Cognitive Science and Immersive Virtual Reality (CS-IVR, Dept. Psychology). The
equipment included the 3-D Vizard Virtual Reality Software Toolkit 4.10 (Worldviz,
LLC, USA) with the Oculus Rift head mounted display (HMD) having two OLED dis-
plays for stereoscopic depth (images = 1920 x 1080; 90° horizontally, 110° diagonally).
The IVR system continuously tracked and recorded participant’s position through a
marker on the HMD. Visual information was updated in real time.
4

2.3 Virtual scenario and virtual stimuli

The virtual room (3 x 2.4 x 3 m) consisted of green walls, white ceiling and grey floor
with a white dashed line from the initial position of the participant to the end of the
virtual room. A total of eight young avatars (four females) were selected among a col-
ony of highly realistic avatars and were used for the present study (Vizard Complete
Characters, WorldViz; USA).
The emotional expression of the face was obtained by modelling the virtual faces
with 3DS Max (Autodesk) following the KDEF database [36]. The sample of avatars
was selected on the basis of a previous pilot study [32] in which 14 participants rated,
on a 9-point Likert scale, how much the faces presented on the PC appeared happy/un-
happy, friendly/threatening, angry/peaceful, and annoying/quite. Following this evalu-
ation, twelve avatars were selected whose facial expressions were: happy (two males
and two females) angry (two males and two females) and neutral (two males two fe-
males).
The selected sample (Fig.1) represented male and female adults aged about thirty
years wearing similar casual clothes and were perceived as representation of Italian
people. Their height was 175 cm (males) and 165 cm (females). Their gaze was kept
looking straight ahead throughout the trials [37].

Fig. 1. Examples of experimental stimuli. On the left, the panel shows examples of angry avatars
with open (top) and closed (bottom) mouth. On the right, the panel shows examples of happy
avatars with open (top) and closed (bottom) mouth.
5

3 Procedure

After presenting the IVR devices, participants were invited to wear the HMD and to
explore the virtual scenario in order to familiarize with the IVR equipment and the
environment. Through the HMD, participants could see the virtual stimuli fully im-
mersed in the virtual scene. During this initial experience they had to describe their
feelings of presence (e.g. participants were asked if they felt present in the virtual space,
if they felt they were acting in the virtual space excluding the real external environ-
ment). After that, participants received instructions about the comfort-task and then
guided to the starting position holding a key-press device in their dominant hand. The
entire experimental flow comprised a four-trial training session to allow the participant
to familiarize with the task. Next, the tasting phase began with a short presentation of
the instructions (2 s) followed by a fixation cross (300 ms) then an avatar (i.e. a male/fe-
male avatar showing angry or happy facial expression) appeared. The instruction was:
“Press the button as soon as the distance between you and the avatar makes you feel
uncomfortable’’. During the experimental session, avatars remained motionless and
participants walked towards them (0.5 ms-1) until they stopped and simultaneously
pressed the button, after which returned to the starting position. After button press, vir-
tual stimuli disappeared. Each virtual stimulus was randomly presented 4 times (total
of trials: 32), the experimental session lasted about 10 minutes. Finally, participants
evaluated their experience with the avatars. They reported they clearly identified their
facial expressions as if they were ‘‘realistic persons’’.

4 Data Analysis

In each trial, the participant-avatar distance (cm) was recorded. The mean distances
were analyzed through a 2x2x2x2 mixed ANOVA with Participant's Gender as between
factor and three within factors: Emotion (2 levels: happy, angry), Avatars’ sex (male,
female) and Mouth (2 levels: open, closed). The Tukey HSD post-hoc test was used,
and the magnitude of significant effects was expressed by partial eta-squared (η²p).

5 Results

A main effect of Emotion appeared (F(1,30)= 22.28, p < .0001, η²p= 0.43). Comfort
distance was larger with angry than happy avatars. A main effect of avatar's gender also
emerged (F(1,30)= 14.85, p < .001, η² p= 0.33), such that comfort-distance was larger
with male than female avatars. Finally, a main effect of mouth was shown (F(1,30)=
16.54, p < .001, η² p= 0.35). Comfort-distance was larger when dealing with avatars
with open than closed mouth. Emotion and mouth significantly interacted (F(1,30)=
8.504, p = 0.007, η²p= 0.22). The effect was due to the fact that participants preferred a
larger distance from angry avatars with open mouth than all other conditions (at least p
< .001). Moreover, no difference emerged with avatars expressing happiness with open
or closed mouth p= 0.92). Participant's gender, avatar's gender and mouth significantly
6

interacted (F(1,30)= 6.13, p = 0.02, η² p= 0.17). The post-hoc analysis showed that male
participants preferred a shorter comfort-distance from female avatars with closed mouth
than all other conditions (at least p < .01). For female participants, comfort-distance
was larger from male avatars with open mouth than all other conditions (at least p <
.01). Finally, emotion, avatar's gender and mouth significantly interacted (F(1,30)=
6.95, p = 0.01, η² p= 0.19) (Fig.2). In presence of happy avatars, distance was shorter
from females with both open/closed mouth (no difference, P= 0.93) than males with
both open/closed mouth (at least p < .05). In presence of angry avatars, distance was
larger from males with open mouth than all other conditions (at least p < .001). With
female angry avatars, distance was larger when they had open than closed mouth (p <
.001).

Fig. 2. Three-way interaction. The graph shows the mean comfort-distance (cm) as a function of
emotions (happy-angry), avatar’s gender (male-female) and mouth (open-closed). Error bars rep-
resent the standard error.

6 Discussion

The inclusion of avatars within IVR environments is known to facilitate interaction and
to improve users' sense of involvement and realism of this virtual experiences [3].
Therefore, understanding which characteristics of the appearance of avatars may in-
crease the social presence can be useful for theoretical and applied purposes (e.g. [3]).
To this end, we have explored the effect of an ancient evolutionary behavior: opening
or closing the mouth for expressing emotions [15-18]. Here we wondered if the primi-
tive (defensive) response to (negative) emotions may be affected by the way the avatar
expresses emotions, i.e. with the mouth open or closed, and, through distance modula-
tion, provide an index of involvement in social-presence. In line with literature, results
showed that participants kept further from angry avatars than happy ones [32] and from
male than female avatars [33, 34]. In addition, male participants reduced distance with
female avatars with closed mouth, while female participants increased distance with
7

male avatars with open mouth. Coherently with previous findings (e.g., [35, 38],
women seem to be more focused on gaining information about the safety/harmful value
of the social context than men. More interestingly for our purpose, the present findings
indicate that all the participants reacted by increasing distance when approached male
angry avatars with open than closed mouth. Notably, no difference emerged between
open and closed mouth when expressing happiness, and this is true both for male and
female participants Indeed, going further previous works (e.g. [38]), interested in clar-
ify gender-related and emotion-related effects and the ecological validity of VR in prox-
emics, here we explored which physical characteristics of virtual conspecifics per se
could have reinforced the sense of presence and co-presence during IVR interactions.
In line with this, our results show that avatars' appearance can enhance the sense of
physical and social presence during IVR experiences and that reactions in response to
emotions can reflect the degree of involvement in virtual interactions. As in the animal
world, participants took advantages from those facial expressions able to enhance sur-
vival (e.g. [24]) reacting with avoiding behaviors against angry avatars showing teeth,
a signal of potential aggression and attack [22, 15, 17, 20, 23]. On the other hand, as
results confirm, in presence of emotions that communicate positivity, sociality and
safety, people respond with approaching behavior, no matter if the other smiles with
open or closed mouth [15, 17, 20, 23, 24]. According to an evolutionary perspective, it
is more important to avoid potentially noxious stimuli, such as predators or other ag-
gressive conspecifics, than to approach positive stimuli, such as smiling conspecifics
[32]. To conclude, if we control the way avatars manifest emotions by paying attention
to their adaptive evolutionary roots, we can improve the capacity of virtual systems to
induce a sense of co-presence, the feeling of being in touch with another in the com-
puter-generated world.
8

References
1. Dolata, U.: The transformative capacity of new technologies a theory of sociotechnical
change. Hoboken: Taylor and Francis (Routledge Advances in Sociology) (2013).
2. Biocca, F., Burgoon, J., Harms, C., Stoner, M.: Criteria and scope conditions for a theory
and measure of social presence. In: 4th Annual International Workshop on Presence (2001).
3. Nowak, K.: Defining and Differentiating Copresence, Social Presence and Presence as
Transportation. Presence 2001 Conference (2001).
4. Riva, G., Mantovani, F., Capideville, C.S., Preziosa, A., Morganti, F., Villani, D., Gaggioli,
A., Botella, C., Alcañiz, M.: Affective interactions using virtual reality: The link between
presence and emotions. Cyberpsychology and Behavior
(2007). https://doi.org/10.1089/cpb.2006.9993.
5. Slater, M.: Place illusion and plausibility can lead to realistic behaviour in immersive virtual
environments. Philosophical Transactions of the Royal Society B: Biological Sciences
(2009). https://doi.org/10.1098/rstb.2009.0138.
6. Slater, M., Wilbur, S.: A framework for immersive virtual environments (FIVE): Specula-
tions on the role of presence in virtual environments. Presence: Teleoperators and Virtual
Environments (1997). https://doi.org/10.1162/pres.1997.6.6.603.
7. Steuer, J.: Defining Virtual Reality: Dimensions Determining Telepresence. Journal of
Communication (1992). https://doi.org/10.1111/j.1460-2466.1992.tb00812.x.
8. Sanchez-Vives, M. V., Slater, M.: From presence to consciousness through virtual reality
(2005). https://doi.org/10.1038/nrn1651.
9. Nowak, K., Biocca, F.: Understanding the influence of agency and anthropomorphism on
copresence, social presence and physical presence with virtual humans. Teleoperators and
Virtual Environments, MIT Pres. 481–494 (2001).
10. Nowak, K.L., Biocca, F.: The Effect of the Agency and Anthropomorphism on users’ Sense
of Telepresence, Copresence, and Social Presence in Virtual Environments. In: Presence:
Teleoperators and Virtual Environments (2003).
https://doi.org/10.1162/105474603322761289.
11. Riva, G., Galimberti, C.: Computer-mediated communication: Identity and social interaction
in an electronic environment. Genetic, Social, and General Psychology Monographs (1998).
12. Biocca, F., Harms, C., Burgoon, J.K.: Toward a More Robust Theory and Measure of Social
Presence: Review and Suggested Criteria. Presence: Teleoperators and Virtual Environ-
ments (2003). https://doi.org/10.1162/105474603322761270.
13. Dennett, D.C.: Brainstorms MIT Press (1978).
14. Dennett, D.C.: The International Stance Cambridgele (1987).
15. Darwin, C.: The expression of emotion in animals and man (1872).
16. Ekman, P.: Universals and cultural differences in facial expressions of emotion. Nebraska
Symposium on Motivation (1971).
17. Ekman, P.: Facial Expressions. In: Handbook of Cognition and Emotion
(1999). https://doi.org/10.1002/0470013494.ch16.
18. Izard, C.E.: Emotions and facial expressions: A perspective from Differential Emotions The-
ory. In: J. A. Russell, J. M. Fernández-Dols (ed.) Studies in emotion and social interaction,
2nd series. The psychology of facial expression, pp. 57–77
(1997). https://doi.org/https://doi.org/10.1017/CBO9780511659911.005.
9

19. Keltner, D., Ekman, P., Gonzaga, G.C., Beer, J., Scherer, K.R., Johnstone, T., Klasmeyer,
G., Snowdon, C.T., Oatley, K., Gabrielsson, A., Juslin, P.N., Reilly, J., Seibert, L.: Expres-
sion of emotion. In: Handbook of affective sciences (2003).
20. Fridlund, A.J.: The new ethology of human facial expressions. In: J. A. Russell, J. M. Fer-
nández-Dols (ed.) Studies in emotion and social interaction, 2nd series. The psychology of
facial expression, pp. 103–129
(1997). https://doi.org/https://doi.org/10.1017/CBO9780511659911.007.
21. Ferri, F., Stoianov, I.P., Gianelli, C., D’Amico, L., Borghi, A.M., Gallese, V.: When action
meets emotions: How facial displays of emotion influence goal-related behavior. PLoS ONE
(2010). https://doi.org/10.1371/journal.pone.0013126.
22. Damasio, A.R.: The Feeling of What Happens: Body and Emotion in the Making of Con-
sciousness (1999).
23. Fridlund, A.J.: Human facial expression: An evolutionary view. San Diego CA: Academic
Press (1994).
24. Schmidt, K.L., Cohn, J.F.: Human facial expressions as adaptations: Evolutionary questions
in facial expression research. Yearbook of Physical Anthropology
(2001). https://doi.org/10.1002/ajpa.20001.
25. Aiello, J.R.: Human Spatial Behavior. In: Stokols, D., Altman, I. (ed.) Handbook of environ-
mental psychology. pp. 389–504. John Wiley and Sons, New York (1987).
26. Argyle, M., Dean, J.: Eye-Contact, Distance And Affiliation. Soins; la revue de référence in-
firmière (1965). https://doi.org/10.2307/2786027.
27. Spencer, R.E.: Book Reviews: The Hidden Dimension by Edward T. Hall. New York: Dou-
bleday and Company, Inc., 1966. Pp. xii + 193. Educational and Psychological Measurement
(1966). https://doi.org/10.1177/001316446602600462.
28. Hayduk, L.A.: Personal space: Where we now stand. Psychological Bulletin
(1983). https://doi.org/10.1037/0033-2909.94.2.293.
29. Kennedy, D.P., Gläscher, J., Tyszka, J.M., Adolphs, R.: Personal space regulation by the
human amygdala. Nature Neuroscience (2009). https://doi.org/10.1038/nn.2381.
30. Tajadura-Jiménez, A., Pantelidou, G., Rebacz, P., Västfjäll, D., Tsakiris, M.: I-space: The
effects of emotional valence and source of music on interpersonal distance. PLoS ONE
(2011). https://doi.org/10.1371/journal.pone.0026083.
31. Graziano, M.S.A., Cooke, D.F.: Parieto-frontal interactions, personal space, and defensive
behavior. Neuropsychologia 44(6):845-859
(2006). https://doi.org/10.1016/j.neuropsychologia.2005.09.011.
32. Ruggiero, G., Frassinetti, F., Coello, Y., Rapuano, M., di Cola, A.S., Iachini, T.: The effect
of facial expressions on peripersonal and interpersonal spaces. Psychological Research, 81
(2017). https://doi.org/10.1007/s00426-016-0806-x.
33. Iachini, T., Coello, Y., Frassinetti, F., Ruggiero, G.: Body space in social interactions: A
comparison of reaching and comfort distance in immersive virtual reality. PLoS ONE, 9
(2014). https://doi.org/10.1371/journal.pone.0111511.
34. Iachini, T., Coello, Y., Frassinetti, F., Senese, V.P., Galante, F., Ruggiero, G.: Periper-
sonal and interpersonal space in virtual and real environments: Effects of gender and age.
Journal of Environmental Psychology, 45,
(2016). https://doi.org/10.1016/j.jenvp.2016.01.004.
35. Uzzell, D., Horne, N.: The influence of biological sex, sexuality and gender role on inter-
personal distance. British Journal of Social Psychology
(2006). https://doi.org/10.1348/014466605X58384.
36. Lundqvist, D., Flykt, A., Ohman, A.: The Karolinska directed emotional faces (KDEF)
(1998). https://doi.org/10.1017/S0048577299971664.
10

37. Bailenson, J.N., Blascovich, J., Beall, A.C., Loomis, J.M.: Interpersonal distance in im-
mersive virtual environments (2003). https://doi.org/10.1177/0146167203029007002.
38. Rapuano, M., Sbordone, F.L., Borrelli, L.O., Ruggiero, G., Iachini, T.: The Effect of Facial
Expressions on Interpersonal Space: A Gender Study in Immersive Virtual Reality. In:
Smart Innovation, Systems and Technologies (Online 2020). https://doi.org/10.1007/978-
981-15-5093-5_40.