=Paper= {{Paper |id=Vol-1382/paper5 |storemode=property |title=Comparing a Social Robot and a Mobile Application for Movie Recommendation: A Pilot Study |pdfUrl=https://ceur-ws.org/Vol-1382/paper5.pdf |volume=Vol-1382 |dblpUrl=https://dblp.org/rec/conf/woa/CervoneSSTR15 }} ==Comparing a Social Robot and a Mobile Application for Movie Recommendation: A Pilot Study== https://ceur-ws.org/Vol-1382/paper5.pdf

Proc. of the 16th Workshop “From Object to Agents” (WOA15) June 17-19, Naples, Italy

Comparing a Social Robot and a Mobile Application
for Movie Recommendation: A Pilot Study

Francesco Cervone, Valentina Sica, Mariacarla Staffa, Anna Tamburro, Silvia Rossi
Dipartimento di Ingegneria Elettrica e Tecnologie dell’Informazione
Università degli Studi di Napoli “Federico II”
via Claudio 21, 80125 Napoli, Italy
{mariacarla.staffa,silvia.rossi}@unina.it

Abstract—Social robots can be used as interfaces to provide real world environments [6]. Moreover, non-verbal behaviors
recommendations to users. While a vast literature compares the serve important functions in affecting the trustworthiness of a
user’s behavior when interacting with a robot with respect to recommendation [7]. In fact, a robot ability to build a trust re-
a virtual agent, in this paper, we conduct a first evaluation on lationship depends on its capacity to help people understand it,
how the user’s choices are affected if the recommendations are in part through non-verbal behavior. Emotion-related signals,
provided respectively by a mobile application or by social robots
with different degree of interaction capabilities. This pilot study
such as those provided by voice pitch changes in speech or
shows that the sole embodiment condition of the robot does not gestures are non-verbal behaviors that influence human trust
imply significant changes in the users’ choices that prefer to [8]. It has been, indeed, well-documented that humans expect
interact with the mobile application. However, the adoption of from humanoid robots socially intelligent responses [9]. This
additional communication channels such as gestures, gaze and leaves the possibility that an agent may influence how humans
voice pitch, which change accordingly to the suggested movie perceive a recommendation through the presence of more or
genre, improves the users acceptability. less communication abilities.
In this paper, we present a pilot study to evaluate the extent
I. I NTRODUCTION of the use of a robotic system in accepting a recommendation
Social robots will be used in the next future in many not with respect to a virtual agent, but to very common
application domains, which span from entertainment and ed- interfaces such as mobile applications. Our experiments aim at
ucation to health–care. In order to be accepted in our houses, evaluating the users’ acceptance of recommendations as well
they should be perceived as trusting, helpful, reliable and as their engagement when the robot or a mobile application are
engaging [1]. This is particularly important in case the robot displaying such advises. In particular, we provided the same
is in charge to convey information to a person (such teaching information contents on recommended movies, but using three
skills, collaborating towards a particular goal or providing different interaction conditions. In the first condition, by using
advises). Social robots, as well as virtual agents, can be used as a mobile application, contents will be provided by text shown
interfaces to provide recommendations. Such embodied social on the mobile screen. In the second one, the same contents
agents make interaction more meaningful than it is when will be provided by using a humanoid robot interacting using
provided by simple interfaces (which do not display actions speech. Finally, in the third condition the humanoid robot will
or speech), because users’ attitude towards social agents is encompass both voice and genre-driven motion primitives.
similar to that they show towards other people.
Recommendation systems aim to provide the user with II. R ELATED W ORKS
personalized advises and suggestions in many different do-
mains, such as books, movies or music. Such suggestions are A vast literature compared the behavior as well as the
provided according to the available information the system acceptance of robots with respect to their virtual counterparts.
has on the user (e.g., his/her preferences or his/her past Embodied robots are consistently perceived as more engaging
interaction with the system). Hence, recommendations can be than a character on a video display, and sometimes as engaging
provided suggesting items similar to items liked by the user as a human. For example, in Kidd and Breazeals [1] work
or liked by similar users. The effectiveness of the provision of subjects were instructed by an agent (either to a human, a
recommendations relies itself on the concept of trust [2] with robot, or a cartoon robot), which showed only its eyes to
respect to the system that proposes the recommendation. Such the subjects. All three visual presentations were accompanied
trust on the recommendation depends upon machine accuracy, by the same vocal instructions. The Authors’ purpose was to
predictability and dependability [3] (e.g., by recommending understand which types of interaction involved more the user
items which are positively evaluated by the users). In literature, (evaluated by a questionnaire), and showed more reliability,
different studies compared the impact of recommendation and usefulness and trust. The results showed that the robot, given
the advises as provided by social robot with respect to virtual its physical presence, was considered as more engaging, cred-
agents [4], [5], by showing that the embodiment condition, ible and informative, as well as being more pleasant as an
as provided by the robot, has more impact with respect to interaction partner. As in [1], in our experiment, we provided
2D/3D virtual agents on a screen. Real robots affect subject the same information contents with very simple and controlled
decision-making more effectively than computer agents in interfaces, but using different interaction modalities.

32
Proc. of the 16th Workshop “From Object to Agents” (WOA15) June 17-19, Naples, Italy

How physical embodiment, as opposed to virtual pres-
ence, affects human perception of social engagement with
an artificial agent was addressed in [4], [6], [10]. In [6]
the Authors evaluated the persuasion effects of a computer
agent and of a robot in various tasks as, for example, in
following indications. The results showed that the user has
shown more confidence and more trust for the physical robot.
User’s behavior in accepting advises was investigated also in
[4]. The results showed that the user preferred to interact
with the robot because it was more effective in providing
recommendations. Shinozawa et al. considered the effect of
persuasion in a laboratory environment comparing a robot and
a computer agent (with a 2D or 3D appearance) displayed on a
monitor [10]. The results showed that the geometric coherence
between a social agent and the environment was an important
factor in the interaction, independently whether it is 2D or 3D.
Conversely to these approaches, in our study, we compared
the effects of adopting for a recommending task a robot with
respect to a mobile application. This is, up to our knowledge, Fig. 1. Client/Server application.
the first attempt to provide such comparison.
Finally, in [5] the Authors studied the impact of the robot and to populate the Ratings Repository. MovieTweetings con-
size with respect to the user reactions in an advertising context. sists of movie ratings contained in well-structured tweets on
The purpose was to understand which robot was more suitable the Twitter.com social network. This information is contained
for interaction for advertising purposes. The results showed in three files: users.dat, ratings.dat and movies.dat, which
that, in the presence of robots of different sizes, the user provide respectively the user identification number, his/her
considers it easier to interact with a smaller robot. associated ratings and a list of movies. The dataset is updated
every day, therefore its size is constantly changing. At the last
III. S YSTEM A RCHITECTURE access, it contained about 35000 users, 360000 ratings and
20000 movies.
In order to evaluate our hypothesis, we developed a
client/server application, where the server provides the recom- The recommendation engine provides rating predictions
mendation service and the possible clients can be a humanoid when the recommendation API is invoked. To achieve this
robot or a mobile application. Clients are in charge to ask for a goal, we used item-based City Block distance, also known as
list of recommendations (in particular of movies) to the server Manhattan distance. In Mahout implementation, the generic
and to show them to users. The social robots and the mobile movie i is represented by a boolean vector:
application will provide the same information, but in different
i = [r1i , r2i , ..., rni ],
ways (i.e., through different communication channels). This
diversity should be reflected in a different perception of where n is the number of users in the dataset and rui = 1 if
the recommendations by the users, and, presumably, it will user u rated the movie i. The distance between two movies
affect their choices. In order to provide recommendations, rated by user u is the sum of the absolute value of the
the Recommendation Engine needs some initial movie ratings differences of the two associated vector components. More
from the users. Hence, independently from the client type formally, the distance between items i and j is:
the users interact with, the initial ratings are performed by n
using the mobile application, which allows users to easily X
d(i, j) = |rui − ruj |.
evaluate movies by means of a friendly graphical interface.
u=1
The main blocks of the developed framework are detailed in
the following subsections.
B. Android Application
A. Movie Recommendation Server On the bottom of Figure 1, the architecture design of the
mobile Android application is depicted. The first duty for the
The server layer of this architecture is characterized by the user, when he/she accesses the application, is to sign up/sign
recommendation system (see Figure 1). It is developed in Java into the system. As explained above, when the interaction
and it is hosted by a Tomcat servelet container. The core of starts, users have first to provide a certain number of movie
this layer is a Web Server, used to store, process and deliver ratings (at least 20 movies). The Training module is dedicated
requested content to clients, which are provided through JSON- to provide an interface to get movie lists and to store movies
based API both to communicate with the humanoid robot and ratings. If a user is in the training stage, he/she can browse
with the android mobile application. The module Recommen- movies by ordering them by most rated or randomly, or search
dation Engine is the core of the recommendation layer. We for a movie (filtering by genre or title, see Figure 2).
adopted the Apache Mahout library1 to predict the user ratings,
and chosen the MovieTweetings [11] dataset to train the system After this first stage, user can get movie recommendations
from the server. When the server gets the recommendation
1 http://mahout.apache.org/ request, once calculated the best movies for the user, it

33
Proc. of the 16th Workshop “From Object to Agents” (WOA15) June 17-19, Naples, Italy

retrieves additional details about the film, like, for example, a) Face Detection: This module is based on a face
the director, writers, actors and genres using OMDb2 web detection/recognition solution provided by OKI and included
service. Fortunately, MovieTweetings data set stores, for each in the Python SDK for NAO. Such module continuously
movie, its IMDb id, which can be used to address the OMDb processes frames from the NAO camera in order to detect a
service. The Android application shows on the screen the human face. Once a face is detected, it provides its position.
recommendations for the users through textual and graphical Moreover, in the third condition, the module continues to
descriptions. provide the user position coordinates to the Motion Controller,
that allows NAO to track the face by moving its head.
b) Speech Recognition: This module gives to the robot
the ability to recognize a predefined words list, and specifically
the usernames and the acceptance/rejection of a recommenda-
tion. It is based on module provided by Aldebaran, which relies
on sophisticated speech recognition technologies provided by
NUANCE for NAO Version 4. Before starting, the robot needs
to receive the list of usernames (UsersList). Then, once the sys-
tem has detected a face through Face Detection, NAO asks for
a username and listens until a word is recognized. Currently,
system does not provide a real authentication when interacting
with the robot because the only way to communicate with NAO
is the speech. Users should sign in through an input system
like a keyboard or a mobile application.
c) Behavior Selector: Through this module, we gener-
ate all the gestures, gaze and the feedback for the user. Once
a user has been recognized, a user tracking system allows the
robot to track the target by moving its head. Movie information
is provided to the user with the Speech Synthesis module
with different speech intonations, but it can be accompanied
Fig. 2. Snapshots from the movie-app training phase.
with arms gestures and facial expressions (e.g., different eyes
colors) generated through Motion Controller module. The
Behavior Selector gets recommendations from the Web Service
and related animations from the Animations Repository. The
C. The Robot Client main task of this module is mapping the movie genre into a
The Robot Client architecture has been designed consid- predefined set of animations and eyes colors. For example, if
ering the adoption of a NAO T14 robot model, consisting in NAO recommends a drama, led eyes become red and gestures
a humanoid torso with 14 degrees of freedom (2 for the head are more serious, while for a comedy led eyes become green
and 12 for the arms) developed by Aldebaran Robotics3 . We and gestures are more joyous. The pitch of the voice is
controlled the NAO platform by means of the standard robotic accordingly manipulated by the Speech Synthesis module.
operating system (ROS) and using the Python programming
language for developing the ROS nodes. NAO is endowed with IV. A P ILOT S TUDY
two main sensors: a camera and a microphone through which
it receives signals from the external environment. Camera The pilot study is conducted by considering three different
frames are processed by a Face Detection module to detect interaction conditions, where participants receive two movie
users presence into its visual field. Sounds obtained from the recommendations for each condition.
microphone are processed by Speech Recognition module.
A. Procedure
As actuators NAO can use the following communication
channels: voice, arms, head and eyes led. The Behavior The testing procedure main steps are: (i) the user provides
Selector module is in charge of providing two different in- new rates for a list of movies (training phase) at the beginning
teraction conditions. In the first case, it presents to users of the interaction; (ii) the recommendation system generates
the recommended movies and their relative information only the top-six recommendations for each user, which will be
through speech, while, in the second case, such description is shown to users through the three interaction conditions in a
accompanied with gestures, gaze, eyes coloring through the random way (two for each condition); (iii) after each test
Motion Controller, and pitch voice changes through Speech condition the participant has to answer to a questionnaire
Synthesis. In this latter case, the Behavior Selector gets motion concerning the specific condition and at the end of the overall
animations from an Animations Repository, based on the genre experiment to a general questionnaire.
of the recommended movie, to execute animated speech, as
will be detailed in the followings. B. Method
2 http://www.omdbapi.com - The Open Movie Database is a free web service The design of this study is a within-subjects, counterbal-
to obtain movie information. anced, repeated measures experiment. The three considered
3 https://www.aldebaran.com/en interactive conditions are the following:

34
Proc. of the 16th Workshop “From Object to Agents” (WOA15) June 17-19, Naples, Italy

Fig. 4. NAO and ENAO conditions.

TABLE I. 18 PARTICIPANTS DATA .
min max avg
Age 22 55 32
Fig. 3. Snapshots from the App recommendation phase. male female
Gender 72% 28%
low high avg
English Level 61% 39% 2.39
Condition 1 (App): in this setting, neither of the robot modali- Robotic Skills 44% 56% 3.17
ties are used. The user only interacts with the mobile applica-
tion that provides to the user two different movies suggestions.
For each movie the app provides the title and additional V. R ESULTS
information by displaying text and images on the screen. For
We hypothesized that the robot as compared with the
each recommended movie, the user has to reply about his/her
application will be more engaging and better liked, and hence
likelihood to see it (see Figure 3).
recommendation provided by the robot should be more likely
Condition 2 (Nao): in this setting, the robot is located on a to be accepted. Moreover, the condition with animated motion
table standing still and waiting for a person to interact with. should be more engaging and better liked with respect to the
When the robot recognizes a face in its filed of view, it greets simple robot.
the person, introduces itself, and asks for a username. NAO
presents the two recommendations by telling the movie title A. Quantitative Analysis
accompanied with the same information provided in Condition
In order to evaluate the degree of acceptance of the
1 (plot, genre, actors, and so on). Finally, it asks the user if
recommendations when provided by different conditions, we
she/he agrees to see this movie and stores the answer.
calculated the selection ratio indicating the number of accepted
Condition 3 (ENao): in this setting, differently from the recommendations with respect to the total number of recom-
previous condition, the robot is endowed with the motion mendations for a each specific condition.
controller module. When the robot is not interacting with
anyone, it simply looks around and waits for a person to talk
with. In this interaction condition NAO, in addition to tell
movie information, gesticulates, changes eyes led color and
voice pitch according to the recommended movie genre (see
Figure 4).

C. Participants

18 subjects participated in this experiment with an average
age of 32 years and a graduate education, for a total of 13
males and 5 females. All the participants were Italian native
speakers with an average English level of 2.39 and Robotic
skills of 3.17 on a likert–scale from 1 to 5. The language
Fig. 5. Percentage of accepted movie recommendations for each Condition.
adopted for the experiment was the English both for text
description and for the robot’s voice synthesizer. The testers
were not informed about the NAO interaction capabilities. In In Figure 5 the selection ratio is expressed in percentage
Table I, personnel data of participants are collected. shows that there is a minimum difference in the acceptance rate

35
Proc. of the 16th Workshop “From Object to Agents” (WOA15) June 17-19, Naples, Italy

between the recommendations provided by App and Nao, while likert scale from 1 to 5. Only for question 6 we explicitly
there is a slightly bigger difference between App and ENao ask for a preference by the users where index from 1 to 3
conditions. This fact is in accordance with our hypothesis that represent respectively the preference for APP, NAO and ENAO.
people are inclined to accept more recommendations provided The questionnaire structure is reported in Table II.
through a more natural interaction, even if the sole embodi-
TABLE II. Q UESTIONNAIRE .
ment condition (Nao) does not imply significant changes in
the testers acceptability level. However, due to the limited Section Question
number of participants and recommendations provided to each Personal Age?
participant these differences, evaluated using ANOVA, are not Information Gender?
English level? (1 to 5)
yet statistically significant, while there is a significant Pearson How familiarized are you with robotic applications? (1 to 5)
correlation between App and Nao conditions (r = 0.43 with Qualitative Q1. How easy was to perform the task? (1 to 5)
Questions Q2. Did the system react accordingly to your expectations? (1 to 5)
p = 0.08 that is significant at p < 0.10) acceptance trends. Q3. How natural is this kind of interaction? (1 to 5)
As future work we will extend such experimentation with a Q4. How satisfying do you find the interactive system? (1 to 5)
greater number of users. Q5. You were sure (5) or unsure (1) of your answers?
Q6. Which mode of interaction you preferred? (1 to 3).
Since users selected for testing are all Italian native speak- Robot-related Q7. The agent was believable (5) or unbelievable (1).
Questions Q8. The agents motions were natural (5) or unnatural (1).
ers, and not all have the same level of familiarity with robotics
applications, we felt it appropriate to consider data by aggre-
Figure 6-(a) shows the mean value of the answers to
gating the results by both the level of English proficiency (e.g.,
the qualitative questions for each interactive condition. Users
the language used to provide recommendations) and the degree
found the interaction with App easier than the interaction
of experience with robots. We thus computed correlations for
with Nao and ENao (Q1 in Table II). ANOVA test endorsed
the acceptance ratio and among conditions couples by grouping
this result by showing that differences between App and Nao
users with a high (from 4 to 5) or low English level (from 1
(F = 6.48 with p = 0.02) and between App and ENao
to 3) and a high or low familiarity with robots (see Section
(F = 3.34 with p = 0.08) were statistically significant. A
V-B):
slightly preference for the interaction with the App was also
• high English level: Pearson showed a negative strong shown by the answers to Q3 and Q4 questions. In this case,
correlation (r = −0.76 with p < 0.05) between Nao the only statistical significant differences were between App
and ENao; and Nao conditions for Q3 (F = 4.25, p = 0.05) and Q4
(F = 3.89, p = 0.06), thus the App was more natural and
• low English level: there is a significant correlation satisfying than Nao interacting with speech.
between App and Nao (r = 0.57 with p = 0.07);
For each question, we computed correlations between App,
• high familiarity: nothing significant; Nao and ENao:
• low familiarity: once again we had a moderate correla- • App-Nao: we notice a moderate correlation for Q2
tion between App and Nao (r = 0.65 with p = 0.08), (r = 0.50, p = 0.03), Q3 (r = 0.44, p = 0.07) and
but in this case also Nao and ENao have a moderate Q4 (r = 0.52, p = 0.03);
correlation (r = 0.65 with p = 0.08).
• Nao-ENao: there is a moderate correlation for Q2 (r =
These results show that for testers with a low English level 0.59, p = 0.01) and Q4 (r = 0.48, p = 0.04) and a
reading text from an application or hearing speech from a strong correlation for Q5 (r = 0.72, p < 0.01);
robot does not have a relevant impact on the decision making,
while for good English skilled participants adding an animated • App-ENao: there are no significant correlations.
behavior changes the acceptance trend. This could be due to If we observe the histogram of question 6 (see Figure 6-
the fact that the users attention in the first case is quite all (b)), a part from the approval ratings average of the qualitative
focused on understanding the text or speech. Moreover, users questions from Q1 to Q5, it is quite evident that the major part
with low familiarity in robotics shows acceptance trend similar of the users prefers to interact with the Humanoid endowed
in both Nao and ENao cases. with emotion-based capabilities. This is probably due to the
fact that the humanoid robot has the potential to portray a rich
B. Qualitative Analysis repertoire of non–verbal behaviors that have familiar social
Our evaluation also takes into account the impressions meaning for users, who perceive the interaction more natural
of users with respect to the interaction with the different and engaging because of the received socially intelligent
conditions. For this aim we propose a qualitative questionnaire responses by the robot. Histogram in Figure 6-(c) shows that
organized in three specific sections: (i) personal information the robotic agent is perceived in the average believeble both
for collecting information about the user (age, gender, english if it shows or not non-verbal feedback, and the agent motion
level, and familiarity with robotics); (ii) Qualitative questions is perceived as natural.
regarding the application easy of use, naturalness and satis- As for the quantitative case, for each pair of conditions,
fiability consisting of 6 questions; (iii) two question specific we try to correlate answers considering only users with high
for the conditions involving the humanoid, dealing with the or low English level or familiarity with robots applications:
sense of trust and movements naturalness of the robot. While
the general information have been asked at the beginning of • high English level: there is a moderate correlation
the tests, the testers have been asked to reply to the specific between App and Nao (r = 0.53, p < 0.01), and
questions at the end of each experiment. We adopted a classical Nao and ENao (r = 0.41, p < 0.01);

36
Proc. of the 16th Workshop “From Object to Agents” (WOA15) June 17-19, Naples, Italy

Fig. 6. Approval ratings average with respect to the qualitative questions.

• low English level: as in the previous case, App and the other two interaction modalities, and a slightly increase
Nao are significantly correlated (r = 0.43, p < 0.01), in the acceptance rate (but not yet significant). In fact, when
as well as Nao and ENao (r = 0.64, p < 0.01); involved in an interaction, humans expect non-verbal signals
from humanoid robots as well as they did with people. Indeed,
• high familiarity: Pearson shows a strong correlation when robotic emphatic responses (Nao) are absent or not
between App and Nao (r = 0.75, p < 0.01) and sufficient, trust decreases.
between Nao and ENao (r = 0.82, p < 0.01). There is
a moderate correlation for App and ENao (r = 0.60, In most cases, there are correlations between App results
p < 0.01); and Nao, and Nao and ENao, but not between App and ENao.
• low familiarity: finally, we have a moderate correlation In our opinion, the leading cause of these results is due to
between App and Nao (r = 0.39, p < 0.01) and the smaller difference between the interaction with the mobile
between Nao and ENao (r = 0.45, p < 0.01). application and Nao condition (e.g., they provide the same
content, but one with text and the other through speech), and
Both for Nao (F = 3.95, p = 0.05) and ENao (F = 4.89, between Nao and Enao conditions (e.g., they share the same
p = 0.03), in the case of low and high familiarity with robots, interface – an embodied agent – but with different interaction
ANOVA shows significant differences between these categories capabilities). Regarding the first and the third conditions, the
of users. In both cases, the mean values of answers of users large difference between the two modes of interaction implies
with high familiarity is greater than other users. There are not no significant correlations between each other.
significant differences in grouping per English skills.
In future works we will extend the pilot study by selecting
more users in order to extend our evaluation and to achieve
VI. C ONCLUSION more significant results.
Social robots have been used in advertisements in public
spaces mainly because of their greater ability to grab customer
attention with respect to displays. Previous research mainly R EFERENCES
investigated the advantage of a physical body in engaging the
user in an interaction with respect to its virtual counterpart. In [1] C. Kidd and C. Breazeal, “Effect of a robot on user perceptions,” in
this work, we compared, in a pilot study, the effect of a social Intelligent Robots and Systems, 2004. (IROS 2004). Proceedings. 2004
IEEE/RSJ Int. Conf. on, vol. 4, Sept 2004, pp. 3559–3564 vol.4.
robot with different communication channels, with respect to a
well–known interface such as a mobile application in providing [2] W. Wang and I. Benbasat, “Trust in and adoption of online recom-
mendation agents,” Journal of the Association for Information Systems,
recommendations, and evaluated the human behavior through vol. 6, no. 3, p. 72, March 2005.
quantitative and qualitative analysis. From the qualitative ques-
[3] S. M. Merritt and D. R. Ilgen, “Not all trust is created equal: Dis-
tionnaire it arises that the users, on average, perceive the positional and history-based trust in human-automation interactions,”
interaction with the mobile application (App) easier than those Human Factors, vol. 50, no. 2, pp. 194–210, 2008.
with the social robot (Nao, ENao) independently from the [4] A. Powers, S. Kiesler, S. Fussell, and C. Torrey, “Comparing a computer
degree of interaction capabilities. Furthermore, App appeared agent with a humanoid robot,” in 2nd ACM/IEEE Int. Conf. on Human-
more natural and satisfying than both Nao and ENao robot. Robot Interaction (HRI), March 2007, pp. 145–152.
This result naturally arises from the fact that the most of the [5] M. Shiomi, K. Shinozawa, Y. Nakagawa, T. Miyashita, T. Sakamoto,
users have more familiarity with mobile applications rather T. Terakubo, H. Ishiguro, and N. Hagita, “Recommendation effects of
a social robot for advertisement-use context in a shopping mall,” Int.
than with robots. Moreover, the sole presence of the robot Journal of Social Robotics, vol. 5, no. 2, pp. 251–262, 2013.
does not provide an improvement in the acceptance rate, while
[6] W. Bainbridge, J. Hart, E. Kim, and B. Scassellati, “The effect of
the additional communication capabilities provided by the presence on human-robot interaction,” in The 17th IEEE Int. Symposium
ENao humanoid robot generate for the users a higher level on Robot and Human Interactive Communication (RO-MAN), Aug
of satisfaction with respect to the expectations compared with 2008, pp. 701–706.

37
Proc. of the 16th Workshop “From Object to Agents” (WOA15) June 17-19, Naples, Italy

[7] C. M. de Melo, L. Zheng, and J. Gratch, “Expression of Moral Emotions [10] K. Shinozawa, F. Naya, J. Yamato, and K. Kogure, “Differences in
in Cooperating Agents,” in Proceedings of the 9th International Confer- effect of robot and screen agent recommendations on human decision-
ence on Intelligent Virtual Agents (IVA), Amsterdam, The Netherlands, making,” Int. J. Hum.-Comput. Stud., vol. 62, no. 2, pp. 267–279, Feb.
Sep. 2009. 2005.
[8] T. B. R and B. Ross, “Emotional expressivity and trustworthiness: The [11] S. Dooms, T. De Pessemier, and L. Martens, “Movietweetings: a movie
role of nonverbal behavior in the evolution of cooperation,” in Journal rating dataset collected from twitter,” in Workshop on Crowdsourcing
of Nonverbal Behavior, A. R. Library, Ed., vol. 23(3), 2003. and Human Computation for Recommender Systems, CrowdRec at
[9] C. Nass and Y. Moon, “Machines and mindlessness: Social responses RecSys 2013, 2013.
to computers,” Journal of Social Issues, vol. 56, pp. 81–103, 2000.