-Social robots can be used as interfaces to provide recommendations to users. While a vast literature compares the user's behavior when interacting with a robot with respect to a virtual agent, in this paper, we conduct a first evaluation on how the user's choices are affected if the recommendations are provided respectively by a mobile application or by social robots with different degree of interaction capabilities. This pilot study shows that the sole embodiment condition of the robot does not imply significant changes in the users' choices that prefer to interact with the mobile application. However, the adoption of additional communication channels such as gestures, gaze and voice pitch, which change accordingly to the suggested movie genre, improves the users acceptability.
Social robots will be used in the next future in many
application domains, which span from entertainment and
education to health–care. In order to be accepted in our houses,
they should be perceived as trusting, helpful, reliable and
engaging [
Recommendation systems aim to provide the user with
personalized advises and suggestions in many different
domains, such as books, movies or music. Such suggestions are
provided according to the available information the system
has on the user (e.g., his/her preferences or his/her past
interaction with the system). Hence, recommendations can be
provided suggesting items similar to items liked by the user
or liked by similar users. The effectiveness of the provision of
recommendations relies itself on the concept of trust [
In this paper, we present a pilot study to evaluate the extent of the use of a robotic system in accepting a recommendation not with respect to a virtual agent, but to very common interfaces such as mobile applications. Our experiments aim at evaluating the users’ acceptance of recommendations as well as their engagement when the robot or a mobile application are displaying such advises. In particular, we provided the same information contents on recommended movies, but using three different interaction conditions. In the first condition, by using a mobile application, contents will be provided by text shown on the mobile screen. In the second one, the same contents will be provided by using a humanoid robot interacting using speech. Finally, in the third condition the humanoid robot will encompass both voice and genre-driven motion primitives.
II.
A vast literature compared the behavior as well as the
acceptance of robots with respect to their virtual counterparts.
Embodied robots are consistently perceived as more engaging
than a character on a video display, and sometimes as engaging
as a human. For example, in Kidd and Breazeals [
How physical embodiment, as opposed to virtual
presence, affects human perception of social engagement with
an artificial agent was addressed in [
Finally, in [
In order to evaluate our hypothesis, we developed a client/server application, where the server provides the recommendation service and the possible clients can be a humanoid robot or a mobile application. Clients are in charge to ask for a list of recommendations (in particular of movies) to the server and to show them to users. The social robots and the mobile application will provide the same information, but in different ways (i.e., through different communication channels). This diversity should be reflected in a different perception of the recommendations by the users, and, presumably, it will affect their choices. In order to provide recommendations, the Recommendation Engine needs some initial movie ratings from the users. Hence, independently from the client type the users interact with, the initial ratings are performed by using the mobile application, which allows users to easily evaluate movies by means of a friendly graphical interface. The main blocks of the developed framework are detailed in the following subsections.
The server layer of this architecture is characterized by the recommendation system (see Figure 1). It is developed in Java and it is hosted by a Tomcat servelet container. The core of this layer is a Web Server, used to store, process and deliver requested content to clients, which are provided through JSONbased API both to communicate with the humanoid robot and with the android mobile application. The module Recommendation Engine is the core of the recommendation layer. We adopted the Apache Mahout library1 to predict the user ratings, and chosen the MovieTweetings [11] dataset to train the system and to populate the Ratings Repository. MovieTweetings consists of movie ratings contained in well-structured tweets on the Twitter.com social network. This information is contained in three files: users.dat, ratings.dat and movies.dat, which provide respectively the user identification number, his/her associated ratings and a list of movies. The dataset is updated every day, therefore its size is constantly changing. At the last access, it contained about 35000 users, 360000 ratings and 20000 movies.
The recommendation engine provides rating predictions when the recommendation API is invoked. To achieve this goal, we used item-based City Block distance, also known as Manhattan distance. In Mahout implementation, the generic movie i is represented by a boolean vector:
i = [r1i, r2i, ..., rni], where n is the number of users in the dataset and rui = 1 if user u rated the movie i. The distance between two movies rated by user u is the sum of the absolute value of the differences of the two associated vector components. More formally, the distance between items i and j is:
n d(i, j) = X u=1 |rui − ruj |.
On the bottom of Figure 1, the architecture design of the mobile Android application is depicted. The first duty for the user, when he/she accesses the application, is to sign up/sign into the system. As explained above, when the interaction starts, users have first to provide a certain number of movie ratings (at least 20 movies). The Training module is dedicated to provide an interface to get movie lists and to store movies ratings. If a user is in the training stage, he/she can browse movies by ordering them by most rated or randomly, or search for a movie (filtering by genre or title, see Figure 2).
After this first stage, user can get movie recommendations from the server. When the server gets the recommendation request, once calculated the best movies for the user, it retrieves additional details about the film, like, for example, the director, writers, actors and genres using OMDb2 web service. Fortunately, MovieTweetings data set stores, for each movie, its IMDb id, which can be used to address the OMDb service. The Android application shows on the screen the recommendations for the users through textual and graphical descriptions.
The Robot Client architecture has been designed considering the adoption of a NAO T14 robot model, consisting in a humanoid torso with 14 degrees of freedom (2 for the head and 12 for the arms) developed by Aldebaran Robotics3. We controlled the NAO platform by means of the standard robotic operating system (ROS) and using the Python programming language for developing the ROS nodes. NAO is endowed with two main sensors: a camera and a microphone through which it receives signals from the external environment. Camera frames are processed by a Face Detection module to detect users presence into its visual field. Sounds obtained from the microphone are processed by Speech Recognition module.
As actuators NAO can use the following communication channels: voice, arms, head and eyes led. The Behavior Selector module is in charge of providing two different interaction conditions. In the first case, it presents to users the recommended movies and their relative information only through speech, while, in the second case, such description is accompanied with gestures, gaze, eyes coloring through the Motion Controller, and pitch voice changes through Speech Synthesis. In this latter case, the Behavior Selector gets motion animations from an Animations Repository, based on the genre of the recommended movie, to execute animated speech, as will be detailed in the followings.
2http://www.omdbapi.com - The Open Movie Database is a free web service to obtain movie information.
3https://www.aldebaran.com/en
a) Face Detection: This module is based on a face detection/recognition solution provided by OKI and included in the Python SDK for NAO. Such module continuously processes frames from the NAO camera in order to detect a human face. Once a face is detected, it provides its position. Moreover, in the third condition, the module continues to provide the user position coordinates to the Motion Controller, that allows NAO to track the face by moving its head.
b) Speech Recognition: This module gives to the robot the ability to recognize a predefined words list, and specifically the usernames and the acceptance/rejection of a recommendation. It is based on module provided by Aldebaran, which relies on sophisticated speech recognition technologies provided by NUANCE for NAO Version 4. Before starting, the robot needs to receive the list of usernames (UsersList). Then, once the system has detected a face through Face Detection, NAO asks for a username and listens until a word is recognized. Currently, system does not provide a real authentication when interacting with the robot because the only way to communicate with NAO is the speech. Users should sign in through an input system like a keyboard or a mobile application.
c) Behavior Selector: Through this module, we generate all the gestures, gaze and the feedback for the user. Once a user has been recognized, a user tracking system allows the robot to track the target by moving its head. Movie information is provided to the user with the Speech Synthesis module with different speech intonations, but it can be accompanied with arms gestures and facial expressions (e.g., different eyes colors) generated through Motion Controller module. The Behavior Selector gets recommendations from the Web Service and related animations from the Animations Repository. The main task of this module is mapping the movie genre into a predefined set of animations and eyes colors. For example, if NAO recommends a drama, led eyes become red and gestures are more serious, while for a comedy led eyes become green and gestures are more joyous. The pitch of the voice is accordingly manipulated by the Speech Synthesis module.
IV.
The testing procedure main steps are: (i) the user provides new rates for a list of movies (training phase) at the beginning of the interaction; (ii) the recommendation system generates the top-six recommendations for each user, which will be shown to users through the three interaction conditions in a random way (two for each condition); (iii) after each test condition the participant has to answer to a questionnaire concerning the specific condition and at the end of the overall experiment to a general questionnaire.
The design of this study is a within-subjects, counterbalanced, repeated measures experiment. The three considered interactive conditions are the following:
Condition 1 (App): in this setting, neither of the robot modalities are used. The user only interacts with the mobile application that provides to the user two different movies suggestions. For each movie the app provides the title and additional information by displaying text and images on the screen. For each recommended movie, the user has to reply about his/her likelihood to see it (see Figure 3).
Condition 2 (Nao): in this setting, the robot is located on a table standing still and waiting for a person to interact with. When the robot recognizes a face in its filed of view, it greets the person, introduces itself, and asks for a username. NAO presents the two recommendations by telling the movie title accompanied with the same information provided in Condition 1 (plot, genre, actors, and so on). Finally, it asks the user if she/he agrees to see this movie and stores the answer. Condition 3 (ENao): in this setting, differently from the previous condition, the robot is endowed with the motion controller module. When the robot is not interacting with anyone, it simply looks around and waits for a person to talk with. In this interaction condition NAO, in addition to tell movie information, gesticulates, changes eyes led color and voice pitch according to the recommended movie genre (see Figure 4).
C. Participants
18 subjects participated in this experiment with an average age of 32 years and a graduate education, for a total of 13 males and 5 females. All the participants were Italian native speakers with an average English level of 2.39 and Robotic skills of 3.17 on a likert–scale from 1 to 5. The language adopted for the experiment was the English both for text description and for the robot’s voice synthesizer. The testers were not informed about the NAO interaction capabilities. In Table I, personnel data of participants are collected.
We hypothesized that the robot as compared with the application will be more engaging and better liked, and hence recommendation provided by the robot should be more likely to be accepted. Moreover, the condition with animated motion should be more engaging and better liked with respect to the simple robot.
In order to evaluate the degree of acceptance of the recommendations when provided by different conditions, we calculated the selection ratio indicating the number of accepted recommendations with respect to the total number of recommendations for a each specific condition.
Fig. 5. Percentage of accepted movie recommendations for each Condition.
In Figure 5 the selection ratio is expressed in percentage shows that there is a minimum difference in the acceptance rate between the recommendations provided by App and Nao, while there is a slightly bigger difference between App and ENao conditions. This fact is in accordance with our hypothesis that people are inclined to accept more recommendations provided through a more natural interaction, even if the sole embodiment condition (Nao) does not imply significant changes in the testers acceptability level. However, due to the limited number of participants and recommendations provided to each participant these differences, evaluated using ANOVA, are not yet statistically significant, while there is a significant Pearson correlation between App and Nao conditions (r = 0.43 with p = 0.08 that is significant at p < 0.10) acceptance trends. As future work we will extend such experimentation with a greater number of users.
Since users selected for testing are all Italian native speakers, and not all have the same level of familiarity with robotics applications, we felt it appropriate to consider data by aggregating the results by both the level of English proficiency (e.g., the language used to provide recommendations) and the degree of experience with robots. We thus computed correlations for the acceptance ratio and among conditions couples by grouping users with a high (from 4 to 5) or low English level (from 1 to 3) and a high or low familiarity with robots (see Section V-B): • • • • high English level: Pearson showed a negative strong correlation (r = −0.76 with p < 0.05) between Nao and ENao; low English level: there is a significant correlation between App and Nao (r = 0.57 with p = 0.07); high familiarity: nothing significant; low familiarity: once again we had a moderate correlation between App and Nao (r = 0.65 with p = 0.08), but in this case also Nao and ENao have a moderate correlation (r = 0.65 with p = 0.08).
These results show that for testers with a low English level reading text from an application or hearing speech from a robot does not have a relevant impact on the decision making, while for good English skilled participants adding an animated behavior changes the acceptance trend. This could be due to the fact that the users attention in the first case is quite all focused on understanding the text or speech. Moreover, users with low familiarity in robotics shows acceptance trend similar in both Nao and ENao cases.
Our evaluation also takes into account the impressions of users with respect to the interaction with the different conditions. For this aim we propose a qualitative questionnaire organized in three specific sections: (i) personal information for collecting information about the user (age, gender, english level, and familiarity with robotics); (ii) Qualitative questions regarding the application easy of use, naturalness and satisfiability consisting of 6 questions; (iii) two question specific for the conditions involving the humanoid, dealing with the sense of trust and movements naturalness of the robot. While the general information have been asked at the beginning of the tests, the testers have been asked to reply to the specific questions at the end of each experiment. We adopted a classical likert scale from 1 to 5. Only for question 6 we explicitly ask for a preference by the users where index from 1 to 3 represent respectively the preference for APP, NAO and ENAO. The questionnaire structure is reported in Table II.
Section Personal Information Qualitative Questions Robot-related Questions Nao-ENao: there is a moderate correlation for Q2 (r = 0.59, p = 0.01) and Q4 (r = 0.48, p = 0.04) and a strong correlation for Q5 (r = 0.72, p < 0.01);
If we observe the histogram of question 6 (see Figure 6(b)), a part from the approval ratings average of the qualitative questions from Q1 to Q5, it is quite evident that the major part of the users prefers to interact with the Humanoid endowed with emotion-based capabilities. This is probably due to the fact that the humanoid robot has the potential to portray a rich repertoire of non–verbal behaviors that have familiar social meaning for users, who perceive the interaction more natural and engaging because of the received socially intelligent responses by the robot. Histogram in Figure 6-(c) shows that the robotic agent is perceived in the average believeble both if it shows or not non-verbal feedback, and the agent motion is perceived as natural.
As for the quantitative case, for each pair of conditions, we try to correlate answers considering only users with high or low English level or familiarity with robots applications: high English level: there is a moderate correlation between App and Nao (r = 0.53, p < 0.01), and Nao and ENao (r = 0.41, p < 0.01); • • • low English level: as in the previous case, App and Nao are significantly correlated ( r = 0.43, p < 0.01), as well as Nao and ENao (r = 0.64, p < 0.01); high familiarity: Pearson shows a strong correlation between App and Nao (r = 0.75, p < 0.01) and between Nao and ENao (r = 0.82, p < 0.01). There is a moderate correlation for App and ENao (r = 0.60, p < 0.01); low familiarity: finally, we have a moderate correlation between App and Nao (r = 0.39, p < 0.01) and between Nao and ENao (r = 0.45, p < 0.01).
Both for Nao (F = 3.95, p = 0.05) and ENao (F = 4.89, p = 0.03), in the case of low and high familiarity with robots, ANOVA shows significant differences between these categories of users. In both cases, the mean values of answers of users with high familiarity is greater than other users. There are not significant differences in grouping per English skills.
VI.
CONCLUSION
Social robots have been used in advertisements in public spaces mainly because of their greater ability to grab customer attention with respect to displays. Previous research mainly investigated the advantage of a physical body in engaging the user in an interaction with respect to its virtual counterpart. In this work, we compared, in a pilot study, the effect of a social robot with different communication channels, with respect to a well–known interface such as a mobile application in providing recommendations, and evaluated the human behavior through quantitative and qualitative analysis. From the qualitative questionnaire it arises that the users, on average, perceive the interaction with the mobile application (App) easier than those with the social robot (Nao, ENao) independently from the degree of interaction capabilities. Furthermore, App appeared more natural and satisfying than both Nao and ENao robot. This result naturally arises from the fact that the most of the users have more familiarity with mobile applications rather than with robots. Moreover, the sole presence of the robot does not provide an improvement in the acceptance rate, while the additional communication capabilities provided by the ENao humanoid robot generate for the users a higher level of satisfaction with respect to the expectations compared with the other two interaction modalities, and a slightly increase in the acceptance rate (but not yet significant). In fact, when involved in an interaction, humans expect non-verbal signals from humanoid robots as well as they did with people. Indeed, when robotic emphatic responses (Nao) are absent or not sufficient, trust decreases.
In most cases, there are correlations between App results and Nao, and Nao and ENao, but not between App and ENao. In our opinion, the leading cause of these results is due to the smaller difference between the interaction with the mobile application and Nao condition (e.g., they provide the same content, but one with text and the other through speech), and between Nao and Enao conditions (e.g., they share the same interface – an embodied agent – but with different interaction capabilities). Regarding the first and the third conditions, the large difference between the two modes of interaction implies no significant correlations between each other.
In future works we will extend the pilot study by selecting more users in order to extend our evaluation and to achieve more significant results.