Decision Support in Tourism through Social Robots: Design and Evaluation of a Conversation-Based Recommendation Approach Based on Tourist Segments

Abstract

Tourism recommendation systems can mitigate the potential impact of choice overload on tourists. Social robots are a promising approach to provide recommendations to tourists through an engaging and intuitive user interface on sites like tourist information ofices. This study investigates whether tourists perceive tourism recommendations provided via social robots as a satisfying and efective experience and whether tourists respond better to a more human or robotic design of social robot interactions. Therefore, an experiment is conducted at a real-world tourist information ofice where 60 tourists are exposed to either the more human or robotic version of the social robot recommender system. Their feedback is collected with a survey. The results show that the social robot is perceived positively across all user-centric evaluation dimensions. This indicates that tourists accept social robots in real-world tourist recommendation situations and would also use them in the future.

Keywords

• eol>Tourism Recommendation System
• Human-Robot Interaction
• Social Robot
• Furhat

1. Introduction

Tourists are exposed to a wide range of options in their search for a suitable destination and local activities, which can lead to choice overload situations [ 1 ]. Recommendation systems can mitigate this choice overload by helping the user make a decision by reducing the number of options for a destination or activity that matches their individual preferences [ 2 ]. An emerging and promising approach to ofer automated recommendations on sites like tourist information ofices is social robots. Social robots can handle complex dialogues and understand and express emotions, e.g., via gestures [ 3 ]. These robots interact with the users via speech technology which allows their use in an intuitive and engaging way. Nevertheless, more evidence is needed whether tourists would perceive recommendations provided via social robots as a satisfying and efective experience. According to the Uncanny-Valley-Theory, the degree of human-likeness determines users’ response to a social robot [ 4 ]. Human-like design can be achieved, for instance, by visual features like gestures, auditory features like speech recognition and synthesis, and mental features like content understanding [ 5 ]. Nevertheless, the Uncanny-Valley-Theory describes the efect that users tend to respond positively to a human-like robot, but if the robot appears too human, the response becomes negative. The study’s objective is to investigate whether users perceive automated tourism recommendations delivered via social robots as satisfying and efective. A particular focus is whether users prefer a rather robotic or a more human-like social robot interaction. It will also be investigated whether the users accept the recommendation of the social robots and whether they have an influence on their holiday planning. Furthermore, decisive factors in the design of the conversation will be investigated, and the users’ perceptions of a more human-like version and a rather robotic version of the social robot will be compared.

2. Theoretical Framework

2.1. Conversation Design with Social Robots

Unlike industry robots, social robots’ main task is to interact socially with humans. These interactions are generally based on complex dialogues, understanding and expression of emotions, and the social robots’ possession of personality and social competencies to a certain degree [ 3 ]. In this study, the social robot Furhat is used to design complex and human-like verbal and non-verbal interactions. This social robot is realized as a head with an animated face using a projector. The interactions are based on facial gestures, the detection of people and their emotions via the camera, and a voice-based multiparty conversational system [ 6, 7 ]. Social characteristics should be considered in the interaction design to create satisfying social robot interaction and avoid frustration and dissatisfaction. In the following relevant characteristics for the interaction design of social robot recommender system are introduced, which can be classified into the categories "Conversational Intelligence", "Social Intelligence" and "Personification" [ 8 ]. Conversational Intelligence This describes the social robot’s ability to behave "humanly" in conversations and consists of the following three factors: [ 8 ] • Proactivity means that a social robot can act independently in the user’s interest [ 9 ] and therefore can grasp the context of the conversation and refer to what is said [ 8 ]. • Conscientiousness describes the ability to follow the conversation and give appropriate answers [ 10 ]. In order to achieve this, the social robot needs to understand the user’s intent [ 11 ]. • Communicability refers to the ability to communicate efectively and eficiently what it was designed for and how it needs to be interacted with [ 12 ]. The robot should be able to present and communicate its capabilities and functions to the user [ 13 ]. It can be achieved by stating the purpose of the robot right at the beginning through an opening message [ 8 ].

For this study, various dialogues were designed before the programming of the interactions. In both robot versions, care was taken to ensure that the robot can assign the correct intent to as many diferent formulations as possible and that it explains its capabilities to the user at the beginning of the dialogue.

Social Intelligence Social Intelligence refers to the individual’s capability to demonstrate an appropriate social behavior for achieving a desired goal [ 8, 14 ]. The following three main concepts were considered for the implementation of the recommender system: • Damage control refers to the robot’s ability to deal with conflicts [ 15]. When talking to a robot, users are more conflict-prone [ 16], invest less time [17] and use vulgar language more often [18]. To counter abuse, the robot can show an emotional reaction [19], can react authoritatively [20] or escalate the conversation to a human employee [21]. • Manners describes the ability to be polite. [ 10 ]. To meet social norms of conversation, the robot should be at least as polite as his counterpart [15]. The ability to make small talk supports the impression of a "polite" robot [22] and is often expected [23]. • Personalization means the ability to tailor functionality, interface, content and behavior to the user [24].

The humanoid version of the social robot in the study was equipped with extensive damage control functionality to be able to react appropriately to insults. Here, an emotional reaction supported by facial expressions was combined with an authoritarian reaction. In addition, emphasis was placed on the humanoid social robot being always polite and being able to give an answer to a question about the robot’s well-being. The personalization of the social robot is mainly achieved through the individual recommendation that is given at the end of the dialogue, which was the case for both versions.

Personification The identity and personality of a social robot are influenced by its appearance and how it speaks [25] as well as the choice of gender, age, and name [ 8 ]. The presence of identity leads to more engagement and perceived human-likeness [26].

The more humanoid version of the social robot has a female face and voice, which sounds neutral in German and has an American accent in English (as the majority of native Englishspeaking tourists in Rothenburg are from the USA). In addition, the robot is designed to be friendly and helpful. The social robot was given the name "Victoria" because this fits its feminine appearance and voice and is also the name of the text-to-speech voice used. The robotic version of the robot was given the same name to exclude influences here but was equipped with a diferent face that visually resembles a robot. In addition, the neural voice was replaced by a monotone-sounding standard voice.

2.2. Types of Tourists

The social robot aims to give users a recommendation for a suitable activity in the city of Rothenburg ob der Tauber at the end of the conversation. Since the creation of a personality profile individually tailored to each person is too complex, the test persons are assigned to a specific tourist type, based on which an activity is then recommended. This goes hand in hand with the theory of segmentation in marketing, as products cannot be specifically adapted to p e e t D h g u o S e c n e i r e p xE llow a h S

Incidental

Cultural

Tourist

Casual

Cultural

Tourist

Sightseeing

Cultural

Tourist

Low

Importance of cultural tourism in High the decision to visit a destination each customer [27]. Assuming that tourism to Rothenburg is generally characterized by the fact that visitors are looking for a cultural experience, a classification of cultural tourists is selected. A widely used theory for classifying cultural tourists is the classification according to McKercher [28]. McKercher divides cultural tourists into five diferent types, each difering in the experience sought and the importance of cultural tourism in deciding on a destination (see Figure 1). For the purposeful cultural tourist, learning about the culture and having in-depth experiences is central. For the sightseeing cultural tourist, the former is also important. However, for them, entertainment is in the foreground. For the casual cultural tourist, culture plays a secondary role, and he or she has a superficial experience at the destination. The fourth type only happens to participate in cultural activities during his holiday and has only a superficial experience. In the fifth case, culture does not play a role in selecting the holiday destination, but this type of tourist has a profound experience at the destination by serendipity.

3. Design and Implementation of the Prototype

To enable a comparison between a humanoid and a robotic version of the recommender robot, two skills were programmed for the Furhat. Both versions ofer the possibility to interact with the social robot in English and German language. A classic menu structure was used to design the dialogues. After a greeting, which also serves to set the language, the user is led to a "main menu" from which the functions can be accessed. The range of functions was determined in accordance with the research goals and the requirements of the Rothenburg Tourism Information Ofice. In the first version, the social robot was able to make a recommendation for activities based on the type of tourist and provide simple information about guided tours. The humanoid and robotic versions of the robot difer primarily in the extent to which the aspects mentioned in the literature have been implemented to achieve the most human interaction possible. For example, the robotic version cannot respond to insults, does not engage in small talk, and gives less varied answers to the user’s questions. In addition, a robotic appearance and a rather monotonous voice was chosen. Also, the knowledge about the dialogue process is limited to its basic functions so that the user cannot, for example, ask for a repetition of the last piece of information.

3.1. The Recommender System

The recommender system is based on McKercher’s findings on cultural tourists and adapts the matrix for classifying cultural tourist types. The user is asked two questions to assign him or her to a tourist type based on the dimensions of the matrix (see Fig. 1) and to make a recommendation. The dimension "experience sought" is determined by asking the user whether he or she seeks an in-depth cultural experience in Rothenburg (yes/no). The second dimension is determined by the question, how important the aspect of culture was when deciding to travel to Rothenburg (high, medium or low). Based on this, the user is presented with a recommendation which, depending on the type, ofers a profound cultural experience or is more for entertainment purposes. The similar cultural tourism type model supported the selection of specific recommendations according to Pröbstle [ 29], which, in contrast to McKercher, explains the individual types in more detail.

Pre-test and Optimizations A pre-test was conducted before the main study to evaluate the experiment setup. Various changes were made to the programming of the social robot to ensure a smooth dialogue and to implement the feedback from the users and the tourism service. In contrast to the recommendation in the literature that the robots should explain their functions at the beginning and thus manage expectations, this was found to be annoying and too lengthy in the pre-test. For this reason, the greeting by the social robot was simplified to only explain its functions when asked. In addition, the dialogue path to get the information about the city tours was shortened to increase eficiency. After the pre-test, the recommender’s functional scope was expanded to provide information on the nearest ATM and toilets as well as with minor small talk functionalities. Further, the recommendation system now provides two additional tourist recommendations per user type if the first recommendation is not considered relevant by the user. In addition, numerous gestures and facial expressions were implemented in the dialogue of the human version of the robot to promote a more natural appearance and to integrate functions such as the confirmation of a statement intuitively by nodding. Finally, in line with the theoretical research on conversational intelligence, a function was implemented that enables the robot to remember and repeat previous user statements during a conversation.

4. Design and Results of the Experimental User Study

A questionnaire is developed to measure the perceived satisfaction and efectiveness of the developed recommender system within the experiment based on a selection of predefined constructs from the Godspeed questionnaire [ 3 ] and the ResQue (Recommender systems’ Quality of user experience) model [30]. During the questionnaire design, it was assumed that tourists would only participate in the study if the questionnaire is short. Therefore, only a subset of constructs and items was selected to shorten the time efort per participant. The selected constructs are illustrated in Table 1.

The experiment was conducted for two days in the tourist information ofice in Rothenburg ob der Tauber. The experiment setup is illustrated in Figure 2. Random tourists were invited to participate as test users and guided through the conversation process. As part of the experiment, 60 people were interviewed who had interacted with the social robot in the tourist information ofice beforehand. Of these, 31 participants had contact with the robotic version of the social robot and 29 participants with the human version. The mean age of the respondents was 44.27, with the youngest being 14 years old and the oldest being 79 years old. From the language perspective, 38 respondents used the German version of the robot, and 22 used the English one. The resulting answers are illustrated in Table 2. The intelligence aspect was rated with an average of 3.93 out of 5.00 and is thus in the upper range. The diferent versions of the robot difer only minimally with a mean diference value of 0.08. The robot was largely perceived as likeable, as the mean value for the question category "likeability" is 4.24. For example, only one person stated that the robot was "unfriendly". The most remarkable diference between the robot versions can be observed in likeability, where the mean for the human version is 4.66 and for the robotic version 3.79. The distribution of the likeability values is illustrated in Figure 3.

Regarding the accuracy of activity recommendation, the social robot was rated 3.84 on average. On the question of whether the robot suggested novel and interesting activities for the user, the average rating is 3.25. While the accuracy of the human version of the robot was rated 0.47 worse on average, the novelty is valued 0.35 better on average. It should be noted that the recommendations are identical in both versions and no adjustments were made to the dialogues. According to the testers, finding an activity they want to do with the recommendation system is mostly easy. The value for this criterion is 3.96, with a standard deviation of 0.93. Overall, 38 out of 60 participants were (very) satisfied with the advice provided by the social robot, which corresponds to a mean value of 3.87 on the rating scale. When it comes to wanting to use a similar recommendation system again in the future, the average score is 3.46 out of 5.00 and 35 out of 60 people rated this question as 4 or 5. The perceived usefulness of the recommendation was rated at 3.31 and is thus in the medium positive range. Within this construct, however, there is wide variation within the used 3 questions. For example, 40 out of 60 people feel supported by the social robot in finding what they like, but only 5 participants said that the recommendations influenced their previous holiday planning. Therefore, the average score for this question is only 2.60 out of 5.00. Looking at the correlations between the individual constructs, few correlations can be identified. The greatest correlation is found in the combination of the accuracy of the recommendation and the satisfaction with the counseling. Here, the correlation value is 0.84. For the aspects intelligence and perceived usefulness, there is also a certain positive correlation in connection with satisfaction with the recommendation (0.72, 0.77). Furthermore, the diferent mean value for the diferent application versions regarding sympathy is confirmed here by a negative correlation (-0.55) between sympathy and version. It is also striking that likeability does not show a meaningful correlation with any of the other aspects mentioned. The questionnaire also ofered the possibility to leave comments on the experience with the social robot in a free text at the end. The evaluation of these comments showed that 10 study participants stated that the robot did not understand them well. Eight participants said the robot had been "very friendly", which fits with the scores from the rest of the questionnaire regarding sympathy. Ten percent (6) of respondents said that the conversation overall was not interactive enough and that they did not have time to explore the robot’s recommendations because of pre-existing plans. Five participants lacked human contact or personal interaction.

5. Discussion

Overall, the user study results indicate that the participants perceive tourist recommendations via social robots well. Nevertheless, two important limitations are not directly evident from the results of the questionnaire data: • Without an active approach by the study organizers, only a fraction of the people would have sought a conversation with the robot on their own initiative. • Without active support during the conversation, the dialogue would not have gone through properly in most cases. This had technical reasons, as the speech recognition could not always provide reliable results at the beginning of the conversation.

When comparing the results of the two versions of the recommender system, the sympathy value for the human version is higher than for the robotic version. However, it also showed that sympathy does not influence the users’ behavioral intentions. The two versions have no remarkable diferences in the other measured constructs. It should be noted, however, that some strategies proposed and implemented in the literature are only useful when the user interacts freely with the robot, which was not the case in this study due to the limitations. Nevertheless, it became clear in the personal conversations with the users that these extended functionalities can positively afect the perception of the social robot if they are suficiently well elaborated. An important result of the survey is that although the advice was very positively received, the tourists’ holiday planning was not influenced by the recommendations, as in their case, the planning was already done before the trip. In future research, validating and deepening these ifndings with a larger sample based on a longer time frame is recommended. Furthermore, the design of the experiment setup should enable a completely independent interaction of the test users through, e.g., improved interaction design and robust speech recognition.

6. Conclusion

The study aimed to evaluate the users’ perception of a tourism recommender system using a social robot and to examine the influence on the users’ behavioral intentions. In addition, the change in users’ perception of a human-like and a robotic version of the social robot was compared. The questionnaire evaluation indicates that the tourists perceived the social robot recommender as satisfying and efective in both versions. The majority of users also stated that they would use such a recommendation service again in the future. However, the recommendation only had a minor influence on the users’ holiday planning. The sympathy value for the human version is higher than for the robotic version, but the perceived sympathy does not influence the users’ behavioral intentions. The diferences between the two versions of the robot in the individual criteria are only marginal and suggest that a lower human-like design of the social robot only has a minor influence on the quality of the decision support. [15] S. B. Daily, M. T. James, D. Cherry, J. J. Porter, S. S. Darnell, J. Isaac, T. Roy, Afective Computing: Historical Foundations, Current Applications, and Future Trends, in: Emotions and Afect in Human Factors and Human-Computer Interaction, Elsevier, 2017, pp. 213–231. doi:10.1016/B978-0-12-801851-4.00009-4. [16] Y. Mou, K. Xu, The media inequality: Comparing the initial human-human and human-AI social interactions, Computers in Human Behavior 72 (2017) 432–440. doi:10.1016/j. chb.2017.02.067. [17] N. Shechtman, L. M. Horowitz, Media inequality in conversation: how people behave diferently when interacting with computers and people, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, Ft. Lauderdale Florida USA, 2003, pp. 281–288. URL: https://dl.acm.org/doi/10.1145/642611.642661. doi:10.1145/642611.642661. [18] J. Hill, W. Randolph Ford, I. G. Farreras, Real conversations with artificial intelligence: A comparison between human–human online conversations and human–chatbot conversations, Computers in Human Behavior 49 (2015) 245–250. doi:10.1016/j.chb.2015.02. 026. [19] P. Wallis, E. Norling, The trouble with chatbots: social skills in a social world, The Society for the Study of Artificial Intelligence and the Simulation of Behaviour, Hatfield, UK, 2005, pp. 29–38. [20] A. Silvervarg, A. Jönsson, Iterative Development and Evaluation of a Social Conversational Agent, in: Proceedings of the Sixth International Joint Conference on Natural Language Processing, Asian Federation of Natural Language Processing, Nagoya, Japan, 2013, pp. 1223–1229. URL: https://aclanthology.org/I13-1174. [21] M.-C. Jenkins, R. Churchill, S. Cox, D. Smith, Analysis of User Interaction with Service Oriented Chatbot Systems, in: D. Hutchison, T. Kanade, J. Kittler, J. M. Kleinberg, F. Mattern, J. C. Mitchell, M. Naor, O. Nierstrasz, C. Pandu Rangan, B. Stefen, M. Sudan, D. Terzopoulos, D. Tygar, M. Y. Vardi, G. Weikum, J. A. Jacko (Eds.), Human-Computer Interaction. HCI Intelligent Multimodal Interaction Environments, volume 4552, Springer Berlin Heidelberg, Berlin, Heidelberg, 2007, pp. 76–83. doi:10.1007/978-3-540-73110-8_9, series Title: Lecture Notes in Computer Science. [22] Q. V. Liao, M. Mas-ud Hussain, P. Chandar, M. Davis, Y. Khazaeni, M. P. Crasso, D. Wang, M. Muller, N. S. Shami, W. Geyer, All Work and No Play?, in: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, ACM, Montreal QC Canada, 2018, pp. 1–13. doi:10.1145/3173574.3173577. [23] G. Dyke, D. Adamson, I. Howley, C. Penstein Rosé, Towards Academically Productive Talk Supported by Conversational Agents, in: D. Hutchison, T. Kanade, J. Kittler, J. M. Kleinberg, F. Mattern, J. C. Mitchell, M. Naor, O. Nierstrasz, C. Pandu Rangan, B. Stefen, M. Sudan, D. Terzopoulos, D. Tygar, M. Y. Vardi, G. Weikum, S. A. Cerri, W. J. Clancey, G. Papadourakis, K. Panourgia (Eds.), Intelligent Tutoring Systems, volume 7315, Springer Berlin Heidelberg, Berlin, Heidelberg, 2012, pp. 531–540. URL: http://link.springer.com/10. 1007/978-3-642-30950-2_69. doi:10.1007/978-3-642-30950-2_69, series Title: Lecture Notes in Computer Science. [24] H. Fan, M. S. Poole, What Is Personalization? Perspectives on the Design and Implementation of Personalization in Information Systems, Journal of Organizational Computing and

References

[1] N. T. Thai , U. Yuksel, Choice overload in holiday destination choices , International Journal of Culture, Tourism and Hospitality Research 11 ( 2017 ) 53 - 66 .

[2] F. Ricci , Recommender Systems in Tourism, in: Z. Xiang , M. Fuchs , U. Gretzel , W. Höpken (Eds.), Handbook of e-Tourism, Springer International Publishing, Cham, 2020 , pp. 1 - 18 .

[3] M. Jung , M. J. S. Lazaro , M. H. Yun , Evaluation of methodologies and measures on the usability of social robots: A systematic review , Applied Sciences 11 ( 2021 ). doi: 10 .3390/ app11041388.

[4] M. Mori , Bukimi no tani [the uncanny valley] . , Energy 7 ( 1970 ) 33 .

[5] N. Pfeufer , A. Benlian , H. Gimpel , O. Hinz , Anthropomorphic information systems , Business & Information Systems Engineering 61 ( 2019 ) 523 - 533 .

[6] S. Al Moubayed , G. Skantze , J. Beskow , K. Stefanov , J. Gustafson , Multimodal multiparty social interaction with the furhat head , in: Proceedings of the 14th ACM International Conference on Multimodal Interaction , ICMI '12, Association for Computing Machinery, New York, NY, USA, 2012 , p. 293 - 294 . doi: 10 .1145/2388676.2388736.

[7] S. Al Moubayed , G. Skantze , J. Beskow, The furhat back-projected humanoid head-lip reading, gaze and multi-party interaction , International Journal of Humanoid Robotics 10 ( 2013 ). doi: 10 .1142/S0219843613500059.

[8] A. P. Chaves , M. A. Gerosa , How Should My Chatbot Interact? A Survey on Social Characteristics in Human-Chatbot Interaction Design , International Journal of Human-Computer Interaction 37 ( 2021 ) 729 - 758 . doi: 10 .1080/10447318. 2020 . 1841438 .

[9] A. Salovaara , A. Oulasvirta , Six modes of proactive resource management: a user-centric typology for proactive behaviors, in: Proceedings of the third Nordic conference on Human-computer interaction , ACM, Tampere Finland , 2004 , pp. 57 - 60 . doi: 10 .1145/ 1028014.1028022.

[10] K. Morrissey , J. Kirakowski, 'Realness' in Chatbots: Establishing Quantifiable Criteria, in: D. Hutchison , T. Kanade , J. Kittler , J. M. Kleinberg , F. Mattern , J. C. Mitchell , M. Naor , O. Nierstrasz , C. Pandu Rangan , B. Stefen , M. Sudan , D. Terzopoulos , D. Tygar , M. Y. Vardi , G. Weikum, M. Kurosu (Eds.), Human-Computer Interaction . Interaction Modalities and Techniques , volume 8007 , Springer Berlin Heidelberg, Berlin, Heidelberg, 2013 , pp. 87 - 96 . doi: 10 .1007/978-3- 642 -39330-3_10, series Title: Lecture Notes in Computer Science.

[11] M. McTear , Z. Callejas , D. Griol , The Conversational Interface, Springer International Publishing, Cham, 2016 . doi: 10 .1007/978-3- 319 -32967-3.

[12] R. O. Prates , C. S. de Souza, S. D. J. Barbosa , Methods and tools: a method for evaluating the communicability of user interfaces , Interactions 7 ( 2000 ) 31 - 38 . doi: 10 .1145/328595. 328608.

[13] F. A. M. Valério , T. G. Guimarães , R. O. Prates , H. Candello , Here's What I Can Do: Chatbots' Strategies to Convey Their Features to Users, in: Proceedings of the XVI Brazilian Symposium on Human Factors in Computing Systems , ACM, Joinville Brazil , 2017 , pp. 1 - 10 . doi: 10 .1145/3160504.3160544.

[14] K. Björkqvist , K. Österman , A. Kaukiainen , Social intelligence- empathy= aggression?, Aggression and violent behavior 5 ( 2000 ) 191 - 200 .