1 'Keep the user in mind!' Persuasive Effects of Social Robot as Personalized Nutritional coach.

Abstract

In this paper we investigate the use of a social robot as an interface for providing personalized information about nutrition. In particular, we evaluated the effect of message adaptation to some demographic traits that are automatically recognized by a social robot, Pepper in this case. The proposed approach is based on soft biometrics and it can estimate several traits simultaneously, such as gender and age of people in the field of view of the robot. Our hypothesis is that adding this capability to a social robot improves the persuasive effects, in terms of perceived informational quality, motivational strength and social believability, and the recall of the message. We performed a preliminary experimental study and its results seem to support the persuasiveness of the personalized nutritional coach.

Keywords

• Personalization
• Social Robot
• Soft Biometrics
• Persuasive technologies

-

Social robots are physically embodied, autonomous agents that communicate and interact with humans on a social and emotional level. They represent an emerging field of research focused on developing a “social intelligence” to maintain the illusion of dealing with a human being [ 1, 2 ]. They are being applied in several domains such as elderly care [ 3 ], autism therapy [ 4 ], education [ 5, 6 ], public places [ 7 ], domestic and work environments [ 8 ]. To be believable, social robots must exhibit social intelligence and adapt their behavior to the situation, therefore they should be endowed with a model of the environment and of the user that may include their profile, emotions, personality and past interactions [ 9 ]. In this regard, robots capable of exhibiting so1 Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) ciability and achieving widespread societal acceptance are being used more and more often in human-centered environments. Due to their ability to enable natural interaction, social robots have a great potential for helping people in their daily activities, especially in tasks that need one-to-one interaction, as is the case of elderly people or children.

Recently, due to the importance of following healthy eating habits, they are being employed in teaching and informing people about nutrition [ 10, 11 ]. It has been shown that a correct diet is related with a decreased risk of cancer and cardiovascular diseases [ 12 ], diabetes and obesity [13; 14]. Social Robots have been employed in this context especially for playing serious games with children [ 15 ].

The long-term goal of this research is to develop a persuasive social robot, playing the role of a nutritional coach, able to induce a behavior change in the health and wellness domains. Such a robot may be especially effective for patients and elderly who want to remain active and live a better and more independent life. Social robots that counsel users on dietary behavior represent an especially promising application in this area. Fruit and vegetable consumption alone plays a protective role in a large number of cancers, and is associated with reduced risk for heart disease, stroke and hypertension, yet only a small percentage of adults meet the guidelines for daily water, fruit and vegetable consumption [ 16 ].

In this context, adaptation of both social behavior and message content to the user is necessary [ 17 ] and, as a first phase of our research, we aimed at making the robot aware of users characteristics and testing whether customizing the robot message to them was an effective way to increase the perception of the information quality and motivational strength of the conveyed message, the degree of recall and the perceived social believability of the robot. This is at the basis of a more complex adaptation of the robot behavior in which persuasive and argumentation strategies will be adapted to the user [ 29 ].

In this paper, we describe our experience with the design and implementation of a dialog module that allows Pepper talking with users about nutrition. To endow the robot with the capability of adapting the dialog to the user a soft biometrics module, able to estimate gender and age, was used [ 18 ]. Then, we performed a preliminary study in which we evaluated the persuasive effects of the adaptation to these traits in terms of perceived 1) informational quality and 2) persuasive effects in terms of robot reliability and the information recalling after the interaction. Preliminary results seem to support the persuasive impact of a personalized nutritional coach.

The paper is organized as follows: in Section 2 a brief overview on soft biometrics analysis, together with the method employed in our system, is provided. Section 3 describes a preliminary study aiming at showing the effects of adaptation on the user perception of the message information content, motivational strength and believability of the social robot. Finally, Section 4 is aimed at a final discussion and proposal for future development.

Soft-Biometrics Analysis

Soft biometric traits refer to physical and behavioral traits, such as gender, age, height and weight, which are not unique to a specific subject, but are useful for identification, and description of human subjects [ 19 ].

Among soft biometric traits, gender and age have been extensively studied. Among the most robust and accurate approaches to gender and age classification, we can find some based on the analysis of texture patterns. Many texture features have been used like LBP, Histogram of Oriented Gradients (HOG) and they usually employ Support Vector Machine (SVM) or k-nearest neighbor as classifiers [ 20,21 ]. Gunay and Nabiyev [ 22 ] used LBP feature as an efficient face descriptor. They divided the faces into small regions from which the LBP histograms are extracted and concatenated into a feature vector. They got 80% age classification rates in FERET database [ 23 ]. Although such local descriptors achieve higher results than holistic methods, their performance is affected by variations in expression, pose, illumination and occlusion.

Recently deep learning approaches are being used successfully in this domain. They mainly use the CNN which is a type of feed-forward artificial neural networks in which the connectivity pattern between its neurons is inspired by the organization of the animal visual cortex [ 24 ]. Yan et al. [ 25 ] proposed an approach which uses the CNN to extract the facial features. Their network has 7 layers and it gives as output 4096 features. For the classification part, they use SVM to classify the face into one of thirteen age groups. Levi and Hassner [ 26 ] proposed a network architecture for both age and gender classification. Rothe et al. [ 27 ] won the ChaLearn LAP 2015 challenge on apparent age estimation, their proposed CNN uses the VGG-16 architecture. They proposed an approach in which first the face is detected from the input image and then extracts the CNN predictions from an ensemble of 20 networks on the cropped face.

In the context of our research, we developed a soft-biometrics module able to recognize age, gender, eyeglasses and beard presence [ 18 ]. In addition, it also recognizes the color of eyes and hairs. For each soft biometric trait, a specific software module has been implemented. In this application, we use only the age estimation and the gender recognition modules. In particular, for age and gender estimation, our system focuses on automatic gender and age classification using deep CNNs. We use a finetuned version of the VGG-16 neural network and, on both tasks, the approach reaches a satisfactory accuracy using unconstrained image dataset2. In particular, the gender recognition has been performed with an accuracy of 85% while age estimation reached an accuracy (+/- 1 year) of 84% on the previously mentioned dataset. In another experiment we tested the performance in real time in the wild having an accuracy of 87.5% for gender recognition and 62.5% for age estimation in which the main confusion problems were in estimating the age of middle-age women [ 18 ]. 2 https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki/ 3

Personalized Information by the Social Robot Pepper

We performed an experiment in real-time with the aim of evaluating the efficacy and the motivational impact of the information about nutrition provided by Pepper. In particular, we tested the difference in the user perception of the information quality and motivational strength of the informative content in case of personalization introduced by adapting the communication of the robot to the recognized soft biometric traits of the subjects. Moreover, we measured the degree of recalling on the informative content and we observed the engagement of the users in interacting with the robot. 3.1

Participants

Forty-five people (23 males and 22 females) aged between 18 and 60 y.o (st.dev=9.09), equally distributed by gender and background were recruited for the experiment. They were motivated in participating to the study by the fact that they would receive vegetables (carrots) and fresh water after the experiment. 3.2

Study Design

In the healthy eating scenario, the subjects interacted with the robot in two different settings: a) no customization (a generic message); b) customization of the message to gender and age. As far as age estimation is concerned, we split in intervals the age of the population typical of the scenario of the experiment and implemented the adaptation of the interaction and the messages accordingly.

In particular, we distinguished among: Young Adults (ages 18-35 years), Adults (ages 36-55 years), and Older Adults (aged older than 55 years). Then, the 45 participants were divided into 2 groups equally distributed in gender and age, of 22 and 23 participants each. The first group interacted with Pepper in the not customized condition (NCC) while the second one in the customized one (CC). The second group was further split into the three groups described above with the following distribution: • • •

Young Adults: 5 females and 6 males Adults: 4 females and 4 males

Older Adults: 2 females and 2 males 3.3

Preparation of Material

The message was generated according to strategies suggested by a human coach expert in nutrition. In this study we refer to user profiles based on age and gender for which we made some assumptions about their presumed goal’s value, consistently with their gender and age. Then, we prepared the scripts for generated the four kinds of messages (Table 1). The first one is a generic and not customized. The second has been customized in order to address the presumed goals of young subjects (i.e. physical appearance, social life, ...).

Type

Not customized Customized

Young (presumed goal: improve physical appearance) Customized

Adult (presumed goal:

prevent health prob

lems) Customized Older Adult (presumed goal: improve health) Hello! + <greeting gesture>. I’m here to provide you with information about the importance of hydrating your body. … Message: Drinking is important for a healthy body. It is recommended to drink at least two liters of water per day to hydrate the body. Proper hydration slows down the body’s aging process. Our organs, in fact, need water to work best. When our bodies become dehydrated, our organs have to work harder, and this can lead to an acceleration in the aging process. Eating fruit and vegetables also helps keeping the body well hydrated. Moreover, these foods give to the body vitamins and minerals.

Hi!+< fist bump gesture> + I’m here to talk with you about the importance of hydrating your body. … Message: Drinking during the day is very important. You must drink at least two liters of water a day. Proper hydration slows down the body's aging process. <if gender=male> Our organs, in fact, need water to work best. Appropriate hydration, about three and half liters a day, can also help keeping the muscles well hydrated, strong and energetic, therefore it is important to drink during physical activity. Remember that the processes that lead to be in a good shape may only occur in a perfectly hydrated body. So, drinking is very important to look in shape! <if gender=female> Appropriate hydration, about two and half liters a day, helps fight water retention which is the antechamber of cellulite. In fact, water retention does not mean having "too much" water in the body but suffering from an imbalanced distribution. People who suffer from it often do not drink enough. To improve this situation, it’s important to drink more and more. So drinking may help you in passing the bikini test! Good morning/evening! + <handshake gesture> + I’m here to talk with you about the importance of hydrating your body. … Message: Drinking during the day is very important. You must drink at least two liters of water a day. Proper hydration slows down the body's aging process and improves its functioning. Drinking water and eating fruits and vegetable may help at reducing several risks for your health since your organ will be more hydrated. <if gender=female>Water will help to remove toxins and your skin will look smoother and more radiant. <if gender=male> Water will help you to prevent urinary and prostate problems. <both gender> Adopt a healthier lifestyle … it is known “prevention is better than cure”.

Good morning/evening! + <handshake gesture> + I’m here to provide you with information about the importance of hydrating your body. … Message: After the age of 50 the thirst stimulus fades. The body has no water reserves and for this it is necessary to rehydrate our body by drinking a lot and eating fruits and vegetables, which contain a significant amount of water. Furthermore, an American study has proved that drinking a good amount of water can decrease the risk of a heart attack. Health authorities and others encourage people to consume 2 or more liters of water a day, that is the right dose for an adult. To this quantity must be added the liquids supplied to the body from fruit and vegetables, for a total of 3 liters per day.

It is best to drink calcium water that prevents osteoporosis.

The third one is directed to middle-age people, caring more for health and disease prevention, and the fourth to older adults. In particular, the second and third messages have been further distinguished between females and males in order to select the right arguments.

In order to assess the user background knowledge and beliefs about correct nutrition a pre-test questionnaire concerning some control variables (like level of information on nutrition and on the importance of adopting dietary behaviors) was prepared. We prepared also a post-test questionnaire aiming at evaluating some characteristics of the provided message (i.e. information quality, motivational strength), the level of recall of the information provided in the message and the social believability of the robot during the interaction. Questions were expressed as statements and evaluated using a 5 points Likert scale. 3.4

Procedure

A realistic scenario was created in one of the open community spaces of our Department for performing the experiment. We set up a kind of healthy eating stand in which Pepper was acting as an expert, thus providing information to people in the domain. To control the interaction and compare results we focus on the importance of drinking and hydrating our body. In this scenario Pepper was running for two days according to two different settings described before. We decided to dedicate one day to the not-customized condition and one day to the customized one.

After receiving a short explanation describing the purpose of the experiment, subjects willing to participate to the experiment signed up an informed consent and filled out the pre-test questionnaire. Then they started to interact with Pepper and received the message when asking about suggestions. After the message and the interaction with Pepper they were asked to fill out the post-test questionnaire.

a) G: Male -Age: 22 b) G: Female -Age: 45 c) G: Female -Age: 20

In the customized condition, after having detected the face of a person entering in its field of view, using its RGB camera in the forehead with a resolution of 1280x960, Pepper estimated gender and age group by running the soft-biometrics module in the Pepper Department Cloud. In this setting, Pepper adapted the greetings by changing the level of formality/friendliness of the employed language because of the recognized age and the style and motivational content of the provided informative messages. Figure 1 shows three examples of interaction. On the left-side column the pictures taken from the Pepper camera are shown together with the annotation of the softbiometrics traits. As you can notice in the right-side picture, Pepper adapts the greeting gesture to the estimated age of the subject. 3.5

Results and Discussion

The analysis of the pre-test questionnaires confirmed our assumptions about the hypothetical users and ensured that differences in the post-text were not due to differences in the healthy eating knowledge of the participants.

To compare the motivational impact of the message in the two output conditions, we analyzed the results of the post-test questionnaire data using a t-test. Results (Table 2) show that the customized interaction and message conveyed through Pepper received an overall better evaluation.

Significant differences occurred in terms of satisfaction and helpfulness. Moreover, the customized condition was perceived as significantly more persuasive and reliable. There were no significant differences in terms of easiness in the understanding and validity of the message content.

About the evaluation of the social believability of the robot we compared the results of the two groups. In the first condition the rating was lower than in the second condition, in which greetings were adapted to the soft-biometric traits and, according to the t-test result (see Table 2), this difference is significant thus showing that the robot is perceived as more believable when it is aware of the characteristics of the type of person present in the environment. However, the recalling degree (in terms of correct answers given by subjects to the 5 questions) does not present a significant difference but a promising tendency that can be reinforced in larger sample (Table 2). 4

Conclusions

Our long-term research goal is to develop a Persuasive Social Robot in domains such as nutrition, wellbeing and health [ 28 ] for helping people in changing their wrong attitudes and habits [ 29 ]. A first step in this direction is represented by the work described in this paper. To this aim, we endowed a social robot, Pepper in this case, with the capability of analyzing soft-biometrics features of people and use this awareness to customize the information provided to users in the domain of nutrition and healthy eating. To test the efficacy of the proposed approach, we set an experiment in a realworld scenario in which Pepper was providing information about correct nutrition. Even if performed on a small number of subjects, the experiment was carried out in the wild and the results show that, when the dialog was customized to the characteristics of the person interacting with the robot, it was better perceived in terms of satisfaction, helpfulness and persuasiveness. Moreover, the robot itself was perceived as more socially believable in the customized condition. These results are in line with classical studies on persuasion which pointed out how a persuasive source can be reliable when it demonstrates to know who is the user and which are his/her needs, by exploiting the empathic and engaging side of persuasion. [30; 31]

In our expectation, adapting the content of the dialog can also affect the retention of the learned information and results seem to go in this direction since the recalling degree (in terms of correct answers given by subjects to the 5 questions) present a promising tendency (Table 2) in this sense. Participants in customized condition recall more than in a not customized one (4.32 vs 4; p=.106).

In a future experiment, we can better explore this result with a larger sample and more ad hoc measures, by also putting in relation the motivational strength, social believability and informational quality with the level of recalling and other behavioral measures. In this sense in fact a customized robot can affect individual healthy attitude simply because users feel a technological ‘trust’ [32] that in this case can be based on persuasive and affective processes of being acknowledged. As far as softbiometric is concerned, we plan to add new traits recognition from the body for making the robot aware of the type of person (i.e. slim vs. fat) who is asking for information and to investigate whether this improve the efficacy of the dialog. However, since it is possible to misclassify age and gender of the user, we plan to investigate also on the effects of wrong customization to these factors.

References

1. Al-Qaderi , Mohammad, and Ahmad Rad . "A multi-modal person recognition system for social robots . " Applied Sciences 8.3 ( 2018 ): 387 . Faria , D. R. , Vieira , M. , Faria , F. C. , Premebida , C. "Affective facial expressions recognition for human-robot interaction" , RO-MAN'17: IEEE International Symposium on Robot and Human Interactive Communication , 2017 .

2. Broekens , J.; Heerink, M. ; Rosendal , H. Assistive social robots in elderly care: review . Gerontechnology 2009 , 8 , 94 - 103 .

3. Palestra , G. , De Carolis , B. , Esposito , F. Proceedings of the Workshop on Artificial Intelligence with Application in Health co-located with the 16th International Conference of the Italian Association for Artificial Intelligence (AI*IA 2017 ), Bari, Italy, November 14 , ( 2017 ).

4. Mubin , O. ; Stevens , C.J. ; Shahid, S. ; Al Mahmud, A. ; Dong , J.J. A review of the applicability of robots in education . J. Technol. Educ. Learn . 2013 , 1 . 209 - 0015 .

5. Saerbeck , M. ; Schut , T. ; Bartneck , C. ; Janse , M.D. Expressive robots in education: Varying the degree of social supportive behavior of a robotic tutor . In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (ACM 2010 ), Atlanta, GA , USA, 10 - 15 April 2010; pp. 1613 - 1622 .

6. Leite , I. ; Martinho, C. ; Paiva , A. Social robots for long-term interaction: A survey . Int. J. Soc. Robot . 2013 , 5 , 291 - 308 .

7. Ahmad , M. I. , Mubin , O. , Orlando , J. A Systematic Review of Adaptivity in HumanRobot Interaction, Multimodal Technologies and Interaction, 2017 , 3 ( 14 ), 1 - 25

8. Kidd C. D. and Breazeal , C. “ Robots at home: Understanding longterm human-robot interaction,” in International Conference on Intelligent Robots and Systems . IEEE, 2008 , pp. 3230 - 3235 .

9. Short , E. , Swift-Spong , K. Greczek , J. Ramachandran , A. Litoiu , A. , Grigore , E. C. FeilSeifer , D. , Shuster , S. , Lee , J. J. , Huang phet, S. al., "How to train your dragonbot: Socially assistive robots for teaching children about nutrition through play," in The 23rd IEEE International Symposium on Robot and Human Interactive Communication . IEEE, 2014 , p. 924 -- 929 .

10. Nicklas , T. A. , Baranowski , T. , Cullen , K. W. and G. Berenson, “ Eating patterns, dietary quality and obesity, ” Journal of the American College of Nutrition , vol. 20 , no. 6 , pp. 599 - 608 , 2001 .

11. Reedy , J. Krebs-Smith , S. M. , Miller , P. E. , Liese , A. D. , Kahle , L. L. , Park , Y. and Subar , A. F. “ Higher diet quality is associated with decreased risk of all-cause, cardiovascular disease, and cancer mortality among older adults,” The Journal of nutrition , vol. 144 , no. 6 , pp. 881 - 889 , 2014 .

12. Faccio , E. , Centomo , C. , & Mininni , G. “ Measuring up to Measure” Dysmorphophobia as a Language Game . Integrative Psychological and Behavioral Science , 45 ( 3 ), 304 , 2011 .

13. Hingle , M. Macias-Navarro , L. , Rezaimalek , A. and S. Going , “ The use of technology to promote nutrition and physical activity behavior change in youth: A review,” The Research Dietetic Practice Group Digest , pp. 1 - 10 , 2013 .

14. Leyzberg , D. , Spaulding , S. , Toneva , M. and B. Scassellati , “ The physical presence of a robot tutor increases cognitive learning gains ,” 2012 .

15. Li R , Serdula M , Bland S , et al. Trends in fruit and vegetable consumption among adults in 16 US states: behavioral risk factor surveillance system, 1990 - 1996 . Am J Public Health 2000 ; 90 : 777 - 81 .

16. de Rosis , F. , Novielli , N. , Carofiglio , V. , Cavalluzzi , A. , De Carolis , B. : User Modeling And Adaptation In Health Promotion Dialogs With An Animated Character . International Journal of Biomedical Informatics , 514 - 531 ( 2006 )

17. De Carolis , B. , Macchiarulo , N. , & Palestra , G. ( 2019 ). Soft Biometrics for Social Adaptive Robots . In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems (pp. 687 - 699 ). Springer, Cham.

18. Dantcheva , A. , Velardo , C. , D'angelo , A , JL Dugelay , Bag of soft biometrics for person identification: New trends and challenges , Multimedia Tools and Applications , 51 ( 2 ): 739 - 777 , ( 2011 )

19. Sun , N. , Zheng , W. , Sun , C. , Zou , C. , & Zhao , L.. Gender classification based on boosting local binary pattern . In Advances in Neural Networks-ISNN 2006 (pp. 194 - 201 ). Springer Berlin Heidelberg ( 2006 ).

20. Lian , H. C. , & Lu , B. L. Multi-view gender classification using local binary patterns and support vector machines . In Advances in Neural Networks-ISNN 2006 (pp. 202 - 209 ). Springer Berlin Heidelberg ( 2006 ).

21. Günay A , NabIyev VV . Automatic age classification with LBP . Proceeding of the 23rd International Symposium on Computer and Information Sciences (ISCIS '08); October 2008 ; Istanbul, Turkey. pp. 1 - 4 .

22. Phillips , P.J. , Wechsler , H. , Huang , J. , Rauss , P.J.: The feret database and evaluation procedure for face-recognition algorithms . Image and vision computing 16(5) , 295 { 306 ( 1998 ).

23. Simonyan K. and Zisserman , A. “ Very deep convolutional networks for large-scale image recognition , ” arXiv preprint arXiv:1409.1556 , ( 2014 ).

24. Yan , C. Lang , C. , Wang , T. Du , X. and C. Zhang , “ Age estimation based on convolutional neural network,” in Proceedings of the 15th Pacific-Rim Conference on Advances in Multimedia Information Processing - PCM 2014 - Volume 8879 , (New York, NY, USA), pp. 211 - 220 , Springer-Verlag New York, Inc., ( 2014 ).

25. Levi G. and Hassner , T. “ Age and gender classification using convolutional neural networks,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops , June ( 2015 ).

26. Rothe , R. Timofte , R. and Gool , L. V. “ Dex: Deep expectation of apparent age from a single image ,” in 2015 IEEE International Conference on Computer Vision Workshop (ICCVW), pp. 252 - 257 , Dec ( 2015 ).

27. Berardina De Carolis, Irene Mazzotta . A user-adaptive persuasive system based on 'arational' theory . Int. J. Hum. Comput. Stud . 108 : 70 - 88 . ( 2017 )

28. Mininni , G. , & Manuti , A. La costruzione discorsiva della cura di sé: culture a confronto sull'idea di benessere . Psicologia della salute . pp. 101 ; ( 2004 )

29. D'Errico , F. , & Poggi , I. Tracking a leader's humility and its emotions from body, face and voice . In Web Intelligence (Vol. 17 , No. 1 , pp. 63 - 74 ). IOS Press. ( 2009 ).

30. D'Errico , F. , Paciello , M. , & Cerniglia , L. ( 2016 ). When emotions enhance students' engagement in e-learning processes . Journal of e-Learning and Knowledge Society , 12 ( 4 ).

31. Castelfranchi , C. , & Falcone , R. Trust theory: A socio-cognitive and computational model (Vol. 18 ). John Wiley & Sons ( 2010 ).