Spoken Conversational Agents for Older Adults: Who Are the Stakeholders and What Do They Expect?

Abstract

In this paper we discuss our road map to design a Dutch spoken conversational agent that helps older adults with self-management of their well-being. We conducted three usability studies with our conversational agent, each time after discussing it with diferent stakeholders. We discuss the challenges incorporating each stakeholder's wishes and needs whilst iteratively designing our agent.

Keywords

• eol>conversational agents
• dialogue design
• stakeholders
• health
• self-management
• older adults

1. Introduction

In our own research we have been develop

ing a conversational agent (CA) that should Self-management has become an important support older adults in self-management of feature in (western) communities, where we their well-being [ 4 ]. Self-management is a colstrive to live independently for as long as we laborative efort between older adults and a can. This is a challenge for older adults who CA, to promote activation and empowerment might face physical limitations, such as re- and adopt health-promoting behaviours [ 5, 6 ]. duced mobility, hearing problems and visual We have explicitly chosen for spoken interacimpairment, or limited support from a social tion because this is a low-threshold form of network. A social robot might support them communication for this target group. Digital with self-management [ 1 ], e.g. with physical literacy is generally low in older adults, so usactivities [ 2 ] or social skills [ 3 ]. ing a chatbot can be a hurdle, while a spoken interface would definitely be more accessible, except for people with speaking or hearing problems. Speaking to a device is certainly easier than typing, which can be problematic, e.g. due to arthritis or visual impairments [ 7 ].

A systematic literature review on conversational agents in the health domain showed that interaction with older adults is seen as one of the big challenges in agent development [ 8 ]. Diferent stakeholders are involved in designing such a CA: care professionals, end-users (older adults), CA developers, fund- as researchers are interested how conversaing agencies and researchers. In this paper tions from a positive psychology point of view we discuss the challenges that we faced in our would benefit end-users [ 16]. We created a own research and the lessons we learned, in small scripted dialogue that would ask about the hope that these might be useful to others activities and social relationships, two imporconducting similar research. tant factors in a persons well-being about their daily activities and quality of life [17].

The conversation took about 2 minutes in 2. Speech-based CAs for total per participant. There was some limwell-being ited Natural Language Understanding (NLU) capability in the Behaviour-based LanguageWe are not the first to use a spoken CA for sup- Interactive Speaking Systems (BLISS) protoporting self-management of older adults [ 9 ]. type for interpreting user answers. For exA CA can be deployed to reduce loneliness or ample, if the user answered a question with provide assistance in daily life [ 10, 11, 12 ]. For “I think that would be cycling.”, the system example, [ 3 ] developed a virtual CA that en- would ask “What would you miss most if you gaged with older adults on a regular basis over wouldn’t cycle?”. a longer period of time to help them improve At the early stage of our research we extheir social skills. A study by [13] looked at a plored how adults with limited experience virtual companion agent for older adults. The with agents interact with our CA. We went authors found that topics that are generally in- to three public Dutch conferences related to teresting include storytelling, the weather, the language and health using a demo stand to family, and future plans they have with a live- evaluate the prototype and dialogue with 56 in companion. Many practical issues remain people. These conferences were in a noisy enwith speech-based systems, and specifically vironment, in spaces with many people and for older adults, speech recognition accuracy provided an ecologically valid test to see if can be improved by pre-processing the au- our system was robust enough in natural condio [14]. Current voice interfaces are trained ditions. for younger audiences, and even with preprocessing, syllable segmentation for older 3.2. Online data collection adults remains an issue [15].

Due to the COVID-19 crisis we had to switch

focus from physical interactions to online in3. Dialogue Design teractions. Therefore we designed a dialogue oriented towards older adults about social reWe conducted a field and an online usability lationships and activities and how the COVIDstudy and co-designed a dialogue with health- 19 crisis impacted them. The dialogue design care professionals from the field. contained questions about social relationships and activities and how the COVID-19 crisis 3.1. Field data collection has impacted this. Then, we contacted 6 older adults (70+) through a telephone call and evalTowards the end of 2019, we conducted field uated the questions in a human-human teleusability studies with the first version of our phone conversation. After correcting the diDutch spoken dialogue system BLISS [ 4 ]. We alogues for unclear, unnecessary and inappropriate questions, we implemented these questions in the BLISS CA. We contacted 10 participants, of whom 7 are older adults, to talk to the CA. Unfortunately, we could not visit the participants and thus set up the CA remotely by running it locally on a computer and set up a video call via Zoom, MSTeams or Skype with the participants, who were using either a tablet or a laptop. Most participants were assisted by a younger family member or were already familiar with video calls. Each conversation took about 10 minutes, and an excerpt of asking about social relationships (family) from one conversation is shown in Table 1. Note that in this conversation the question of the agent on line 11 does not make much sense given the previous response of the user. # 1

After these two data collections, we invited

two domain experts, healthcare workers, to evaluate and improve our dialogue designs and co-design a better dialogue that is more suited for older adults in a healthcare facility. Based on the feedback, we improved the dialogues to include more self-disclosure and backchanneling by the system. An example of this is shown in Table 2. We tested these dialogues with 7 healthy adults, of whom 4 were over the age of 70, and the others between 50 and 70 years old.

4. Stakeholders

Designing and testing a spoken CA requires

the help by and collaboration with many different stakeholders. Each stakeholder has different wishes and needs they bring to the table. The first and foremost stakeholders are the older adults, the end-users of our CA.

They need to be comfortable with the technol- more possibilities for other stakeholders to ogy and this needs to be accessible to them. It ask for support because of new discoveries in can take up to two months before people ac- research could benefit all stakeholders. cept social robots, and even take half a year be- Finally, we as researchers want to evalufore a CA will be integrated in its environment ate the use of the CA, the user experience, [18]. [12] mention the following things that the conversation flow, long-term usage and are important to the older adults: retaining test novel ideas to publish and develop knowlcontrol, being encouraged for tasks or activ- edge. We design prototypes based on theories ities, safeguard their privacy and usefulness and/or datasets and evaluate them with endand quality of social skills of an agent. In our users. Only recently more focus has shifted to online study, older adults also commented on co-designing with end-users, actively involvthe voice lacking clarity and empathy, which ing them in the whole design process of CAs. is also found in earlier research [19, 20]. Researchers might still have problems explain

The second type of stakeholders are the ing to other stakeholders what is possible and healthcare professionals. Their main interest what is not, and to fully accommodate other is providing the best care they can in which stakeholders’ needs, especially the end-users. the CA must be a support tool and not a hassle. All these stakeholders’ expectations impact In the co-design session, the healthcare work- the design of the CA and we have to combine ers who were in direct contact with clients the wishes and needs of all parties. In fact, we expressed the need for a CA that would (just) experienced that this can be very challenging engage, entertain and distract clients; they during our data collections. preferred an intervention that can help with boredom and loneliness. They did not yet see the usefulness or applicability of a personal 5. Conclusion user profile that could be collected through such an engaging CA, in contrast to what The application of interactive and intelligent some of healthcare management would like CAs ofers many possibilities to the healthto see. care field. We believe that a personalised CA

Thirdly, there are CA developers who im- can help both clients and professionals: implement state-of-the-art technologies in the proved self-management for clients, and a betCA. Their interest is in seeing how their tech- ter insight into clients’ needs and wishes for nology is accepted by the end-users and it- professionals. Discussions with healthcare eratively improve the technology. However, management have confirmed this. much (research) software becomes outdated The healthcare workers, the professionals due to shifts in maintenance and time and/or who are in contact with the clients on a daily funding is too limited for achieving the re- base, are the gatekeepers of these intervensearch goals. tions. Their main focus is on providing good

Fourthly, funding agencies often require a care for the client, and especially during the clear valorisation and societal and scientific COVID-19 pandemic their time is pressed. As impact of the research, focusing on novel re- such, they are naturally looking for solutions search. They often only see the global part of a that provide an immediate improvement. project and have to deal with many projects at There is an inverted mismatch between the the same time. They are usually only involved current needs and wishes of healthcare produring writing of proposals and reviewing of fessionals and end-users, and the design and the project mid-way and at the end. Adding technological possibilities of our CA for researchers and CA developers. Professionals in direct contact with older adults are satisfied with a system that (just) engages and gives attention, while healthcare management is also interested in the longer-term benefits of collecting valuable data about clients, in order to provide even better personal care. Additionally, CA developers are the most knowledgeable about the CA’s capabilities and are interested in usability of their latest technology.

Mindset broadening seems to be required.

Acknowledgments

This work is part of the research

programme Data2-Person with project no. 628.011.029, which is (partially) financed by the Dutch Research Council (NWO).

We thank the reviewers for their valuable suggestions. Social support agents for older adults: health: a mixed methods study, BMJ longitudinal afective computing in the Open 6 (2016) e010091. doi:10.1136/ home, Journal on Multimodal User In- bmjopen-2015-010091.

terfaces 9 (2015) 79–88. [18] M. M. A. de Graaf, S. B. Allouch, J. A. [11] B. Spillane, E. Gilmartin, C. Saam, B. R. G. M. van Dijk, Why Would I Use This in Cowan, V. Wade, Adele: Care and com- My Home? A Model of Domestic Social panionship for independent aging., in: Robot Acceptance, Human–Computer ICAHGCA@ AAMAS, 2018, pp. 18–24. Interaction 34 (2019) 115–173. doi:10. [12] C. Tsiourti, E. Joly, C. Wings, M. B. 1080/07370024.2017.1312406, Moussa, K. Wac, Virtual assistive com- publisher: Taylor & Francis. panions for older adults: qualitative field [19] J. Oliveira, G. S. Martins, A. Jegundo, study and design implications, in: Pro- C. Dantas, C. Wings, L. Santos, J. Dias, ceedings of the 8th International Confer- F. Perdigão, Speaking robots: The chalence on Pervasive Computing Technolo- lenges of acceptance by the ageing sogies for Healthcare, ICST (Institute for ciety, in: 2017 26th IEEE International Computer Sciences, Social-Informatics Symposium on Robot and Human Interand . . . , 2014, pp. 57–64. active Communication (RO-MAN), 2017, [13] L. P. Vardoulakis, L. Ring, B. Barry, C. L. pp. 1285–1290. doi:10.1109/ROMAN.

Sidner, T. Bickmore, Designing rela- 2017.8172470, iSSN: 1944-9437. tional agents as long term social com- [20] J. James, C. I. Watson, B. MacDonpanions for older adults, in: Interna- ald, Artificial Empathy in Social tional Conference on Intelligent Virtual Robots: An analysis of Emotions in Agents, Springer, 2012, pp. 289–302. Speech, in: 2018 27th IEEE Interna[14] S. Kwon, S.-J. Kim, J. Y. Choeh, Prepro- tional Symposium on Robot and Human cessing for elderly speech recognition of Interactive Communication (RO-MAN), smart devices, Computer Speech & Lan- 2018, pp. 632–637. doi:10.1109/ROMAN. guage 36 (2016) 110–121. doi:10.1016/ 2018.8525652, iSSN: 1944-9437. j.csl.2015.09.002. [15] G. Son, S. Kwon, Y. Lim, Speech rate control for improving elderly speech recognition of smart devices, Advances in Electrical and Computer Engineering 17 (2017) 79–85. Publisher: Stefan cel Mare

University of Suceava. [16] M. E. P. Seligman, Positive psychology, positive prevention, and positive therapy, in: Handbook of positive psychology, Oxford University Press, NY, US, 2002, pp. 3–9. doi:10.1017/

CBO9781107415324.004. [17] M. Huber, M. v. Vliet, M. Giezenberg, B. Winkens, Y. Heerkens, P. C.

Dagnelie, J. A. Knottnerus, Towards a ‘patient-centred’ operationalisation of the new dynamic concept of

References

[1] M. Rijken , M. Jones , M. Heijmans , A. Dixon , Supporting self-management, in: Caring for People with Chronic Conditions: A Health System Perspective , 1st ed., McGraw-Hill Education (UK), 2008 , pp. 116 - 142 .

[2] R. Kocielnik , L. Xiao , D. Avrahami , G. Hsieh, Reflection companion: A conversational system for engaging users in reflection on physical activity , Proc. ACM Interact. Mob. Wearable Ubiquitous Technol . 2 ( 2018 ). doi: 10 .1145/ 3214273.

[3] R. Ali , E. Hoque, P. Duberstein , L. Schubert , S. Z. Razavi , B. Kane , C. Silva , J. S. Daks , M. Huang , K. Van Orden , Aging and engaging: A pilot randomized controlled trial of an online conversational skills coach for older adults , The American Journal of Geriatric Psychiatry ( 2020 ). doi: 10 .1016/j.jagp. 2020 . 11 .004.

[4] J. van Waterschoot , I. Hendrickx , A. Khan , E. Klabbers, M. de Korte, H. Strik , C. Cucchiarini , M. Theune , BLISS: An agent for collecting spoken dialogue data about health and wellbeing , in: Proceedings of the 12th Language Resources and Evaluation Conference , European Language Resources Association, Marseille, France, 2020 , pp. 449 - 458 . URL: https://www. aclweb.org/anthology/2020.lrec- 1 . 57 .

[5] M. S. Goldstein , The persistence and resurgence of medical pluralism , Journal of Health Politics, Policy and Law 29 ( 2004 ) 925 - 946 . doi: 10 .1215/ 03616878 -29-4-5-925.

[6] K. Farrell , M. N. Wicks , J. C. Martin , Chronic disease self-management improved with enhanced self-eficacy , Clinical Nursing Research 13 ( 2004 ) 289 - 308 . doi: 10 .1177/1054773804267878, publisher: SAGE Publications Inc.

[7] S. J. Czaja , N. Charness , A. D. Fisk , C. Hertzog , S. N. Nair , W. A. Rogers , J. Sharit , Factors predicting the use of technology: Findings from the center for research and education on aging and technology enhancement (create) ., Psychology and aging 21 ( 2006 ) 333 .

[8] J. L. Z. Montenegro , C. A. da Costa , R. da Rosa Righi, Survey of conversational agents in health , Expert Systems with Applications 129 ( 2019 ) 56 - 67 . doi: 10 . 1016/j.eswa. 2019 . 03 .054.

[9] R. Jaber , D. McMillan , Conversational user interfaces on mobile devices: Survey , in : Proceedings of the 2nd Conference on Conversational User Interfaces , CUI '20 , Association for Computing Machinery, 2020 , pp. 1 - 11 . doi: 10 .1145/ 3405755.3406130.

[10] L. Ring , L. Shi , K. Totzke , T. Bickmore,