How Can Intelligent Conversational Agents Help? The Needs of Geriatric Patients and Their Caregivers Gerhard W. Eschweiler Leo Wanner Geriatric Center at the University Hospital Catalan Institute for Research and Advanced Studies and University of Tübingen University Pompeu Fabra Tübingen, Germany Barcelona, Spain gerhard.eschweiler@med.uni-tuebingen.de leo.wanner@upf.edu ABSTRACT to informal caregivers, most of them family members, which of- The population in Europe grows older; only between the years ten results in a physical, emotional or mental burnout. Another 2007 and 2017 the number of people aged 65 and older increased consequence has been a massive recruitment of affordable work by 2.4%. With the increasing age, the time that an average indi- force from (first of all) Eastern Europe, which bears qualification, vidual suffers from illnesses, loss of autonomy, deterioration of language and cultural problems. mental capacity, etc., and is thus in need of care in their old age In the light of this situation, the exploration of alternative solu- also increased substantially. This calls for qualified care personnel. tions seems inevitable. One of these solutions may be grounded in However, qualified personnel is scarce. To alleviate the situation, the use of intelligent agent technologies. A considerable amount of embodied conversational agents (ECAs) that are able to interact work has been done on the design and realization of intelligent em- with care recipients, their carers and medical personnel would be a bodied conversational agents (ECAs) that act as social companions great asset – for instance, by acting as social companions, coaches of elderly. Overall, ECA companions can be attested a great poten- or medical assistants. In any of these roles, ECAs must be guided tial from the technical viewpoint – even if they are still far from by the needs of the targeted interaction partner and act within having the skills needed to assist elderly as humans do. However, a well-defined strict ethical and legal framework. Especially the there is also another challenge: in order for ECAs to be appropriate interaction with geriatric patients requires an acquaintance with for social care, the researchers working in the field must know well the basics of geritatric medicine, its ethical aspects and the available the needs of elderly and their caregivers, i.e., have at least basic models that help to assess them. knowledge on basic care, gerontology and old age medicine, also termed “geriatric medicine”, and the legal and ethical frameworks KEYWORDS into which geriatric medicine is embedded. This is not always the case. This paper aims to introduce researchers working on ECAs intelligent conversational agents, geriatric applications, commu- into the specifics and needs of geriatric medicine. In Section 2, we nicative structure, thematicity, prosody, text-to-speech, human- outline the basic notions of geriatric medicine. In Section 3, we machine interaction then introduce the needs of elderly and in Section 4 we discuss how 1 INTRODUCTION ECAs could address these needs. Section 5, finally, presents some conclusions. The population in Europe grows older. According to Eurostat, over the last decade (i.e., between the years 2007 and 2017), the number of people aged 65 and older increased by 2.4%.1 On the one side, 2 BACKGROUND this is good news since this means that the expectancy of life of In this section, we present the basic characteristics of geriatric an average individual in Europe keeps growing. On the other side, medicine and geriatric patients that will allow us to derive the old by no means always means healthy. In general, the time that an needs of medical professionals and the care recipients alike and average individual suffers from illnesses, loss of autonomy, deterio- identify some of the central ethical concerns that must be taken ration of mental capacity, etc., and is thus in need of care in their old into consideration when targeting solutions that shall address these age increased substantially. Thus, according to the German Federal needs. Statistical Office, 2015 saw in Germany 2.6 million elderly in need of care; 1.86 million of them live in their own households. In UK, The Independent reports that “more than 1.4 million people over the 2.1 Geriatric Patients and Geriatric Medicine age of 65 are struggling without the help they depend on to carry First of all: who is a “geriatric patient” and what makes them recep- out basic tasks such as getting out of bed, going to the toilet and tive for ECA services? Unfortunately, there is no generally accepted washing themselves”.2 This calls for qualified care giving personnel. and used definition of the term. According to the Geriatrics Sec- However, qualified personnel is scarce. Currently, only in Germany, tion of the European Union of Medical Specialists (UEMS, Union there is a shortage of tens of thousands of professional care givers. Européenne des Médecins Spécialistes in French), a geriatric patient One consequence of this shortage has been the shift of the burden is mostly older than 70 years and suffers from multiple active disor- 1 https://ec.europa.eu/eurostat/statistics-explained/index.php/Population_structure ders. We can adopt this definition as our working definition. _and_ageing#The_share_of_elderly_people_continues_to_increase 2 Independent, 9 July 2018 “Number of elderly people deprived of vital support hits A dedicated branch of medicine, namely the geriatric medicine, record high, finds report” focuses on geriatric patients. The UEMS definition of geriatric 1 medicine was coined in 2008.3 Geriatric medicine is a specialty the WHO)6 as a progressive cognitive decline caused by a patholog- of medicine concerned with physical, mental, functional and social ical process in the brain. The cognitive decline continues at least for conditions in acute, chronic, rehabilitative, preventive, and end- six months and causes a loss of ability to manage the ADLs without of-life care in older patients. This group of patients is considered support or help. About 70% of dementia is related to the Alzheimer to have a high degree of frailty and active multiple pathologies, Disorder (AD), which is caused by neurodegeneration of cortical which requires a holistic approach. Geriatric medicine therefore synaptic connections, neurons, and deposition of Amyloid plaques exceeds organ-orientated medicine like cardiology or urology. It and neurofibrillary tangles. This cortical degeneration first affects offers additional therapy in a multidisciplinary team setting, the the encoding of memories and later the loss of semantic knowl- main aim of which is to optimize the functional status of the older edge about locations, persons, tools and complex tasks [6]. Mild person and improve the quality of their life and their autonomy. and moderate dementia may also show a number of non-cognitive The multidisciplinary team includes not only physicians and nurses, symptoms that equally need to be assessed for better treatment and but also physiotherapists, speech therapists, social workers, etc., care of the patient, among them, in particular: (i) problematic (e.g., in order to address the mentioned frailty and multiple pathology repetitive asking, crying or wandering about) and aggressive (e.g., of geriatric patients. Frailty implies a combination of at least three spitting at or beating of fellow citizens) behavior, (ii) depression, out of the following five symptoms, whose phenotype is a vulner- (iii) reduced quality of life, (iv) paranoid thinking like feeling of able person with a lack of resources to compensate stress, acute being robbed, and (v) limited ADLs. infections or injuries [31]: (i) involuntary weight loss (over 10% in one year); 2.2 The Scope of Geriatric Medicine (ii) objectified muscle weakness (determined by manual force Geriatric medicine does not only focus on the treatment of the measurement in terms of a hand grip4 ), loss of muscle mass diseases of geriatric patients; it is also concerned with the preven- (sarcopenia), or osteoporosis (bone loss), often followed by tion of typical geriatric diseases. A number of studies show that fractures, falls and immobility; physical and cognitive activities are crucial in the context of this (iii) subjective (mental, emotional, or physical) exhaustion; prevention. Thus, a recent epidemiological study from Japan re- (iv) immobility, instability, gait and stance with fall prone; vealed that physically active subjects who walk more than one hour (v) decreased physical activity (in terms of basic and/ or instru- per day have a 28% lower risk to develop dementia, compared to mental activities of daily living (ADLs)). subjects walking less than 30 minutes per day, even if the latter are monitored with respect to age-related symptoms, body mass The aim of the geriatric medicine with respect to frailty is three- index or stroke indicators [27]. In a larger study with more than fold: (i) prevention of frailty, (ii) assessment of the degree of frailty, 11,000 Japanese subjects, the risk of functional impairments was and (iii) treatment of the frailty aspects. Multiple pathology may measured after a follow-up of nine years for three activities: (i) include somatic pathologies and mental pathologies. The somatic cognitive activity, (ii) walking, and (iii) social activities. Models pathologies may be manifold and the same as those from which of the estimated mediating effects showed that cognitive activity other patients may suffer, although geriatric patients over-proport- accounted for 9.3%, time spent walking for 8.3%, psychological state ionally suffer from acute vascular events such as a myocardial for 4.6%, and social support for 2.8% of the reduced risk of incident infarction and stroke. Patients suffering from heart failure might functional disability [21]. lose their ability to climb steps or to walk longer distances; patients An additional important factor appears to be body weight. Ac- suffering from a stroke might suffer from impairments in reading cording to a study by Norton et al. [20], a reduction in obesity or speaking (cf. aphasia). prevalence and thus diabetes prevalence in the mid-age reduces Two of the main mental pathologies in elderly that are not re- dementia prevalence by more than 10%. lated to vascular events are Parkinson disease and dementia of In general, recommendations for physical, cognitive and social Alzheimer-type, caused by the loss of distinct neurons in the brain. activities are found in all medical guidelines. These recommenda- The Parkinson disease5 is a neurodegenerative disorder which has tions are buttressed by cohort studies. The gold standard is however as typical symptoms slowness of movements (bradykinesia), an provided not by cohort studies, but, rather, by forward-looking increased muscle stiffness (rigor), decreased mimic movements (hy- randomized controlled trials (RCTs). Recently, the FINGER study pomimia), and instability when standing (postural instability) or showed prospectively in an RCT that multimodal lifestyle inter- starting to walk. Most patients suffer early on from loss of smell, ventions in Finland, including healthy diet, physical exercises and obstipation and nightmares or restless legs before they show the mental activities, were able to delay cognitive decline with mod- motoric symptoms since neuronal degeneration starts in vegetative erate effect only [19]. Unfortunately, RCTs are expensive and thus nerves before motoric signs are present. For more details, see [11]. rare. Dementia is a clinical syndrome, defined worldwide in the ICD 10 (The international classification of disorders 10th Revision, by 2.3 Ethical Aspects in Geriatric Medicine The importance of ethics in geriatric medicine requires a clear 3 http://uemsgeriatricmedicine.org/www/land/definition/english.asp#, last access on definition of the ethical requirements towards each individual so- 2018-07-12. lution and mechanisms for the control of the fulfilment of these 4 Hand grip depends also on motivation, but has real impact on the activities of daily requirements at the time of the implementation. living (ADLs) like to cut a piece of meat or to carry a bag. 5 For the first time described in 1817 by the English physician James Parkinson. 6 http://apps.who.int/classifications/icd10/browse/2016/en 2 2.3.1 Ethical requirements towards solutions in geriatric medicine. Geriatric medicine, including home and social care of elderly, deals with personal data and privacy of vulnerable persons, which must be protected from disclosure and misuse. Therefore, the European and member state legislations have implemented strict ethical data safety guidelines that must be followed by both humans and ma- chines. For ECAs and ECA-related research, these guidelines are of special relevance since they need to “know” the elderly with whom they interact and whose momentary emotional and mental state they perceive. Only then will they be able to act as personal assistants in a way one would expect from a human caregiver. How- ever, this presupposes, on the one hand, access to personal data such as age, gender, disorders, handicaps, family status, cultural and social environment, etc., and, on the other hand, perception and analysis of mimics, gestures, and voice during the interaction. Furthermore, to “have a memory”, an agent must be able to access Figure 1: The MEESTAR model (from https://www.nks- the history of previous interactions, health and mood conditions of mtidw.de/dokumente/meestar-studie-englisch) the elderly and other relevant data, while due to better performance, easier maintenance, accessibility and other technical criteria, many services, including agents, are nowadays cloud-based. Technical But by no means should a technology attempt to substitute hu- solutions that store and process personal data must thus ensure man care. The decisions concerning the design and functionality of data protection and give the users the choice to agree to the sharing an ECA are thus also moral decisions and require an assessment of their data to an extent that will depend on the type of assistance from this viewpoint. A framework is therefore needed for the as- they desire. In particular, ethical guidelines must be followed that sessment of the ethical standards of these decisions. The MEESTAR foresee that any subject must, for instance, be fully aware about model [14, 30] illustrated in Figure 1,7 which was developed for the nature and volume of their personal data that an agent acquires the assessment of Ambient Assisted Living solutions, but which during the interaction or accesses prior to interaction and give an can be readily applied to assistance systems in geriatrics as well, informed consent to the acquisition and use of their personal data. is such a framework. MEESTAR foresees four different outcomes In case of advanced dementia, informed consent can only be given of the assessment: (i) the solution is unproblematic; (ii) the solu- by a legal guardian, as patients suffering from moderate, let alone tion is critical from the ethical perspective, but the critical aspects severe, dementia are not able to give informed consent. Therefore, can be addressed in the practical application; (iii) the solution is the first question for a researcher working on the development of ethically very problematic and needs to be permanently monitored ECAs for elderly should be: “Does my ECA research really need or completely abandoned; and (iv) the solution is to be rejected patients with dementia or could also cognitively fit or only mildly from the ethical perspective. The assessment is to be carried out impaired elderly be addressed as users?” along seven ethical dimensions: care, autonomy, safety, justice, pri- It is to be noted that if the ECA is an CE-approved medical prod- vacy, participation, and self-conception at the social, organizational uct, it can only be used in its specific indication for patients with or individual levels. Let us, in what follows, briefly interpret the specific diagnoses listed in the ICD-10 (international classification description of these seven dimensions provided in [14] from the of disorders) and cannot be applied in another medical diagnostic viewpoint of an ECA, without going into the detail of the levels.8 field since this would be “off-label”, which means that a doctor In accordance with Manzeschke [14], the evaluation of the care would need to document very carefully the reasons why he or she dimension must assess whether (and if yes, to what extent) an ECA uses the ECA or the medical product outside its diagnosis-related negatively influences the self-conception of a subject and/or their indication. relationship to the external world and whether it creates a depen- It is also to be kept in mind that in 2020 the risk categories for dency or limits the freedom of decision (assuming “a patronizing software applications as medical products will be further detailed. or negatively paternalistic” role). In the context of the autonomy dimension, we must assess whether the appropriate balance is kept 2.3.2 Assessment of the ethical standards of a technical solution. between the right of an individual to autonomy, the fact that an The goal of any ECA (as of any virtual assistant) in the context of individual’s criteria concerning autonomous decision making and geriatrics should be, firstly, to provide better care of elderly, but, acting may have become questionable or even untenable, and the secondly, also to secure their empowerment and social engagement right of an individual to care and support. in order to avoid the risk that they become dependent from an The safety dimension concerns the security and self-confidence expensive technology, impoverish or even lose their social life. of an individual and the role of the ECA in it. It thus assesses That is, ECAs should target to: whether the intervention of the ECA can lead to a more passive 7 ‘MEESTAR’ stands for “Modell zur ethischen Evaluierung sozio-technischer Arrange- • increase quality of life of the subjects; • protect human dignity at any age; ments” ‘Model for ethical evaluation of socio-technical arrangements’. 8 https://www.nks-mtidw.de/infomaterial/ethische-fragen-im-bereich- • enable autonomy and participation. altersgerechter-assistenzsysteme 3 stance of an individual in the sense that they delegate some tasks instrument that may address these needs. In Section 4 we assess whose execution is beneficiary to them, to a subjective feeling of then which of these needs can be, in fact, satisfied by ECAs. safety without objectively increasing safety, or whether the con- tribution of the ECA to the safety of an individual interferes with privacy or autonomy. 3.1 Needs of Geriatric Patients The privacy dimension shall assess whether an ECA records According to Section 2.1, the needs of geriatric patients revolve visual and audio information during interactions in intimate con- around frailty and handicaps (both somatic and mental). In the texts (e.g., bedroom, bathroom, dressing room, etc.) and thus may context of frailty, the patient is helped by an accurate assessment violate the personal sphere or offend the honor of an individual of the degree of their frailty with respect to all five symptoms such that the individual may feel ashamed or humiliated. Dignity listed in Section Section 2.1, as well as by support of the prevention is very important. Most subjects do not accept video monitoring or and treatment of the determined frailty. The degree of frailty is recording in rooms in which they might be exposed – even for fall often captured in terms of a “frailty index”. A number of frailty detection. indices (FIs) are known from the literature; cf., e.g., [4, 13, 26]. One The justice dimension shall assess whether the access to the of the most influential FIs is the Barthel Index, which measures services of the ECA is fair and in accordance with age-related the functional capacity of an individual on a 100 point scale in ten services. Furthermore, it is concerned with the cost distribution of predefined Activities of Daily Living (ADLs) [12], which include, the services and the question on the responsibility assigned, e.g., to among others, help needed with grooming, toilet use, feeding, etc. the care taker and to the social insurance. More recent indices, such as the Frailty score from the European In the context of the participation dimension, we need to assess SHARE-Study [25], CSHA [24], and FI-VIG [1] go beyond the ADLs. to what extent the services of the ECA in question facilitate (or For instance, the FI-VIG index also covers Instrumental Activities impede) an active participation of older persons in the working of Daily Living (IADLs), which target the capacity of an individual and social life and whether the kind of participation promoted to manage money and medication and use the telephone, as well as by the ECA is in accordance with the personal preferences of the the cognitive, emotional, social, nutritional, etc. capacities, which individual. cover all five frailty symptoms listed in Section 2.1. The capacity of The self-conception dimension, finally, reflects on the personal an individual with respect to a specific activity and the individual’s and social constraints that an ECA and its services impose on an in- frailty symptoms are assessed, as a rule, manually by medical and dividual and, related to this, to what extent the ECA is personalized, care specialists. i.e., takes the personal preferences and needs of an individual into To prevent and/or reduce frailty, targeted exercises and healthy account, instead of addressing standardized default user profiles. diet are recommended. The exercises and diet depend on the health Other ethically relevant questions that are not directly subsumed condition of a person and should thus be proven by a medical under the above seven dimensions concern, for instance, the accu- specialist. racy and quality of the services (e.g., information, recommendations, The needs of elderly with respect to their somatic disorders are instructions, etc.) of an ECA. An ECA is composed of several soft- not that different from the needs of other patients, although intense ware modules, each of which can fail to a certain degree. Consider, medical assistance may be required due to the susceptibility of for instance, speech recognition or reasoning over the statement elderly, for instance, to acute vascular events; see Section 2.1. In the of a user, where, e.g., unclear pronunciation (e.g., due to a poorly case of Parkinson Disease, as one of the main mental pathologies fitting denture) or use of irony can lead to misunderstandings and from which elderly may suffer, medication and physiotherapy are thus wrong reactions. The question on the liability thus comes up in of outmost importance. In the case of dementia (the other main case the ECA provides wrong information, misjudges the obtained mental pathology by which elderly may be affected), medication information and symptoms obtained from the user, or fails to detect may also be needed to treat, in particular, non-cognitive symptoms, changes in the addressed health conditions of the user. Any of these but personalized care is the key. failures may result in expensive and/or invasive procedures, with In general, (basic) care is of high relevance to the majority of serious consequences for the user. elderly, as is social companionship. It is crucial for elderly to have There are also several issues of data safety and legal aspects in the someone to talk, to exchange memories and information (e.g., about interaction of humans with ECAs. Thus, when an elderly individual places, times, people, etc.). Cognitive stimuli are also crucial. For develops a deep immersion experience with the ECA, they tend instance, often elderly love to solve crossword puzzles or Sudoku. to share personal information about other individuals (relatives, Bingo is also very attractive; in addition, Bingo has the advantage to friends, neighbors, etc.). This information may be of private nature train attention and reaction time, which are impaired most during (and thus violate their privacy) or apt to damage a third party’s aging. Curiosity for new technological devices might be a good reputation. indicator for mental health. However, no generalization is possible; each individual has their own preferences and their own needs. 3 BASIC NEEDS IN GERIATRIC MEDICINE The health issues of elderly sketched in Section 2.1 give us some 3.2 Needs of Caregivers hints on the needs of all three involved parties: geriatric patients, The needs of professional and informal caregivers and thus also care givers and medical personnel. In this section, we summarize the way in which ECAs can support them are rather different. Let these needs in general terms, without having in mind ECAs as an us, therefore, discuss them separately. 4 3.2.1 Needs of professional caregivers. As pointed out in the is a very helpful strategy to maintain emotional wellbeing despite Introduction, there is a significant shortage of professional care- increasing impairments and handicaps in old age. givers. Apart from an increased involvement of informal caregivers (discussed in the next subsection), this leads, on the one hand, to 3.3 Needs of Medical Personnel an increased recruitment of less qualified personnel (often with While the European legislation and ethical guidelines practically language barriers), and, on the other hand, to a higher workload exclude the delegation of diagnoses or treatment of diseases to of the well-formed caregiver personnel. To ease the pressure of intelligent agents, such agents can provide assistance to medical the higher workload, caregivers often ask for simplification of the specialists as intelligent symptom assessment instruments whose obligatory documentation in the care record of the health conditions outcome can be then used by specialists for diagnoses respectively of each care recipient and the procedures they carry out on them treatment, with the general goal to support the compression of (such as washing, dressing, changing bandages, etc.). morbidity to the last months before death beyond the late 80s or 90s The needs of ad hoc recruited less qualified personnel may be of many subjects [8]. Such an assessment can be time consuming of different kinds, for instance: (i) to be coached with respect to and medical specialists are always short of time. The needs of basic care tasks and potentially also handling of the basic care geriatric specialists with respect to symptom assessment cover equipment; (ii) in case of a language barrier, to receive support in the whole spectrum of characteristics and symptoms of geriatric the language of the caretaker; and (iii) to be informed about the patients. To begin with, there is the assessment of frailty, which is needs, preferences, etc. of the caretaker. essential to obtain a general picture of the conditions of an elderly. 3.2.2 Needs of informal Caregivers. Informal caregivers involved Geriatric specialists also routinely test elderly with respect to in care of geriatric patients are in their vast majority family mem- early symptoms of Parkinson disease and cognitive impairments, bers. In a recent study, in which 122 family members were inter- which may be related to Alzheimer’s and other types of dementia. viewed with respect to the main negative personal consequences of These tests are time consuming, but cannot be delegated to, e.g., the care of someone from their family (in this case, a patient who nurses since they require experience and profound expert knowl- suffered an acute stroke), the distribution of the main worries was edge. For instance, the most common cognitive impairment tests as follows; cf., [22]: are the paper- and pen-bound tests such as the Mini-mental State Examination (MMSE) by Folstein et al. [7] or the clock drawing test, (1) not to have enough time for own needs and interests (57%); which take less than 10 minutes. But there are also more elaborated (2) suffer from sleep disturbances (38%); tests that also cover attention and executive domains like the MoCA (3) feel tired and have no energy (34%); (Montreal cognitive assessment) [18], which takes more than 10 (4) have an irritated and aggressive relationship with the partner minutes and can be done, again, only by trained assessors. (34%); In addition to the aim to identify as early as possible early symp- (5) miss common activities (with the partner) outside the house toms of cognitive impairment, its prevention is a central issue in (30%); geriatric medicine – in particular, because causal therapies of such (6) be responsible for paperwork and finances (29%); diseases as Alzheimer’s are still out of sight for the next years. (7) be afraid that something bad will happen (24%). As pointed out in Section 2.2, a recent FINGER study in Finland To support informal caregivers, in this study a telephone-based shows that multimodal lifestyle interventions, including healthy counseling service was implemented as a randomized controlled nutrition, exercise and mental activity were able to moderately trial. However, this service resulted in no significant benefits com- delay cognitive decline. Some aspects have been shown to be of pared to a control group without counselling. On the other hand, higher relevance. For instance, control of obesity is a major topic as in the recent TABLU project,9 a tablet-based prototype for infor- overweight patients suffering from diabetes mellitus have a signifi- mal caregivers has been developed and evaluated. The prototype cantly increased risk of developing dementia, both with Alzheimer’s contains four modules: (1) introductory training course for care, pathology or of the vascular type [5]. Unfortunately, such lifestyle (2) video library of care, (3) written contact, and (4) video phone interventions are not easy to implement since each individual has conversations with a professional caregiver. The modules 2, 3, and 4 their own experience in life style changes. It requires a strong mo- have been positively tested by 41 caregivers, selected among more tivation and/or acute psychological strain, as it is given following a than 600 candidates, for up to 6 months. An online video from stroke or a heart attack. This might be a “teachable moment” which module 2 on principles of mobility10 has been furthermore watched increases plasticity to new habits or new technologies (like tread more than 48,000 times, which can be also interpreted as a positive mills or activity trackers and calories counting devices). evaluation outcome. Overall, it is to be noticed that many problems perceived by 4 HOW CAN ECAS ADDRESS THE NEEDS? caregivers are situated at a psychological level (as 3, 5, and 7 above) Let us now examine how ECAs can address the needs of the different or cannot be solved (as, e.g., 1 or 2) by technological devices because parties involved in geriatric care. the caregiver feels exclusively responsible for the care recipient. The acceptance of facts or circumstances which cannot be changed 4.1 ECAs and the needs of geriatric patients 9 TABLU: Technological Systems of Assistance Enable Independent Living, sponsored The general rule is that geriatric patients want to be treated with by the German BMBF (http://www.tablu.de/index.html). respect and the same effort as younger subjects, even if they are 10 https://www.youtube.com/watch?v=Qrttx1j6EUI slower in getting familiar with digital devices and the terminology 5 of the digital community.11 However, in contrast to the other two 4.1.2 Tasks of the ECAs. Ethical concerns currently prevent the types of users of ECAs (i.e., caregivers12 and medical personnel), involvement of ECAs in such sensitive care tasks as identification when addressing geriatric patients, close attention must be paid to of reasons for indisposition and taking measures against it, or daily the design of the avatar that embodies the ECA. medication intake control. We see the role of the ECAs in interac- tion with geriatric patients, first of all, as social companions. They 4.1.1 Design of the avatar. Since the current generation of el- should intervene to animate elderly to engage in physical and social derly is still not familiar with PCs, tablets and smart phones, the activities and stimulate them cognitively in order to make them interaction of the ECAs with them should most appropriately be abandon their physical and mental comfort zone (such as passive speech-based. In the context of all applications, the design of the movie watching and other passive entertainments) and thus stay avatar that embodies the ECA is crucial. In its default mood, it physically and mentally active. should have a friendly, sympathetic appearance.13 The cultural, In these interventions, ECAs can act as intelligent sensor devices social and age/gender related features of the appearance should that support the subject in their interaction with the environment in reflect the personal preferences of the elderly and thus be personal- case of sensory disabilities and/or as a social companion. Consider, ized. However, the observation of the design guidelines might still for instance, age-related macular degeneration (AMD). AMD is the not be sufficient. ECAs have been proposed as a natural computer leading cause of central blindness or low vision among the elderly interface for humans, which should be easy to understand by hu- in industrialized countries [3]. Patients suffering from AMD are mans who are not familiar with computer programs or computer often not able to read written material without expensive reading devices. But it might be difficult for an elderly to communicate aids. ECAs might help centrally blind persons by reading aloud the with an ECA, as most elderly are not familiar with human-like but newspaper after the user has read aloud the headline of the article not yet human faces, mimics, gestures and voices of an avatar. If or the death announcement in the local newspaper. Patients with the ECA is designed to resemble as much as possible a human, its glaucoma also have problems to see obstacles in the periphery of acceptance might be below the expectance. The sensation of an their vision and need guiding systems with optical landmarks or elderly might be in the “uncanny valley” [16] since they might not feedback by vibrations or sound, which might be monitored by an be sure whether the ECA or robot they are looking at and talking ECA with optical sensors, in analogy to a guide dog. to is a real person or an artificial agent. Some patients suffering from Alzheimer’s disease and related de- From a neurobiological point of view, it has been demonstrated mentias (ADRD) have serious problems in recognizing faces. They that young Japanese subjects show different brain activation pat- might become unable to recognize the partner or even their chil- terns during a repeated interactive display of an android, robot and dren. Patients suffering from this condition, referred to as “Capgras human. Violation of the expected norm could be measured by fMRI syndrome” [2], develop paranoia or confusion about their own iden- and by N400 waves in the EEG. The key assumption is that brain tity. The consequence is a tremendous stress in patients and also in activity is higher for a stimulus that does not coincide with an ex- caregivers. In this case, an ECA should not have a humanoid face pected (or predicted) norm or that is not explained by a generative as this might even worsen the condition. A familiar voice (e.g., of a neural model of the external causes for sensory states [9]. Recent known radio speaker) might be more helpful to provide serious and studies buttress the hypothesis that the “uncanny valley” could be personalized information. However, to the best of our knowledge, explained by the violation of the assumption what is human. They there are no experiences so far with subjects suffering from the have measured different N400 brain waves in congruent human-like Capgras syndrome in ECA or AAL settings. movements of a real human, a mechanical robot, and a realistic In its role of a social companion, the ECA could suggest specific robot [28]. social or cultural activities, offer news on specific topics, recom- In a very recent study that used pupillometry, the uncanny valley mend diets, etc. and conduct small talk on topics known to be of has been confirmed via the pupillary reaction of young subjects interest to the patient. The ECA can be also of use to alleviate during their confrontation with robotic and human emotions: when the sensation of loneliness, which elderly often perceive as a very confronted with “uncanny” robots, their pupils were less dilated negative condition. The feeling of loneliness may come up when a than when confronted with more human-like robots [23]. Although patient objectively lacks company or when they do not receive the there are no data available for seniors or patients suffering from attention they claim from their social environment. For instance, cognitive impairment,14 the uncanny valley might be wider in older in the case of moderate (or even mild) dementia, patients tend to persons than in younger persons, reaching extremes in cognitive talk again and again about the same events of the past (behavior impaired or in paranoid subjects, as some of them suffer from referred to as “perseveration”) that left a deep impression in them, propasognosia, i.e., inability to identify faces, or even have the which may lead to a negative reaction of their conversation partners. delusion that a familiar person has been replaced by an identical ECAs may be very “patient listeners”, as dogs and cats are. Their impostor (cf. the mention of the Capgras-Syndrome in Section 4.1.2). role could consist in dedicated affirmative reactions that encourage 11 This has consequences for the formulation of the informed consent to be signed by the patient to continue with their story. subjects in experiments with ECAs. In all of these tasks, personalization is crucial. The ECA should be 12 In the case of relatives of geriatric patients as caregivers, the design of the avatar knowledgeable about the personal characteristics and preferences may also be of relevance. of the patient and also take the reaction of the patient in previous 13 As verified in the KRISTINA project http://kristina-project.eu/en/, the subjects are interactions for future interactions into account. Obviously, this very sensitive to “unnatural” or inappropriately serious facial expressions of an avatar. 14 Importantly, our results imply that the mechanisms underlying the perception of implies the consideration of data privacy and data protection issues, other individuals are predictive in their nature. both of the patients themselves and of the individuals who might be 6 mentioned or commented upon in the conversations. This concerns, as already pointed out above, they bear a significant share of the in particular, family members, friends, neighbors, etc., who might workload related to geriatric care. see themselves exposed to public. In any case, an ECA should first Informal caregivers may profit from the same coaching functions build up trust by providing serious and reliable information, before of an ECA as discussed above in Section 4.2.1. In addition, when it begins to intervene in the personal life of a subject. The first steps ECAs exercise their role of a social companion of a care recipient, in the interaction of elderly with the ECA should be supervised they also help to mitigate one of the most negative consequences by an expert. The time of assistance by an expert can be reduced from which informal caregivers suffer, namely not to have enough to nearly zero within weeks or substituted by a technology affine time for own needs and interests (cf. Section 3.2.2). relative. Apart from “classical” geriatric rehabilitation at rehabilitation 4.3 Supporting medical personnel hospitals or on outpatient basis, serious gaming and networks of Intelligent agents, including ECAs, can support medical personnel humans in change of life style might be a target of ECAs and smart in the context of geriatric medicine first of all in tasks related to technologies in general. It is still under debate whether older pa- patient monitoring and health condition assessment. For instance, tients will be willing and able to also join those virtual communities. the five frailty symptoms presented in Section 2 can be controlled Experimental setups will be needed to obtain a clearer view in this by an agent: respect. (i) involuntary weight loss can be measured and protocolled using standard procedures like scales; 4.2 ECAs and the needs of caregivers (ii) objectified muscle weakness, loss of muscle mass and strength In accordance with the differentiation of the needs of professional can be measured by pressure sensors on smart objects in and informal caregivers, the involvement of ECAs may be also households, such as, e.g., a roller shutter belt or a window different. handle; (iii) subjective exhaustion can be asked about by an ECA or 4.2.1 Professional caregivers. ECAs can be of use to both, the derived from changes of speech loudness and speed, pitch less qualified and the overstrained qualified care personnel. For the of the voice, mimics or gestures; first, an ECA could serve as a coach and intermediator. As a coach, it (iv) immobility, instability, and fall-prone gait and posture can can train the caregiver in the basic care procedures (including, e.g., be assessed by balancing in a virtual environment rather washing, bandaging, erecting, etc.), the use of medical or supportive than in standard clinical environments that check the ability devices, practices of interaction with care recipients, etc. This can and duration of semi-tandem standing with open and closed be done in terms of interactive demonstrations, monitoring and eyes; correction, and/or guidance. ECAs can also search for relevant (v) decreased physical activity (in terms of basic and / or in- material in the web and offer to care personnel a summary of it – as strumental everyday activities) can be, again, inquired in is done by the KRISTINA agent for Alzheimer’s related multilingual interviews led by an ECA or be measured by digital activity information [29]. trackers such as smart watches or by movement integration As intermediator, an ECA can provide to the caregiver the per- in ECAs. sonal dietary, social, or daily life routine preferences of a care recip- ECAs can be also help in the diagnosis of age-related diseases. ient, their health conditions, medication, etc. In case of language Thus, early symptoms of Parkinson include reduced perception and barriers, advanced ECAs can also serve as interpreters. Migration distinction of odors and disturbed sleep. Agents could carry out is a big issue in the EU. For instance, in Germany less trained care- odor exposure tests and track sleep parameters by smart sensors. givers come from Poland and Romania; they are not familiar with The Aachener Aphasie Test (AAT)15 is the standard assessment the regional language, the habits or dishes. Multilingual ECAs might tool in Germany for the diagnosis of aphasia (which is a wide-spread bridge the language barriers. phenomenon after a stroke, more prevalent in elderly), but it can For overstrained qualified care personnel, an ECA can also as- only be performed by skilled speech therapists, psychologists or sume some routine communication tasks with the patient and also neuro-linguists. As recently shown by K´’onig et al. [10], artificial act as provision instance of information related to the patient, ob- intelligence can help to interpret the semantic verbal fluency (SVT) tained, e.g., in interaction with family members or with the patient in a quick and efficient way. In this work, automatically extracted themselves. Certain health-related information (such as, e.g., body clusters and switches were highly correlated in SVT with man- temperature, heart rate, number of made steps, etc.) is more reliably ually established values and could separate healthy controls and obtained using sensors, rather than in a verbal conversation with patients with probable Alzheimer’s dementia and with mild cogni- the patient. For this purpose, bio sensor data need be provided to tive impairment with a good area under the curve (AUC) of 0,94 for the knowledge processing module of the ECA. In any case, if ECAs healthy controls (HC) compared with subjects with – Alzheimer are deployed to assist care personnel, they should not be perceived dementia (AD) and less between HC – and subjects suffering from as controllers or surveillants. Most of care personnel object to be mild cognitive impairment (MCI) AUC of 0,76. tracked by their companies and employers. Furthermore, ECAs can be supportive in the assessment of cogni- tive impairment, which is nowadays measured by neuropsycholog- 4.2.2 Informal caregivers. Empowerment of informal caregivers ical screening and assessment tools. For instance, recent studies (cf. should be a central goal of tools and digital devices like ECAs since, 15 https://www.testzentrale.de/shop/aachener-aphasie-test.htm 7 [17]) showed that when the “classical” clock drawing test (cf. Sec- substitute for human-to-human interaction, which is essential to tion 2) is done on a tablet with an “active” pen, it offers additional elderly to overcome loneliness and cognitive and emotional decline; information. Thus, the time in air before the subjects draw the digits ECAs can only act as complementary assistants. As assistants they and the hands is significantly longer in subjects with mild cognitive can be also be of use in different scenarios to caregivers and medical impairment compared to healthy control persons, as they hesitate professionals that can be situated in private households, care homes to set the pen to the surface. This hesitation cannot be monitored or hospitals. by the “classical” paper and pen version of the clock drawing task. No matter which role an ECA assumes data security and privacy There is a huge potential in pen-based digital cognitive testing, but also compatibility with care records is crucial. The user must especially if an agent is able to classify the performance and save own their data and be able to define which data are private and can valuable time for nurse and physicians. be only shared with relatives, which data can be shared with medical As speech is early affected in several types of dementia and and care professionals and which data can be made public. With the also in major depression, there are several quick tests such as the Digital Single Market Law “Communication on enabling the digital phonematic and semantic verbal fluency tasks, which count the transformation of health and care in the Digital Single Market; number of correct words starting, e.g., with ‘M’ (phonematic) or all empowering citizens and building a healthier society” from 25 animals (semantic) a subject can give within a minute. But there is April2018,16 the European Commission provided a legal fundament much more information in such a spoken row of words, which can for the use of ECAs in the geriatric context. be controlled by intelligent technologies: animals can be categorized Recent initiatives such as the SOLID project17 launched by T. by their frequency from very frequent like dog to infrequent like Berners-Lee furthermore indicate solutions to the challenge of the grasshopper. The speed of speech and the number of hesitation storage and use of private data. As Berners-Lee writes in his open markers (such as ummm. . . and aeh. . . ), repetitions, restarts, etc. also letter on Sept 28th 2018: “It gives every user a choice about where give information on the verbal abilities of the speaker. ECAs could data is stored, which specific people and groups can access select be programmed to apply geriatric assessment tools and questions elements, and which apps you use. It allows you, your family and of different aspects: colleagues, to link and share data with anyone”.18 (i) for depressive symptoms: geriatric depression scale (GDS), Finally, with the increasing maturity of ECAs, the opportunities (ii) for quality of life: quality of life in Alzheimers disease, for their successful commercialization in the geriatric sector (and in (iii) for cognition: mini mental state examination (MMSE), and the health sector in general) also grow. Business models should be (iv) for ADL/ IADL: Katz activities of daily living and Lawton versatile and adaptive. A personalized entertainer, an interpreter of instrumental activities of daily living, a caregiver from another culture, a promoter of healthy food with (v) For detection of problematic and aggressive behavior in the expertise in regional recipes, an innovative neuropsychological case of mild and moderate dementia, the revised memory assessment assistant, or a life quality and health conditions moni- and behavior problem checklist (RMBPC). toring device – all of them are potentially successful applications. Besides easy and quick assessments, the transfer of this infor- mation into adequate care and medical procedures is essential for REFERENCES [1] J. Amblàs-Novellas, J.C. Martori, J. Espaulella, R. Oller, N. Molist-Brunet, M. dementia-friendly hospitals and cities. A study from T´’ubingen has Inzitari, and R. Romero-Ortuno. 2018. Frail-VIG index: a concise frailty evaluation shown that visual rehabilitation with reading training is beneficiary tool for rapid geriatric assessment. BMC Geriatrics 18, 29 (2018). [2] J. Capgras and J. Reboul-Lachaux. 1923. Illusion des sosies dans un delire sys- to seniors suffering from macula (retinal spot of best visual acuity) tematisé chronique. Bulletin de la Societe Clinique de Medicine Mentale 2 (1923), degeneration; it also improved their impaired mood [15]. Such a 6–16. visual rehabilitation could also be coached by an ECA which dis- [3] J.B. Christoforidis, N. Tecce, R. Dell’Omo, R. Mastropasqua, M. Verolino, and Costagliola C. 2011. Age related macular degeneration and visual disability. Curr plays the reading material, assesses the reading and gives feedback Drug Targets 12, 2 (2011), 221–233. on improvements in reading speed and pronunciation. [4] A. Clegg, J. Young, S. Iliffe, M.O. Rikkert, and K. Rockwood. 2013. Frailty in elderly people. Lancet 381 (2013), 752–62. [5] DGN. 2016. S3 Leitinie Demenzen. Deutsche Gesellschaft fuer Neurologie. 5 CONCLUSIONS [6] G.W. Eschweiler, T Leyhe, S Kloeppel, and M Huell. 2010. New Developments in the Diagnosis of Dementia, doi: 10.3238/arztebl.2010.0677. Dtsch Aerztebl Int 107, Our aging society faces a continuously growing number of geriatric 39 (2010), 677–683. patients and an increasing shortage of professional and informal [7] M.F. Folstein, S.E. Folstein, and P.R. McHugh. 1975. “Mini-mental state”. A caregivers. No reversion of this tendency can be expected in the practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12, 3 (1975), 189–198. near future. This calls for intelligent technologies such as ECAs as [8] J.F. Fries. 1980. Aging, Natural Death, and the Compression of Morbidity, part of the solution. However, no off-the-shelf ECA will work. ECAs doi:10.1056/NEJM198007173030304. New England Journal of Medicine 303, 3 have to be adapted to the special needs of elderly. Firstly, we must (1980), 130–135. [9] K. Friston. 2010. The free-energy principle: a unified brain theory? doi: be aware about the heterogeneity of elderly from fit, to pre-frail 10.1038/nrn2787. Nat Rev Neurosci 11, 2 (2010), 127–138. and frail subjects. Apart from physical differences among elderly [10] A. Koenig, N. Linz, J. Troeger, M. Wolters, J. Alexandersson, and P. Robert. 2018. Fully Automatic Speech-Based Analysis of the Semantic Verbal Fluency Task, mental changes from cognitively fit to mild cognitive impairment doi: 10.1159/000487852. Dement Geriatr Cogn Disord 45, 4–5 (2018), 198–209. and dementia occur. This decline separates subjects who can give informed consent and can adapt to new types of interaction from 16 https://ec.europa.eu/digital-single-market/en/news/communication-enabling- subjects in need of guardians and caregivers. Only the first group of digital-transformation-health-and-care-digital-single-market-empowering subjects should be addressed by ECAs. However, in the case of this 17 https://solid.mit.edu/ 18 https://www.inrupt.com/blog/one-small-step-for-the-web group , we must keep in mind that ECAs cannot be a fully-fledged 8 [11] S. Lerche, M. Hobert, K. Brockmann, I. Wurster, A. Gaenslen, S. Hasmann, G.W. Eschweiler, W. Maetzler, and D. Berg. 2014. Mild Parkinsonian Signs in the Elderly – Is There an Association with PD? Crossectional Findings in 992 Individuals. PLoS One 9, 3 (2014). [12] F. Mahoney and D. Barthel. 1965. Functional evaluation: The Barthel Index. Md Med J 14 (1965), 61–65. [13] T.K. Malmstrom, D.K. Miller, and J.E. Morley. 2014. A comparison of four frailty models. J Am Geriatr Soc 62, 4 (2014), 721–726. [14] A. Manzeschke. 2015. MEESTAR: Ein Modell angewandter Ethik im Bereich assis- tiver Technologien. Steiner, Stuttgart. [15] A. Mielke, K. Wirkus, R. Niebler, G.W. Eschweiler, N.X. Nguyen, and S. Trauzettel- Klosinski. 2013. The influence of visual rehabilitation on secondary depressive disorders due to age-related macular degeneration. A randomized controlled pilot study. Ophthalmologe 110, 5 (2013), 433–440. [16] M. Moro, K.F. MacDorman, and N. Kageki. 2012. The Uncanny Valley, doi:10.1109/MRA.2012.219281. IEEE Robotics & Automation Magazine 19 (2012), 98–100. [17] S. Mueller, O. Preische, P. Heymann, U. Elbing, and C. Laske. 2017. Increased Diagnostic Accuracy of Digital vs. Conventional Clock Drawing Test for Discrim- ination of Patients in the Early Course of Alzheimer’s Disease from Cognitively Healthy Individuals. Front Aging Neurosci 9 (2017), 111. [18] Z.S. Nasreddine, N.A. Phillips, V. Bédirian, S. Charbonneau, V. Whitehead, I. Collin, J.L. Cummings, and H. Chertkow. 2005. The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53, 4 (2005), 695–699. [19] T. Ngandu and et al. 2015. A 2 year multidomain intervention of diet, exercise, cognitive training, and vascular risk monitoring versus control to prevent cogni- tive decline in at-risk elderly people (FINGER): A randomised controlled trial. Lancet 385, 9984 (2015), 2255–2263. [20] S. Norton, F.E. Matthews, D.E. Barnes, K. Yaffe, and C. Brayne. 2014. Potential for primary prevention of Alzheimer’s disease: an analysis of population-based data. Lancet Neurol 13, 8 (2014), 788–794. [21] T. Otsuka, Y. Tomata, S. Zhang, K. Sugiyama, F. Tanji, Y. Sugawara, and I. Tsuji. 2018. Association between social participation and incident risk of functional disability in elderly Japanese: The Ohsaki Cohort 2006, doi: 10.1016/j.jpsychores.2018.05.004. J Psychosom Res 111 (2018), 36–41. [22] K. Pfeiffer, D. Beische, M. Hautzinger, J.W. Berry, J. Wengert, R. Hoffrichter, C. Becker, R. van Schayck, and T.R. Elliott. 2014. Telephone-based problem-solving intervention for family caregivers of stroke survivors: A randomized controlled trial, doi: 10.1037/a0036987. J Consult Clin Psychol 82, 4 (2014), 628–643. [23] M. Reuten, M. van Dam, and M. Naber. 2018. Pupillary Responses to Robotic and Human Emotions: The Uncanny Valley and Media Equation Confirmed, doi: 10.3389/fpsyg.2018.00774. Front Psychol 23, 9 (2018), 774. [24] K. Rockwood, X. Song, C. MacKnight, H. Bergman, D.B. Hogan, and I. McDowell. 2005. A global clinical measure of fitness and frailty in elderly people. CMAJ 173, 5 (2005), 9–13. [25] R. Romero-Ortuno, C.D. Walsh, B.A. Lawlor, and R.A. Kenny. 2010. A frailty instrument for primary care : findings from the survey of health, ageing and retirement in Europe (SHARE ). BMC Geriatr 10, 57 (2010). [26] S. Searle, A. Mitnitski, E. Gahbauer, T.M. Gill, and K. Rockwood. 2008. A standard procedure for creating a frailty index. BMC Geriatr 8, 24 (2008). [27] Y. Tomata, S. Zhang, K. Sugiyama, Y. Kaiho, Y. Sugawara, and I. Tsuji. 2017. Changes in time spent walking and the risk of incident dementia in older Japanese people: The Ohsaki Cohort 2006 Study. Age Ageing 46, 5 (2017), 857–860. [28] B.A. Urgen, M. Kutas, and A.P. Saygin. 2018. Uncanny valley as a window into predictive processing in the social brain. Neuropsychologia 114 (2018), 181–185. [29] L. Wanner, E. André, J. Blat, S. Dasiopoulou, M. Farrús, T. Fraga, E. Kamateri, F. Lingenfelser, G. Llorach, O. Martínez, G. Mediskos, S. Mille, W. Minker, L. Pragst, D. Schiller, A. Stam, L. Stellingwerff, F. Sukno, B. Vieru, and S. Vrochidis. 2017. KRISTINA: A Knowledge-Based Virtual Conversation Agent. In Advances in Prac- tical Applications of Cyber-Physical Multi-Agent Systems, The PAAMS Collection, Y. Demazeau, P. Davidson, J. Bajo, and Z. Vale (Eds.). Springer, Cham, 284–295. [30] K. Weber. 2015. MEESTAR: Ein Modell zur ethischen Evaluierung sozio-technischer Arrangements in der Pflege- und Gesundheitsversorgung. Steiner, Stuttgart. [31] Q.-L. Xue. 2011. The Frailty Syndrome: Definition and Natural History. Clinical Geriatric Medicine 27, 1 (2011), 1–15. 9