Introduction

By 2050, the number of individuals over the age of 85 is projected to be three times more than today [26]. In this scenario, most older adults will need physical, social, and cognitive assistance. Indeed, aging has a considerable impact on the health of older adults in terms of cognitive and EMPATHY: 3rd International Workshop on Empowering People in Dealing with Internet of Things Ecosystems. Workshop co-located with AVI 2022, June 06, 2022, Frascati, Rome, Italy. physical impairments, which influence the abilities to complete and perform basic activities of daily living, such as cooking, shopping, managing the home, bathing, and dressing. Nowadays, a large proportion of cognitive assistance is provided by informal caregivers, usually family members. These caregivers often experience a negative impact on their psychological, emotional, and physical well-being due to the high workload [3]. Given the high health care expenditure at older ages, and their effects on family caregivers, new technologies to assist older adults with cognitive impairments are urgently needed.

Non-pharmacological interventions, such as physical training, cognitive training, and social stimulation activities have been used to mitigate the cognitive decline by maintaining or improving cognitive abilities, social well-being, and quality of life of older adults [3,5]. However, traditional interventions require experienced instructors who may be unavailable. Assistive technologies can provide useful support to address this problem. They are technologies that aim to assist different types of users during their rehabilitation. They can help older adults maintain their independence during daily routines and can also be an important instrument during their rehabilitation [15].

In recent years, humanoid robots have increased their similarity to human behaviour starting from gestures and facial expressions to the ability to understand questions and provide answers. Thanks to such humanlike characteristics, the interaction between people and robots is becoming more natural. The behaviour of such robots can also be personalized through end-user development approaches, such as the use of trigger-action rules and associated support [10]. A recent literature review [16] indicates that the humanoid robot is an interactive technology still not sufficiently investigated for supporting the cognitive stimulation of older adults.

In this paper, we present a novel approach based on a Pepper humanoid robot, with the purpose to obtain games for cognitive stimulation of older adults. We consider this kind of technology in this context because it can potentially promote seniors' cognitive, physical, and emotional well-being and also reduce the workload of the healthcare system [25]. A humanoid robot is a system that can employ different interaction strategies, such as verbal and non-verbal communication, the use of facial expressions, communicative gestures, and sensors. These capabilities are essential to creating social and emotional interaction with users to increase their acceptability and users engagement, which may increase the possibility to reach the goal of assistance in less time and with better results [3].

Using robots to support and assist patients can be a valuable tool to help them during their cognitive training. In such context, digital cognitive training through serious games may potentially benefit those with cognitive impairments more than traditional training due to enhanced motivation and engagement. In the literature, different studies show how digital games can obtain positive results in stimulating older adults and helping them improve their cognitive abilities compared to traditional training [24].

Combining a humanoid robot and a serious game can be an exciting solution to obtain measurable progress in cognitive functions and stimulate the user to continue the training program. We aim to offer novel digital training through serious games designed using personally relevant material from older adults' life. They will be based on elements associated with their biography, thus making interactions personalised, relevant, and more engaging.

The Remind Approach

The psychological well-being of older adults may be affected by some age-related conditions, such as approaching death, loss of family members, and reduced autonomy. A meta-analysis [2] indicates that the practice of life review, even more than reminiscence (reminiscence involves describing a memory itself, while life review is based on discussing what a memory means), is a good instrument for improving the psychological well-being of older adults: in fact, its effect sizes are comparable to those of cognitive-behavioural therapy. Serrano et al. [17] found that the practice of autobiographical memory improved the mood of the elderly by improving their life satisfaction. Furthermore, Damianakis et al. [6] report that interventions that contextualize history, personality, and life experiences can contribute to improving both communication and social interactions between family members and between family members and formal caregivers.

Based on previous experiences [12], we have started the development of a new prototype in which the serious games installed on the humanoid robot will motivate older adults by engaging them in playful situations that draw on their personal memories, with which they can interact. Indeed, such serious games are designed using personally relevant material from older adults' life. Specifically, the games are based on elements associated with the biography of the older adults, thus making interactions more relevant and more likely to keep them engaged while enhancing their well-being as well. In the following, we describe the approach proposed for a platform able to support cognitive stimulation through humanoid robots, the multimodal app for collecting memories, and the first set of games able to exploit such memories through the humanoid robot.

The REMIND Platform

Biographical information is exploited in a group of games that aim to support tasks related to Activities of Daily Living (ADL), which seem to have some potential in terms of both utility and cognitive stimulation according to a recent systematic review [16] that has analyzed various game types employed in this domain. An example game is one whose object is carrying out cooking activities, such as replication of recipes: an activity that can stimulate multiple cognitive domains simultaneously (memory, attention, planning), but that can also be adapted and calibrated to stimulate in a more targeted way each of the cognitive domains mentioned above. In addition, the evocation and detection of a user's emotional state are becoming a crucial element in the aim of developing more effective interfaces between humans and computers, especially in applications such as games and e-learning tools [14]. This is typically done by analyzing emotions while playing, which can be performed by using sensors in a wristband to detect relevant information. For this purpose, we aim to infer emotional states from physiological measures. Galvanic skin response, skin temperature, heart rate, and Inter-beat (RR) intervals are all valuable in inferring emotional states. We consider some of these measures detected also through an Empatica E4 wristband in order to assess how engaged participants feel while playing the games. The humanoid robots will serve as personal trainers, proposing exercises and communicating through various modalities, and challenging users in cognitive games relevant to their daily life (e.g. by memorizing scheduled events or names of family members and friends).

The solution aims to allow doctors and caregivers to configure the exercises and choose the most suitable games to stimulate the cognitive abilities of end users and enhance their experience. The REMIND platform is based on a modular architecture allowing the deployment of the multimodal serious cognitive games on a humanoid robot, which can stimulate interest and engagement from seniors that would not be possible with other types of smaller and more limited robots thanks to its human-like appearance and behaviour.

On the one hand the older adults, by interacting with the biographical app, provide relevant biographical data that are mainly used to customize the games, which thereby will be highly personalised for them. On the other hand, seniors will also interact with the games in order to stimulate their cognitive abilities; the data produced during the interactive sessions will be exploited by the Game Logic (to improve the adaptation of the game according to the data gathered in previous game sessions) as well as by the Analytics services, which will improve its analysis by also including such data. The caregivers will be able to interact with a Game personalisation tool, which allows them to provide further relevant personalisations for the interaction with the games. In addition, the caregiver will also interact with an Analytics tool, to have both overview and detailed information regarding user performance and state. The goal is to develop end-user configurable games. Game developers implement the features, functionalities, and graphics of the core game. The caregivers will be able to configure the game, according to the skill and learning level of the end users. For this purpose, previous programming experience is not required, as the game's configuration will be performed through intuitive user interfaces. In addition, the games include a custom tracking system, which tracks the data about user performance (such as time, date, pass/fail, score) and other important metrics necessary for the analytics of serious games (comments, number of errors, completion level, skill level achieved etc.).

The Biography App

A responsive Web application has been developed to collect older adults' memories. It can receive them both through graphical and vocal interaction. At the beginning, users are asked whether they want to enter a new memory or review those previously entered. A first set of possible categories of memories has been identified, and shown at the user access (see Figure 1, left). They can refer to music, specific events, games that the user liked to play, locations, food, and hobbies. When a memory category is selected, then the user can provide the information associated with the specific memory. For example, Figure 1 (right) shows the user interface associated with entering a memory related to a hobby. The user has to indicate a name characterizing the hobby, then provide a description (it can be done either vocally or by keyboard), the user can also indicate the age at the time of the hobby, and then provide a list of activities required by the hobby. Such list can then be used by the Pepper game for specific exercises.

The Games with the Humanoid Robot

The prototype aims to stimulate cognitive functions through play sessions, which should last 15-20 minutes. The games present various exercises such as completion of a memory with the main element missing, choosing from a multiple choice of elements, and in case of a correct answer the memory is reread to the older adult. The exercises should be useful for making the participants think and reason to provide the correct answer. An example is a memory: "In 1950 I was in Paris with my daughter Eleonora and I ate at the Angel restaurant", and Pepper will introduce it with some missing elements, asking for example "Do you remember the missing element of your memory in 1950 when you were in Paris with your daughter Eleonora and ate in the …. restaurant"?

Another exercise is to reorder a list of activities to perform a task (hobby, job or home procedures), starting with an unordered list (this stimulates executive functions and procedural memory). For example, considering the list of home activities when users wake up, Pepper proposes the following list: make the bed, rinse your face in the bathroom, do morning exercises, have breakfast, go to the newsstand; and the older adult should reorganise it.

One exercise is about the association of a memory to a main connected element. For example, Pepper can present three memories and three locations and ask the user to indicate for each memory the corresponding location. A further exercise considers the year of a memory, and asks the user to select which important event in the world happened that year from a set of events that are listed.

Another exercise asks questions about an element drawn from the memories already saved for a specific older adult. Suppose the memory is about something that happened in the city of Pisa. Then there can be questions such as: What are the main squares or places of interest in Pisa? What is the name of the river that crosses Pisa? What are the names of the inhabitants of Pisa? In the case of memories about a song, then the questions can be about the year of the song or the singer.

We also plan to use memories uploaded by relatives or people close to a participant, tell the memory to the older adult, and pose questions or ask them to complete sentences regarding the relative's memory. For example, Pepper says "I have a memory from your daughter Eleonora, do you want me to tell you about it? The memory is when she visited Florence, she saw the most famous monuments and entered the Uffizi to see the works inside; her favourite was Birth of Venus ". Then, Pepper asks questions such as "What was Eleonora's favorite work in the Uffizi?"

Conclusions

In this paper, we introduce a novel approach to personalizing serious games for cognitive stimulation of older adults delivered through a humanoid robot. It is based on a multimodal Web app to collect memories of the older adults, and then such content is exploited in a set of games implemented for a Pepper robot.

In future work, we plan to validate the approach with a user test involving a group of people aged 65+, participating in a Train the Brain programme in which they regularly attend cognitive stimulation exercises in a care centre. We also plan to develop a customization and analytics environment to further extend the possible personalizations of games.