Reminiscence plays an important role in the lives of older adults [8]. Many perfect the art of storytelling and enjoy its social benefits. The telling of stories of past events and experiences defines family identities and is an integral part of most cultures. Losing the ability to recollect past memories is not only disadvantageous, but can prove quite detrimental, especially to many older adults. In this paper we introduce MemoryLane, a Personal Digital Assistant (PDA) based application being developed to enhance the reminiscence capabilities of older adults. Using abilities and preferences as a basis, MemoryLane employs Artificial Intelligence (AI) techniques to adapt its multimodal interface to accommodate the needs of differing older users and to compose and recount user life-cached multimedia as memory stories.
The population count of older people is steadily increasing,
especially in the more economically developed countries of the
world. In Ireland the census of 2006 recorded an increase over the
previous ten years in excess of 54,000 in the number of persons
aged 65 years and over [
Conference’04, Month 1–2, 2004, City, State, Country.
Copyright 2004 ACM 1-58113-000-0/00/0004…$5.00.
of the natural ageing process ensuring that the number of cases
increases steadily as the older population grows. This varies
among individuals, affecting abilities such as memory and
planning, and often impeding reminiscence. Assistive
technologies exist which support older adults with memory
impairment and act as reminder systems that often liaise with
carers [
There are several bodies of literature which relate to this research: reminiscence, HCI for older adults and usability studies and the measurement instruments employed in user evaluations and testing. In this section we briefly discuss relevant key literature. 2.1
Research is being conducted into various aspects of reminiscence.
Chaudhury [
Developing technologies for older adults is an exacting science,
often varying from established HCI research processes. Zajicek
[
Many research tools, metrics, instruments and scales have been
developed in an attempt to measure the intangible emotions,
feelings, opinions and well-being of older adults.
2.3.1 Measuring Reminiscence & Social Well-Being
Haight [
Since its inception in 1986, Davis’ [
Heerink et al. [
Two ethnographical studies were conducted to elicit requirements
to assist in the design and implementation of MemoryLane. Due
to the known benefits of reminiscence among older adults [
The first study conducted with a sample of 15 participants aged 65+ investigated current PDA usability levels among older adults. Each participant was interviewed in familiar surroundings in a one-to-one structured interview format. Questionnaires were employed to record the participants’ opinions of and preferences for different colours, sizes and number of interface components. Participants were given a demonstration of how to interact with a PDA by a researcher, followed by observation of their capability in attempting to complete pre-set interactive PDA tasks as depicted in Figure 1. While the study afforded a rich insight into the preferred sizes and colours of on-screen buttons and text, it became clear from the outset that the participants found the PDA extremely complicated to use and had difficulty even knowing where to start with no one finding the interface instinctive or intuitive. This was evidenced by the level of assistance requested and given. Despite the functionality of a PDA being demonstrated beforehand, not one of the participants could carry out even the most basic of tasks unaided, participants all referred to ‘not knowing what to do next’, and many frequently objected to cluttered screens and complex menu structures. In addition to this, participants who had poorer levels of vision or dexterity often complained of cumbersome scroll bars and impracticable text and button sizes. However, the potential and portability of a PDA appealed to the majority of participants who remarked on it being ‘small enough’ to fit into a handbag or breast pocket. However, most referred to being ‘too old to learn how to use one now’. This would imply that many older adults possess a genuine interest in engaging with mobile technologies and that a PDA has a certain appeal. However, due to complex interfaces many choose not to experiment with such devices. These findings suggest that adopting AI approaches to both create intuitive applications which guide user navigation and which would dynamically adapt their interfaces to support varying user abilities would certainly provide older users with a more fruitful PDA interaction experience.
The second study was designed to investigate the reminiscence capabilities, patterns and preferences of older adults. The findings from this study influenced the choice of reminiscence topics selected for MemoryLane to ensure that it produces topical and pleasing memories for users. Informal focus groups were conducted with older adult samples which examined both episodic/autobiographical1 and procedural2 memories and elicited valuable oral histories. We examined how older adults recalled their past experiences singularly in isolation, socially in groups of their peers and also with younger people, such as family members. Reminiscence discussion was initially conducted without the aid of props to investigate participants’ powers of (un-aided) recollection of past events. This independent discourse was followed by further reminiscence sessions during which users were encouraged to consider various cultural probes, e.g., artefacts, photographs, newspapers. and a specially compiled Memory Scrapbook as pictured in Figure 2 to investigate if this improved their reminiscence experience. Participants found the sessions both stimulating and enjoyable, and all agreed that their powers of reminiscence were richly enhanced, many evoking long-forgotten histories, when using the memory prompts. This provided a strong argument for the popularity and usefulness of developing MemoryLane as a portable memory companion. The results of the study found that in the main, genders frequently wished to reminisce and discuss similar gender specific topics. However, there were some topics which seemed to be of universal interest to both genders. The more popular reminiscence topics are listed in Table 1. These results inspire the category options from which the user can select the topic for a memory story. 1 Personalised memories of events, times, places and associated emotions. 2 Long term ‘how to’ memory of skills and procedures.
MemoryLane design follows the USID methodology [
MemoryLane is a hybrid system which incorporates the AI techniques of Case-Based Reasoning (CBR) and Rule-Based Reasoning (RBR) for decision making, generation of data and also to address the usability problems encountered by older adults when using mobile devices. The data flow of MemoryLane’s architecture is given in Figure 3. User abilities and preferences are input to the system to form a unique user profile and the information stored in this profile is consulted for all future decision making for the duration of that user’s interaction. MemoryLane has two primary objectives: (1) multimodal interface configuration, and (2) dynamic generation of appropriate and entertaining memory stories.
The first intelligent aspect of MemoryLane is concerned with configuring the interface on the basis of its current user’s preferences and abilities, thus tailoring to individual needs, e.g. a visually impaired user would be compensated with an audio enriched interface. To achieve this, the user is required to enter a rating for their perceived ability for four different modalities: hearing, vision, speech and dexterity. Users must rate themselves as having normal, reduced or very poor levels of these abilities. The four ratings entered by the user are stored as part of that user’s unique profile and are linked with the interface input and output elements. Hearing determines the volume level, speech, the usage of automatic speech recognition (ASR), vision governs the use of text to speech (TTS) and frequency and sizes of text and images. While both vision and dexterity govern the size and choices of on-screen buttons and menus available to that user. As seen in Table 2, there are three possible categories (default, enhanced and superior) for each modality. Each of these categories has a pre-set interface specification. In this way the interface can be adapted to suit the varied needs of older adults. For example, a user profile purporting normal hearing, very poor vision, normal speech with reduced dexterity as shown in the shaded cells of Table 2, would be provided with a tailored interface that would accommodate his/her needs. The default hearing specification would provide a default volume level for all audio and TTS, the default speech specification would involve default levels of ASR. The superior vision specification would ensure that TTS would relay all on-screen prompts aloud and display all on-screen text, images at the maximum size. However, in this instance, because vision was rated as very poor, the enhanced dexterity specification would be upgraded to superior, thus ensuring that all on-screen buttons and menus were of an appropriate size. All system default levels are based on the results of the prior PDA requirements analysis.
The second intelligent aspect of the system is concerned with intelligently generating dynamic ‘memory stories’. The user’s life-cached multimedia items provide story content and are output in accordance with the user’s preferences and abilities. The system offers the user a choice of categories such as family, holidays, weddings or history from which they can select the topic for the new memory story. Once a selection is made, the system locates all stored multimedia objects which are tagged as (a) belonging to that user, and (b) belonging to the chosen category. Appropriate multimedia items, based on the likes and dislikes of the user, are selected from this pool for inclusion in the memory story. This multimedia, including TTS and non-speech audio if deemed applicable, are synchronized and fused into a memory story for simultaneous output through multithreading.
MemoryLane is deployed on a PDA which equips users with the ability to re-live bygone days, and the portability to relay them to others. MemoryLane operates across a Client/Server architecture on a bespoke local area network (LAN) as seen in Figure 4. The user’s client PDA stores the multimedia items and hosts the MemoryLane application. This application connects to a hosting server which provides system functionality through the public and private web methods of a web service. The server also hosts a back end database which stores user profiles and system information and the web service facilitates interrogation of this database. A speech engine also located on the server provides a TTS facility for the production of speech synthesis from string variables. This supports multimodal interaction in the utterances of on-screen prompts to assist the user if required and in the conveyance of stories. To further enhance multimodal user interaction MemoryLane will also incorporate ASR. MemoryLane may be deployed using the Internet for communication at a later phase. At this stage a LAN is required for user evaluations in areas which do not currently have Internet access. In accordance with the findings of the previously conducted pilot studies, the initial interface for MemoryLane is designed to be both intuitive and instinctive to the user while also being of visual appeal. The layout is consistent, deliberately plain, avoiding scroll bars or ambiguous clutter. The default colour scheme is of neutral tones. The interface has minimal screen objects at any one time and yet provides full functionality. The sizes of the screen objects (e.g. buttons or text) are decided by MemoryLane to suit the needs and preferences of the user. Iterative evaluations of MemoryLane with target user groups will improve and enhance this initial prototype.
The user is greeted with a welcome screen as shown in Figure 5(a). To log-in the user must select (press) their photo from a set of photos of six potential users. MemoryLane then establishes which user has logged on and immediately retrieves the stored profile for that user. The interface is then adjusted to reflect the profile details, tailoring it to the abilities and preferences of that user. The user’s image is displayed in the top left of the screen throughout the duration of their interaction and personalised messages are displayed. The logged on user proceeds to either view memories or edit their profile. A Help button is continuously available in the bottom right of the screen and explains any ambiguities with on-screen text, and speech if deemed applicable. The user can exit MemoryLane at any time via the Exit button in the bottom left of the screen. This button is replaced by a Go Back button positioned in the same location on all subsequent screens which provides the user with a means of navigating through MemoryLane and return to the Exit button to leave.
In Figure 5(b), we can see that ‘Nellie’ has logged and has chosen the Change Profile option. She is now presented with the choice of editing her preferences or abilities held in her recorded profile. The preferences option facilitates control over certain interface options such as choosing a preferred colour scheme or skin for the interface, or perhaps opting for the use of icons and symbols instead of text. The abilities option allows the user to change the level recorded for their hearing, vision, speech and dexterity, as previously discussed in section 4.1., the user can select from normal, reduced or very poor for each ability. Changing the level for an ability will instantly be reflected in the multimodal interface, e.g., increased or decreased font size, button size, volume levels or amounts of ASR and on-screen text read aloud to the user.
The user begins the reminiscence experience and is offered the option of viewing a previously seen and saved memory story from the stored album, or creating a new memory story using combinations of their stored multimedia, e.g., photographs, video clips, music, sounds, letters or poems. If the user chooses to view a memory from the album, they are presented with a selection of thumbnail images where each image represents a stored memory in the album. Selecting (pressing) an image causes it to be played in full. The new memory option allows the user to select a topic for the new memory story as seen in Figure 6(a) and previously discussed in section 4.2. The user can then view the ensuing memory story via the bespoke user interface as shown in Figure 6(b). Memory stories last anywhere between one and three minutes, during which the user has the options to pause, stop or replay the memory, and to also maximise the viewing screen if desired. The options to rate a memory story and save to the album are offered after each showing. An example memory story might simultaneously include background music, a series of rotating photographs and a voice over narration. (a) (b)
MemoryLane will learn from the user during interaction and record this information as part of the user’s profile. Should the user express dislike for a particular story then MemoryLane will learn to avoid this particular multimedia combination for future memory stories. Similarly if the user rates the memory story highly MemoryLane will learn that this is a popular combination of multimedia and increase further usage of these items in future memory stories. For example, a memory story may choose to exclude a particular topic which could cause discomfort, e.g., the mention of a recently deceased loved one. Similarly, MemoryLane could highlight positive events, a wedding perhaps and also devise historical stories based on known topics of interest to its user. In addition, if the user repeatedly requires help at the various stages of the interaction, MemoryLane will become proactive and will begin to automatically offer help in problem areas for that user. MemoryLane will also record how often favourite saved memory stories were retrieved from the user’s album and viewed again. As a user interacts with MemoryLane over a period of time its knowledge of that user will increase accordingly. MemoryLane can then offer more precise and accurate memory stories in a way that the user finds entertaining using interface components that the user finds easy to understand, navigate and control. The more the user interacts with MemoryLane the more it will learn from him/her.
This research introduces a hybrid method of decision-making specifically for a mobile platform applying AI techniques in the development of a multimodal PDA-based application called MemoryLane. MemoryLane accommodates user-specific abilities and preferences for multimodal input and output, and also performs fusion and synchronisation of life-cached multimedia for story generation. MemoryLane provides older adults with an intelligent, portable memory device to enhance their reminiscence experiences. The MemoryLane prototype is currently being implemented and will be enhanced and improved with iterative evaluations with target user groups. The final prototype will be rigorously tested with a fresh sample of end users. This will eliminate the possibility of any prior knowledge of, or familiarity with the system, and therefore give an accurate account of the intuitiveness of the interface and MemoryLane’s usability. This final testing will also use real data (i.e., the users’ own multimedia items) to effectively measure (a) the appeal of MemoryLane (its functionality and usability) and (b) the support that it offers to older adults whilst reminiscing. Some of the measurement instruments and tools previously mentioned in section 2.3 will be employed in this testing to measure and validate the success of MemoryLane. Detailed analysis of the ensuing results will follow.
The authors would like to extend their appreciation to the user study participants for contributing to the research. The research is supported by The Department for Employment and Learning (DEL).