Memorising and recalling information are integral components of our cognition. However, we often encounter everyday cognitive lapses that manifest as moments when the mind encounters temporary obstructions. They frequently involve the elusive sensation of having information "on the tip of one's tongue" yet being unable to retrieve it [1]. While modern technology has surpassed human capabilities in numerous tasks, the quest to develop technology that enhances human cognition performance remains an enduring challenge [2].
Psychology and cognitive science offer a rich repertoire of techniques proven to enhance cognitive performance, including the ability to store and retrieve information from memory. One method is the use of mnemonic devices, which entails establishing a connection between the information we try to memorise and a stimulus that is more likely to facilitate its retrieval. The link mnemonic method, which we will delve into later, is one such a strategy. However, many more mnemonic techniques exist such as the peg and loci method, the keyword mnemonic method, the major system among the most known, all of which have found practical applications. For instance, a recent study used the keyword method with immersive technologies for vocabulary acquisition [3].
Despite, in the course of an ordinary day, individuals grapple with various cognitive challenges, often unaware of the above mentioned strategies that could ease their cognitive burdens. Clinch and Mascolo [2] have explored domains where technology could intervene to offer assistance, highlighting the potential for leveraging evolving technologies to address these challenges effectively. According to them, the second most common semantic memory lapse is the inability to recall someone's name. This lapse can lead to self-embarrassment and inconvenience in various social situations. To the best of our knowledge, very few studies have addressed the problem of forgetting names and faces through an efficient digital tool. We hypothesise that such a tool could substantially aid individuals in mastering mnemonic systems. In this survey and position paper, we explore the digital and traditional methods for addressing everyday cognitive failures, elucidate the identified gap and propose a digital system designed to enhance semantic memory with regard to the retrieval of face-name associations.
The existing literature predominantly emphasises interventions for individuals with medical memory impairments [2,4]. However, it is also valuable to investigate cognitive processes in healthy populations, as cognitive lapses are pervasive across population [2].
Cognitive failures, as introduced by Broadbent et al. in 1982 [5], manifest as disruptions in intended actions, whether they involve mental or physical tasks, indicating a general vulnerability to lapses in cognitive control. The Cognitive Failures Questionnaire (CFQ) and other instruments have been developed to assess an individual's predisposition to cognitive failures, providing vital insights into the subjective experience of such lapses. The CFQ itself delves into errors stemming from perception, memory, and misdirected actions. Additional instruments are presented in the systematic review [6], including the Cognitive Slippage Scale [7] and the Prospective and Retrospective Memory Questionnaire [8] focusing on memory failures. Confidence in memory has also been assessed using self-reported diary studies as in [9,2].
Carrigan et al.'s systematic review on cognitive failures in the everyday lives of healthy population [6] furthermore examines the construct of cognitive failures as well as the factors impacting the frequency of cognitive lapses' occurrence. Broadbent et al. gave 3 aspects of the construct of cognitive failures (memory, perception, and action). However, later studies have revealed more internally-consistent aspects of the CFQ as memory, distractibility, blunders, and memory for names [10]. Finally, the systematic review categorises the possible influencing factors in two main groups: stable and variable (details visible in Table 1). According to the authors, stable factors have a stronger correlation with cognitive failures compared to the variable factors.
Chan's study [4] illuminates various memory failures, covering semantic, episodic, and procedural memory (retrospective) as well as prospective memory (future plans and intentions) [9]. These studies have highlighted the vulnerability of each of these memory systems to
Although it is frequently disregarded in research, this variable may have an impact.
breakdowns, frequently referred to as everyday instances of memory failure [4,9]. Prospective memory lapses, documented in two diary investigations [9,2], were the most prevalent type of memory failure. Common incidents included forgetting object locations or failing to complete tasks like sending emails or making purchases, as addressed in [4]. Semantic memory lapses were ranked as the second most frequent. In [2], the primary reported semantic memory lapse was "failure to recall someone's name", documented in a three-week study with 14 participants, resulting in 82 documented semantic failures. Terry's earlier diary study [11], involving 50 participants over several weeks, recorded 30 incidents of forgotten names. These studies indicate that remembering names is a common challenge for a lot of people and as such it could be addressed with technology.
Clinch's study [2] advocates addressing common memory failures with technological interventions, aligning with Chan's recent work [4]. Chan explores two main approaches: internal and external memory aids. Internal aids involve mental strategies, like mnemonics, which demand time and effort. External aids encompass tangible tools which are user-friendly but not always accessible.
A mnemonic device can be defined as a strategy for organising and/or encoding information through the creation and use of cognitive cuing structures [12]. The sole purpose of a mnemonic device is to enhance recall performance. They gained popularity in the 1970s after the general acceptability of cognitive processes as a legitimate area for research [13]. According to the classification from 1981 [12], they can be classified into 2 main groups: Peg type and Chain type. The sub-categories of the Peg Type are Method of Loci and Peg-word Mnemonic, while the sub-categories of the Chain type are Story Mnemonic and Link Mnemonic.
The Method of Loci involves memorising specific locations (loci) in a building (the memory palace). The loci are then associated with the items to be remembered as visual images. The recall is achieved by mentally traversing these locations of the palace [14]. The Peg-word technique uses concrete objects as anchors for remembered images, often employing schemes like rhyming or encoding numbers to remember both the peg words and their order ("one is a bun, two is a shoe, three is a tree"). Both the Method of Loci and Peg-word Mnemonic yield similar recall results and can be used for remembering key pieces of information of a talk, lists of items or tasks. They provide a cognitive cue structure permanently stored in memory for associating and later recalling information.
The Link Mnemonic offers an alternative to Peg-type mnemonics for remembering lists of items. It involves forming visual associations between each pair of consecutive words in the list, creating a chain of interconnected mental images. The story mnemonic involves incorporating each word in the list into a self-created story as they are presented. Variations of these methods also exist. In [1], Lorayne, a professional mnemonist, introduces a successful application of the Link Mnemonic, also known as the Link Method. One of the scenarios includes the recall of faces and names (Figure 1). The method involves several encoding phases: Memory augmentation systems use a wide range of approaches to support memory. Mainly they can be divided into training systems and assistance systems. The training systems are further divided into process-based and strategy-based. They stem from research in psychology that has developed and studied internal memory aids. The assistance systems are divided into reminder systems, life-logging systems and just-in-time systems [4]. They provide memory assistance to the user and have their roots in the idea of the "memory prosthesis".
Recent advancements in storage, machine learning, and wearable tech have sparked innovative ideas for enhancing human memory with assistance systems. While promising improved recall and freed cognitive resources, they also introduce a vulnerability: heavy reliance on these technologies may lead to dysfunction when it is absent. In the worst case, it can irreversibly alter cognition, resulting in poorer performance without it, potentially manifesting as memory loss or distortions in remembered events, skills, or knowledge [16]. Conversely, training systems not only avoid the risk of cognitive degradation but can also reduce age-related cognitive decline and lower the risk of cognitive diseases like Alzheimer's [17,4].
Digital strategy-based training systems make mnemonic strategies accessible on the-go and train users to apply these strategies. The Method of Loci has attracted the attention of quite a few researchers for creating a strategy-based training system as the Physical Loci [18], vMPeg [19] and NeverMind [20].
To our knowledge, there are no existing training system that instruct users in applying mnemonic strategies specifically to name and face recall. While various "memory prostheses" like Google Glass [21], Vimes [22], Pal [23], and "Haven't We Met Before" [24] assist with face-name recall, they do not try to enhance human memory. Additionally, systems using the Loci method train semantic memory for facts, but the Link method is more suitable for encoding names [1]. Given that forgetting names and faces is a common semantic failure [2], addressing this directly is a promising research direction, especially since the literature shows positive results. For example, Lorayne's face-name mnemonic, assessed in [25], significantly outperformed a control group when explained to students. In a recent evaluation by [26], the simplified face-name mnemonic surpassed three other groups in a between-subject design. Implementing a digital training system that would enhance the verbal-only instructions by showing the actual imagery projected upon the face of the individual should therefore enhance memorisation of names. Unlike traditional verbal instructions, a technology system can simu-late the application of the method in real-life situations, providing a more effective learning experience [26]. Further, we believe that our method will provide better results since recent research shows that showing external images enhances short and long-term memory [3].
Recognising the well-established advantages associated with strategic training systems [4,16], we have identified the following gap in the current body of literature:
Currently, there are no strategy-based digital training systems designed to enhance semantic memory with regard to the retrieval of face-name associations.
However, the design space of a technologically supported training system is vast. Based on the idea from Figure 1 as well as the future directions proposed in [26] we propose to build a desktop training system based on the face-mnemonic strategy. Our first step towards designing the training system, is to implement a system that will augment faces on conversational videos with an overlaying animation providing interaction of the keyword with the (prominent feature of the) face (Figure 2d). Our key inquiry involves assessing the efficacy of such augmented videos in improving recall and the feasibility of implementing them. If the outcomes will be positive, our focus will shift towards developing the training system, which will integrate the augmenting system method and will train the users how to apply the mnemonic strategy independently.
To achieve this, we propose to build a desktop system to support learners in remembering names. The system will have 4 different modes to augment the faces in real world videos with digital content (not in real time). Each mode will incorporate a different set of instructions as seen in Table 2 for remembering the presented names based on the simplified face-name mnemonic method [26]. The name is already transformed into an imaginable object.
Imagine the object interacting with the face.
The name is already transformed into an imaginable object and shown on the video. Observe its interaction with the face.
To our knowledge, memory training systems focusing on the Link method for name and face recall do not exist. Existing solutions, including AI-driven systems and wearables, often foster user dependency without strengthening human memory. To address this gap we propose to build a digital system designed to explore the feasibility to enhance semantic memory with regard to the retrieval of face-name associations. With this system we plan to conduct a user study to evaluate the proposed mnemonic training effectiveness. Finally, our aim is to build a memory enhancement training system, which would train users to use mnemonics on their own.
This research was funded by the Slovenian Research Agency, grant number P1-0383, P5-0433, IO-0035, J5-50155 and J7-50096. This work has also been supported by the research program CogniCom (0013103) at the University of Primorska.