Interactive systems are firmly becoming fundamental in daily life, providing users with tools to enhance their autonomy, communication, self-organization, and overall well-being. Due to their helpful impact on users' lives, interactive systems are increasingly being applied as supportive products for individuals with mild intellectual impairments. Thus, delivering accessible and intuitive interactive solutions tailored to the needs of these individuals is a way to support their autonomy.
Nowadays, intellectual impairments are characterized by significant limitations in intellectual
functioning and adaptive behavior. These limitations afect an individual’s ability to learn, reason, and solve
daily life dificulties. Such impairments can range from mild to severe and typically manifest before
adulthood, impacting several skills such as communication, self-care, and social relations. Individuals
with mild intellectual impairments often require some level of support to carry out daily tasks [
In Spain, intellectual impairments afect a significant part of the population, which requires extensive
support procedures. In 2022, approximately 28,684 people had a recognized degree of intellectual
impairment [
CEUR
ceur-ws.org
For instance, Plena Inclusión has launched initiatives assisting more than 100,000 families across the
country [
Besides these supportive initiatives, technology could also ofer additional opportunities to enhance the independence of individuals with intellectual impairments. In particular, interactive systems such as mobile applications have become powerful tools for organizing daily routines, navigating environments, and managing personal tasks. These widely used technologies could empower individuals with mild intellectual disabilities by fostering autonomy and also provide caregivers with specific tools to monitor, guide, and assist individuals more efectively.
Based on these considerations, this paper presents the findings from the final stage of a design
project carried out to develop a supportive mobile application for individuals with mild intellectual
impairments and their caregivers: Eleuteria App. This paper specifically examines the final Deliver
stage of our design project, which was structured according to the Double Diamond Model [
The rest of the paper is organised as follows: Section 2 presents related work followed by Section 3 which details the structured methodology we propose to carry out the Deliver stages. Section 4 describes Eleuteria App, while Section 5 discusses the process done for the Deliver stage. Finally, conclusions and future work are presented in Section 6.
Various studies have explored the importance of user testing and heuristic evaluations when developing mobile applications for individuals with accessibility impairments. In these cases, user testing involves working directly with people who have impairments to see how they use the apps and what challenges they face. Expert evaluations such as heuristic evaluation done by designers represent, on the other hand, a significant tool to check the apps’ usability and accessibility before the final implementation.
Regarding user testing, Benson-Goldberg et al. [
Recent studies also focused on evaluating supportive technologies adopting expert-driven usability and heuristic evaluations to refine app interfaces. The aim is to detect interface and user flow issues to be solved before delivering supportive products to individuals with impairments. For instance, the FIND (Friendly Indoor Navigation Device) app underwent iterative expert evaluations focusing on users with visual, hearing, cognitive, and mobility impairments [13]. These evaluations demonstrated the value of combining heuristic feedback with inclusive design principles to produce a more adaptable and userfriendly interface. Moreno et al. [14] introduced a specialized heuristic evaluation instrument tailored to check the accessibility of video conferencing platforms. To apply this instrument, an accessibility evaluation was carried out by six experts across four popular platforms: Zoom, Google Meet, MS Teams, and Cisco Webex. Alongside the general expert assessments, a blind user performed an additional review specifically focused on screen reader compatibility. This dual evaluation method provided insights into the platforms’ accessibility from both a general and a specific assistive technology perspective. In addition, the findings revealed that although each platform met certain accessibility requirements, none achieved full accessibility, highlighting considerable room for improvement. Furthermore, Guasch et al. [15] introduced the Easy Communicator (ECO) application, a mobile Augmentative and Alternative Communication (AAC) [16] tool designed to assist individuals with complex communication needs. The authors aimed to develop ECO using a User-Centered Design (UCD) [17] approach to ensure usability and meet essential accessibility requirements for both end users and professionals, such as educators and therapists. The evaluation focused on the Communication Manager module of ECO, with a formative evaluation conducted by experts to assess usability and accessibility. A heuristic questionnaire, based on a checklist of 16 accessibility checkpoints, was used to validate the tool. These checkpoints, organized into six categories, were evaluated by experts using a scale of “Yes,” “No,” “Partial,” and “NA” to assess compliance with accessibility standards. The findings showed positive results, indicating strong perceived usability and that the application met key accessibility requirements. Expert feedback also provided valuable suggestions for refining the tool’s features and improving its accessibility. These insights, along with a forthcoming summative evaluation involving real users, will inform the next version of the ECO application.
Although prior studies have established that user testing and heuristic evaluation serve complementary roles (e.g., to capture real user experiences and to identify usability issues through expert analysis), this work goes beyond by focusing specifically on how these methods can be efectively integrated to address the specific needs of individuals with intellectual impairments. Unlike previous approaches that treat these methods in isolation, our study demonstrates how their combined application leads to more inclusive and context-sensitive mobile applications that both meet accessibility guidelines and align with the real life of individuals with mild intellectual impairments.
The methodology of this paper is based on the Design Thinking framework [18]. Design Thinking is a
user-centered framework focused on problem-solving through a visual and creative approach, ensuring
that solutions align with users’ needs and deliver meaningful experiences. Design Thinking processes
often incorporate the Double Diamond Model [
The first diamond of the Double Diamond Model is dedicated to identifying and understanding user challenges while exploring possible solutions. It falls within the problem space and consists of two key stages: Discover and Define. The Discover stage prioritises user research, aiming to gain deep insights into their needs, obstacles, and experiences. Then, in the Define stage, the collected data is analysed to clearly articulate the core problem and establish the best strategy for addressing it.
Conversely, the second diamond shifts focus to developing and implementing a tangible solution, placing it within the solution space. This phase includes the Develop and Deliver stages. The Develop stage involves designing and refining an interactive prototype with functionalities tailored to user requirements. Finally, the Deliver stage is dedicated to testing the prototype with users, gathering feedback, and fine-tuning its design and functionality before its first release.
This paper explores the final stage of a design process guided by the Double Diamond model, aimed
at delivering Eleuteria App, a supportive system designed to enhance the autonomy and independence
of individuals with mild intellectual disabilities and support the work of their caregivers. The process
began with the Discover stage, where the specific needs and challenges of a real group of users with
intellectual impairments were identified [
Building on the prior stages, this paper focuses on presenting a structured methodology for the Deliver stage of a supportive system tailored to individuals with mild intellectual impairments and their caregivers. Leveraging principles from Design Thinking framework and the Double Diamond Model, we outline a four-step methodology, detailed in Table 1. This methodology integrates user testing with heuristic evaluation to guide the final development of accessible and supportive mobile applications, ensuring that they are both technically robust and meaningfully inclusive for users with impairments.
The name Eleuteria, derived from the Ancient Greek word for “freedom” and “autonomy”, perfectly matches the app’s mission: to empower individuals with mild intellectual impairments by promoting their independence and self-suficiency. The Eleuteria App (see Figure 2) is designed as a virtual assistant that ofers dual configurations: one tailored for individuals with mild intellectual impairments 2 and another for their caregivers3.
As a supportive virtual friend, Eleuteria App provides individuals with a comprehensive set of tools to enhance daily life planning and foster greater autonomy. Simultaneously, it ofers caregivers an integrated suite of features to organise their job responsibilities, monitor the progress of those under their supervision, and maintain a systematic follow-up. Developed under a Spanish initiative, the app’s 2An interactive prototype of Eleuteria App for individuals with mild intellectual impairments can be accessed here. 3An interactive prototype of Eleuteria App for caregivers can be accessed here.
Assess in- Definition of a tasks list to evaluate the prototype: De- Test the prodividuals’ fine a list with all the tasks required for individuals with totype with interactions mild intellectual impairments to carry out while testing the individuals to with the app prototype. identify areas to identify Single testing sessions with individuals: Recruit a group for improveenhance- of individuals and conduct individual testing sessions, ask- ment in its ments for ing each user to interact with the prototype following the usability and design and Task List and apply the Thinking-Aloud Protocol. After accessibility. workflow each session, gather feedback and have the user complete optimization. the SUS (System Usability Scale) questionnaire.
Iterative adjustment of the prototype: Between app testing sessions, make adjustments to the prototype to address any interaction issues identified.
SUS Questionnaire score analysis: Analyse the SUS to obtain the score, which will provide valuable insights into the overall usability of the app.
Adjust the final prototype of the app.
Adjust the prototype of the app.
Prototype modification: Review the feedback from the testers to ensure all issues have been addressed and determine if further modifications are needed based on their suggestions.
for the evaluator to assess during the heuristic evaluation.
Heuristic evaluation [21] by product designers: Recruit a group of professional product designers and ask them to review the prototype following the Heuristic Checklist to verify the ease of navigation, consistency, and simplicity of the prototype.
Definition of list of issues: Define a list of critical issues to resolve before the first release of the system.
Modify the interactive prototype in Figma.
Review the
prototype
before the
first release.
oficial language is Spanish. Thus, based on previous research [
GIFs to assist those who may struggle with reading or writing. • Mis rutas (“My maps”): A location-based feature that helps users find friends (Figure 2 (b)), favorite locations, and navigate safely. It also serves as a tool for caregivers to track the user’s whereabouts when needed. • Voy a ser independiente (“I’m going to be independent”): A tool that helps individuals organise and complete tasks, incorporating a gamified approach to encourage greater independence and goal achievement. Caregivers can post tasks (Figure 2 (f)) and assign them to the individual, and individuals earn points for completing them, fostering autonomy. Over time, these points contribute to building independence in daily activities like navigating the city or living on their own. • Voy a trabajar (“I’m going to work”): A feature that assists individuals in finding job opportunities suited to their skills and interests. It provides guidance for applying and preparing for work, allowing users to update their job profiles, apply for relevant positions, and upload their CVs. Caregivers can ofer feedback on the user’s profile and post new job listings. • Mi hucha (“My piggy bank”): A financial tool designed to help individuals monitor their savings and set financial goals. In addition, “My piggy bank” includes a feature called “What can I buy?” (Figure 2 (c)) which helps individuals understand what they can aford with specific amounts of money (e.g., 10€, 20€, 100€, 1.000€). This feature is designed to support financial literacy by ofering concrete examples of potential purchases corresponding to each amount. • Mejor relajarse (“Better to relax”): A mindfulness tool designed to assist individuals in relaxing and managing stress through guided exercises, breathing techniques, and calming activities. Caregivers can also monitor usage to ensure it’s not being overused and to keep track of the individual’s stress levels or mental well-being. • Perfil/Ajustes (“Profile settings”) : This feature allows users to personalize and manage different aspects of the app’s functionality to suit their preferences. It includes options to modify notifications, change the login password, and log out of the app.
During Step 1 and Step 2 of the Deliver stage, the two distinct configurations of the application,
the one for individuals with mild intellectual impairments and the other for their caregivers, were
tested. This dual testing approach enabled an evaluation of user interfaces and accessibility features for
individuals, as well as tools for organising care and monitoring progress for caregivers. In addition, to
enhance user engagement, each configuration was paired with an avatar: Peter Parker representing the
individuals and Harley Quinn representing the caregivers. As highlighted in the previous stage [
This section of the paper details the four steps carried out during the Deliver Stage of our design process, following the methodology outlined in Table 1. The process began with two user testing steps, during which caregivers and individuals with intellectual impairments interacted with their own configuration of Eleuteria App. Their feedback was incorporated into a third step focused on adjusting the prototype. Hence, during the final step, the prototype underwent a heuristic evaluation by product designers. Each step of the process is thoroughly described in the following subsections. In addition, the final two subsections highlight the lessons learned during the Deliver stage and the ethical considerations applied in our study respectively.
This subsection outlines the testing performed with the prototype by a group of six caregivers. The evaluation aimed to assess the usability and ease of interaction of the caregiver configuration of the Eleuteria App. Participants interacted with this version of the app and then completed a 10-question System Usability Scale (SUS) questionnaire to measure overall usability and user satisfaction.
Before the testing, we defined a list of tasks which caregivers had to complete during the interaction
with Eleuteria App. The Task List (Table 2), used during the test, was based on the list applied in the
tests done in the Develop Stage [
Access the chat from the home page and check the conversation with MJ and Peter Parker, then return to the home page.
On the home page, access the notifications and from there: assess the task completed by Peter Parker and assign the task to Ned Leed. Return to the home page.
On the home page, access job features to create a new job ofer. Return to the home page.
On the home page, access the job profile of Peter, MJ, and Ned Leed, and from there: view the information on their CV and make a new recommendation. Return to the home page. On the home page, access the maps and locate the individuals.
On the home page, access the user profile to change the password and log out.
The application was evaluated by six professional caregivers employed as educators, instructors, and social integrators at Fundación AMÁS4 in Leganés (Madrid). Each caregiver was instructed to interact with the app using a predefined task list (see Table 2). The testing sessions were conducted online (via Google Meet) by an evaluator who explained the testing procedure verbally and provided the caregiver with the links to access the task list, the interactive prototype of the Eleuteria App configured for caregivers, and the SUS questionnaire form (to be filled out at the end of each session).
The testing sessions were conducted online using Google Meet. Tasks were explained verbally to the participants, and supporting links or materials were shared through the chat feature.
The demographic data of the professionals involved in the evaluation are presented in Table 3. Along with the demographic data, we provide details on their job duties and the profiles of the individuals they supervise. This contextual information is crucial for understanding how their professional roles and daily interactions shape their use of a supportive system as Eleuteria App, and the importance of its features to their caregiving tasks. Additionally, we gathered information about the caregiver’s experience with using smartphones, as this is necessary for the aims of our design project. The Eleuteria App is specifically developed to assist individuals with mild intellectual disabilities in organizing daily activities and enhancing their independence. Gaining insight into each caregiver’s background allows us to more efectively assess how well the app fits their context and ensures that its functionalities are aligned with both the users’ needs and the caregiver’s supportive role.
The testing was carried out in individual sessions over four separate days. Participants C1, C2, and C3 took part on the first day, followed by C4 on the second day, C5 on the third, and C6 on the fourth and final day.
Caregivers interacted with the Eleuteria App using the Thinking-Aloud Protocol, a method where participants express their reflections, decisions, and actions aloud while performing a task. This helps researchers understand their thinking, identify any challenges, and spot usability issues. The thinkaloud comments were recorded by the evaluator, who took detailed notes of the caregiver’s reactions and the sentences they articulated during the interaction. In spite of the remote nature of the study, the
She supports individuals aged She uses smartphone and between 19 and 65 who need tablet for her job duties and medium assistance, some of them her smartphone for personal have mobility and hearing impair- purposes. ments too. She uses smartphone, tables and PC almost every day.
ID
Caregiver and He supports individuals aged Workshop In- between 19 and 60 who need structor medium assistance, and with language impairments too.
She often uses her smartphone for communication but she prefers not to use PC or tablet.
He often uses his smartphone and PC.
Caregiver and Social integrator
She supports individuals aged She uses every day PC for between 19 and 60 who need her job duties and her smartmedium assistance, and with dif- phones. ferent kinds of mobility impairments too.
Caregiver and She supports individuals aged be- She uses quite often the Workshop Re- tween 19 and 60, most of them are smartphone for communicasponsible able to complete some daily tasks tion, but she prefers not to use
independently. PC. connection remained stable throughout, and there was no impact on the observation or interpretation of the caregiver’s behavior.
The data obtained through the thinking-aloud technique were analyzed using a qualitative approach. The evaluator reviewed the notes and audio recordings to identify recurring patterns and insights in the caregivers’ verbalizations. While a formal thematic analysis was not conducted, the results were summarized qualitatively, focusing on key observations related to usability issues, emotional reactions, and interaction strategies. This approach allowed for an precise understanding of user experience without applying a rigid framework. This approach was even applied during the testing session with individuals.
During the test, the six caregivers voiced their reflections, actions, and reactions as they used the app, giving us valuable insights into how the app worked for them and whether it met their needs. For instance, C1, when trying to find all the profiles of individuals under her supervision in the home page (Figure 2 (d)), couldn’t locate all the content of the carrousel (user must swipe on the left to visualise all the information) and explicitly mentioned having dificulties. C2, on the other hand, expressed dificulty in finding the notifications. Besides, C6 struggled with the grey icons on the action bar, thinking they were deactivated (she knew grey in interfaces is typically associated with disabled features).
At the end of the testing and once they fulfilled the SUS questionnaire form, caregivers also shared their opinions on Eleuteria App. Participants C1, C2, and C5 expressed satisfaction with the app, commenting that it appeared very useful and had the potential to improve their work. All of them found its features valuable and suitable to their daily tasks. C3 mentioned that, although she is not currently confident using smartphones, she would be willing to learn to incorporate the Eleuteria App into her work. C4 raised concerns regarding how some of his colleagues might interact with the app. While he, being younger and tech-used, was keen to use it, he stated that older colleagues might struggle with the technology or be less willing to adopt it. Besides, C6 ofered a more slight perspective, stating: “Although I generally struggle with technology, I found the app to be practical and intuitive in some aspects, though others were more challenging for me. For instance, I had dificulty locating the chat icon on the Action Bar, but I recognize that this may be due to my limited experience.”.
As the app was tested over four separate days, feedback from participants C1, C2, and C3 was reviewed and addressed at the end of the first day. The resulting updates were implemented and evaluated with C4 on the second day. Further refinements were made based on C4’s input and tested with C5 on the third day, followed by a final round of evaluation with C6 on the fourth day.
During the first day of testing, we observed that C1 had dificulty identifying how to swipe through the carousel of supervised individuals. In response, we redesigned the carousel by removing the top arrow (Figure 3 (a)) and adding a colored background to indicate that additional content followed (an application of the Gestalt principle of continuity5 (Figure 3 (b)). We also modified the notification icon after C3 struggled to locate it. The small red dot was (Figure 4 (a)) replaced with a larger red number to improve visibility and clarity (Figure 4 (b)). These changes were implemented before the second day of testing. During subsequent sessions, caregivers experienced no issues accessing the list of supervised users or the notifications, suggesting the adjustments were efective. 5The principle of continuity in Gestalt Psychology suggests that elements aligned along a continuous path are perceived as part of a whole. In a carousel, this principle applies when content extends beyond the visible screen, encouraging users to interact and explore more by scrolling or swiping, creating a sense of ongoing sequence and prompting further engagement.
Finally, on the last day of testing, based on C6’s feedback about the gray icons on the action bar, which were perceived as inactive or disabled, we chose to shelve this visual adjustment to Step 3, when the prototype was refined prior to heuristic evaluation.
The app tested by caregivers was evaluated with an average System Usability Scale (SUS) score of 72 (SD = 13.4), indicating that the overall usability of the application is above average. The SUS is a well-known tool used to measure how user-friendly an interactive system is, with scores going from 0 to 100. Since the average score is around 68, getting 72 shows that the caregivers found the app clear, helpful, and simple to use.
This result shows that the app mostly matches what caregivers need and expect. The score of 72 indicates that the app has been well received by caregivers, providing a strong foundation for continued development and future improvements. 5.2. Step 2: Testing the prototype with individuals with mild intellectual impairments This session describes the prototype testing carried out with a group of five individuals with intellectual impairments, using a process similar to the one performed with caregivers. Thus, the primary objective of this second step remained to evaluate the prototype’s usability and ease of interaction. However, in this case, individuals explored the Eleuteria configurations tailored specifically to their needs. Once completed their interaction with the app, they were asked to answer the SUS questionnaire to assess its overall user experience.
Before the testing, as we did in Step 1, a task list (Table 4) was prepared based on the list used in
the Develop Stage [
As in Step 1, the aim in this case was to simulate real-world scenarios for individuals with intellectual impairments, to evaluate how easily they could navigate the app and complete their tasks. This approach provided a realistic assessment of how intuitively users from each group could interact with the app’s interface and perform their assigned activities. By replicating authentic use cases, the testing aimed to identify potential usability issues and evaluate the app’s overall accessibility. The focus was not only on ease of navigation but also on how efectively the app supported users in achieving their tasks, ensuring that it met their specific needs and expectations in practical, everyday contexts. 5.2.2. Single testing sessions with individuals with intellectual impairments The application was evaluated by a group of five individuals with intellectual impairments who regularly participate in various activities organised by the Fundación AMÁS, such as washing, ceramics, and screen printing workshops. Each participant was asked to test the prototype by following a specific set of tasks outlined in the task list (see Table 4). The testing sessions were conducted online by an evaluator in a Google Meet call; therefore, participants were instructed verbally by the evaluator and provided with links to access the task lists, interactive prototypes, and the System Usability Scale (SUS) questionnaire, which they completed at the end of the testing.
The demographic data of the individuals involved in the evaluation are presented in Table 5. In addition, the same table provides details about the life of each participant (e.g., his intellectual impairment’s level and his personal information such as whether he lives or travels independently) and his familiarity with smartphone use. These data are necessary to be collected, given the purpose of our design project, particularly in relation to the Eleuteria App which is specifically designed to support individuals with mild intellectual impairments in managing daily tasks and fostering independence. Understanding Access the landing of the app by consulting the descriptive screens, then access the home page of the app and read the descriptions of the five main features of Eleuteria.
Access the chat from the home page and check the conversation with MJ and Harley Quinn, then return to the home page.
Access the “I’m going to be independent!” functionality, consult the list of tasks and access the “Shopping” task details to request it. Check that it is assigned to you and return to the home page.
Access the “My maps” functionality, take the routes to go to your friend Ned Leed and to your favourite place: Leganés Central Station. Return to the home page.
Access “My piggy bank” to check your income, make a new deposit, and create a goal. Return to “My piggy bank” initial screen to access the “What do I buy?” and check what can be purchased with diferent amounts of money. Return to the home page.
Access the “I’m going to work!” functionality to consult the job “Gardening” and apply to this job ofer then access your job profile to edit the information and load a new CV. Return to the home page.
Access the “Better to relax” functionality and choose “I want some time for myself” and start the meditation.
Check the notification from the home page.
On the home page, access the user profile to change the password and log out. each user’s context allows us to evaluate the app’s applicability better and ensure that its features are appropriately tailored to their needs and abilities.
The testing was conducted over four days. I1 took part on the first day, I2 on the second day, followed by I3 and I4 on the third day. Finally, I5 completed the testing on the last day.
Like the caregivers, individuals with mild intellectual impairments interacted with the app using the Thinking-Aloud technique. The evaluator documented the comments by taking notes of the individuals’ reactions and spoken remarks during the interaction. The connection was consistently stable during all the online sessions, allowing for straightforward observation and interpretation of individuals’ behavior without any disruptions.
For instance, I1 asked where the notifications could be accessed, as she had not noticed the icon located in the top-right corner of the screen (as happened to C1 in Step 1). I2 commented that the “I’m going to be independent” functionality is very interesting. However, what stood out most was her enthusiastic reaction while exploring the “What can I buy?” feature within “My piggy back” feature. Upon discovering that a guitar could be purchased for €50, she exclaimed: “I didn’t know I could buy a guitar for 50 euros! ”.
I3, on the other hand, highlighted the “Relax” functionality as particularly beneficial, mentioning its applicability to his personal experiences with managing anger. He shared: “When I get angry, I hit the trash bins on the street to avoid arguing with people. This relaxing tool of Eleuteria App would help me to not act like that. If I did that at the foundation, they’d probably kick me out”. He also stated that some texts on the app (the functionalities description) were too small for him, which made them dificult for him to read and understand. However, he also stated that the feature “My Maps” of Eleuteria in his opinion is better than the Moovit App especially for locating his friends.
During interaction with the app, I4 said aloud: “I like using Eleuteria because it’s really fun”. Even I5 expressed enthusiasm for the app saying: “How cool! It’s really well explained! The money section is very useful because it can help me a lot. During this test, I’ve learned a lot about this kind of technology that I didn’t know before, and yes, it could really help me. I would like to use it! ”.
ID Age Afected by mild intellectual impairments, he lives alone but needs support with shopping, food management, and hygiene supervision. He takes part in the washing workshop in the foundation. He has a smartphone, a computer and a tablet; he is used to access his Instagram account every day on his smartphone.
She has mild intellectual impairments and She uses just her smartphone to access her epilepsy. Presents dificulties using public social network accounts (e.g., Facebook, transport alone. She takes part in the ce- WhatsApp, and Instagram). ramics workshop in the foundation.
He is afected by mild intellectual disabil- He daily uses his smartphone, computer, ities. He lives alone, but he has issues and tablets but he mostly uses his smartmanaging money, food, and cleaning. In phone, especially for messaging his famthe foundation, he takes part in the wash- ily and friends and he uses the app ing workshop. Moovit (public transport app) when mov
ing around the city.
Afected by moderate intellectual impairments, she has dificulty managing money, using public transport alone, and food management. She takes part in the gardening workshop in the foundation.
She is afected by mild intellectual impairments causing her dificulty managing money, food, and using public transport alone. In the foundation, she takes part in the screen printing workshop.
She only has a smartphone but she does not have social networks (e.g., Facebook or Instagram). However, she accesses Spotify, YouTube and Gmail almost daily.
She often uses the computer to listen to music and play video games. In addition, she uses her smartphone to access her social network accounts (e.g., Facebook, Instagram, and X).
At the end of the user testing sessions, individuals with intellectual disabilities were asked to complete the usability questionnaire. Unlike the caregivers, however, a diferent protocol was adopted for this group to better accommodate their needs and ensure accessibility. Specifically, the evaluator conducted the questionnaire as an interview rather than providing it in written form. More specifically, to better accommodate individuals with mild intellectual impairments, the evaluator adapted the standard SUS (System Usability Scale) protocol during the usability evaluation. Instead of reading the SUS questions exactly as written, the evaluator rephrased each item using more accessible and specific language, and contextualized the questions specifically to the Eleuteria app. For example, for the original SUS item “I think that I would like to use this system frequently”, the evaluator asked: “Would you like to use the Eleuteria app in your daily life?” If the participant responded positively, the evaluator followed up by asking them to rate their response on a scale from 1 to 5, where 1 meant “not at all” and 5 meant “very much”. This approach made the questions more relatable and easier to understand, as referring directly to Eleuteria (rather than generically to “the system”) helped participants connect the questions to their experience. In this way, the SUS protocol was adapted to better suit the cognitive needs of the user group, ensuring the validity and relevance of the responses. The evaluator recorded the responses on behalf of the participants. This adapted protocol proved efective, as it allowed individuals to fully understand the questions and express their opinions clearly. As a result, their feedback on the usability of the Eleuteria App could be reliably collected and considered in the evaluation.
At the end of the first day of testing, we decided to modify the notification icon to enhance its visibility. This decision was based on feedback form I1 who had dificulty locating it in its original form. To address this, we replaced the small red dot with a larger red number, making notifications more prominent and easier to notice (as we did in Step 1 for caregivers’ configuration (Figure 4)).
Additionally, in response to I3’s comment about the small text size, we decided not address this feedback and we did not change text size in the prototype of the app. This decision came because Eleuteria App was designed in accordance with the WCAG 2.1 (Success Criterion 1.4.4: Resize Text) [22] which ensure readability and accessibility for a wide range of users. Additionally, the Eleuteria app for individuals includes a screen reader to assist users with visual impairments, especially on screens that contain a large amount of information (Figure 2 (b)). Hence, following the modifications made after the first day of testing, we did not observe any interaction issues related to the shape or visibility of the notification icon among the other participants. I2, I3, I4, and I5 were able to locate and understand the notification feature without dificulty, suggesting that the updated design successfully addressed the concerns raised earlier.
The app tested by individuals with intellectual impairments received an average System Usability Scale (SUS) score of 72.5 (SD = 14.5), indicating that the overall usability of the application is above the average value of 68. This result shows that the app’s design and interaction flow generally meet the needs and expectations of its target users. While the score indicates good user satisfaction, it also suggests that there may be areas for improvement, such as refining interface elements or improving accessibility to better support end users. Overall, a SUS score of 72.5 confirms that the app is well-received and provides a strong basis for further development.
Based on the user testing conducted by caregivers and individuals with mild intellectual impairments in Step 1 and Step 2, respectively, we proceeded to Step 3 of the methodology to incorporate the feedback from the testers and update the Eleuteria app prototype.
At the conclusion of Step 1, we decided to address C6’s feedback regarding the action bar. C6 noted that the grey colour of the unselected icons led to confusion (Figure 5 (a)), making them appear as if they were disabled. In response, we modified the design in Step 3 by changing the style of the unselected icons. Instead of using a solid colour, we introduced an outline around the unselected icons, while the selected icons remained fully coloured, making the options clearer and more intuitive (Figure 5 (b)).
The action bar was modified in both configurations—those for individuals 6 and for caregivers7. Following the testing in Step 2, no further changes were deemed necessary on the individual configuration, and thus, this third step of the Deliver Stage was concluded.
Heuristic evaluation is a usability inspection method where experts assess an application or system against a set of established usability principles, commonly known as heuristics. This method was 6Updated prototype for individuals with mild intellectual impairments can be accessed here. 7Updated prototype for caregivers can be accessed here. popularized by Jakob Nielsen and his colleagues, who outlined a set of 10 principles [21] that serve as a benchmark for evaluating user interfaces.
Based on Nielsen’s 10 usability heuristics, we developed a checklist to conduct a heuristic evaluation of the Eleuteria App and recruited three product designers for reviewing the prototype. Their evaluation produced a list of issues that must be resolved before the first release of Eleuteria App. Each part of the work done in this step of the process is detailed in the following subsections.
The Heuristic Checklist (Table 6) outlined in this section is based on Jakob Nielsen’s 10 Usability Heuristics, a widely recognized framework for evaluating the usability of digital interfaces. The 10 Nielsen’s usability heuristics focus on fundamental aspects of user interaction, from the visibility of system status to help and documentation, helping experts to ensure that the interactive systems they are evaluating are intuitive, eficient, and user-friendly for their target audiences.
For each of Nielsen’s usability heuristics, we have defined two or more targeted questions that product designers should consider during their evaluation. This checklist is particularly valuable because it provides a structured approach to identifying usability issues, ensuring that all critical aspects of the app’s user interface are thoroughly assessed. By addressing specific questions related to each heuristic, experts can systematically evaluate the app’s functionality, ease of use, and overall user experience. Moreover, the checklist helps to ensure consistency in the evaluation process, allowing for a more comprehensive and objective analysis of the Eleuteria app’s design. This process ultimately supports the refinement of the app, aligning it more closely with best practices for usability and enhancing its efectiveness for users with mild intellectual impairments and their caregivers.
To conduct the heuristic evaluation, we recruited three professional product designers (see Table 7) and invited them to interact with the Eleuteria App prototype. The evaluation was carried out remotely; therefore, the designers were provided with links to access the prototype of Eleuteria App updated in Step 3 (they received two links: one to the prototype for individuals with intellectual impairments and another for the equivalent for their caregivers).
Each product designer tested the app independently. Upon completing their evaluation, they were asked to provide structured feedback using the heuristic checklist we developed. Following their interaction with the app, all designers agreed that Eleuteria has a meaningful impact on its target users and complies with most of Nielsen’s heuristic principles (specifically H1 through H7 and H9). However, they all also pointed out that several screens contained an excessive amount of informative text, which could overwhelm or bore users—indicating a lack of compliance with H8, particularly question 8.1. Additionally, they reported the absence of personalization options within the app, which could limit user engagement and adaptability.
Based on this feedback, we compiled a list of usability issues that should be addressed before the first release of Eleuteria App. The list is outlined in the following subsection.
Based on the heuristic evaluation conducted by the three product designers, two main issues emerged that could impact user engagement with Eleuteria App. Therefore, we create a table detailing the issues designers found and possible solution for them (Table 8).
Based on the feedback gathered during the heuristic evaluation conducted by the three product designers, we identified two key areas where the application can be improved. First, some screens—particularly in the onboarding flow—contain a large amount of text, which may be challenging for some users. However, for the first release of the application, these screens will include a screen reader feature, ensuring that the content remains accessible. In the future, we could explore presenting this information
ID 1.1: Does the app keep users informed about the status of the actions they are performing? (e.g., loading indicators, action confirmations) 1.2: Is there clear feedback for all users after performing a task, such as confirming a change, or completing a task? Match Be- 2.1: Does the app use simple and familiar language for users and caregivers? For tween System example, does the app avoid complex technical terms? and the Real 2.2: Are actions and instructions aligned with the daily context of users with intellecWorld tual disabilities and their caregivers? 2.3: Are visual metaphors and common symbols used that are also understandable for individuals with intellectual disabilities? User Control and Freedom 3.1: Can all users easily undo their actions, especially if an accidental error is made? 3.2: Is it easy for all users to navigate backwards or cancel tasks when necessary? Consistency 4.1: Is the app consistent in its design? For example, does it use the same terms, and Stan- buttons, and layout across all screens? dards 4.2: Are accessibility standards followed to ensure the app is easy to understand and use for people with intellectual disabilities and their caregivers? (e.g., high contrast colours, large text)? Error Preven- 5.1: Is the app designed to prevent potential user errors? (e.g., eliminating options tion that could cause problems) 5.2: Are interactions clear and easy to follow, minimising the possibility of all users making mistakes? 5.3: Are clear warnings provided before performing critical actions? Recognition Rather Than Recall Flexibility and Eficiency of Use 6.1: Does the app use clear icons and labels so users don’t have to remember information between screens? 6.2: Are important instructions or steps always visible or accessible to avoid users having to remember details? 7.1: Does the app ofer options to personalise the experience, such as changing text size, toggling accessibility features, or setting shortcuts? 7.2: Are there alternative interaction methods (e.g., voice commands, gesture control, hotkeys) to facilitate use by users and caregivers? Error Preven- 8.1: Does the app have a clean and clear design, without unnecessary distractions or tion visual elements that could overwhelm the user or caregiver? 8.2: Do the screens contain only the essential information to avoid overwhelming the user with excessive options or messages? 8.3: Is the layout of elements consistent and does not create visual confusion? Help Users Recognize, Diagnose, and Recover from Errors 9.1: When an error occurs, does the app provide a clear and understandable message for the user and caregiver? 9.2: Are errors explained simply, and are solutions or suggestions provided to correct them? 9.3: Is additional support provided (such as a help or contact button) in case of persistent issues? Help and Doc- 10.1: Does the app have an easily accessible and understandable help section for all umentation users? 10.2: Is help available in diferent formats (e.g., text, audio, video) to accommodate the needs of users with intellectual disabilities? 10.3: Is information provided on how to personalize the app so users can adjust settings according to their needs?
Product Designer working in a consulting firm as User Experience (UX) Researchers.
UX Designer and Digital Accessibility Evaluator. Experience in designing digital gamification solutions.
UX/UI Designer, Prototyping Expert, and UX Researcher.
Several screens, partic- Long texts are present on the onboarding pages where the ularly the initial one, app and its features are explained. In the tested prototype, contain an excessive an icon for automatic text reading was included but not amount of informa- yet functional. However, in the final version of the app, tion, which may over- this functionality will be implemented, providing valuable whelm or bore users. support for users by enhancing accessibility.
The app does not provide information on how to personalize settings, which limits users’ ability to adjust the configuration according to their individual needs.
For a future release of the application, the implementation of accessibility settings will be considered for each configuration of Eleuteria App. The app has been designed in alignment with accessibility standards, and these settings will further enhance usability for all users. through alternative formats, such as short videos or visual aids. For now, the implemented screen reader functionality is considered suficient for ensuring accessibility.
Secondly, the product designers highlighted the lack of accessibility settings, such as options to adjust font size or personalize text display. While these features are not included in the current version, they will be prioritized for inclusion in a future release of the app.
With these considerations in mind, we conclude that the design phase for the first release of Eleuteria App is complete. We are now ready to proceed to the implementation phase, where we will develop a ifrst functional MVP to be deployed and tested with end users.
The Deliver stage of the project ofered several valuable insights that informed both the design and implementation of the Eleuteria App.
One key lesson emerged during Step 1, where it became evident that elder caregivers often have limited experience with smart devices. Feedback from participants C3, C4, and C6 highlighted the need to account for difering levels of digital literacy. This underlines the importance of integrating targeted training or user support to promote broader acceptance and more efective use of the app across a diverse caregiver population. Supporting users with varied technological familiarity is essential to ensure inclusive and practical deployment in real-world professional settings. However, all participants were able to complete the assigned tasks independently, with the exception of a single instance in which one participant requested clarification regarding a specific feature (e.g., participant C6 and the grey icons). No additional assistance was required during the sessions. This minimal intervention ensured that the results primarily reflect the participants’ natural interaction with the system, preserving the validity of the observed usability issues.
Additionally, both Step 1 and Step 2 demonstrated the value of the iterative design process. This approach enabled us to refine the app progressively based on direct user input, ensuring that updates addressed real-time feedback and more accurately aligned with user needs. The changes implemented early in the testing phase proved efective in resolving initial usability issues, ultimately enhancing the app’s accessibility and intuitiveness.
Regarding the SUS questionnaires, we found that adapting the protocol to the needs of users with intellectual disabilities was efective. In particular, a more direct and empathetic approach—where an evaluator asked the questions in a simplified and supportive manner—proved to be both eficient and successful in collecting meaningful user data.
On the other hand, evaluation was useful because it provided a structured and eficient way to identify usability issues early in the design process, allowing us to uncover potential barriers—such as information overload and lack of personalization guidance—before user testing. It ofered valuable expert insights that helped prioritize areas for improvement and informed decisions for future development stages.
This study received approval from the Committee of Ethics in Research of Universidad Carlos III de Madrid (N CEI23 09 MORENO, May 8th, 2023). Thus, the study was conducted in accordance with ethical guidelines to keep the integrity of the research process and safeguard the well-being of all participants. Caregivers, individuals with mild intellectual impairments, and professional product designers participated in the study as volunteers. Before starting with the study, all participants were fully informed about the study’s objectives, their involvement, and their right to withdraw at any time without any repercussions. In addition, to ensure confidentiality, demographic information was gathered and presented anonymously, ensuring no personal identifiers were disclosed. Observations and feedback were securely documented and solely utilized for the purpose of the research.
This paper has proposed a structured methodology to carry out the Deliver Stage of the Double Diamond Model to implement Eleuteria App, an interactive system for supporting individuals with mild intellectual impairments and their caregivers. By applying our structured methodology for the Deliver stage, we refined the app’s functionality and interface based on direct feedback received during two user testing step with real end-users (e.g., six caregivers and five individuals with mild intellectual impairments). This iterative approach allows for continuous improvement and ensures that the app evolves in response to real user experiences, making it more intuitive and accessible with each iteration. Since the tests provided valuable insights into the experiences and challenges faced by our target audience, our ifndings highlighted the importance of conducting interactive user testing with the actual end-users before implementing supportive systems. In addition, the user testing sessions reveled valuable insights even on research aspects that required significant adaptation to individuals with impairments. For example, our experience with the SUS questionnaire demonstrated that standard usability instruments may not always be suitable for individuals with intellectual impairments. The necessity of translating technical terms—such as the generic reference to the “system” in the original questions—into clear, context-specific language became evident. This adaptation is essential for obtaining accurate and meaningful feedback, reinforcing that usability tools must be adapted when working with diverse user groups. Additionally, while no specific task was identified as particularly dificult by participants, some features were especially well-received. In particular, individuals expressed strong appreciation for the geolocation function (“Mis mapas”) and the piggy bank feature (“My piggy bank”), which they found engaging and useful. These positive reactions highlight areas of the app that resonate with individuals and contribute to a more enjoyable experience. Thus, the main focus for future work will be the full implementation of the Eleuteria App. This next step will involve developing the complete application and conducting an initial trial in a real-world setting. This trial will allow us to evaluate the app’s performance, usability, and overall impact on users in a practical environment, providing valuable insights for further refinement and development.
This work received funding from: Ministerio de Ciencia, Innovación y Universidades de España, “Convocatoria de la Universidad Carlos III de Madrid de Ayudas para la recualificación del sistema universitario español para 2021-2023, de 1 de julio de 2021, en base al Real Decreto 289/2021, de 20 de abril de 2021 por el que se regula la concesión directa de subvenciones a universidades públicas para la recualificación del sistema universitario español”. This work has also been supported by grants PID2023-148577OB-C21 (Human-Centered AI: User-Driven Adapted Language Models-HUMAN_AI) and PID2023-148577OB-C22 (Human-Centered AI: User-Driven Adaptative Interfaces-HumanAI_UI) funded by MICIU/AEI/10.13039/501100011033 and by FEDER/UE.
During the preparation of this work, the authors used Grammarly software for grammar and spelling checks. [11] R. Ruzi, Y. Pan, M. L. Ng, R. Su, L. Wang, J. Dang, L. Liu, N. Yan, A speech-based mobile screening tool for mild cognitive impairment: Technical performance and user engagement evaluation, Bioengineering 12 (2025). doi:10.3390/bioengineering12020108. [12] M. Zeiler, N. Dietzel, F. Haug, J. Haug, K. Kammerer, R. Pryss, P. Heuschmann, E. Graessel, P. L.
Kolominsky-Rabas, H.-U. Prokosch, A User-Centered Design Approach for a Screening App for People With Cognitive Impairment (digiDEM-SCREEN): Development and Usability Study, JMIR Human Factors 12 (2025). doi:https://doi.org/10.2196/65022. [13] F. Shahini, V. Nasr, M. Zahabi, A friendly indoor navigation app for people with disabilities (FIND), in: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 66, SAGE Publications Sage CA: Los Angeles, CA, 2022, pp. 1922–1926. [14] L. Moreno, P. Martínez, A. Díaz, H. Ochoa, B. Corsetti, Assessing standards-driven accessibility in top video conferencing platforms, Universal Access in the Information Society (2024) 1–18. doi:10.1007/s10209-024-01176-1. [15] D. Guasch, I. Martín-Escalona, J. A. Macías, V. Francisco, R. Hervás, L. Moreno, S. Bautista, Design and evaluation of ECO: an augmentative and alternative communication tool, Univers. Access Inf.
Soc. 21 (2022) 827–849. doi:10.1007/S10209-021-00819-X. [16] Augmentative and Alternative Communication (AAC) — asha.org, https://www.asha.org/public/ speech/disorders/aac/, 2025. [Accessed 09-04-2025]. [17] What is User Centered Design (UCD)? — interaction-design.org, https://www.interactiondesign.org/literature/topics/user-centered-design, 2025. [Accessed 09-04-2025]. [18] K. Moran, M. Brown, Design Thinking: Study Guide, https://www.nngroup.com/articles/designthinking-study-guide/, 2024. [Accessed 09-04-2025]. [19] J. Nielsen, Thinking Aloud: The #1 Usability Tool, https://www.nngroup.com/articles/thinkingaloud-the-1-usability-tool/, 2012. [Accessed 08-04-2025]. [20] M. Soegaard, System Usability Scale for Data-Driven UX, https://www.interaction-design.org/ literature/article/system-usability-scale, 2025. [Accessed 08-04-2025]. [21] K. Moran, K. Gordon, Heuristic Evaluations: How to Conduct, https://www.nngroup.com/articles/ how-to-conduct-a-heuristic-evaluation/, 2023. [Accessed 08-04-2025]. [22] W3C Consortium, WCAG, Success Criterion 1.4.4: Resize Text (Level AA), 2024. URL: https: //www.w3.org/WAI/WCAG21/Understanding/resize-text.html.