Senile dementia is one of the most common sufferings in the elderly population as brain functions start to deteriorate, making their day-to-day routine increasingly dependent on care staff. We need to build rapport between the Elderly and technology tools to allow aged people have an easier way to complete daily tasks without losing their independence and allowing them to work out to maintain their health and mobility with as less intervention with rehab staff as possible via remote monitoring using Internet of Things to detect anomalies in the elderly daily routine to prevent other age-related diseases in an early stage. This article proposes an elderly assistance ecosystem with focus on senile dementia using IoT and applying Software Engineering for the architecture design side allowing to make a more detailed and suited implementation.
Despite the Technological advances being adopted by more people every time [
Most modern technology corporations do not design products for the elder users daily living,
not to mention ease of use of Graphic User Interfaces (GUI) for them, making it harder to adopt
modern gadgets and devices that could make their life easier [
Elder people could develop a low self-esteem after retirement because of the contrast with
their new daily routine, diminishing their physical and social activity to a large degree. All persons
in retirement should maintain their dignity and the right to be useful; the users should be able to
discover his own limitations and capacities. The Quality of life concept is nearly linked to social,
mental and physical wellbeing [
Senile dementia is known for being in the top 3 causes of death in the elderly, just after cancer
and hearth diseases [
As technology has a big presence in our daily routine, and it has been an important part of our
entire lives, in a direct or indirect way. The younger generations are able to adapt quickly to
technology created to be friendly and intuitive for almost any user. User Experience (UX) is an
almost new knowledge area in the modern devices and applications development; there are
notable changes among device generations and their interactions with users. There is an evident
good UX design in the field of Smart Assistants (Google Assistant, Amazon Alexa, etc.) allowing
recognition of multiple accents in a language, the fulfillment for users is prominent [
This work has been formed in seven sections: Second section, named Background, contains basic definitions of the technologies related, Third section contains some work with VR devices for treating senile dementia patients, Fourth section contains the problem outline, with other works using the integrated technologies of this approach, Fifth section describes a Digital Ecosystem Model for the development of Mixed Reality Environments, showing a general architecture for possible scenarios and organizing the model by interaction layers in order to assist the patients with senile dementia; A case study is presented in section six, applying several technologies in a real scenario. Finally, the conclusions are described in section seven.
VR, also known as virtual environments, consists in an ambience surrounding a person with
sound and images that can that transmit the feel of being present physically in that virtual
environment [8]. Thanks to the versatility of current Game engines, Virtual Environments
development can be very straightforward compared to some Game engine generations ago,
making them reliable tools to enhance UX in combination with the daily routine [
Augmented Reality (AR) refers to a set of technologies that permit the users seeing most of the real environment surrounding them through a device capable of adding extra graphical information over real objects, adding virtual layers to the real environment view. AR can make some activities easier for people applied to the daily life, by highlighting objects with colors or showing notifications.
Mixed Reality (MR), as shown in Figure 1, consists in a combination of real objects and the interaction with virtual ones, giving the ability of using corporeal objects with 3D objects rendered on a MR headset [11]. The use of physical peripherals can be more adequate for elder users, making them feel comfortable with the feedback found in the haptic vibration.
The current electronic miniaturization techniques allow the production of reliable and powerful mobile microcomputers as the Raspberry Pi, being the perfect choice for reliable devices to be the brain unit for custom and open-source home automation systems [12] to control different electronic devices as sensors, electric motors and actuators with Arduino as low-level hardware interface.
The concept of Digital Ecosystem can be defined as loose networks of interacting organization that are digitally connected and enabled by modularity, and that affect and are affected by each other’s offering [14], in this way, a digital ecosystem can be integrated by the interaction among Internet of Things, Virtual Environments, AR/VR headsets, Wearable devices, using Smartphones, Tablets and Smart Assistants as intermediaries with the elder patient.
The use of AR/VR applied to Senile Dementia is an almost new area in the treatment of Mental
Diseases. Due the cost reduction in Virtual reality headsets and the increase in power for mobile
devices, VR applications have become affordable for more people like ever before.
The creation of virtual environments experience have had an exponential growth combining
modern engines engines like Unity [
Since 2012 a growth in the VR apps for dementia had been noticed [15] without the necessity of taking the patients out of the nursing home allows to explore a great variety of scenarios and study the reaction of the elders to virtual nature environments as a forest [8]. A proposal of an adaptive VR environment can be found, using Real-Time feedback from the user to change what the user can see and interact with and monitor brain activity [9].
The combination of Virtual environments and work out allow the elderly to avoid routine and
“visit” (in a virtual way) other places outside the asylum they live in [
Diseases as Alzheimer’s have promising results when combining Virtual Reality [16] showing the benefits of a tight integration with MR and theoretical models [17] using reminiscence therapy.
Project CogARC [18] integrates real objects with AR add-ons and a visual detection software installed on a tablet for the elder patient to play small games; even when they obtained positive results, in the part of negative feedback shows that the interactive aspects for some games have some flaws.
HalleyAssist [19] uses a complex ecosystem trying to prevent the majority of elder patient’s
health related ailments combining sensor sets to achieve a better detection of anomalies in
movement according to patterns stored in the knowledge base; on the other side, this work leaves
the software and user interface design to make easier for the elder to adopt this kind of devices
and applications for their day-to-day living [
Microsoft HoloLens, as a MR headset [
DCPAR project [
Eldergames Project has a MR environment with Tabletop games in it, this to train cognitive functions [11]; this work deepens in the emotional aspects of the player interactions with each other, having positive results and proving that a good design in UX for elder patients is feasible.
Several areas of opportunity can be found when talking about MR applied to Elderly patients:
• Mobility issues: one of the most frequent ailing in the elder population caused by
muscular, cardiovascular, metabolism and brain deterioration related to age [
The reason to define a digital ecosystem arquitecture is because of the presence of different technologies interacting with each other using multiple communication protocols that could produce confusion among all the kinds of devices connected [14]. The use of multiple distinct environments will be required for the different patient scenarios, in this work the proposal consist in a digital ecosystem architecture combining AR, VR, MR, and tools we can find in software design like the Unified Modeling Language (UML) [22] and System Modeling Language (SysML) [23], using the Human-Computer Interface [24] to adapt User Interfaces (UI) and User Experience (UX).
We are looking for a way to easily adapt to the most possible kinds of mental diseases starting from a generic architecture and follow the evolution of the participants sufferings.
We define Direct Interaction as all the devices the patient can interact with in a physical way (tablets, smartphones, AR/VR headsets, body-attached wearable devices), including biometric sensors to monitor the heart rate, step counter with the help of a smartphone for fall detection [25]. Some kinds of dementia can cause the patient to forget that is wearing a VR headset or any other kind of wearable device, so the need of a non-intrusive gadget is a must to make the experience comfortable for the elderly.
Tablets and smartphones allow the elder users them to be aware of their own medical situation and a medium to receive calls and notifications from relatives and care staff to maintain a constant interpersonal contact, meanwhile the sensor set found in smartphones can register accidental falls, movement patterns and daily activities for pattern analysis.
AR/VR Headsets running routines in virtual environments will permit the patient to have multiple therapies for physical workout, alternating the virtual scenario to make the experience less monotonous.
The area of Indirect interaction will be integrated by devices the patient can interact with and connected to a server in the local network to process data locally. Usually, wearables have an incredibly low processing, battery, and storage capacity, so a more powerful device is needed to collect all the local information.
Microcomputers like Raspberry Pi [12] have shown their powerful processors that can be used
for data readings from wearables and other monitoring local network devices as: cameras for
pattern recognition and live remote monitoring; smart assistants as Google Home or Alexa [
In this layer of interaction, the patient can trigger simple commands in the voice Assistant to change lights or even adjust the room temperature; in case these parameters are not dependent of the patient, other near relatives can make the respective adjustments to the room control.
The remote interaction is made through the Internet connection, this is the way caring staff, using audio and video, and distant relatives, using social network notifications, in a and can maintain contact and monitor the elder patient, permitting them to know the status of their physical situation and to make direct calls using the smart Assistant speaker or mobile devices.
The medical staff can receive alerts when an anomaly is detected with the patient that is being monitored and make audio calls directly to the mobile phone, tablet or smart assistant speaker to give the patient feedback of their situation to immediately take actions.
An encrypted connection is needed to protect all the data collected from patients when is stored in the applications servers, where all the big data processing will be done. This is where anomalies in the patient’s behavior can be detected following differences in the movement or voice patterns, allowing to detect other diseases related to the Elderly in an early stage.
Server infrastructure will be needed to host more users from different places, including hospitals, nursing homes or the patient’s home and classify by type of disease and sufferings; this will allow to detect anomalous patterns and early symptoms in patients that share the same suffering. All the ecosystem server infrastructure needs to be ready to adopt methods for big data processing and algorithms that will classify the data.
The target audiences are the elderly patients in public nursing homes that belong to the “Integral Development for the Family” institute (DIF) in Aguascalientes, México. An interview with the caring staff is necessary to study the elders work out routines, recreational activities, sleeping schedule, main interests, other aging ailments not causally related to the Senile Dementia, general user preferences, openness to using technological gadgets and wearable devices, etc.
The Mixed reality infrastructure scenario is showed in Figure 4 where we can show in more detail the process of the information management and all the ways the care staff can receive the information and monitor the patient.
The Senile Dementia disease we are going to focus on is the Alzheimer’s disease [26], adapting virtual environments with therapy techniques and follow the evolution of the patients participating in the project. Adaptations to devices will be needed in case some patients have physical or mental limitations, but the idea is to have the devices ready for the most possible scenarios.
Xiaomi mi Band (Figure 5) is an especially useful wearable to monitor the steps per day and the heart rate of the patient at a low cost with an acceptable reading quality for 15 to 40 USD and with a great battery duration that is from one to three weeks depending on the generation of the wearable and the features that are being used.
A smartphone is needed for every single Mi Band device to store the data we can retrieve from the patients via Bluetooth Connection. The Mi Band can collect data from one to two weeks until the battery is completely depleted.
The readings will give us an idea of how many steps the patient walks a day and adjust their exercise routine when it is necessary. A simple heart rate monitor is integrated in the smart band and could help the caring staff to have more frequent heart rate readings without visiting the patient.
For the first stage of the project the selected VR headset is the Oculus Rift (Figure. 6) as part of the interface between the patient and the virtual environment. For further stages, the objective is using the Cardboard (Figure 2) headset with a smartphone, as a low-cost VR headset and easy to replicate in case the headset needs spare parts.
Amazon Alexa and Google Home (Figure 7) are reliable voice-controlled Smart Assistants with
great cloud voice processing algorithms [
Using APIs of the most popular Social Networks like Facebook, Instagram or X (aka Twitter) or even some Instant Messaging applications like Telegram can send notifications to the interested relatives and friends concerning the patient’s condition; in a similar way, medical staff will receive special notifications but with biometric information instead of social network interactions to know the status of the patient in a more real-time oriented communication.
The first step for data analysis is to build a knowledge base with all the data collected from the devices around the patient; with the data retrieved from all the patients’ routines for data mining will be developed to detect patterns in the information using statistics and heuristics.
In a further stage there will be a possibility to detect anomalies in movement and some sufferings in an early phase, giving the medical staff the opportunity to treat diseases opportunely.
The digital ecosystem modeling is a necessary part of the interconection of devices and the type of interplay with the actors involved. Each device role is essential in the environment to maintain a consistent connectivity between layers. Comprehend the specific case of each user will permit us to have a wide view of the needs for the different kind of dementia present in the elderly.
Interactions between elder users and virtual environments though mixed reality can offer an inmense expansion of the elderly’s daily social interaction without leaving their homes, allowing elders to get out of the monotony and follow their daily therapy sessions in an easier way; the AR part could help the elders to have a better perception of their home environment when being left alone.
The UX design will allow an increasing number of elder patients to have a more bearable experience in rehabilitation and get out of the routine they live in the nursing homes. Virtual environments are great tools with AR/VR Headsets altogether to achieve the users engagement with the ecosystem. We need to present our elders the technologies they have been missing for years, or even decades, to make their daily living a more enjoyable experience at a relative lowcost and keeping their independence as much as possible. [8] Moyle, W., Jones, C., Dwan, T., Petrovich, T.: Effectiveness of a Virtual Reality Forest on People With Dementia: A Mixed Methods Pilot Study. The Gerontologist. 58, 478–487 (2018). https://doi.org/10.1093/geront/gnw270. [9] Kritikos, J., Alevizopoulos, G., Koutsouris, D.: Personalized Virtual Reality Human-Computer Interaction for Psychiatric and Neurological Illnesses: A Dynamically Adaptive Virtual Reality Environment That Changes According to Real-Time Feedback From Electrophysiological Signal Responses. Front. Hum. Neurosci. 15, 596980 (2021). https://doi.org/10.3389/fnhum.2021.596980. [10] Persky, S., McBride, C.M.: Immersive Virtual Environment Technology: A Promising Tool for Future Social and Behavioral Genomics Research and Practice. Health Communication. 24, 677–682 (2009). https://doi.org/10.1080/10410230903263982. [11] Gamberini, L., Martino, F., Seraglia, B., Spagnolli, A., Fabregat, M., Ibanez, F., Alcaniz, M., Andres, J.M.: Eldergames project: An innovative mixed reality table-top solution to preserve cognitive functions in elderly people. In: 2009 2nd Conference on Human System Interactions. pp. 164–169. IEEE, Catania, Italy (2009). https://doi.org/10.1109/HSI.2009.5090973. [12] Jamil, M.M.A., Ahmad, M.S.: A pilot study: Development of home automation system via raspberry Pi. In: 2015 2nd International Conference on Biomedical Engineering (ICoBE). pp. 1–4. IEEE, Penang, Malaysia (2015). https://doi.org/10.1109/ICoBE.2015.7235916. [13] Kishino, F., Milgram, P.: A Taxonomy of Mixed Reality Visual Displays. IEICE TRANSACTIONS on Information and Systems. E77-D, 1321–1329 (1994). https://ci.nii.ac.jp/naid/110003209335/ [14] Valdez-De-Leon, O.: How to Develop a Digital Ecosystem – a Practical Framework. TIM
Review. 9, 43–54 (2019). https://doi.org/10.22215/timreview/1260. [15] Garcia, L., Kartolo, A., Methot-Curtis, E.: A Discussion of the Use of Virtual Reality in Dementia.
In: Eichenberg, C. (ed.) Virtual Reality in Psychological, Medical and Pedagogical Applications. InTech (2012). https://doi.org/10.5772/46412. [16] Clay, F., Howett, D., FitzGerald, J., Fletcher, P., Chan, D., Price, A.: Use of Immersive Virtual Reality in the Assessment and Treatment of Alzheimer’s Disease: A Systematic Review. JAD. 75, 23–43 (2020). https://doi.org/10.3233/JAD-191218. [17] Tsao, Shu, Lan: Development of a Reminiscence Therapy System for the Elderly Using the Integration of Virtual Reality and Augmented Reality. Sustainability. 11, 4792 (2019). https://doi.org/10.3390/su11174792. [18] Boletsis, C., McCallum, S.: Augmented Reality Cubes for Cognitive Gaming: Preliminary Usability and Game Experience Testing. IJSG. 3, (2016). https://doi.org/10.17083/ijsg.v3i1.106. [19] Forkan, A.R.M., Branch, P., Jayaraman, P.P., Ferretto, A.: HalleyAssist: A Personalised Internet of Things Technology to Assist the Elderly in Daily Living. 10 (2019). https://doi.org/10.24251/HICSS.2019.507 [20] 18 19Cardona Reyes, H., Muñoz Arteaga, J.: Multidisciplinary production of interactive environments to support occupational therapies. Journal of Biomedical Informatics. 63, 90– 99 (2016). https://doi.org/10.1016/j.jbi.2016.08.002. [21] Schall, M.C., Rusch, M.L., Lee, J.D., Dawson, J.D., Thomas, G., Aksan, N., Rizzo, M.: Augmented Reality Cues and Elderly Driver Hazard Perception. Hum Factors. 55, 643–658 (2013). https://doi.org/10.1177/0018720812462029. [22] Gomaa, H.: Designing concurrent, distributed, and real-time applications with UML. In: Proceeding of the 28th international conference on Software engineering - ICSE ’06. p. 1059.
ACM Press, Shanghai, China (2006). https://doi.org/10.1145/1134285.1134504. [23] Huang, E., Ramamurthy, R., McGinnis, L.F.: System and simulation modeling using SYSML. In: 2007 Winter Simulation Conference. pp. 796–803. IEEE, Washington, DC, USA (2007). https://doi.org/10.1109/WSC.2007.4419675. [24] Dillon, A., Watson, C.: User analysis in HCI — the historical lessons from individual differences research. International Journal of Human-Computer Studies. 45, 619–637 (1996). https://doi.org/10.1006/ijhc.1996.0071. [25] Tsinganos, P., Skodras, A.: A smartphone-based fall detection system for the elderly. In: Proceedings of the 10th International Symposium on Image and Signal Processing and Analysis. pp. 53–58. IEEE, Ljubljana, Slovenia (2017). https://doi.org/10.1109/ISPA.2017.8073568. [26] Bucciarelli, A.: Alzheimer’s disease. Mercury Learning and Information, Dulles, Virginia; Boston, Massachusetts; New Delhi (2016).