This paper presents VERA, a sensor-based platform for assessing and rehabilitating Alzheimer's patients, grounded in the embodied cognition paradigm. VERA integrates wearable technologies with motion capture and machine learning to collect and analyse motor-cognitive biomarkers for early diagnosis. The system also supports personalised rehabilitation through sensorimotor feedback in a virtual environment. VERA's design aligns with theories of cognition emerging from body-environment interaction, ofering a novel, theoretically grounded approach to cognitive care.
Recent advances in neuroscience and cognitive science highlight the body’s central role in understanding
and supporting cognition. The embodied cognition paradigm, developed in response to classical symbolic
models, posits that cognitive processes are deeply rooted in the body’s interaction with the environment.
Perception and action are not just outputs but are integral to cognition. The body is thus a fundamental
site of cognitive activity, not just a vessel for symptoms [
The VERA (Virtual Environment for Rehabilitation and Assessment) project adopts this framework to create a sensor-based platform for assessing and rehabilitating Alzheimer’s patients. Drawing on enactive and ecological aspects of embodiment, the system integrates wearable technologies—sweatsensing patches, haptic gloves, pressure-sensing insoles—with motion capture and machine learning tools. This enables fine-grained, real-time analysis of motor behaviour, detecting subtle anomalies linked to early cognitive decline. From this perspective, dysfunctional motor patterns reflect the cognitive system’s disruption. VERA identifies motor-cognitive biomarkers by analysing these embodied traces through feature selection and classification. The body becomes a measurable interface for detecting and monitoring cognitive deterioration.
Beyond assessment, VERA leverages sensory-motor stimulation to engage cognition actively. Rehabilitation protocols adapt dynamically to patient-specific feedback, enabling a closed-loop interaction. This approach aligns with theories viewing cognition as emerging from sensorimotor coupling shaped by biological, cultural, and environmental factors. Grounded in embodied cognition, VERA enhances diagnostic accuracy and supports personalised rehabilitation.
This paper presents the system architecture (Figure 1), clinical protocol, data fusion and learning strategies, and discusses the role of virtual environments in cognitive care.
The architecture employed in the VERA project consists of four devices designed to collect data from
patients afected by Alzheimer’s disease. These devices include Sweat-Sensing Patches, Haptic Gloves,
Pressure-Sensing Insoles, and a Real-Time Localization System (RTLS)—a system of cameras and
bodyworn markers used to collect motion data [
The system infrastructure is entirely cloud-based, with all services hosted on secure European servers. This ensures compliance with data protection regulations such as the GDPR, and allows for high availability, scalability, and geographic redundancy.
The system integrates multiple wearable devices to capture data during user interaction. Each device
plays a specific role in the data acquisition process, contributing to a comprehensive understanding of
the user’s physical state:
• Sweat-Sensing Patches to continuously monitor the concentration of electrolytes in sweat, such as
sodium, potassium, and chloride. There is a potential correlation between electrolyte imbalances
and alterations of the autonomic nervous system in patients with Alzheimer’s and Parkinson’s
disease [
VERA will include a motion capture system to collect detailed data about human movement. This will allow for precise three-dimensional reconstruction of body movements, including joint trajectories and segment orientations. The recorded motion data will be processed for research and clinical evaluation, particularly in the study of motor control and neurodegenerative disorders.
The system includes a web-based application that helps clinical operators manage patient sessions, assign tasks, and monitor sensor data in real time (see Figure 1, "Data assessment"). Sensor data are transmitted to a local unit, pre-processed, and then sent via MQTT to a remote server, where they’re stored in a structured database through RESTful APIs. This setup supports scalable data flow, modular processing, and centralised patient information access.
Software development follows DevOps principles, with code managed on GitLab using version control and automated CI/CD pipelines. This ensures continuous testing, integration, and delivery of updates, supporting reliability and rapid development.
Collected data will also support machine learning to identify behavioural patterns, assess motor function, and aid in early diagnosis or monitoring of neurodegenerative diseases like Alzheimer’s and Parkinson’s.
The web interface is intuitive and supports structured execution of sessions. Operators can manage patient profiles, launch tasks, and monitor real-time data from wearable devices. Tasks trigger data streams that are automatically recorded. Upon task completion, the operator moves to the next step. The interface minimizes cognitive load and operational errors, ofering a straightforward workflow and real-time status indicators.
The data collected from wearable devices and tracking systems will be stored in a relational database, such as MySQL, managed through an Object-Relational Mapping (ORM) system. This approach enables structured and consistent data management, allowing for high-level database interactions while maintaining the flexibility required for future developments of the project.
To ensure long-term data availability and security, automated backup mechanisms and disaster recovery strategies will be implemented. These systems are designed to prevent data loss in the event of hardware failures or other unexpected incidents.
The database organization is designed to support eficient data archiving and easy access to information collected during assessment sessions. Access control measures and data protection protocols will be applied in compliance with data privacy regulations, safeguarding sensitive patient information.
This section outlines the structure and methodological details of the experimental study. It describes the protocol adopted for task execution and the criteria used to recruit and select participants.
The assessment protocol consists of a battery of 13 motor and cognitive-motor tasks designed to evaluate
gait characteristics [
Participants must have a prior diagnosis of Alzheimer’s Disease (AD) or Mild Cognitive Impairment (MCI), confirmed by neuropsychological assessment, with MMSE scores between 20 and 24. We will also include a control group of cognitively healthy individuals. All participants must provide informed consent directly or via a legal representative. Individuals with psychiatric comorbidities or lacking decisional capacity will be excluded. Data will be anonymised using unique alphanumeric codes, in compliance with privacy regulations. Recruitment will occur at University of Palermo and University of Malta, enrolling 400 participants total (200 per site).
Although the VERA platform shows promise through integrating sensors, machine learning, and virtual reality, several challenges remain. Feature selection and data fusion must balance accuracy and eficiency while ensuring interpretability. Using shallow, deep, and ensemble models raises concerns about transparency and clinical relevance. Additionally, virtual environments must ensure long-term engagement and usability. The following sections outline these open issues and suggest directions for improving adaptability, personalisation, and user-centred rehabilitation.
Feature selection is crucial in analysing data from multiple sources to ensure model efectiveness,
personalisation, and adaptability [
VERA manages heterogeneous data - time series, accelerometry, forces, physiological signals, videostreams - and derived features. Defining appropriate fusion techniques and understanding their roles, benefits, and limitations are essential.
Multimodal sensor data analysis enables user models via machine learning to assess patient status and
personalize rehabilitation protocols [
Shallow models (SVMs, Random Forests) work well for structured data and classification, while k-NN with Dynamic Time Warping supports real-time time-series motion analysis. Furthermore, shallow models’ interpretability favors clinician oversight, while online learning ensures continuous adaptation.
Deep learning handles high-dimensional data: DNNs learn complex patterns, CNNs extract spatial features, and RNNs (LSTM, GRU) capture temporal dependencies. Ensemble methods improve robustness: meta-models optimize predictions, bagging reduces variance, and weighted voting emphasizes reliable signals, aiding real-time classification in noisy clinical contexts.
Combining shallow, deep, and ensemble techniques within a multimodal framework enables precise assessment, personalized therapy, and adaptive rehabilitation.
Virtual reality (VR) technologies are integral to clinical practice, ofering immersive environments that boost patient engagement and support functional recovery. In Alzheimer’s rehabilitation, VR combines cognitive stimulation with physical interaction, enhancing memory, attention, and motor planning. Within the VERA platform, the virtual environment will act as a dynamic, adaptive component of the embodied cognition framework. Based on sensor-derived biomarkers identified during assessment, the system will personalise rehabilitation protocols, adjusting motor tasks in real time to target specific deficits, e.g., balance or gait issues will lead to tailored VR exercises at home. Rehabilitation will extend beyond clinical settings, as patients will perform gamified therapeutic activities remotely while staying connected to the platform. This remote, interactive approach will support continuous, scalable, cost-efective care. Gamification will increase motivation and adherence, transforming repetitive tasks into engaging challenges, which is crucial for individuals with cognitive impairment.
VERA proposes an innovative, sensor-based platform for the assessment and rehabilitation of Alzheimer’s and MCI patients, rooted in embodied cognition. Combining multimodal data, machine learning, and virtual reality enables early detection of motor-cognitive impairments and delivers adaptive, personalised therapy. The integration of wearable devices, real-time tracking, and a gamified VR environment supports patient engagement and continuity of care. However, several open issues remain. These include challenges in feature selection, data fusion, AI model design and transparency, design and integration of the VR environment, and the clinical validation of assessment tools. Addressing these challenges will be essential to realise the full clinical potential of VERA and promote its adoption in everyday neurorehabilitation.
This work is supported by the VERA project (Cod. C1-1.1-20, INTERREG VI – A Italia Malta). We thank all project partners for their valuable contributions and collaboration: Leonardo Abbene, Mario Barbagallo, Antonino Buttacavoli, Donato Cascio, and Giuseppe Raso (UNIPA); Vincent Buhagiar, Paulann Grech, and Alexei Sammut (UM); Neville Calleja, Mark Camilleri, and Anton Grech (MHA); Nicola Coppedè, Giuseppe Tarabella, and Andrea Zappettini (CNR); Vincenzo Conti, Ligia Domingez, Moreno la Quatra, and Valerio Salerno (UKE).
The author(s) have not employed any Generative AI tools.