Enhancing upper limb mobility through gamified tasks and Azure Kinect: a preliminary study in post-stroke subjects

Abstract

Stroke is a leading cause of long-term disability as well as death worldwide, where aging is among the most significant nonmodifiable factors. Motor impairments related to post-stroke hemiplegia, resulting from the loss of specific brain functions following the acute event, commonly affect both upper and lower limbs, leading to a deterioration in perceived quality of life as daily activities become unsafe and difficult to perform. Various rehabilitation strategies and therapies are commonly adopted in the hospital setting during the post-acute phase to recover, at least partially, major motor functions and improve physical mobility to ensure the patients' safety in daily life. However, these functions should be stimulated continuously and frequently through maintenance activities in order not to lose the level of functional recovery achieved and avoid subsequent hospitalizations for new rehabilitation treatments. This paper proposes the use of gamified tasks in a virtual environment to enhance upper limb mobility. Gamified tasks are performed using a single RGB-D camera-based vision system (specifically, Microsoft Azure Kinect DK) suitable for easy deployment in home environments. Non-invasive body tracking models are employed to capture 3D upper limb trajectories in real time and measure, through objective parameters, the unilateral and bilateral movements required by each task. Preliminary results on a small cohort of post-stroke subjects show a general progress in upper limb mobility and coordination, in agreement with an improvement in some clinical severity scores and tests. This suggests that the proposed solution is suitable for continuous stimulation of upper limb function and performance monitoring over time in the home environment, contributing to the improvement of the patient's general motor condition and increased physical well-being in daily life.

Keywords

• eol>Upper limb rehabilitation
• Azure Kinect
• home monitoring system
• Artificial Intelligence 1

1. Introduction

Annual reports on stroke show an increasing incidence in the global population despite advances in prevention, treatment, and wellness [ 1 ], [ 2 ]. Among the known risk conditions, age represents one of the more critical non-modifiable factors causing incidence to double with age [ 3 ]. The physical and neurological consequences of the acute event cause long-term functional deficits, leading to a significant burden on the healthcare systems [ 4 ] and reduced quality of life for stroke

0000-0001-5381-4794 (C. Ferraris); 0000-0003-4061-8211 (G. Amprimo); 0000-0001-9034-7101 (L. Vismara); 0000-0001-9072-7454 (A. Mauro); 0000-0003-0547-0143 (G. Pettiti) © 2023 Copyright for this paper by its authors.

Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).

CEUR Workshop Proceedings (CEUR-WS.org) survivors [ 5 ]. Motor disabilities are common after stroke, limiting overall mobility with a direct impact on activities in daily life and active social involvement [6]. One of the more disabling conditions is the hemiparesis of the contralateral upper limb, which affects more than 80% of stroke survivors, as acute or chronic limitation of mobility, control, and coordination and impairs everyday daily actions (e.g., reaching and picking) [7].

Various complex rehabilitation treatments are promptly activated after the acute phase to recover lost functions, activate compensatory strategies, and increase patients’ autonomy in daily life. For example, rehabilitation protocols focus on gait, posture, and balance to avoid fall risks and ensure patient safety [8]-[10]. Regarding upper limbs, several studies have shown that exercise therapies play a crucial role in stroke rehabilitation, and several ad-hoc strategies are planned (goal-oriented, task-oriented, repetitive task training) with different duration, workload, and feedback to suit the patient’s condition [6]. For example, bilateral training is a recent rehabilitation strategy that relies on the knowledge that the non-paretic upper limb can stimulate the movement of the paretic upper limb during simultaneous movements, with significant benefits on motor coordination [11].

In recent years, various technological solutions have been proposed for upper limb rehabilitation of post-stroke addressing various degrees of motor impairment severity: these include assistive devices [12] and robots [13][14], and more innovative methodologies such as virtual reality [15][16], serious/exergames/gamification [13][17], and camera-based solutions [18]-[22].

Along this line of research, we present an activity monitoring system, based on gamified tasks and suitable for home environments, to stimulate upper limb mobility and promote improvement/maintenance of motor functions with a focus on range of motion, motor control, and coordination. To this end, gamified tasks propose unilateral and bilateral movements, with configuration settings that take into account the subjects’ motor conditions. This work used one of the platforms developed in the REHOME project [23], the Motor Rehabilitation and Exergames platform (MREP) [ 24 ], which implements a suite of assessment tasks, gamified tasks, and rehabilitative exergames for monitoring upper and lower limb performance in subjects with motor and cognitive impairments originating from neurological disorders. Specifically, this preliminary study considered two of the available gamified tasks and one of the assessment tasks (i.e., walking) to focus only on upper limb mobility in post-stroke subjects and evaluate the potential benefits for arm swing during walking [ 25 ]. Preliminary results confirm an improvement in overall upper limb mobility for both the paretic and non-paretic sides in terms of speed and number of movements per minute, increased movement coordination, and an implicit adaptation of the non-paretic side to the performance of the paretic side, as stated in [6]. Moreover, these results agree with the overall improvement shown by the clinical scales at the end of the experimental protocol and an objective reduction of arm swing asymmetry during walking. This confirms that the proposed solution can stimulate movements and detect (monitor) changes in upper limb performance over time. Considering the increased incidence of stroke events associated with aging and that residual deficits require continual and prolonged rehabilitation treatments, the activity monitoring system could be a helpful support tool in an aging society, particularly in decentralizing specific healthcare services from hospitals to home settings. In addition, it is important to note that, in this paper, we reported a specific application case study of a system particularly suitable for the elderly population. Its main features and versatility also make it relevant for healthy aging applications as a tool to stimulate motor function in contrast to the functional physiological decline related to aging, thus promoting physical exercises and early detection of functional alterations.

2. Materials and Methods

2.1. Tasks and vision system characteristics

In this study, only a specific subset of the tasks and exergames offered by MREP was analyzed, choosing from the motor tasks most suitable for the recruited post-stroke volunteers. The experimental protocol involved the following tasks: • Gait (G): This task allows assessment of alterations in gait patterns, including spatiotemporal features, dynamic stability, and rhythmic arm swings. Subjects were asked to walk on a 6-m long path, facing the RGB-D camera, to the best of their ability. This setting allows the estimation of relevant gait parameters over a relatively short area to be suitable for home settings, as in [ 26 ]. The 6 m path devoted to the G task is undoubtedly more suitable for hospital or outpatient settings with greater available spaces. However, the walking ability is analyzed over a shorter distance (4.0-4.5 m), which is more suitable for home scenarios. In addition, the G task was included in the experimental protocol only to demonstrate the effects of exergames on arm swing during walking: only the gamified tasks could be proposed for home protocols, maintaining the G task only for pre- and post-treatment comparison in clinical settings. • Lateral Weightlifting (LWL): This task, commonly performed in physiotherapeutic sessions, is offered in a gamified version (i.e., in a virtual gym environment) to assess the mobility of the left and right upper limbs, as in [ 27 ]. Subjects were asked to perform, facing the RGB-D camera, a predetermined number of lateral adduction/abduction movements with the arms to the best of their ability in terms of range of motion and speed. This task allows estimating some relevant mobility parameters of lateral movements related to each arm separately or both arms simultaneously. • Frontal Weightlifting (FWL): This task is designed like LWL [ 27 ]. The difference lies in the type of movement required. In fact, subjects were asked to perform, facing the RGB-D camera, a predetermined number of frontal up/down movements with their arms to the best of their ability in terms of range of motion and speed. This task allows the estimation of relevant mobility parameters of frontal movements related to each arm separately or both arms simultaneously.

Including both frontal and lateral arm movements allows to emphasize the differences in execution and control movements in the two directions, since post-stroke survivors exhibit more difficulty in the lateral direction, as often evidenced [ 28 ][ 29 ]. In addition, simultaneous bilateral arm movements have shown the potential to reactivate the damaged hemisphere, contributing to increased strength and motor function of the paretic limb [ 30 ]. FWL and LWL can be performed in standing or sitting position, 2.5-3 meters away from the RGB-D camera, to address the patient’s dynamic instability and ensure safety while performing the task. In addition, both gamified tasks are customizable through a configuration file (i.e., number of movements or minimum arm angle) to cope with the patient’s condition. For now, the configuration file is set by therapists; however, in the next future, it can be set by automatic artificial intelligence algorithms based on the assessed performance of patients and progress over time. In addition, several gamification elements were considered during the design phase of the gamified tasks to enhance patient engagement and experience. For example, the game scenario (virtual gym environment) allows participants to be immediately involved in the exercise to be performed (weightlifting) and to identify with the virtual character (avatar). In addition, using the body to control the avatar without external aids or devices encourages simple, active, and autonomous participation by providing immediate visual feedback of the interaction with the game. Sounds enrich the game by highlighting the movements performed, thus increasing emotional engagement. At the same time, text and voice messages (via text-to-speech functions) complement the user interface and guide participants in completing tasks and levels. In the future, gamified tasks will be enriched with other gamification elements (point rewards, timed challenges, new levels) to reward performance and encourage treatment continuity in the medium and long term.

As mentioned earlier, the selected tasks were proposed through a vision system based on the Azure Kinect [ 31 ] and the non-invasive 3D body tracking library that leverages Deep Learning methodologies [ 32 ]. SDKs for using Azure Kinect facilities, available in C++, were first ported to the Unity (C#-based) environment, the game engine used to design and develop the entire MREP exercises suite. The body tracking library was used both to capture body movements and interact with the game scenario in real time and non-invasive manner by analyzing the trajectories of specific joints among the 32 available that make up the 3D skeletal model. The user interface, consisting of text messages and audio support, guides the user in the execution of all tasks but also allows a supervisor to intervene, if necessary, by starting and stopping the proposed exercises [ 27 ].

For this experimental study, a ZOTAC© ZBOX EN52060-V (16 GB RAM, NVIDIA GeForce RTX 2060 6GB, 9th generation 2.4GHz quad-core processor) was used to run the MREP software, while Azure Kinect was configured as follows: 30 fps for both color and depth streams, 1080p resolution for the depth stream, and Narrow Field of View (NFV) to capture body movements farther from the camera and with a wider frontal viewing angle to ensure optimal body tracking [ 33 ].

2.2. Participants and experimental protocol

For this preliminary study, a small cohort of 11 volunteer post-stroke participants was recruited from the Division of Neurology and Neurorehabilitation at San Giuseppe Hospital (Istituto Auxologico Italiano, Piancavallo, Verbania, Italy). Participants were post-acute or subacute stroke, with hemiparesis on one side of the body (six on the right and five on the left), with minor disability of the upper and lower limbs (ability to walk). The only exclusion criterion was cognitive impairment with Mini-Mental State Examination (MMSE)<26. No exclusion criteria related to age, sex, side, dominance, or therapy were adopted. The local ethics committee approved the study as part of the REHOME project. All participants were instructed on the experimental protocol and instrumentation. Then, they signed an informed consent before being admitted to the study.

The experimental protocol included an initial clinical assessment session (T0) in which clinical staff assessed general motor status using traditional scales and functional tests commonly used in post-stroke. These included the Berg Balance Scale [ 34 ], Trunk Impairment Test scale (TIS) [ 35 ], Time Up-and-Go test (TUG) [ 36 ], and shoulder joint mobility assessment [ 37 ]. In the same session, the instrumental gait motor task (G) was proposed to participants to assess gait information before starting the subsequent sessions based on gamified tasks. The same assessment was repeated at the end of the gamified sessions to compare the motor condition before and after the overall protocol (TF). The gamified sessions were organized over two weeks, three sessions per week, for a total of six sessions (R1-R6).

The experimental protocol was administered to all participants under the same environmental conditions and the supervision of the clinical staff. All participants were able to complete the experimental protocol correctly and as planned, except for one subject who withdrew after the second gamified session and was therefore excluded from the subsequent analysis.

2.3. Functional parameters and data analysis

Data analysis was performed with MATLAB® from the 3D trajectories of specific joints of the skeletal model collected during the three selected motor tasks (i.e., G, LWL, and FWL). Initial preprocessing was applied to all skeletal model joints, which included a resampling procedure (50Hz) to remove frame rate jittering in the camera acquisition phase and low-pass filtering (5Hz) to focus on the voluntary motion frequency band and remove high-frequency noise interference. The resampled and filtered trajectories were then used to estimate ad-hoc functional measures.

Concerning G, several traditional spatiotemporal parameters were estimated, as well as parameters related to dynamic stability and arm swing during walking. The methodological approach to gait analysis was the same as in [ 25 ][ 26 ], where forward and backward arm swing trajectories were estimated with respect to the trunk segment in the walking direction. Mean gait parameters were estimated at T0 and TF to detect the improvement in performance at the end of the experimental protocol in agreement with the clinical tests.

For LWL and FWL, parameters were estimated from specific body segments determined using some skeletal model joints. The following body segments were considered: upper limb segment between the wrist and clavicle joints (UPPL); trunk segment between the neck and pelvis joints (TRUNK); arm segment between the clavicle and elbow joints (ARM); and forearm segment between the elbow and wrist joints (FORE). Angle measurements were determined between the UPPL and TRUNK segments (upper limb angle) and between the ARM and FORE segments (elbow angle). Based on the movements required by the gamified tasks, the upper limb angle was estimated in the corresponding movement axes: sagittal axis for LWL (adduction-abduction movements) and transversal axis for FWL (up-down movements). Figure 1 shows the location of the joints and body segments involved in the data analysis for LWL and FWL.

Other secondary parameters, in particular speed and rate, were estimated from the primary angular measures. Angular measures were estimated for paretic and non-paretic limbs for both unilateral and bilateral execution. The complete list of functional parameters considered for this study is given in Table 1.

SIMILLWL1,2 Similarity index (-) 1 Parameters that refer to the overall task. All the other parameters are computed separately for the affected and non-affected side. 2 Parameters estimated only for the bilateral execution of FWL and LWL.

The ARMSYMG is an index assessed to highlight arm swing asymmetry during walking. It was calculated as in [ 25 ]: more negative values indicate more pronounced asymmetry between the maximum swing angles of the upper limbs.

The SYNC and SIMIL metrics (for both LWL and FWL tasks) aim to highlight the differences, in terms of temporal and spatial execution, between the 3D trajectories of the upper limbs during simultaneous bilateral movements. These summary indices provide an immediate indication to monitor the improvement of motor control, symmetry, and coordination in post-stroke subjects along with the other single-arm parameters. The SYNC metric is defined as in [ 27 ] and considers the time lag between upper limb trajectories above and below the minimum angular threshold configured for the exercise, finding correspondence in bilateral movement cycles. According to its definition, values close to 0 indicate bilateral movements with good time synchronization; increasing values indicate unsynchronized bilateral movements. The SIMIL metric refers to the similarity of the 2D closed shapes that enclose the trajectories drawn by the upper limbs (in particular, WRIST joint), according to the two main directions of motion, with respect to a reference point (in this case, NECK joint). It is estimated using Procrustes analysis [ 38 ] implemented in MATLAB (procrustes function) that returns an index of dissimilarity between the shapes that enclose left and right trajectories. The scaling parameter of the procrustes function was disabled to maintain information on any different excursion between the paretic and nonparetic sides. According to its definition, values close to 0 indicate bilateral movements with good shape similarity; increasing values indicate dissimilar shapes during bilateral movements. Figure 2 shows an example of the shapes drawn during the LWL simultaneous execution.

Mean parameters and metrics were estimated for the first and second weeks to detect performance improvement and trends in the gamified tasks.

3. Results

3.1. Participants clinical and demographic data

The clinical and demographic characteristics of the participants who correctly completed the experimental protocol (10 participants) are shown in Table 2.

Five subjects habitually use assistive devices during walking (tripod, walking stick). However, they were all able to amble without them during task G. The same participants preferred to perform gamified sessions in a sitting position. All subjects performed the proposed instrumental and gamified tasks correctly, as scheduled by the experimental protocol. At the end of the study, 20 G sessions, 60 LWL sessions, and 60 FWL sessions were available for data analysis. Regarding LWL and FWL, it is important to remember that, for each gamified task, 60 trials were collected for the non-paretic arm, 60 trials for the paretic arm, and 60 trials for both arms simultaneously. One subject was not able to complete the bilateral tasks in most of the planned sessions: the data were discarded, so only 54 trials were considered for the analysis of LWL and FWL bilateral tasks.

Data analysis revealed that all the clinical metrics (TIS, TUG, BERG, and paretic shoulder mobility) indicate an overall improvement in motor performance at the end of the experimental protocol (TF). Specifically, TIS score increased (i.e., improved) by 20.9% (TF=13.30±4.30 pts); TUG time decreased (i.e., improved) by 13.4% (TF=23.17±15.28 s); BERG score increased (i.e., improved) by 9.8% (TF=23.17±15.92 pts); and paretic shoulder mobility increased (i.e., improved) by 11.76% (TF=147.78±29.49 deg).

3.2. Results on gait analysis

This analysis aims to show the differences in mean G parameters between T0 and TF at the end of the experimental protocol. Table 3 shows the percentage changes over the cohort of participants.

The results reveal no substantial differences in gait patterns, whose parameters are relatively stable. There is a slight reduction in walking speed (-8.5%) and step length (-2.9%), while the stance phase shows minimal improvement (-0.4% in stance phase duration). The dynamic stability (TSWAYG) shows a minimal inter-group worsening (+2.93% of instability). Regarding the arm swing, the maximum swing angle decreased for both sides (-16.5% on average). However, interestingly, this result is associated with a significant overall reduction in arm swing asymmetry (T0=-16.21±13.70, TF=-12.54±9.76), suggesting lower amplitude but greater coordination in arm swing movements. Considering that the gamified tasks stimulate upper limb movements, the result on arm swing asymmetry is consistent with the proposed exercises and seems to confirm an overall practical benefit for the upper limb. The result on asymmetry agrees with the mean improvement in shoulder mobility assessed by clinicians at TF (T0=132.22 ± 33.46 deg vs. TF=147.78±29.49 deg). However, it should be considered that the participants had different clinical pictures, and each of them responded differently to the experimental protocol. This justifies the stable gait parameters, suggesting the need for ad-hoc gamified tasks for the lower limbs and balance to appreciate the same improvement obtained on arm swing asymmetry.

3.3. Results on LWL and FWL: unilateral execution

This analysis aims to detect trends in FWL and LWL parameters between the first and second weeks of the experimental protocol. Table 4 shows the percentage changes over the cohort of participants.

As expected, the mean parameters estimated from upper limb movements show a significant difference between the paretic and non-paretic sides for FWL and LWL. The comparison shows a significant improvement in the velocity parameters, both in terms of movement speed and number of movements per minute. The improvement is substantial for both gamified tasks, especially for RATEFWL and RATELWL. In contrast, the other parameters show negligible variation between the two weeks. The results also suggest that the frontal movements (in FWL) allowed higher upper limb angles (UPANGFWL > UPANGLWL) than the LWL. On the contrary, the lateral movements (in LWL) facilitated the maintenance of adequate upper limb extension, as suggested by the higher elbow angles (ELANGLWL > ELANGFWL). The results confirm a positive trend for all participants in upper limb motor performance for both the paretic and non-paretic sides, suggesting that prolonged treatment could produce many benefits to upper limb mobility with positive effects on overall motor condition. Again, the results, particularly on the paretic arm, agree with the average improvement in shoulder mobility assessed by clinicians at the end of the experimental protocol (T0=132.22 ± 33.46 deg vs. TF=147.78±29.49 deg).

3.4. Results on LWL and FWL: bilateral execution

This analysis aims to detect trends in FWL and LWL parameters between the first and second weeks of the experimental protocol. Table 5 shows the percentage changes over the cohort of participants.

The results in Table 5 confirm the same outcome observed for unilateral execution, with relevant improvement in movement speed and number of movements in a more complex execution that demands motor control and coordination. This suggests that also bilateral tasks confirm previous indications of increased shoulder joint mobility evidenced by clinical evaluation in TF. Another significant outcome derives from the analysis of SYNC and SIMIL metrics (Table 6).

Table 6 highlights a significant improvement (SYNC index reduced by 49.2%) in temporal synchronization of bilateral movements for FWL and a slight deterioration for LWL (+7.5%). However, in both gamified tasks, the SYNC index is relatively low (<0.4). In contrast, only a slight improvement (-0.2%) in the SIMIL index has been observed for FWL and a slight deterioration (+4.6%) for LWL. Conversely from FWL, in LWL, the SIMIL index suggests more dissimilarities between the shapes drawn during the movements, denoting a greater general difficulty in coordinating movements during simultaneous lateral execution.

Another interesting outcome can be observed by comparing the upper limb performance during unilateral and bilateral execution (Table 7).

As Table 7 shows, during the bilateral execution, the maximum angle of the upper limb is less than the angle of unilateral execution for all conditions examined except for the paretic arm in FWL. This emphasizes the greater complexity of simultaneous movement execution and control. In addition, a form of implicit adaptation emerges from the analysis, in which the non-paretic arm appears to adapt to the performance of the paretic. This behavior could be reversed by extending gamified sessions for a longer period. However, the results seem to confirm a positive trend for all participants in upper limb motor performance, even in bilateral execution, suggesting that prolonged treatment could produce many benefits for upper limb control and coordination, with consequent positive effects on overall motor condition.

4. Conclusions

This study investigated the potential of gamified tasks (i.e., exergames) as an easy-to-use and engaging tool for improving upper limb mobility. As a case study, a small cohort of post-stroke subjects was involved in a two-week experimental protocol that included six training sessions with gamified tasks in a virtual environment to enhance user experience and engage participants in a fun and playful real-world scenario. The gamified tasks were proposed through a vision system based on a single RGB-D camera (specifically, Microsoft Azure Kinect DK) and its innovative body tracking algorithm that relies on deep learning approaches. This solution was developed as part of the REHOME project, with the primary objective of designing a telerehabilitation and telemonitoring platform, thus suitable for the home environment and for people with motor and cognitive deficits related to neurological disorders.

Post-stroke is one of the physical conditions that could benefit from this type of solution. Stroke survivors promptly undergo in-hospital rehabilitation after the acute phase to begin recovery of motor functions impaired by the event as soon as possible. Despite this, most patients would need continuous and frequent maintenance activities to avoid losing the functional recovery achieved, but this is not feasible in a hospital setting. Telemonitoring and telerehabilitation solutions could fill this gap, and exergames could prove to be important in ensuring continuity of treatment, facilitating the execution of specific physical exercises, stimulating the achievement of new rehabilitation goals, and ensuring greater adherence to treatment through a fun and engaging approach.

This study focused on upper limb mobility, stimulated through two gamified tasks requiring the execution of arm-lifting movements. To exert joint mobility, frontal and lateral lifting movements were included to stress the upper limb motor function differentially. In addition, the gamified tasks proposed unilateral (i.e., with only one arm at a time) and bilateral (i.e., with simultaneous and synchronized movements of both arms) execution modes to solicit not only range of motion but also motor control and coordination. Another relevant feature of the gamified tasks is their reconfigurability according to the subject's motor condition: when motor function improves, a higher level can be set (e.g., the number of movements required or the minimum amplitude of movements) or, conversely, the level can be reduced if the patient shows difficulty. During the experimental study, the clinical supervisor increased the game level for some participants by augmenting the number of arm-lifting movements for paretic and non-paretic arms. In contrast, the game level remained unchanged for other more impaired participants throughout the treatment.

Regarding motor condition, the clinical evaluation at the end of the experimental protocol indicates an overall improvement in the participants, resulting from the clinical scales and tests. The improvement relates to several motor functions, including shoulder joint mobility, posture (TIS scale), balance (BERG scale), and walking (TUG test), as discussed in Section 3.1.

The results for gamified tasks follow this trend as expected, especially for upper limb mobility, since gamified tasks exclusively solicit the upper limbs. Regarding unilateral execution, the most significant improvement is related to velocity parameters in terms of speed and rate (i.e., number of movements per minute) for both frontal and lateral execution. The parameters related to upper limb and elbow angles are relatively stable (Table 4). However, this result is also clinically relevant, as it was obtained with a significant increase in execution speed. Regarding bilateral execution, the same trend was observed, with significant improvement in velocity parameters and stability in angle parameters (Table 5). In addition, the synchronization metric for FWL shows a relevant improvement in the second week: the same is not true for LWL (Table 6), probably due to the greater difficulty in motor coordination during lateral execution. Finally, the comparison of unilateral and bilateral executions highlights an implicit adaptation of the nonparetic arm to the performance of the paretic arm, as suggested by a lower upper limb angle (Table 7).

The improvements in upper limb performance reflect the significant reduction in arm swing asymmetry during walking. In contrast, the other traditional gait analysis parameters appear stable at the end of the two weeks: this was expected, however, since only the upper limbs were directly stressed by this experimental protocol. Moreover, the results seem to indicate only partial improvement of motor condition and in specific domains, in contrast to clinical assessments that show overall improvement. However, two aspects must be kept in mind: 1) instrumental assessment measures and quantifies specific parameters and does not provide a qualitative assessment of performance as with clinical scales; 2) a more extended protocol would probably be needed to appreciate the same improvements in terms of measurement of individual parameters.

Nevertheless, the results obtained are positive and encouraging, especially from the perspective of using the proposed solution as a tool for monitoring and training/maintenance of motor function in the home environment. However, it will be necessary to extend the analysis to a larger group of subjects, not necessarily post-stroke, and over a more extended period to confirm the effectiveness of the proposed solution. In addition, in future studies, we will evaluate the possibility of automatically configuring gamified tasks through artificial intelligence algorithms that consider the subject's condition and motor performance to adjust game levels appropriately, avoiding emotional stress (anxiety, distrust, demoralization) but stimulating the subject to improve constantly. Future developments will also include the integration of new gamified tasks for hand dexterity to enhance the full motor function of the upper limb. As mentioned earlier, the purpose of this study was to evaluate a trend toward improvement in upper limb motor function using ad-hoc gamified tasks, continuing the exploration of the potential of such innovative approaches to support traditional physiotherapy treatments, as evidenced by several studies and reviews in the literature [ 39 ]-[42]. However, a point-by-point comparison with other studies is not possible, mainly because of the different protocols, participants, games, and motor functions elicited. In addition, many of these studies are clinical trials that measure the effectiveness of exergames in improving motor performance only through pre- and post-treatment clinical scales and not through the comparison of functional parameters estimated directly from the exergames, as in our case. In conclusion, specific and quantifiable potential benefits emerge from the presented study, especially for remote follow-up, in line with the state of the art, current trends, and future perspectives highlighted by several recent studies in the literature.

Acknowledgements

This work was supported by the project “ReHOME – ICT solutions for tele-rehabilitation of cognitive and motor disabilities in neurological disorders”, grant from ROP Piemonte (Italy), POR FESR 2014-2020. [6] S.M. Hatem, G. Saussez, M. Della Faille, V. Prist, X. Zhang, D. Dispa, and Y. Bleyenheuft. ”Rehabilitation of Motor Function after Stroke: A Multiple Systematic Review Focused on Techniques to Stimulate Upper Extremity Recovery”. Front Hum Neurosci. (2016) 13:10:442. [7] Y. Bleyenheuft and A.M. Gordon. ”Precision grip in congenital and acquired hemiparesis: similarities in impairments and implications for neurorehabilitation”. Front. Hum.

Neurosci. (2014) 8:459. [8] C. Beyaert, R. Vasa, and G.E. Frykberg. ”Gait post-stroke: Pathophysiology and rehabilitation strategies”. Neurophysiol Clin. (2015) 45(4-5): 335-55. [9] A. Darekar, B.J. McFadyen, A. Lamontagne, and J. Fung. ”Efficacy of virtual reality-based intervention on balance and mobility disorders post-stroke: a scoping review”. J Neuroeng Rehabil. (2015) 10: 12:46. [10] C.L. Yang, R.A. Creath, L. Magder, M.W. Rogers, and S. McCombe Waller. ”Impaired posture, movement preparation, and execution during both paretic and nonparetic reaching following stroke.” J Neurophysiol. (2019) 121(4): 1465-1477 [11] H. Akremi, J. Higgins, R. Aissaoui, and S. Nadeau. ”Bilateral motor coordination during upper limb symmetric pushing movements at two levels of force resistance in healthy and poststroke individuals.” Hum Mov Sci. (2022) 81: 102913 [12] I.M. Barbosa, P.R. Alves, and Z.C. Silveira. ”Upper limbs’ assistive devices for stroke rehabilitation: a systematic review on design engineering solutions.” J Braz. Soc. Mech. Sci.

Eng. (2021) 43: 236 [13] A. Adomavičienė, K. Daunoravičienė, R. Kubilius, L. Varžaitytė, and J. Raistenskis. ”Influence of New Technologies on Post-Stroke Rehabilitation: A Comparison of Armeo Spring to the Kinect System”. Medicina (2019), 55: 98. [14] H.M. Qassim, and W.Z. Wan Hasan. ”A Review on Upper Limb Rehabilitation Robots”. Appl.

Sci. (2020), 10: 6976. [15] R. G. Lupu, N. Botezatu, F. Ungureanu, D. Ignat, and A. Moldoveanu, "Virtual reality based stroke recovery for upper limbs using leap motion," 2016 20th International Conference on System Theory, Control and Computing (ICSTCC), Sinaia, Romania, 2016. 295-299. [16] W.-S. Kim, S. Cho, J. Ku, Y. Kim, K. Lee, et al. ”Clinical Application of Virtual Reality for Upper Limb Motor Rehabilitation in Stroke: Review of Technologies and Clinical Evidence”. J. Clin.

Med. (2020), 9: 3369. [17] Reis, A. et al. ”Developing a System for Post-Stroke Rehabilitation: An Exergames Approach”.

In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. Users and Context Diversity. UAHCI 2016. Lecture Notes in Computer Science (2016), 9739.

Springer, Cham. [18] A. Dash, A. Yadav, A. Chauhan, and U. Lahiri. ”Kinect-Assisted Performance-Sensitive Upper

Limb Exercise Platform for Post-stroke Survivors”. Front Neurosci. (2019) 13: 228. [19] A. Scano, F. Molteni, and L. Molinari Tosatti. ”Low-Cost Tracking Systems Allow Fine Biomechanical Evaluation of Upper-Limb Daily-Life Gestures in Healthy People and PostStroke Patients”. Sensors (2019). 19(5): 1224. [20] M. Weiss Cohen, and D. Regazzoni. ”Hand rehabilitation assessment system using leap motion controller.” AI & Soc (2020) 35: 581–594. [21] L. Vismara, C. Ferraris, V. Votta, et al. ”Tele-Monitoring and Tele-Rehabilitation of the Hand in Hemiplegic Patients: A Preliminary Study”. In: Bettelli, A., Monteriù, A., Gamberini, L. (eds) Ambient Assisted Living. ForItAAL 2020. Lecture Notes in Electrical Engineering (2022) vol 884. Springer, Cham. [22] G. Fuertes Muñoz, R.A. Mollineda, J. Gallardo Casero, and F. Pla. ”A RGBD-Based Interactive

System for Gaming-Driven Rehabilitation of Upper Limbs”. Sensors (2019) 19(16): 3478. [23] C. Ferraris, et al. "Telerehabilitation of cognitive, motor and sleep disorders in neurological pathologies: the REHOME project" 2022 IEEE Symposium on Computers and Communications (ISCC), Rhodes, Greece (2022):1-6. [42] A. Gelineau, A. Perrochon, L. Robin, J.C. Daviet, S. Mandigout. ”Measured and Perceived Effects of Upper Limb Home-Based Exergaming Interventions on Activity after Stroke: A Systematic Review and Meta-Analysis.” Int J Environ Res Public Health. (2022) 19(15): 9112.

References

[1] J. Kim , T. Thayabaranathan , G.A. Donnan , et al. ” Global Stroke Statistics 2019 ”. International Journal of Stroke ( 2020 ), 15 ( 8 ): 819 - 838 .

[2] A. Bustamante , T. García-Berrocoso , N. Rodriguez , V. Llombart , M. Ribó , C. Molina , and J. Montaner . ” Ischemic stroke outcome: A review of the influence of post-stroke complications within the different scenarios of stroke care” . Eur J Intern Med . ( 2016 ) 29 : 9 - 21 .

[3] A.K. Boehme , C. Esenwa , and M.S. Elkind . ” Stroke Risk Factors, Genetics, and Prevention” . Circ Res . ( 2017 ) 120 ( 3 ): 472 - 495 .

[4] M. Katan and A. Luft . ”Global Burden of Stroke”. Semin Neurol . ( 2018 ) 38 ( 2 ): 208 - 211 .

[5] M.J.M. Ramos-Lima , I.C. Brasileiro , T.L. Lima , and P. Braga-Neto . ”Quality of life after stroke: impact of clinical and sociodemographic factors” . Clinics ( 2018 ) 73 : e418 .

[24] C. Ferraris , I. Ronga , R. Pratola , G. Coppo , et al. ” Usability of the REHOME Solution for the Telerehabilitation in Neurological Diseases: Preliminary Results on Motor and Cognitive Platforms .” Sensors ( 2022 ) 22 : 9467 .

[25] C. Ferraris , G. Amprimo, G. Masi , L. Vismara , et al. ” Evaluation of Arm Swing Features and Asymmetry during Gait in Parkinson's Disease Using the Azure Kinect Sensor” . Sensors ( 2022 ) 22 ( 16 ): 6282 .

[26] V. Cimolin , L. Vismara , C. Ferraris , G. Amprimo, G. Pettiti , et al. ” Computation of Gait Parameters in Post Stroke and Parkinson's Disease: A Comparative Study Using RGB-D Sensors and Optoelectronic Systems .” Sensors ( 2022 ) 22 ( 3 ): 824 .

[27] G. Amprimo , G. Masi , L. Priano , et al. ” Assessment Tasks and Virtual Exergames for Remote Monitoring of Parkinson's Disease: An Integrated Approach Based on Azure Kinect .” Sensors ( 2022 ) 22 : 8173 .

[28] G. Marchesi , G. Ballardini , L. Barone , P. Giannoni , et al. ” Modified Functional Reach Test: Upper-Body Kinematics and Muscular Activity in Chronic Stroke Survivors .” Sensors ( 2021 ) 22 ( 1 ): 230 .

[29] A. Prat-Luri , P. Moreno-Navarro , J.A. García , D. Barbado , F.J. Vera-Garcia , J.L.L. Elvira . ”Do Initial Trunk Impairment, Age, Intervention Onset , and Training Volume Modulate the Effectiveness of Additional Trunk Exercise Programs after Stroke? A Systematic Review with Meta-Analyses .” Int J Environ Res Public Health. ( 2020 ) 17 ( 23 ): 8714 .

[30] R. Bruchez , M. Jequier Gygax , S. Roches , J. Fluss , D. Jacquier , et al. ” Mirror therapy in children with hemiparesis: a randomized observer-blinded trial” . Dev Med Child Neurol. ( 2016 ) 58 ( 9 ): 970 - 8 .

[31] Azure Kinect DK . URL: https://azure.microsoft.com/it-it/products/kinect-dk/ (accessed on September 2023 )

[32] Liu , Z. 3D Skeletal Tracking on Azure Kinect . 2019 . URL: https://www.microsoft.com/enus/research/uploads/prod/2020/01/AKBTSDK.pdf (accessed on September 2023 ).

[33] L.F. Yeung , Z. Yang , K.C. Cheng, D. Du, and R.K. Tong. ” Effects of camera viewing angles on tracking kinematic gait patterns using Azure Kinect, Kinect v2 and Orbbec Astra Pro v2 .” Gait Posture ( 2021 ) 87 : 19 - 26 .

[34] M. Kudlac , J. Sabol , K. Kaiser , C. Kane , and R.S. Phillips . ” Reliability and Validity of the Berg Balance Scale in the Stroke Population: A Systematic Review . ” Phys. Occup. Ther. Geriatr . ( 2019 ) 37 : 196 - 221

[35] G. Verheyden , A. Nieuwboer , J. Mertin , R. Preger , C. Kiekens and W. De Weerdt. ” The Trunk Impairment Scale: a new tool to measure motor impairment of the trunk after stroke . ” Clin Rehabil . ( 2004 ) 18 ( 3 ): 326 - 34 .

[36] Z. Sawacha , E. Carraro, P. Contessa , A. Guiotto , S. Masiero and C. Cobelli . ” Relationship between clinical and instrumental balance assessments in chronic post-stroke hemiparesis subjects .” J Neuroeng Rehabil . ( 2013 ) 10 : 95 .

[37] J.M. Soucie , C. Wang , A. Forsyth , S. Funk , M. Denny , K.E. Roach and D. Boone . ” Range of motion measurements: reference values and a database for comparison studies .” Haemophilia. ( 2011 ) 17 ( 3 ): 500 - 7 .

[38] D.G. Kendall. “ A survey of the statistical theory of shape” Statist . Sci. ( 1989 ) 4 : 87 - 120 .

[39] M. Rodríguez-Hernández , B. Polonio-López , A.I. Corregidor-Sánchez , J.L. Martín-Conty , A. Mohedano-Moriano , J.J. Criado-Álvarez. ” Effects of Specific Virtual Reality-Based Therapy for the Rehabilitation of the Upper Limb Motor Function Post-Ictus: Randomized Controlled Trial .” Brain Sci. ( 2021 ) 11 ( 5 ): 555 .

[40] N. Norouzi-Gheidari , A. Hernandez , P.S. Archambault , J. Higgins , L. Poissant , D. Kairy . ”Feasibility, Safety and Efficacy of a Virtual Reality Exergame System to Supplement Upper Extremity Rehabilitation Post-Stroke: A Pilot Randomized Clinical Trial and Proof of Principle .” Int J Environ Res Public Health. ( 2019 ) 17 ( 1 ): 113 .

[41] P.P.B. Henrique , E.L. Colussi , A.C.B. De Marchi. ” Effects of Exergame on Patients' Balance and Upper Limb Motor Function after Stroke: A Randomized Controlled Trial .” J Stroke Cerebrovasc Dis. ( 2019 ) 28 ( 8 ): 2351 - 2357 .