=Paper=
{{Paper
|id=Vol-3616/paper3
|storemode=property
|title=A virtual reality game for hand rehabilitation after stroke
|pdfUrl=https://ceur-ws.org/Vol-3616/paper3.pdf
|volume=Vol-3616
|authors=Amal Bouatrous,Nadia Zenati,Abdelkrim Meziane,Chafia Hamitouche
|dblpUrl=https://dblp.org/rec/conf/rif/BouatrousZMH23
}}
==A virtual reality game for hand rehabilitation after stroke==
https://ceur-ws.org/Vol-3616/paper3.pdf
A virtual reality game for hand rehabilitation after
stroke
Amal Bouatrous1,4,† , Nadia Zenati2,† , Abdelkrim Meziane3,† and
Chafiaa Hamitouche4,†
1
Computer Science Department USTHB University, BP 32, Bab Ezzouar, 16111, Algiers, Algeria
2
Centre for Development of Advanced Technologies (CDTA), Algiers, Algeria
3
Research Center on Scientific and Technical Information (CERIST) 05, Rue des 3 frères aissiou - Ben Aknoun
-Algiers,Algeria
4
IMT-Atlantique Bretagne- Pays de la Loire, Brest, France. LaTIM, INSERM, UMR 1101, Brest, France
Abstract
This paper presents a serious game represented by a virtual environment, that is intended for the
functional rehabilitation of the hand of stroke patients. The game was designed in close collaboration
with specialized clinicians, and developed using a Leap Motion controller as a hand tracking device and
the Unity 3D game development platform. The game simulates fruit harvesting in order to train the
movement of the bidigital hand grip. The game has been enjoyed by patients undergoing functional
hand rehabilitation.
Keywords
Serious Games, Virtual Reality, Hand Rehabilitation, Leap Motion Controller, Stroke
1. Introduction
Functional rehabilitation in rehabilitation centers requires major improvements in terms of
rehabilitation tools, and this is due to the fact that conventional clinical protocols for functional
rehabilitation suffer from boredom, which affects patients’ motivation and consequently the
recovery of their motor functions. Virtual rehabilitation is one of the new therapies that have
been proposed to enhance functional rehabilitation protocols. It is a virtual reality therapy
that involves virtual environments, which is a software system that simulates real-world tasks
in rehabilitation therapy for stroke patients and helps them to effectively assess and measure
patient performance[1]. Serious games are increasingly used for motor rehabilitation[2] after
stroke. They involve incorporating gamification into rehabilitation exercises. Gamification is
the process of applying codes and mechanisms from the world of video games to areas where
they were not intended. Thus, gamification of rehabilitation exercises (exergames) consists in
making rehabilitation exercises enjoyable. In order to play, participants must have a physical
activity or exercise, which requires them to move the body parts involved in the exercise in
RIF 2023: The 12th Seminary of Computer Science Research at Feminine Constantine, Algeria, March 9, 2023
†
These authors contributed equally.
Envelope-Open amalbouatrous@gmail.com (A. Bouatrous); nzenati@cdta.dz (N. Zenati); ameziane@cerist.dz (A. Meziane);
chafiaa.hamitouche@imt-atlantique.fr (C. Hamitouche)
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
order to accomplish the required tasks. In this paper, we present a serious game that simulates a
traditional clinical exercise for hand motor rehabilitation. The game was designed and developed
in close collaboration with specialized clinicians, and it was clinically tested on post-stroke
patients with hand motor disabilities.
2. Related Works
Upper limb motor deficit is a common symptom of stroke[3], affecting the quality of life of these
patients, and improving upper limb function remains a major component of rehabilitation for
stroke patients. Therefore, a variety of interventions have been developed for this purpose[4]. In
recent years, commercial virtual reality (VR) consoles, also known as exergames, have emerged
as a new form of intervention for stroke rehabilitation. These systems allow individuals to
participate in tasks ranging from exercises to performing meaningful activities in a virtual
environment[5]. S.-H. Yang et al.[6] proposed a virtual reality game for bilateral hand training
for home rehabilitation, the movement of the less affected hand was measured by a hand sensor
glove using a flexion sensor. The affected hand is then trained by the exoskeleton using the
measured hand movements. M. G. Lansberg et al.[7] showed that the Smart Glovea,which
is a commercially available noninvasive system consisting of a wearable glove and a tablet,
includes 45 games that mimic real-life activities or are based on invented scenarios. Each game
is designed to focus on specific sets of hand and wrist movements, may help increase the dose
of outpatient rehabilitation. A. de los Reyes-Guzmán et al.[8] presented seven therapeutic
virtual reality applications for the rehabilitation of upper limb manipulative abilities in patients
with neurological disorders, manipulated using the Leap Motion Controller. Q. U. Ain et al.[9]
showed that repetitive use of the hemiparetic upper limb through Xbox Kinect-based upper
limb rehabilitation training, in addition to conventional therapy, has promising potential to
improve upper limb motor function in stroke patients. K. N. K. Fong et al.[10] presented a
TS-VR program based on seven general hand function tasks used in activities of daily living that
require upper extremity movement, using a Leap Motion Controller VR device and the Unity3D
program for training distal hand function. M. Weiss Cohen and D. Regazzoni.[11] proposed
an approach to tracking hand rehabilitation exercises for stroke patients using a leap motion
controller as a hand tracking device. Pilatásig et al.[12] presented an interactive system for
hand and wrist rehabilitation using the leap motion device and Unity3D software.
Our system focuses on the development of a low-cost interactive tool, consisting of a simple
game based on a usual activity that takes place in a familiar environment for post-stroke patients,
in order to attract them and maintain their motivation.
3. Materials and methods
3.1. The proposed serious game
By working closely with clinicians specialized in functional rehabilitation, we have selected a
fine hand movement which is the Bidigitale grasp, that involves grasping relatively small objects
between the thumb and the index finger of the hand. We chose a fruit harvesting scenario to
create a virtual environment that simulates this exercise, which appears to be a familiar activity
for all categories of patients. Thus, in the game, the patient is encouraged to harvest fruit from
their trees and place them in a basket, grasping them between the thumb and index finger of
their hand.
3.2. Hardware and Software
The main objective of this work is to propose a non-invasive and low cost tool for motor
rehabilitation of the hand, in particular the training of a fine hand movement which is the
bidigital grasp. Due to its low cost, small size compared to other devices, and especially its ease
of use, lack of markers, and exciting aspects of its technology[13], the Leap Motion Controller
LMC optoelectronic system shown in Figure 1 was selected. It captures the movement of both
hands and controls a virtual environment. Thus, the LMC captures the fine movements of
the hand, so it is potentially more suitable for training fine movements of the upper limb and
hand functions[13]. Because of its ease of use and extensive documentation[14], the Unity 3D
platform, which enables the development of content for virtual reality and the creation of two-
and three-dimensional games, was used to build the virtual environment in which the game
takes place.
Figure 1: The Leap Motion device.
3.3. Gameplay
The game scenario takes place in a virtual grove made up of fruit trees bearing small fruits such
as cherries (see Figure 2). Due to the limited field of view of the LMC, we considered a set of
nine fruits per tree. In order to play, the player must perform the following tasks:
1. A pointing task: consists of moving the hand towards the fruit and pointing it with the
index finger, in order to be able to pick it up.
2. A grasping task: consists of bringing the two fingers of the thumb and index finger as
close together as possible in order to pick up the fruit that the index finger is pointing to,
by grasping it between the two fingers concerned.
3. A deposit task: consists of moving the thumb and index finger away from each other, in
order to deposit the fruit that is between the two fingers in a basket by releasing it from
the two fingers involved.
The interaction model is avatar-based, so that the user’s hand is represented in the virtual
environment by a virtual hand (see Figure 3).
Figure 2: The virtual scene representing the game.
3.4. Game Settings
To successfully complete the game, the patient must be able to grasp the fruit between the
thumb and index finger of their hand after reaching it, and then drop it after grasping it. Thus,
as previously stated, in order to grasp the fruit, the patient must bring their two fingers together
(see Figure 4), and to release it, they must spread them apart (see Figure 5). This movement
was measured in our game by the Euclidean distance between the tips of the two fingers. The
distance must be greater than a certain predefined threshold to grasp the fruit, and it must be
less to release it. The difficulty of the game is determined by this threshold.
The game’s difficulty level can be tailored to the patient’s initial motor status. Before starting to
play, the patient calibrates the game by performing the pinch gesture (see Figure 6). Behind
the scenes, the system computes the minimum and maximum amplitudes of the patient’s hand
movement. These values will then be used to create thresholds against which the patient’s
Figure 3: Representation of the user’s hand in the game scene.
Figure 4: The task of grasping the fruit.
performance during the game will be compared. The entire calibration procedure lasts 30
seconds.
Figure 5: The task of dropping the fruit.
Figure 6: The calibration scene of the game.
3.5. Exercises History
We considered some parameters to measure during the game in order to quantify the patient’s
performance. The Table 1 depicts these parameters. Indeed, each patient has their own file in a
database, at the end of the game, these parameters are recorded in their file, and the therapist
can consult these data to determine the patient’s level of motor skills.
Table 1
Game Settings
Parametrey Meaning
Time The time that elapses between the beginning and the end of the game.
For each success, a reward is given in the form of a number of coins that appear
Score_S
in the scene(see Figure 7),the score is the sum of all the awards given.
Score_RF Represents the number of fruits harvested
A list of nine items, each linked to a fruit and representing the patient’s
motion skills
performance in completing the tasks required to harvest each associated fruit.
Figure 7: The representation of rewards in the game.
4. Preliminary Tests and Discussion
The usability and acceptability of the proposed game were tested on a group of post-stroke
patients with hand motor impairment. The LMC hand motion sensor was connected to a PC
computer (Intel(R) Core™i7-3520 M CPU @ 2.90 GHz 290 GHz, NVIDIA GeForce 840M graphics
Figure 8: The Hardware configuration of the game.
card, 12GB RAM). The complete hardware architecture of the system is shown in Figure 8. The
patients who played the game found it enjoyable and beneficial to their hand rehabilitation.
Patients in particular, reported that the proposed game’s features were very well integrated,
and that it was an innovative and motivating tool for them, as it can help them better engage
in their rehabilitation sessions. However, because the number of patients who participated in
the clinical evaluation was so small, more patients must be recruited to properly assess the
acceptability and the usability of the proposed game. Figure 9 shows a patient playing the game.
Indeed, the selection of patients who can use the game was conducted by the therapists.
Figure 9: A post-stroke patient playing the game.
5. Conclusion and Future Works
In this article, we presented a serious game for the functional rehabilitation of the hand. It is a
simulation of a clinical exercise for a fine hand movement, namely the Bidigitale grip. The game
was designed and developed in close collaboration with clinicians specialized in functional
rehabilitation. A first evaluation of the acceptability of the tool on patients has been performed.
The game was very well received by the patients. They indicated that it was a new tool that
could help them recover their functionality. Nonetheless, the clinical evaluation sample was
too small to objectively assess the tool’s acceptability and usability by patients. The addition of
new patients is required, which we intend to do in the near future.
Acknowledgment
The authors would like to thank everyone who contributed to the completion of this work.
Beginning with the clinicians of the Douera University Hospital in Algiers, Pr Houria Kaced, Dr
Amine Brahimi, and Dr Sara Ait Ziane, with whom they collaborated throughout the game’s
setup process. They would also like to thank Dr. Olivier Rémy-Néris, Dr. Myriam Thiebaut-
Clerin, and Ms. Sandrine Chedet of the CHRU of Brest in France. They would like to thank
the patients who took part in the device’s clinical evaluation. Finally, they thank Dr. Chayma
Zatout (CERIST) for her participation in the system’s evaluation on patients, as well as the great
efforts she made to accompany them during this phase, assisting them.
References
[1] N. F. Rakib, N. H. Mahmood, N. Ramli, N. A. Zakaria, and M. A. A. Razak, “Preliminary
Results of Hand Rehabilitation for Post Stroke Patient using Leap Motion-based Virtual
Reality,” in 2020 IEEE Student Conference on Research and Development (SCOReD), Batu
Pahat, Malaysia, Sep. 2020, pp. 259–262. doi: 10.1109/SCOReD50371.2020.9250985.
[2] F. Pereira, S. Bermúdez i Badia, R. Ornelas, and M. S. Cameirão, “Impact of game mode in
multi-user serious games for upper limb rehabilitation: a within-person randomized trial
on engagement and social involvement,” J NeuroEngineering Rehabil, vol. 16, no. 1, p. 109,
Dec. 2019, doi: 10.1186/s12984-019-0578-9.
[3] J.-S. Kwon, M.-J. Park, I.-J. Yoon, and S.-H. Park, “Effects of virtual reality on upper extremity
function and activities of daily living performance in acute stroke: A double-blind random-
ized clinical trial,” NRE, vol. 31, no. 4, pp. 379–385, Nov. 2012, doi: 10.3233/NRE-2012-00807.
[4] J.-H. Shin, S. Bog Park, and S. Ho Jang, “Effects of game-based virtual reality on health-related
quality of life in chronic stroke patients: A randomized, controlled study,” Computers in
Biology and Medicine, vol. 63, pp. 92–98, Aug. 2015, doi: 10.1016/j.compbiomed.2015.03.011.
[5] A.-V. Nguyen et al., “Virtual reality exergaming as adjunctive therapy in a sub-acute stroke
rehabilitation setting: facilitators and barriers,” Disability and Rehabilitation: Assistive
Technology, vol. 14, no. 4, pp. 317–324, May 2019, doi: 10.1080/17483107.2018.1447608.
[6] S.-H. Yang et al., “An Instrumented Glove-Controlled Portable Hand-Exoskeleton for
Bilateral Hand Rehabilitation,” Biosensors, vol. 11, no. 12, p. 495, Dec. 2021, doi:
10.3390/bios11120495.
[7] M. G. Lansberg et al., “Home‐based virtual reality therapy for hand recovery after stroke,”
PM&R, p. pmrj.12598, May 2021, doi: 10.1002/pmrj.12598.
[8] A. de los Reyes-Guzmán et al., “RehabHand: Oriented-tasks serious games for upper limb
rehabilitation by using Leap Motion Controller and target population in spinal cord injury,”
NRE, vol. 48, no. 3, pp. 365–373, Apr. 2021, doi: 10.3233/NRE-201598.
[9] Q. U. Ain et al., “Additional Effects of Xbox Kinect Training on Upper Limb Function in
Chronic Stroke Patients: A Randomized Control Trial,” Healthcare, vol. 9, no. 3, p. 242, Feb.
2021, doi: 10.3390/healthcare9030242.
[10] K. N. K. Fong, Y. M. Tang, K. Sie, A. K. H. Yu, C. C. W. Lo, and Y. W. T. Ma, “Effects of
task-specific virtual reality training using a Leap Motion Controller on hemiparetic upper
extremity in patients with stroke,” In Review, preprint, Oct. 2020. doi: 10.21203/rs.3.rs-
55924/v2.
[11] M. Weiss Cohen and D. Regazzoni, “Hand rehabilitation assessment system using leap
motion controller,” AI & Soc, vol. 35, no. 3, pp. 581–594, Sep. 2020, doi: 10.1007/s00146-019-
00925-8.
[12] M. Pilatásig et al., “Interactive System for Hands and Wrist Rehabilitation,” in Proceedings
of the International Conference on Information Technology & Systems (ICITS 2018), vol. 721,
Á. Rocha and T. Guarda, Eds. Cham: Springer International Publishing, 2018, pp. 593–601.
doi: 10.1007/978-3-319-73450-7_56.
[13] M. Iosa et al., “Leap motion controlled videogame-based therapy for rehabilitation of elderly
patients with subacute stroke: a feasibility pilot study,” Topics in Stroke Rehabilitation, vol.
22, no. 4, pp. 306–316, Aug. 2015, doi: 10.1179/1074935714Z.0000000036.
[14] M. Alimanova et al., “Gamification of Hand Rehabilitation Process Using Virtual Real-
ity Tools: Using Leap Motion for Hand Rehabilitation,” in 2017 First IEEE International
Conference on Robotic Computing (IRC), Taichung, Taiwan, Apr. 2017, pp. 336–339. doi:
10.1109/IRC.2017.76.