In this study, we present a virtual reality system for upper limb rehabilitation inspired in the Box and Block Test of Manual Dexterity. The system is designed to recover gross and fine disorders in upper extremities. To evaluate our system, ten children trained with the system. The system captured time of completion of grasping and moving each element. Outcomes indicates that is is possible to track the progression of each subject with quantitative data. In a near future, we will test our system in children with rare diseases that difficulty mobility in upper extremities.
The main goal of physical rehabilitation is to help people to return to the functional performance of
everyday activities by recovering lost motor skills. To do so, medical specialists seek to promote motor
learning, defined as "a set of internal processes associated with practice or experience that lead to
relatively permanent changes in the capacity of a motor skill." [
Virtual rehabilitation must consider four fundamental variables of motor learning: observational
learning, practice, augmented feedback and motivation [
Virtual rehabilitation requires high-intensity, repetitive and task-specific training [
Literature shows that Leap Motion can be used to meet specific needs of for four specific
psychological domains such as: autism spectrum disorder, attention-deficit/hyperactivity disorder,
dementia, and mild cognitive impairment [
Based on the International Classification of Functioning, Disability and Health (ICF) [10] from the World Health Organization we focus on Fine hand use (d440), that refers to performing the coordinated actions of handling objects, picking up, manipulating and releasing them using one's hand, fingers and thumb. The choice of the exercises and the validation of the functionality requirements were carried out by two specialists in physical therapy and a pediatric neurologist. 3.2.
The system addresses a 2D rehabilitation environment that controls the execution of movements in the upper limbs through the Leap Motion Controller (LMC) device. LMC is an optical device without markers, with sub-millimeter precision, oriented to gestural movement control. It detects movements made with the hands, fingers and even objects that are around its field of vision. LMC allows gesture and position tracking for use in realistic stereo 3D interaction systems especially concerning direct selection of stereoscopically displayed objects [11]. LMC is a small, rectangular device (13 mm × 13 mm × 76 mm) that weights 45 g. LMC consists of three IR (wavelengths 850 nm emitters) and two IR cameras. It’s dual platform (Macintosh/Windows), connects to a computer via USB 3.0 connection. It has a full-functioning SDK that allow to rebuild the 3D scene and track the positions of the hands and fingers. The supports a variety of programming languages, such as C++, Python, Unity, Unreal [12]. The effective range of the controller extends from approximately 25–600 mm above the device [6].
The system was designed taking into account the characteristics of motor learning: observational learning, feedback, motivation and practice. The objective of the system is to prepare a pizza with two ingredients (cheese and meet). The user must pick up the ingredient and brought it to the pizza, one by one. For each ingredient placed, a point will be added. There is a special sound whether there is a hit or a miss. Given that each subject would have a different condition, the system will be adapted to the different needs of the patient. The system allows to configure the hand to use (left or right hand) and allows to configure the number of ingredients that are displayed (Figure 1).
The system's architecture has four layers. The first two layers have the purpose of "motion capture" and integrate the SDK of the LMC and its respective drivers. LMC was integrated with the Ultra Leap Unity Packages plugin and Ultra Leap Hand tracking software (V5.2+). The third layer consider three modules for interaction. 1) Service Provider, for referring the position of the virtual hands; 2) Physic Engine, it gives physical and cinematic properties to the game objects, 3) Animator Component: gives the construction of virtual hands in avatar format. Third layer also has UNITY scripts that serve as a listener, sound manager, game logic management and data storage. Finally, in the last layer we have the database. Also, to develop a realistic environment, the display screen was placed along a table to represent a kitchen environment (Figure 2).
A simple structure with aluminum bars was designed and built to position the LMC (Figure 3). After several tests, the result was that the LMC should be located 80 cm above the display screen. 3.4.
The participants were composed of children ranging from 9 to 13 year-old (six girls and four boys, with a mean age of 11±1.56). We evaluate the completion time for each element, the global completion time and the assessment. The inclusion criteria are the following: 1) without cognitive deficits; 2) without motor disorders; 3) with previous experience with Virtual Environments (VE); and 3) no knowledge in Computer Graphics or two-dimensional rehabilitation video games. Informed consents and assets were obtained from the parents or guardians with the agreement to participate in the research. The study was approved by the ethical committee and accomplished the ethical standards of the Declaration of Helsinki (DoH). The study was performed at the Center for Research, Technology Transfer and Software Development in a public university. Before the beginning of the sessions, the researcher explained the instructions to perform the exercise correctly. During the session the participant must try to place all 20 elements on the pizza. The system captures the interaction process with each element: the time it takes to pick it up, the number of failures and, the time it takes to leave it on the pizza.
The outcomes obtained in this study showed that it is possible to capture the time of completion of grasping objects and moving objects. Having a record of this type would allow the medical specialist to see the progress quantitatively in terms of functionality in each session and to be able to make the necessary adjustments in terms of the amount of practice required. Our proposal is intended to be a complementary tool for physical rehabilitation. Our work was inspired by the Box and Block Test of Manual Dexterity [14], where subjects are asked to move as many blocks from one side of a box to another within one minute. The main difference with the test is that we focus on completing the movements without time pressure and we seek to analyze the improvement in each movement.
There are previous experiences with the use of Leap Motion for fine motor rehabilitation therapies. The work of Postolache et al. [6] evaluates the usability of the game which focuses on the messages on the screen, its location, the ease of follow-up, among other things. Fluet et al. [8] present a home-based virtual rehabilitation system oriented to arm, wrist and hand where results showed adherence to home exercise. Furmanek et al. [9] established the correlation structure between the carrying and grasping components remains similar if a reaching-to-grasp movement is performed in a physical environment and in a virtual environment. Although our work is also oriented to motor hand rehabilitation our proposal focuses on describing the game as an training tool.
This work showed a virtual reality as a support for upper limb motor rehabilitation in children. The development had the constant advice of medical specialists in physical therapy who identified the activities that needed to be promoted according to the ICF international classification. The development considered the required attributes of virtual rehabilitation systems for therapies: observational learning, feedback, repetition and motivation. With respect to other immersive-type systems, Leap Motion has the advantage that it does not cause discomfort in addition to its characteristics that there is no physical contact with the skin, which is an advantage when it comes to of children The limitations of the study are given by the limited number of people who tested the system. As future work we will carry out the validation with the children, expanding the sample.
This contribution was funded by the Universidad Nacional de San Agustin de Arequipa under the contract IB-42-2020-UNSA- project “Virtual Rehabilitation System (VR) for motor and cognitive improvement in children with Epileptic Encephalopathy. CEPIVIRT.” The authors thank the Clínica San Juan de Dios de Arequipa and its specialists in the area of physical therapy, physical rehabilitation and pediatric neurology. 7. References Review”. Journal of stroke and cerebrovascular diseases : the official journal of National Stroke Association, 31(1), 106174, 2021. [6] G. Postolache, F. Carry, F. Lourenço, D. Ferreira, R. Oliveira, P. S. Girão, O. Postolache. “Serious Games Based on Kinect and Leap Motion Controller for Upper Limbs Physical Rehabilitation”. In Modern Sensing Technologies, 147-177, Springer, Cham, 2019. [7] J. M. Oliveira, R. C. Fernandes, C. S. Pinto, P. R. Pinheiro, S. Ribeiro, V. H. de Albuquerque. “Novel Virtual Environment for Alternative Treatment of Children with Cerebral Palsy”. Computational intelligence and neuroscience, 2016. [8 ]G. G. Fluet, Q. Qiu, J. Patel, A. Cronce, A. S. Merians, S. V. Adamovich. “Autonomous Use of the Home Virtual Rehabilitation System: A Feasibility and Pilot Study”. Games for health journal, 8(6), 432–438, 2019. https://doi.org/10.1089/g4h.2019.0012 [9] M. P. Furmanek, L. F. Schettino, M. Yarossi, S. Kirkman, S. V. Adamovich, E. Tunik. “Coordination of reach-to-grasp in physical and haptic-free virtual environments”. Journal of neuroengineering and rehabilitation, 16(1), 1-14, 2019. [10] World Health Organization.. International Classification of Functioning, Disability, and Health:
Children & Youth Version: ICF-CY. World Health Organization, 2007. [11] F. Weichert, D. Bachmann, B. Rudak, D. Fisseler. “Analysis of the accuracy and robustness of the leap motion controller”. Sensors, 13(5), 6380-6393, 2013. [12] LeapMotion,https://www.leapmotion.com/ [13] V. Mathiowetz, S. Federman, D.Wiemer,. Box and block test of manual dexterity: norms for 6–19 year olds. Canadian Journal of Occupational Therapy, 52(5), 241-245, 1985.