The current demographic ageing in Europe is the result of a relevant economic, social, and medical development. Nevertheless, at the same time, it is also leading to a significant increase in the demand for long-term care (LTC), especially for seniors. One viable way to offer qualified cares at home, while at the same time containing costs, is to exploit digital technologies as enablers of a constant interaction between seniors and assisting personnel. In particular, (video) games have already been identified as a viable way to foster motivation and engagement in the long term. While technical solutions to provide at home LTC has already been proposed, the scientific community is still working on general methodologies to streamline the process on the caretaker's side. In this paper, we focus on a software application to design and propose physiatric exercises from a remote location. These exercises can be extremely tailored on the requirements of each patient and can be monitored in an automated way exploiting off-the-shelf gaming technologies such as Microsoft Kinect. The proposed solution aims to shorten the feedback loop between patient and caretaker in order to increase the quality of the therapy and improve the recovery time.
Author Keywords HCI; visual interfaces; healthcare; long-term care; LTC.
INTRODUCTION
It has been estimated that, by 2025, people over 60
worldwide will be 1.2 billion, and by 2050 they will reach 2
billion (in 2000 they used to be ‘only’ 600 million). Also, in
2050, in Europe, the number of over 60 will equal the 40%
of the total population, and the 60% of the population in
GHItaly18: 2nd Workshop on Games-Human Interaction, May 29th, 2018,
Castiglione della Pescaia, Grosseto (Italy)
Copyright © 2018 for the individual papers by the papers' authors. Copying
permitted for private and academic purposes. This volume is published and
copyrighted by its editors.
working age – that is to say 15–64 years old [
Designing a serious game for rehabilitation [
With the above goal in mind, this paper is focused on proposing a visual interface which will be viable for a medical operator and where an exercise can be easily defined while taking into account patient’s monitoring data. This exercise can then be deployed remotely as a physiatric serious game and be part of a long-term rehabilitation therapy.
RELATED WORK
The idea of using games to foster medical therapies has
already been accepted long before computers and electronic
devices became a way to convey entertainment [
In [
Other contributions try to be more general and proposes full
frameworks [
A more comprehensive solution to host and manage
exergames, and to monitor patients’ performance from a
remote location is represented by Care@Home [
The Care@Home Project The Care@Home project has been developed to allow remote interaction, on a daily basis, with elders (especially those living alone) requiring LTC. Care@Home integrates a number of sensor devices in the patient environment and allows to assign exercises, verify progresses in mobility, and monitor health/environmental parameters. In particular, the Kinect gaming device has been exploited as a movement and body measurement sensor as well as for its easy integration in a gaming platform. A set-top-box is installed in the elder home to manage the Kinect together will all other environmental sensors and to interface the household with a caretaker’s control center (see Fig. 1). Care@Home has been implemented as a distributed architecture whose purpose is to provide an efficient, reliable, and scalable real-time communication between sensing equipment located in the home of each patient and a central control system. A schema of the hi-level architecture is reported in Fig. 2. In the picture, we can observe that this architecture is composed by many components; some of them are located inside the caretaker’s organization datacenter (shaded area), while others are dislocated where convenient: either in the caretaker’s office or in the patient’s household. In particular, on the patient side, a DeviceProxy is in charge to manage the connection and collect data from the local sensors. Moreover, the DeviceProxy is storing all the information related to the exercises to be “dispensed” to the patient.
When an exercise is performed, the Kinect comes into play and starts tracking the patient’s position. A sequence of positions is presented on the screen as dots overlapped to the avatar reporting the current body position (see Fig. 3). Each exercise requires from the patient to reach a sequence of positions with her body. If, while performing an exercise, the tracking is reported too off of the required asset, the dots are drawn in red, to provide immediate visual feedback and an alarm is raised on the screen; see Fig. 4 for an example. Each exercise may implement its own checking policy and focus on monitoring specific postures, which may be critical for a given patient.
CREATING EXERCISES As already mentioned, an important feature required from a remote rehabilitation service is a convenient way to define new exercises tailored on each patient’s needs and performances. When designing such a tool and its interface, we must also take into account that the creation of exercises will be performed by a medical operator; this implies that any description methodology requiring to write code is not viable. Moreover, the physiotherapist must be able to perform all operations of storage, retrieval, and deployment of exercises without asking the assistant of technical staff. The proposed solution In order to create an efficient yet very intuitive interface, we resolved to use a Natural User Interface (NUI) also on the caretaker side. As a consequence, Microsoft Kinect is used to let the caretaker define the tracking positions of each exercise. The caretaker can track her own body to define an exercise and then edit the result. This approach proved to be very well accepted by medical staff because physiotherapists are already used to show to their patients how to perform each exercise. The only learning curve required from the caretaker is about the interface for editing and deployment, which is used in a second stage. Once an exercise is decorated with all required metadata (e.g., number of repetitions, reference patient, and description), it can be stored in an online database located in the caretaker’s datacenter. From there, the PhysioServ application server will take care to push the exercise toward the correct DeviceProxy.
The caretaker’s database can also be used to retrieve existing exercises and tailor them for different patients due to new, hopefully improved, conditions.
Editor Architecture The exercise editor has been implemented using Windows Presentation Foundation (WPF): a Kinect-compatible framework for visual applications distributed by Microsoft. In WPF, the interface is defined by means of an XML file (taking xaml as extension). A specific class, usually implemented in C#, manages the interface as described in the xaml file and bridges the GUI to external elements and the Kinect hardware. The hi-level architecture of the exercise editor is reported in Fig. 5. In this architecture, the Main Window class (with its xaml file) is responsible to manage the main application panel. From within this panel, we can initialize all data structures, perform data management tasks, and drive Kinect sensors. The Main Window interface exposes standard data management functionalities for exercises (new, open, save, and so forth) as well as players control (start, stop, record). The Metadata Editor class is responsible to manage all additional data for the exercise. These data include both contextual information, such as how many repetitions are required and when the exercise should be performed, as well as generic information such as medical notes from the caretaker. Screenshots of the Metadata Editor windows are proposed in Fig. 6 and Fig. 7. In the Metadata Editor, it is also possible to select single joints (Fig. 7) in order to provide additional directions for the exercise software. These directions may be about a joint which is not supposed to be moving or a limb bending which should not exceed a given angle, as in the alarm raised in Fig. 4.
Both Metadata Editor and Main Window use two service classes designed to store data about the whole exercise and each single joint, respectively. Exercise and Joints will be serialized and stored in a backend database for deployment and/or later retrieval.
The last class in our architecture is Player. The Player class takes care of all functionalities related to capturing the caretaker’s movement using Kinect and replaying the edited exercise before saving it in the backend database. Moreover, the Player can also retrieve an existing exercise from the database and let the caretaker create a customized version. Figure 8 shows the Player window during the recording of an exercise. In the figure, it is possible to see a left panel with a real-time tracking of the caretaker body while performing the exercise (the red border means that recording is in progress) and a right-hand panel with a hierarchical representation of the exercises database in the backend. The exercise database is available from the Player window in order to ease the process of browsing and playing out existing exercises. The database is represented in a hierarchical way with a tree branching based on the interested limbs or body part for each exercise. This data organization proved to allow for quick retrieval from medical staff without requiring to fill an explicit search form. The last element of the Payer window, in its lower section, is a selector to pick a subset of the joints which are important for monitoring. This way, the recording will be limited to the selected subset and the rest of the body will not be used to evaluate the patient’s performance. Once finished the recording, the Player window can be closed to perform further processing and deployment from the Main Window.
Creating and Managing Exercises The workflow, while creating an exercise, can be divided into a sequence of steps. During the first step, the caretaker will record the required movement, select the relevant joints, and add all important metadata to the exercise. A serialized version of the Exercise instance can then be saved locally in a binary format. This local save will be then pushed to the backend database and indexed using the metadata provided by the caretaker.
Once in the backend database, the information will be managed by the PhysioServ application server: basing on the indexing metadata, each exercise will be pushed to one (or more) set-top-boxes in patients’ households when connection will take place. We cannot assume the client side of the architecture is always connected and sync is performed based on opportunity. When the exercise is deployed on a set-top-box, the patient will see the Exercise de-serialized and instantiated inside a rehabilitation exergame.
CONCLUSION AND FUTURE WORK In this paper we discussed about exergames to support rehabilitation of patient on LTC. While it is already very clear the importance of (video) games for rehabilitation, we are still missing a complete and streamlined solution to dispense gaming activity as part of a therapy. In the context of Care@Home, an infrastructure for remote administration of LTC at home, we addressed here the problem of defining exercises with a functional and easy-to-use application which is feasible for non-tech-savvy caretakers. This application exploits Kinect on the medical personnel’s side to define an exercise via body tracking and let the user decorate it with metadata useful for archiving and deployment.
The proposed solution is currently under testing and preliminary unstructured feedback from medical staff seems promising.
As a future extension, we are planning to implement an adaptation layer to let each exercise to be instantiated inside a generic game, as in the early prototype shown in Fig. 9. In this prototype, the player/patient is required to raise alternatively the right and left leg for a given number of times. Our intention is to insert several kinds of leg-related exercises into this game without bounding movements or requiring the modification of game rules and mechanics. ACKNOWLEDGEMENTS We would like to thank our student Ana Maria Alexandru for her contribution in defining the first interface and guidelines for this application.