In the last years, social robots have been used in rehabilitative therapies in different applications. One reason for this necessity to research such kinds of therapies with social robots came from the worldwide lack of medical staff to provide enough therapy to affected patients. Typically, such applications were not created to include long-term clinical trials and affected patients, to achieve a clinically proven improvement of their health status. Here we present a research project where a social humanoid robot is guiding through the course of rehabilitation exercises for stroke patients with different kinds of handicaps. We display the challenges that arose during the interdisciplinary creation of these tasks, with the beforementioned issue of using therapies to let the patients exercise at their performance limit. We first highlight the research problem and the opportunity for a social robot in this domain. Then we showcase the related work and list our research questions for this project. Finally, we discuss our results, research plan and future experiments.
Use only styles embedded in the document. For paragraph, use Normal. Paragraph text. Paragraph 1.1.
2020 Copyright for this paper by its authors. robot system as with trained therapists. Additionally, we want to discover, what are the important factors of such a system and what could be improved in a future (clinically or commercially) usable SAR stroke-therapy setup. 1.2.
The research questions of this doctoral project are part of the E-BRAiN project. This project wants to discover how far social robots can be used to motivate stroke patients in neurorehabilitation therapies. The research questions of this doctoral thesis are: • • • • •
How should the system be designed to perform the four diverse therapies from one system? How is such a social robot therapy system accepted? What are the system features patients desire in such a system? By what actions of the system can the patient better stay motivated and engaged?
How can the interaction between the patient and system be modeled?
Feingold-Polak et al. [
The following, other approaches of robot therapy have not been targeted for a long-term therapy use
or are not developed for stroke patients. Gonzalez et al. [
Another application with an autonomous NAO robot was developed by Görer et al. [
Shao et al. [
Regarding supporting technologies to model the Human-Robot Interaction and possibly make it
easier to work on it, Van der Bergh et al. [
There is not much work done on specific applications for social robots in this domain. But a relevant
work has been presented by Sutherland et al. [
To answer the research questions from section 1.2 and for the realization of an autonomous SAR therapy, we have been developing a robot therapy setup to undertake experiments. We started to evaluate the system in a study with participants in a rehabilitation therapy setting and validation by medically trained personnel. To evaluate such a SAR system, we needed to first imitate “human therapist-patient” therapies into program-“scripts”, which then can be executed by the system. The specifications for these were gathered with cognitive walkthroughs together with stroke rehabilitation experts.
During the therapies of patients with the robot setup their performance is observed and recorded. After their therapies patients evaluate the implementation of our system in questionnaires and qualitative interviews.
Here we present the stroke therapies which we have been implementing to work with the robot, the robot system and the so far achieved results so far. 3.1.
We have been implementing four groups of therapies, as seen in Figure 1. Patients were initially classified depending on their health state and chance for recovery. Arm ability training (AAT) [10] aims for relatively minor arm movement problems, mirror therapy (MT) for predominantly one-sided arm paralysis and Arm basis training (ABT) [11] depicted is used for severe arm paralysis. In addition, neurovisual therapy (NVT) is offered for patients with a neglect handicap. A neglect handicap typically results in a lack of field of vision of the patient’s eyesight. Patients are not aware, that they have a lack of their sight.
While the ABT, MT and AAT are mostly computer scripts, which let the robot streamline a certain program of text and speech commands, during the NVT the patient interacts additionally with a touch monitor the whole time.
The NVT is a new development of a therapy, adapted and similar to [12], and has yet to be tested for general acceptance and clinical effectiveness. For the nature of the tasks, it has built in certain features to react on patient speech and precise touch on photos. healthy arm. Bottom left corner: Examples of the movements of the ABT therapy. Bottom right corner Example of the visual exploration task of the neurovisual therapy (NVT).
Robot Control
Pepper
MQTTServer
Management of Therapies
We use the robot Pepper [13] for our experiments, as seen in Figure 3. In addition, several cameras, microphones, a confirmation touch panel, a pc and a database with information about the patient and the helper are connected. The computer unit is directing the dynamic dialogue structure and the actual content of robot feedback. The system receives explicit confirmation from the physical touch panel and in some parts speech input from the patient. The components communicate with via the MQTT-protocol [14].
On the software side, for the “Management of Therapies” we build a website to add patients, set up and administer therapy sessions and control a therapy session. Furthermore, since we used AndroidTablets, we had to develop a suitable app with a reasonable layout for the mostly older adults participating as patients. Both the tablet and the robot will display images, videos and the instructions texts on the display. The patients will also be able to confirm certain actions by touching elements on the screen.
The dialogue is built on a finite-state-machine [15], which sends out messages to all other devices, as shown in Figure 2. Inside each device, an interpreter reads these messages and displays the contents of the message.
From the patient’s point of view, at the beginning of the session, he/she will sit down and interact with the nearby tablet/monitor to start the therapy.
We are currently conducting therapies with the beforementioned system and aim for at least 30 patients. Patients are invited to participate and will be screened for a suitable therapy of ours. Then, they are required to come to these therapy sessions to the Universitätsmedizin Greifswald, Germany.
Each patient will have a 4 week-therapy, in a random order with a two-week human-therapist program and a two-week SAR program. The goal of the E-BRAiN project is to explore the success rate and for ways how to motivate patients during the therapies with the SAR. In the real application, before the first robot session, a human therapist will do an extensive introduction. The medical experts will perform the Fugl-Meyer assessment [16] with the patients to collect the starting condition of the patient. This test measures the patient’s level of impairment and the real success of the therapy will also be measured with this test.
For the actual technical evaluation, we use a questionnaire with the ALMERE-Model [17] for the acceptance of the robot system. The ALMERE-model is a building upon the foundations of the
Month/Year 09/19 – 09/20 09/20 - 09/21 09/21 – 12/22 technology acceptance model (TAM) and on the Unified theory of acceptance and use of technology (UTAUT)-model. It has been developed precisely for applications like ours, i.e. social robot acceptance with elderly adults, thus it appears to be the most promising approach. Inside the actual questionnaire, we will add certain open questions, to broaden our understanding, how the system is perceived by the patients.
In a second step, we will additionally conduct usability interviews with frameworks like [18] with the patients, to gather more precise feedback on certain elements of the session. For this, we would like to know more about e.g. the speech speed of the robot, the clearness of the given instructions, reaction time of the system.
The robot system with all four therapies are done and the therapy study with the aim for at least 30 patients is running since the 09/2021. Currently last preparations are done to prepare and finalize the technical questionnaires and qualitative interviews. 5.2.
So far three different papers have been published as the first author. One work [15] during the early development of the therapy system describes the software approach for the system. More precisely it explains on more detailed parts how the dialog and contents of the therapy are put together and how the system components work together.
Another work [19] focused on a theoretical approach, how a simple rule-system could be used to provide motivational feedback during the ABT-therapy.
Another work [20] was done to investigate for possible user models of a patient and a (human) therapist. For this, we tried a lesser known method of “repertory-grid” technique with the goal to explore implicit knowledge about the patients and therapists. We asked the therapists of the E-BRAiN project about their past patients and of their impression with other therapists. We gathered around 170 attributes for patients and 45 for therapists. We list the patient and therapist attributes online available at [21]. These attributes and co-developing of therapy applications with medical experts for similar therapies inside the E-BRAiN project may help to design the robot better. Currently we make use of them, but we can greatly increase our personalization efforts of the therapy.
The paper discussed the idea of creating a system for performance-oriented stroke therapies with the help of a social humanoid robot. Some research questions were identified. We hope that we can show the following things as a contribution to the HRI area: (1) the acceptance of a SAR in such a style of a therapy (2) the learnings and improvements to include for such a system (3) a proof that social robots can be a helpful performer as instructor compared to a human therapist during the recovery of a strokesurvivor.
Additionally, we made the first steps for a smarter robot therapy. For the next iteration, we plan to include an enhanced version of the user models and test our digitalized therapy exercises [22]with real patients.
We are looking forward for the DC of the EICS conference to learn from the experience of others in
engineering and modelling such medical therapy systems. Even though our patient study has already
started, we may still integrate points to consider like aspects of the evaluation and other things.
Furthermore, we also have other projects, especially to enhance the text-based domain specific language
“DSL-CoTaL” [
This joint research project “E-BRAiN - Evidence-based Robot Assistance in Neurorehabilitation” is supported by the European Social Fund (ESF), reference: ESF/14-BM-A55-0001/19-A01, and the Ministry of Education, Science and Culture of Mecklenburg-Vorpommern, Germany. The sponsors had no role in the decision to publish or any content of the publication.