The main goal of this paper is to try to increase the comfort and well-being of older people through the employment of some kind of automated processes that simplify daily work. So, this paper presents a prototype of an a ective personal robotic trainer which, together with a non-invasive sensor, allows caregivers to monitor certain physical activities in order to improve their performance. In addition, the proposed system also takes into account how the person feels during the performance of the physical exercises and thus, determine more precisely if the exercise is appropriate or not for a speci c person.
The continuous increase of the amount of elderly people [
With the usage of wearable sensors the detection of emotions is possible attending to bio-markers such as pulse, blood pressure and muscular response. The acquisition of the emotional states of the care-receiver can be used to infuse the cognitive assistant of emotional responses to interact smoothly with the care-receivers. The objective of this feature is to create a nity and a ection of the carereceivers with the cognitive assistant, improving the acceptation values of them towards digital devices.
Cognitive assistants can be composed of just software (interfaced through common devices like
a smartphone) or in partnership with speci c hardware, like assistant robots. Some examples are
presented in depth by Martinez-Martin et al. [
to visually identify the care-receivers' physical movements and classi es them using deep learning
methods. Using wearable devices (like wristbands) with accelerometers and gyroscopes are the works
of [
We present the EmIR 3.0, an advancement over the previous versions, where the newest feature is to be a trainer capable of recommending, detecting and classifying human-performed activities. Apart from this, the robot is now able to freely navigate and its body is fully changed and is taller, being able to interact better with the care-receivers.
The rest of the paper is structured as follows: section 2 presents a related work section; section 3 explains the proposed approach; nally, section 4 gives some conclusions and possible future works. 2
EmIR 3.0 is a low cost robot (see Figure 1) which has been designed divided into two layers. There
is a low-level, reactive layer using an Arduino Mega 2560, which allows motor control and access to
ultrasonic sensors for obstacle detection, allowing the robot to have a reactive behaviour. The
highlevel layer is controlled using a raspberry pi 3 b+. allowing to have a higher behaviour controlling a
7-inch LCD screen, on which it is possible to visualise the face of the robot along with the di erent
physical activities that the robot can recommend. EmIR 3.0 has been built in a modular way,
facilitating the incorporation of new elements such as lidars, environmental sensors, etc. . . It also
has a camera that allows you to identify people and their emotions [
EmIR 3.0 capabilities can be split into two parts: recommending and classifying. The exercise recommendation follows the care-receivers pro le, thus their medical condition. The classi cation is composed by the limbs and body position detection and, using deep learning methods, classi cation and model veri cation. This leads to a comprehensive report of the exercise performance and the care-receiver health condition. 2.1
Activities Recommendation The activities have been selected by physiotherapists, who have helped to determine which exercises focus on the upper-body. This resulted in the following exercises: 1. Wall Push Up: (strength) The performer should stand at arm's length in front of a wall.
Lean forward a little and place the hand palms on the wall at shoulder width, moving the body towards the wall and back, while keeping the feet still. 2. Sit to stand: (strength) Using a chair, the performer should sit on the edge of it, feet apart at the hip level. Lean forward slightly and stand up slowly, maintaining the head leveled. 3. Mini squats: (strength) The performer has to rest its hands on the back of the chair for stability and stand with the feet spread across the hips. Then bend the knees while being comfortable.
Keep the back straight at all times. Gently stand up, squeezing the buttocks as you do so. 4. Back Leg Raises: (balance) The performer has to slowly lift the right leg backwards, without bending the knees, while the chest moves slightly toward the front. Hold that position for one second and then gently lower the leg. 5. Clock Reach: (balance) The performer has to imagine that it is on top of a clock, facing 12 o'clock. Holding on a chair at its side, the performer must lift the leg opposing the hand holding the hand, and with the free arm, lift pointing at 12 o'clock and move to 3 (or 9) o'clock and back. We have used AVNET to detect if the activities are properly done. As some of the activities to be performed involve chest movement, the sensor was placed in a harness which is used by the patient as can be seen in Figure 2.
Activities Classi cation Once de ned the exercises to classify, the following step is to create our dataset. This was necessary because there is no public dataset, which measures these kind of exercises. Ten persons were used to which a total of six repetitions per exercise were captured during 10 sessions, creating a database of 25.000 data. These data were then partitioned into three sub-datasets, one for training with 80% of the data, 10% for teas and, nally, 10% for validation.
Once the dataset was created, it was normalised between [
It can be observed that we have a success rate per exercise between 50 to 60%. this is due to small movements generated by patients, movements such as moving to the sides during the performance of exercises. As well as small spasms or involuntary movements. To try to solve this problem, we are getting more samples and we will try to lter or eliminate those involuntary or voluntary movements of patients. 3
Cognitive assistants and assistive robots are now ready to be used in a home environment. This is the care of EmIR 3.0. It is responsible of monitoring the performance of physical exercises and interact with the care-receivers and recognizing emotions and perform activities recommendation based on the monitoring of the exercises.
As future works, we aim to test EmIR with patients and workers of a daycare centre. The validation will be performed through the recommended exercises under the supervision of caregivers.
This work was partly supported by the Spanish Government (RTI2018-095390-B-C31) and FCT|Fundac~ao para a Ci^encia e Tecnolog a through the Post-Doc scholarship SFRH/BPD/102696/2014 (A. Costa) and UID/CEC/00319/2019.
Fig. 3: Confusion matrix obtained from classi cation.