Robots for the elderly are a particular category of home assistive robots, helping people in the execution of daily life tasks to extend their independent life. Such robots should be able to determine the level of independence of the user and track its evolution over time, to adapt the assistance to the person capabilities and needs. We present an heterogeneous information management framework, allowing for the description of a wide variety of human activities in terms of multi-modal environmental and wearable sensing data and providing accurate knowledge about the user activity to any assistive robot.
Home assistive robotics addresses the design of robots to be deployed in domestic
environments, to assist the residents in the execution of daily life tasks. Robots
for the elderly are a particular category of home assistive robots, which rely on
social interaction with the person and aim at extending the elderly independent
life [
Human Activity Recognition (HAR) systems for elderly-care are devoted to
the identification, among all actions executed by a person during a day, of
specific activities of interest, the Activities of Daily Living (ADL), which require
the use of different cognitive and physical abilities and are used by
gerontologists to estimate the level of autonomy of a person [
We address the problem of endowing robots for the elderly with the ability of monitoring the Activities of Daily Living, by designing a HAR system which allows for a seamless integration with the robot planning system. We propose the integration of multiple sensing strategies in a single framework, to compensate the weaknesses of each modality and increase the recognition reliability.
The abstract is organized as follows. Section 2 details the system architecture. Preliminary experimental results are analysed in Section 3. Conclusions follow. 2
We set up a test bed in an apartment located in the city of O¨ rebro (SWE), in the elderly care facility A¨ ngen. The apartment, shown in Figure 1, is composed of fully furnished living-room, bathroom, bedroom and kitchen. (a) (b)
We propose the multi-modal monitoring system with the architecture shown in Figure 1 for the reliable detection of the activities: transferring (denoting the motions of sitting down, standing up, lying down, getting up); feeding (eating, drinking); food preparation; indoor transportation (climbing stairs, descending stairs, walking). The system makes use of: (i) a wrist-placed inertial sensor; (ii) a waist-placed inertial sensor; (iii) a network of Passive Infra-Red sensors; (iv) RFID tags, pressure sensors and switches; and (v) a temporal reasoner.
The data extracted by the wrist sensor are used to detect occurrences of
gestures [
Listing 1.1 reports the models of sitting and standing. The field Head defines the entity it refers to and the name of the model, separated by a ::. As an example, Head Human::SitDown() indicates that whenever the reported constraints are satisfied, the reasoner should infer that the activity of sitting has been executed by the person. The field RequiredState defines the sensor values which correspond to the execution of the motion. The field Constraint defines the temporal relation between each sensor value of interest and the activity.
Listing 1.1. DDL models for the activities sit down and stand up. ) ( S i mp l e O p e r a to r ( Head Human : : SitDown ( ) ) ( R e q u i r e d S t a t e r e q 1 G e s tu r e : : S i t ( ) ) ( R e q u i r e d S t a t e r e q 2 P o s t u r e : : S i t t i n g ( ) ) ( R e q u i r e d S t a t e r e q 3 C h a i r : : On ( ) ) ( C o n s t r a i n t OverlappedBy ( Head , r e q 1 ) ) ( C o n s t r a i n t During ( Head , r e q 2 ) ) ( C o n s t r a i n t EndEnd ( Head , r e q 3 ) ) ( S i mp l e O p e r a to r ( Head Human : : SitDown ( ) ) ( R e q u i r e d S t a t e r e q 1 G e s tu r e : : S i t ( ) ) ( R e q u i r e d S t a t e r e q 2 P o s t u r e : : S i t t i n g ( ) ) ( R e q u i r e d S t a t e r e q 3 Armchair : : On ( ) ) ( C o n s t r a i n t OverlappedBy ( Head , r e q 1 ) ) ( C o n s t r a i n t During ( Head , r e q 2 ) ) ( C o n s t r a i n t EndEnd ( Head , r e q 3 ) ) ( S i mp l e O p e r a to r ( Head Human : : StandUp ( ) ) ( R e q u i r e d S t a t e r e q 1 G e s tu r e : : Stand ( ) ) ( R e q u i r e d S t a t e r e q 2 P o s t u r e : : S ta n d i n g ( ) ) ( R e q u i r e d S t a t e r e q 3 Human : : SitDown ( ) ) ( C o n s t r a i n t MetByOrOverlappedBy ( Head , r e q 1 ) ) ( C o n s t r a i n t S t a r t s ( Head , r e q 2 ) ) ( C o n s t r a i n t MetByOrAfter ( Head , r e q 3 ) )
We performed preliminary tests defining sequences of sensor values to analyse the reasoner inferences they trigger. In Figure 2, the timeline of the context variable Human is computed by the reasoner and list all corresponding recognized activities, as indicated by the Head fields. The other timelines report the sensor values (i.e., gesture, posture, location and objects sensors). At each time instant, the reasoner samples the sensors, keeping track of all modelled activities which are consistent with the sensors readings up to that instant (i.e., those that could be the one currently being executed). As time passes, the number of possible activities progressively reduces, until it converges to the one effectively performed, if it is among the modelled ones, or to none. All sensors or context variables statuses supporting an inferred activity are marked with a blue filling.
Figure 2 reports the simulated sensor readings related to a person who drops a heavy bag on the kitchen chair, then walks to the living-room and sits on the armchair. Although the chair pressure sensor is activated by the bag (for t = [16; 35]), the wearable gesture and posture sensors do not signal any sitting motion, therefore preventing the reasoner from making an erroneous inference. Later on, when the person sits on the armchair, environmental and wearable sensors agree on indicating that the person sat down, therefore triggering the correct recognition of the sitting motion. The example also highlights one advantage deriving from overloading rules. Since the two modelled sitting actions (i.e., sitting down on the kitchen chair and sitting down on the armchair) are defined as SitDown, it is possible to define a single model for the standing up motion, constrained by the previous occurrence of any sitting action. In this abstract, we introduce the idea of a multi-modal monitoring system, which combines information retrieved via different monitoring approaches and we prove, in simulation, that the integration of wearable and environmental information is beneficial for the purposes of human activity monitoring. Future work will focus on the set up of a test bed apartment in an elderly care facility in O¨ rebro, Sweden, to test the performance of the system under realistic conditions.