Decline in daily functioning usually involves the reduction and discontinuity in daily routines; entailing a considerable decrease of the quality of life (QoL). This is especially relevant for people that need assistance, as for instance elderly or disabled people and may also hide pathological (e.g., Alzheimer) and/or mental (e.g., depression or melancholia) conditions. Thus, there is the need to intelligent systems able to monitor users' activities to detect emergencies, recognize activities, send notifications, and provide a summary of all the relevant information. In this paper, we present a sensor-based telemonitoring system that addresses all that issues. Its goal is twofold: (i) helping and supporting people (e.g., elderly or disabled) at home; and (ii) giving a feedback to therapists, caregivers, and relatives about the evolution of the status, behavior and habits of each monitored user. Some features of the system have been evaluated with two healthusers in Barcelona and results show good performance. Finally, the system has been adopted and installed in several end-users' homes under the umbrella of the projects SAAPHO and BackHome.
In the literature, various studies and systems aimed at detecting and
overwhelming the worsening in daily activities have been proposed. Several methods are
limited to measuring daily functioning using self-report such as with the modified
Katz ADL scale [
The rest of the paper is organized as follows. Section 2 presents the architecture of the sensor-based solution as well as the intelligent monitoring system. Section 3 shows the evaluation performed to test the availability and reliability of the proposed solution as well as the results coming from the adoption of the system in two real scenarios. In Section 4, we conclude with the main results of this work pointing out its future directions. 2 2.1
Advanced telemonitoring systems entail the composition and orchestration of heterogeneous distributed technologies and services. In Figure 1, we sketch the high-level architecture of the proposed system. As shown, its main components are: home; healthcare center; middleware; and intelligent monitoring system. The sensor-based system is able to monitor indoor activities by relying on a set of home automation sensors and outdoor activities by using an activity tracker, namely Moves1. Moreover, through environmental sensors, the system is able to detect emergency situations.
At home, a set of sensors are installed. In particular, we use presence sensors (i.e., Everspring SP103), to identify the room where the user is located (one sensor for each monitored room); a door sensor (i.e., Vision ZD 2012), to detect when the user enters or exits the premises; electrical power meters and switches, to control leisure activities (e.g., television and pc); and pressure mats (i.e., bed and seat sensors) to measure the time spent in bed (wheelchair). The system is also composed of a network of environmental sensors that measures and monitors environmental variables like temperature, but also potentially dangerous events like gas leak, fire, CO escape and presence of intruders. All the adopted sensors are wireless z-wave. They send the retrieved data to a collector (based on Raspberry pi). The Raspberry pi collects all the retrieved data and securely redirects them to the cloud where they will be stored, processed, mined, and analyzed. We are also using the user’s smartphone as a sensor by relying on Moves, an app for smartphones able to recognize physical activities and movements by transportation. The user interacts with the overall system through a suitable interface aware of end-user needs and preferences.
The middleware, which acts as a SaaS, is composed by a secure communication and authentication module; API module to enable the collector transmitting all the data from sensors to make them available to the activity monitoring module; and further utilities such as load balancing and concurrency.
In order to cope with the data necessities of the actors of the system (i.e., therapists, caregivers, relatives, and end-users themselves), an Intelligent Monitoring (IM) system has been designed.
The healthcare center receives notifications, summaries, statistics, and general information belonging to the users through a web application. 2.2
IM aims to continuously analyze and mine the data through 4-dimensions: detection of emergencies, activity recognition, event notifications, and summary extraction. In order to cope with these objectives, the IM is composed of the following modules (see Figure 2): PP, the pre-processing module to encode the data for the analysis; ED, the emergency detection module to notify, for instance, in case of smoke and gas leakage; AR, the activity recognition module to identify the location, position, activity- and sleeping-status of the user; EN, the event notification module to inform when a new event has been detected; and SC, the summary computation module to perform summaries from the data. Pre-processing IM continuously and concurrently listens for new data. The goal of PP is to pre-process the data iteratively sending a chunk c to ED according to a sliding window approach. Starting from the overall data streaming, the system sequentially considers a range of time |ti - ti+1| between a sensor measure si at time ti and the subsequent measure si+1 at time ti+1. Thus, the output of PP is a window c from ts to ta, where ts is the starting time of a given period (e.g., 8:00 a.m.) and ta is the actual time. Thus, each chunk is composed of a sequence of sensor measures s; where s is a triple < ID, v, t >, i.e., the sensor ID, its value and the time in which a change in the sensor status is measured. Figure 3 shows an example of a chunk composed by four sensors measures.
Emergency Detection ED module aims to detect and inform about emergency situations for the end-users and sensor-based system critical failures. Regarding the critical situations for the end-users, simple rules are defined and implemented to rise an emergency, when specific values appear onc (e.g.; gas sensor ID, smoke sensor ID). Regarding the system failures, ED is able to detect when the enduser’s home is disconnected from the middleware as well as a malfunctioning of a sensor (e.g., low battery). The former is implemented by a keepalive mechanism in the Raspberry pi. If no signals are received from the Raspberry pi after a given threshold, an emergency is risen. The latter is implemented by using a multivariate gaussian distributions of sensor measurements on c. If the corresponding total number of measures is greater than a given threshold, an emergency is risen. Each emergency is a pair < si, lεi > composed of the sensor measure si and the corresponding label lεi that indicates the corresponding emergency (e.g., fire, smoke). Once the ED finishes the analysis of c, the list of emergencies ε is sent to the middleware, whereas c, filtered from the critical situations, is sent to AR. Activity Recognition In the current implementation, the system is able to recognize if the user is at home or away and if s/he is alone; the room in which the user is (no-room in case s/he is away, transition in case s/he moving from a room to another); the activity status (i.e., active or inactive); and the sleeping status (i.e., awake or asleep).
To recognize if the user is at home or away and if s/he is alone, we implemented
a solution based on machine learning techniques [
To measure the activity status, we rely on the home automation sensors. By default, we consider as “active” the status of the user when s/he is away (the corresponding positions are saved as “no-room”). On the contrary, when the user is at home, AR recognizes s/he as “inactive” if the sensor measures at time ti that user is in a given room r and the following sensor measure is given at time ti+1 and the user was in the same room, with ti+1 − ti greater than a given threshold θ . Otherwise, the system classified the user as “active”.
Finally, sleeping is currently detected by relying on the presence sensor located in the bedroom and the pressure mat located below the mattress. In particular, we consider the presence of the user in that room and no movements detection (i.e., the activity status is “inactive”) together with the pressure of the mattress. Thus, the output of AR is a triple < ts, te, l >, where ts and te are the time in which the activity has started and has finished, respectively, andl is a list of four labels that indicates: the localization (i.e., home, away, or visits), the position (i.e., the room, no-room, or transition), the activity status (i.e., active or inactive), and the sleeping status (i.e., awake or asleep). To give an example, let us consider Figure 4 where the same chunk of Figure 3 has been processed by AR.
Event Notification By relying on a set of simple rules, EN is able to detect events to be notified. Each event is defined by a pair< ti, l > corresponding to the time ti in which the event happens together with a label l that indicates the kind of event. In particular, we are interested in detecting the following kind of events: leaving the home, going back to home, receiving a visit, remaining alone after a visit, going to the bathroom, going out of the bathroom, going to sleep, and awaking. Following the example, in Figure 3, an event is the pair < 2014 − 02 − 2410 : 31 : 55, going to the bathroom >.
Summary Computation Once all the activities and events have been classified, measures aimed at representing the summary of the user’s monitoring during a given period are performed. In particular, two kinds of summary are provided: historical and actual. As for the historical summary, we decided to have a list of the activities performed during (i) the morning (i.e., from 8 a.m. to 8 p.m.), (ii) the night (i.e. from 8 p.m. to 8 a.m.), (iii) all the day, (iv) the week (from Monday morning to Sunday night), as well as (v) the month. In particular, we monitor: sleeping time; time spent outdoors; time spent indoors; time spent performing indoor activities; time spent performing outdoor activities; number of times spent in each room; and number of times that the user lefts the house. As for the actual summary, we are interested in monitoring: the room in which the user is; if the user is at home, or not; the number of times that s/he leaves the home; sleeping time; activity time; and number of visits per room.
As a final remark, let us note that all emergencies, activities, notifications, and summaries are stored in a database to be available to all the involved actors. 3
The proposed solution has been developed according to a user-centered design approach in order to collect requirements and feedback from all the actors. For evaluation purposes, the system has been installed in two healthy-user homes in Barcelona. Moreover, the system has been used in the SAAPHO2 project to monitor elderly people and in the BackHome3 project to monitor disabled people. 3.1
Before installing the system at real end-users home, its evaluation was undertaken
by a control group of healthy users. Two healthy users participated in the study
as a control group (1 female, M=32.5 years). The evaluation has been performed
from November 2nd, 2014 to December 21st, 2014 for a total of 34 days. The
performed testing activity was focused on evaluating some of the features of AR
and EN. In particular, we evaluated: the performance of the hierarchical approach
(AR) as well as the ability in recognizing the events of leaving the home and
receiving visits (EN); and the ability in recognizing the sleeping activity (AR).
As for the evaluation of the hierarchical approach, we first trained both classifiers.
To measure the performance, we compared the overall results with those obtained
by using a rule-based approach at each level of the hierarchy. Results are shown
in Table 1 and point out that the proposed approach outperforms the rule-based
one with a significant improvement. The interested reader may refer to [
To evaluate in the ability of EN in correctly detecting the notification about the number of times that the user leaves the home and the number of received visits, we daily ask the users to answer to the questions: “How many times did you go out from home?” and “How many times did you receive visits at your home?”. Then, we compared the answers given by the user with the number of detection by EN. Figure 5 sketches the results whereas, in Table 2, the first two columns show the cosine similarity and the accuracy for each of the considered notifications. As it can be noted, the system is able to recognize quite well the number of times the user leaves the home as well as the number of visits that s/he receives, thanks to the proposed hierarchical approach. 2http://www.saapho-aal.eu/ 3http://www.backhome-fp7.eu/backhome/ As for the evaluation in the ability of AR to recognize the sleeping activity, we daily ask the users to answer to the questions “What time did you stand up from bed?” and “What time did you go to sleep?”. Then, we compared the answers given by the user with the sleeping time calculated by AR. Figure 6 sketches the results whereas, in Table 2, the last column shows the cosine similarity and the accuracy. Let us note that, due to the fact that we are not relying on further information such as luminosity to understand if the user is on the bed doing something (e.g., reading) or if s/he is really sleeping and that the user in the questionnaire is giving the time in which s/he turns-off the light, we are considering a bias of 5400 secs to consider the user as awake. 3.2
SAAPHO was an European R&D project aimed at integrating health, social,
and security services seamlessly in the same architecture [
9.9), 60-74 years. Regarding the gender both in Spain and Slovenia, 66.6% of
the participants were women. Moreover, 100% of the participants had experience
using computers; 66.6% had experience in using tablet PC and all of them had
Internet at home. ED has been used in SAAPHO to detect fire and smoke, AR
and EN to detect events, such as inactivity and toileting4. This information has
been then sent to the end-user through a suitable interface in a smart portable
device. In particular, it is possible to check on real time the status of environmental
sensors (e.g., temperature, humidity), to trigger an alarm (e.g., when smoke or
gas lekeage is detected), to view the list of the recent home notification events
(e.g., sharp increase/decrease of temperature, prolonged lack of movements) as
well as to configure the home sensors. In [
BackHome is an European R&D project that focuses on restoring independence
to people that are affected by motor impairment due to acquired brain injury or
disease, with the overall aim of preventing exclusion [
In this paper, we presented a sensor-based system aimed at detecting emergencies, recognizing activities, sending notification as well as collecting the information in a summary. The goal of the implemented system was to help and support people that need assistance and to constantly give a feedback to therapists, caregivers, and relatives about the evolution of the status, behavior and habits of the corresponding user. The system has been evaluated with 2 healthy-users to assess if 4SC has not been implemented due to the self-managing purpose of the project; i.e., no caregivers were involved. the user is at home or away, the number of times the user leaves the home; if s/he receiving visits as well as the number of received visits; and the sleeping activity. Results show that the system performs well in all of these tasks. Moreover, under the umbrella of SAAPHO and BackHome, the system has been installed and tested in 6 homes of elderly people and in 3 homes of disabled people, respectively. As a final remark, let us note that the intelligent monitoring system could be extended by adding new functionalities into the modules depending on the requirements of the corresponding use-case(s).
As for the future work, we are currently improving the sleeping activity recognition by relying also to a sensor that measures luminosity in addition to presence. Moreover, we are setting up new tests to evaluate the ability of the system in recognizing when the user is active or inactive, relying also on the information coming from Moves. We are also planning to recognize more activities, such as cooking and eating. Finally, in order to assess quality of life, we are interested in measuring the sleep quality and in making some studies on the (virtual or physical) social interactions of the users.
The research leading to these results has received funding from the European Community’s, Seventh Framework Programme FP7/2007-2013, BackHome project grant agreement n. 288566, and AAL (Ambient Assisted Living)/Call 3, SAAPHO project grant agreement n. 2010-3-035.