We consider use of an ambient sensor network, installed in Smart Homes, to identify low level events taking place which can then be analysed to generate a resident's profile of activities of daily living (ADLs). These ADL profiles are compared to both the resident's typical profile and to known “risky” profiles to support evidence-based interventions. Human activity recognition to identify ADLs from sensor data is a key challenge, a windowbased representation is compared on four existing datasets. We find that windowing works well, giving consistent performance. We also introduce FITsense, which is building a Smart Home environment to specifically identify increased risk of falls to allow interventions before falls occurs.
Like most countries world-wide, the United Kingdom is facing an ageing population with many people living longer. Ten million people in the UK are currently over 65 with a further increase of 5.5 million projected over the next 20 years. Three million people are aged over 80 which is expected to double by 2030 [Population Division and Affairs, 2015]. An ageing population puts additional strains on health and social services with both a smaller proportion of working population available to support services, and with the elderly having more complex medical needs. In this changing scenario it is important that we help people with mobility or social needs to live independently for longer, and so reduce their reliance on more expensive health care solutions.
In this paper we examine the potential for Smart Homes to support assisted living environments by monitoring health trends. A mixed approach is proposed which exploits the patterns of activities identified by sensors to infer information about the health of the residents. We explore the use of everyday, low-cost ambient sensors installed in new-build Smart Homes with the aim of supporting tenants to live independently for longer. Specifically, we identify and discuss the main challenges with an ongoing project to design and deploy a real-world health monitoring system that senses and predicts the level of risk of falling attributed to Smart Home residents. Data is captured by a range of sensors installed in specially-designed, technology-enabled “FitHomes”, targeting specific activities identified as pre-cursors to falls. An outline solution is developed and initial experiments are undertaken to evaluate alternative approaches to classifying activity with low level, raw data inputs from multiple sensors. 2
Activities of Daily Living (ADLs) are events in daily life which would be considered intrinsic to a person’s ability to live independently. ADLs include being able to dress oneself, get out of bed, and feed oneself. Katz [Katz et al., 1963] originally proposed the term along with a scale for rating a person’s independent ability using their performance in simple ADLs. The concept of losing ADLs as we age has influenced future research in the field by identifying that specific ADLs are more indicative of reduced capability than others. Observing variances in ADL performance can aid the identification of degenerative mental and physical capability. Vestergaard [Vestergaard et al., 2009] identified a relationship between performance in a 400-meter walk test and subsequent mortality rates. This test is usually performed in hospital which allows a physician to also consider other metrics from the test. These include but are not limited to, whether or not a break was taken, variation in lap times and existing health conditions. However, lab-based testing is time consuming, costly and impractical for many patients, especially those with mobility issues. In addition, some studies have been able to identify risk of falling in the elderly using gait velocity alone [Stone and Skubic, 2013; Jiang et al., 2011; Montero-Odasso et al., 2005]. So while gait and other expressions of movement are indicative of many underlying conditions, measuring all aspects of gait, such as swing and stride length, requires specialist equipment. Gait velocity, however, can be measured using simpler equipment and still provides excellent insight into subject movement. For instance, Rana [Rana et al., 2016] performed a study in which gait velocity was estimated using simple infra-red motion sensors. We plan to adopt this approach and, while lab-based testing can provide higher accuracy, envisage that accessible in-home testing can contribute to early detection of health problems.
Housing installations with ubiquitous simple sensors offer an opportunity to provide continuous behavioural and physiological monitoring of residents. These simple sensors can range from binary magnetic switches [Tapia et al., 2004], to IoT-monitored motion sensors [Suryadevara and Mukhopadhyay, 2012], all of which can provide insight into behavioural and physiological expressions. ADLs can then be modelled by identifying temporal patterns in these sensor outputs. Several manually annotated datasets taken from Smart Home installations have been produced for the purpose of activity recognition [Tapia et al., 2004; Van Kasteren et al., 2008; Cook and Schmitter-Edgecombe, 2009]. We use these existing datasets in our experiments to evaluate the effectiveness of alternative classification approaches. 3
FitHomes is an initiative, lead by Albyn Housing Society Ltd (AHS), that aims to support independent living with the supply of custom-built Smart Homes fitted with integrated noninvasive sensors. Sixteen houses are being built and near completion at Alness near Invergordon. These houses are part of a development cycle with further FitHomes planned to be built within the Inverness area within 3 years. FITsense 1 is a one year project that aims to exploit the sensor data to develop a prototype fall prediction system for the residents of FitHomes.
A key consideration in designing a Smart Home focused on health monitoring is the choice, mix and location of sensors to use in order to provide a cost-effective solution that is also acceptable to residents. AHS have conducted initial research and it is clear that their tenants want an unobtrusive system. Both video and wearables are considered too intrusive for continuous in-home use; video due to privacy issues and wearables due to the ongoing overhead associated with 24 hour operation. As a result, the focus in this project is on simple everyday sensors, many of which are already widely used in security and automation applications. FITsense is an applied project and with this approach we can establish the limits of existing technology now, rather than developing new solutions for the future. A further benefit is provision of a low cost solution from the hardware perspective but with additional challenges for the data analysis.
FITsense aims to identify increased risk of falls and so a key focus for monitoring is to identify activity levels, patterns and speeds. However, monitoring can go beyond just movement to consider other factors that have been shown to be related to falls, including dehydration, tiredness and mental health. Gaining information on these additional factors requires monitoring to also capture data on more general activities such as eating & drinking behaviours, sleep patterns, and toileting & grooming habits. With these criteria in mind a range of sensors have been selected for the FitHomes, that include: IR motion sensors that capture movement in each room; contact sensors to capture room, cupboard, and fridge door opening/closing; pressure sensors that identify use of the bed and chairs; IR beam break sensors used to identify gait speed; electricity smart meters to identify power usage pattern; float sensors identifying toilet flushing; and humidity sensors to identify shower use.
Most of the sensors chosen have a binary output that simply activate when the event they are monitoring takes place 1www.rgu.ac.uk/fitsense e.g. a door opening; however others output give continuous readings provided at fixed polling rates. The data fusion task across multiple sensors with different output modes becomes one of the main challenges in employing large numbers of sensors. We employ a two stage process. The first stage is to generate activity profiles. This requires pattern identification to create meaningful representations from the raw sensor data that capture the residents’ activities e.g. sleeping, dressing, showering, cooking, and then to assemble these activities into personalised daily and weekly profiles. The second stage is the analysis of these profiles to enable both the identification of changing trends in the resident’s activities over time and to make comparisons with data collected from other similar residents. Changes in the Smart Home resident’s own activity patterns over time can then be used to detect deterioration in health, while comparisons with the patterns of other Smart Home residents can provide benchmark measures of health. The data thus supports evidence-driven intervention tailored to the resident and their specific circumstances. 4
Human Activity Recognition (HAR) to identify ADLs is challenging in Smart Home scenarios because large volumes of data is generated from multi-modal sensors in real time making patterns associated with specific activities difficult to identify. Figure 1 shows a diagram with example sensor activations for motion sensors in a hall, kitchen and lounge together with pressure sensors on the chair and bed. Simple events can be inferred from this data to generate activities. A mix of approaches will be adopted to identify activities and to then generate daily activity profiles. For the simple activities shown (e.g. time sitting, time in bed, number of toilet visits, number of room transitions) only one or two sensor activations are required to identify the activity; a rule-based approach with simple rules is sufficient and where effective this approach will be adopted.
More complex activities can only be recognised by the interaction of several sensors e.g. food preparation, showering, disturbed sleep. For these more complex activities a Machine Learning (ML) approach will be adopted. HAR typically employs a windowing approach to create a single aggregated vector representation on which ML (e.g. kNN, Support Vector Machines or Naive Bayes) can be applied for classification. These approaches can work well but are perhaps less able to handle the data fusion scenarios from Smart Homes because of difficulties in selecting appropriate time windows for different activities; and due to the loss of information when the sequence of events is not maintained, by aggregating within a window. We investigate the performance of a windowing approach. 5
Identifying ADLs in themselves does not give an indication of health. However, it has been shown that functional assessment is an effective way to evaluate the health status of older adults [Cook et al., 2015]; ADLs are lost as we age and in FITsense the plan is to monitor changes in ADL activity as an indicator of deteriorating health and increased risk of falls. A Case-Based Reasoning (CBR) approach is adopted. In our scenario, a set of ADL templates (together with contextual information) is used as the problem representation to retrieve similar profiles from a case base of existing profiles. Solutions will identify interventions, where required, and their previous outcomes.
Figure 2 presents an overview of our approach. Low-level, time-stamped events identified by the sensors are transformed into a daily user profile. The profiles are a set of ADLs with mixed data types: some ADLs are binary, e.g. disturbed sleep; some ADLs are counts, e.g. number of room transitions or stand up from seat count; some are cumulative daily time spans, e.g. time sitting; while others are numeric, e.g. average gait speed. Whatever the data type a similarity measure is associated with each ADL so that comparison can be made between them. A set of daily ADL profiles for a resident can then be compared with those in the case base, on the right of Figure 2. Retrieval of similar profiles labelled as at risk identifies the need to recommend intervention, and falling similarity with the user’s own previous profiles identifies changing behaviours. Importance in determining similarity for FITsense is given to ADLs known to correlate with falls. For other health conditions the similarity knowledge could be refined to reflect specific conditions e.g. gait for falls, erratic behaviour for Dementia, general physical activity level for obesity, etc.
A key challenge is to identify “risky” or deteriorating behaviour. Labelled positive cases (identifying a fall is likely) are rare because people don’t fall that often. The initial approach is to generate template solutions with guidance from health care professionals. Then, as real data becomes available, we can learn/refine/supplement these hand-crafted templates with the addition of real experiences as they occur in the data generated both by the user and by others. 6
The initial task is to assess effectiveness at classifying ADLs from raw sensor data. Live data is not yet available from the FitHomes, so in this evaluation existing datasets are used. Four publicly available datasets are used in our experiments: CASAS2 (adlnormal), Van Kasteren3 (kasteren) and two from the Massachusetts Institute of Technology4 (tapia1/2). These datasets share similar properties to that expected from FitHomes. They all capture binary sensor activation data from the homes and have been labelled with class information, i.e. the ADL identified during the specified time period.
Table 1 gives an overview of the structure of the datasets. These are relatively small datasets with between 120 and 295 instances, reflecting the high cost of manual labelling. The number of attributes varies between 14 and 76 reflecting differences in the number of sensors present in different installation set ups. Likewise, there are differences in the number of activities being monitored (i.e. classes) depending on the focus of the particular study; tapia in particular has a large number of different activity labels, some of which would not be relevant for predicting falls. Some activities are more popular than others and as a result most datasets do not have balanced class distributions. The window-based representation we use is a fixed-length vector which does not change with varying activity lengths. If we count the number of sensors in the installation there will be one attribute for each sensor. The attribute value being a count of the number of times the
sensor is activated during an activity timespan. The solution is a single class label, namely the labelled activity. Popular ML algorithms that delivered good performance on these datasets were selected from the Weka library to run with default settings on the window-based representation of each dataset [Hall et al., 2009]. These were compared to Conditional Random Fields (CRFs) also run with default settings on the CRF++ toolkit. Both tools make use of different data formats, so each dataset was converted to ARFF (for Weka), and CSV (for CRF++). Given the limited data available, LeaveOne-Out cross validation was applied on all experiments and average accuracy results were recorded.
k-NN: Using the IBk lazy classifier (with k=3).
The performance of BayesNet, k-NN, SVM, J48 and CRFs are compared. The results can be seen in Table 2 with the highest accuracy achieved on each dataset in bold.
High accuracies, generally in excess of 90%, are achieved on adlnormal and kasteren compared to highs of 61% and 47% on tapia1 and 2 respectively. The differences reflect that both tapia datasets present a much harder classification task with over 20 fine grained activities, many of which are hard to distinguish even with over 70 sensors. Adlnormal and kasteren have fewer activities being identified (5 and 7 respectively) and fewer sensors (39 and 14 respectively). Kasteren in particular is more in line with the type of activities and sensor network we plan for FITsense.
There is not a clear winner. BayesNet, k-NN and J48 provide good performance on the simpler datasets (adlnormal and kasteren); k-NN gives highest accuracy on kasteren which, having the fewest sensors and shortest activity sequences, is likely to have few noisy attributes. BayesNet gives highest accuracy on adlnormal which is distinguished by having long sensor sequences associated with activities. CRF gives highest accuracies on the more complex tapia datasets, which seems to indicate that the relationship between sensor activations becomes more important for distinguishing similar activities from each other. 7
In this paper we have presented a Smart Home approach to monitoring health with a particular focus to predicting increased risk of falls for residents at 16 assisted living houses being built in Scotland. Simple ambient sensors are employed to monitor activities of daily living. We propose a two stage approach in which activities are first classified based on low level sensor data inputs. Daily/weekly activity profiles are then assembled for each resident and compared to their own past data and known risky profiles.
Key contributions of the work are: outlining a novel solution for identifying the risk of falls for Smart Home residents; and evaluating window-based representations for activity recognition from the low-level, data inputs delivered from sensors.
Acknowledgements This work was part funded by The Scottish Funding Council via The Data Lab innovation centre. Thanks also to Matt Stevenson at Carbon Dynamic and Angus Watson at NHS Highland, Inverness for their support of the FITsense project.