Detecting Signatures of Early-stage Dementia with Behavioural Models Derived from Sensor Data

Abstract

There is a pressing need to automatically understand the state and progression of chronic neurological diseases such as dementia. The emergence of state-of-the-art sensing platforms offers unprecedented opportunities for indirect and automatic evaluation of disease state through the lens of behavioural monitoring. This paper specifically seeks to characterise behavioural signatures of mild cognitive impairment (MCI) and Alzheimer's disease (AD) in the early stages of the disease. We introduce bespoke behavioural models and analyses of key symptoms and deploy these on a novel dataset of longitudinal sensor data from persons with MCI and AD. We present preliminary findings that show the relationship between levels of sleep quality and wandering can be subtly different between patients in the early stages of dementia and healthy cohabiting controls.

-

Dementia is a progressive neurological condition affecting cognition and behaviour with a significant impact on activities of daily living. It is one of the major causes of disability and dependency among elderly population with approximately 50 million people with the disease world wide. Alzheimer’s disease (AD) is the most common cause of dementia. Mild cognitive impairment (MCI) refers to a decline that has minimal impact on activities of daily living [ 13 ]. Patients with MCI can continue to live independently [ 26 ]. Not all patients with MCI will convert to dementia, but they are at increased risk. MCI has an annual conversion rate to AD of between 5 and 10% [ 27 ].

Making an accurate, reliable diagnosis of dementia is a challenge. Patients may find cognitive tests stressful, which impacts their performance. This may be exacerbated in a new, unknown, clinical environment with an unfamiliar clinician. Performance on cognitive tests also does not give an indication of how someone is managing in the real world. Although diagnosis may include bio-markers measurement, in addition to this being invasive and expensive, these do not have adequate temporal resolution to understand disease progression and deterioration since AD has high variance across patients.

Some symptoms are prevalent across dementias. Confusion can lead to unpredictable behaviour and activity abandonment during normal daily routines. Interestingly, behavioural interactions between persons with dementia (PwD) and their carers/family are also affected by disease progression and efforts by PwD to allay confusion and uncertainty often result in them persistently seeking the company of persons whom they trust. In this work we intend to address shadowing [ 23 ] (persistently staying in the company of a trusted carer/family), wandering [ 24 ] (going from room to room with nonspecific intent) and disturbed sleep [ 15 ] (a common symptom for PwD).

Progression of dementia and MCI are characterised by an increase in symptoms that will affect normal daily activity and behaviour. It is important to remember that these symptoms affect normal daily activity and behaviour. In understanding the dependence between normal behaviour and the expression of symptoms, an opportunity to understand and quantify disease state indirectly via behavioural change is exposed. Modern sensing technology offers promise in monitoring daily behaviour, extracting symptom expression rates and measurement of state and progression of the disease [ 7, 30 ]. In contrast to normal clinical evaluation that occur in foreign environments, ours is achieved from the patients’ residences.

This paper outlines novel computational behavioural analysis algorithms for modelling disease state and progression. The potential for detecting unseen behavioural bio-markers of the early stages of dementia is exposed by analysis on the longitudinal, in-home sensor data. Section 2 reviews prior work on smart homes and dementia. Our data collection and modelling procedures are outlined in Section 3. Results are presented in Section 4 and we conclude in Section 5. 2

Related Work

Numerous smart systems have been designed and developed to monitor the well-being and health status of elders, such as the GATOR Tech Smart House [ 14 ], the AWARE home [ 1 ], the Microsoft’s EasyLiving project [ 4 ] and the MavHome Project [ 6 ]. In the dementia domain, [ 22 ] introduced a support system to aid doctors diagnosing dementia. Participants only needed to perform a selection of Instrumental Activities of Daily Living (IADL) in a smart home environment. However, the main limitation is that this approach relies on data collected in a laboratory environment and not in a real world setting.

Along these lines, [ 3 ] undertook a study on data acquired by The ORegon Centre for Aging and TECHnology (ORCATECH) at the Oregon Health and Science University, that investigated the walking speed and general activity in the homes of participants in order to detect dementia. The extension presented in [ 2 ] modelled the presence of the participants in different rooms using Poisson processes to demonstrate statistical differences between different states of cognition. Localisation has been found to be a common ground for different works. For instance [ 18 ] and [ 29 ] use an Ultra Wideband device to try to measure wandering behaviours. However, Radio Frequency (RF) identification may be limited by its detection range since unpowered RF devices operate only over a meter’s distance. Also, [ 10 ] uses a relatively complex sensor network consisted of RF tracking and motion and heading sensors for monitoring and studying behavioural patterns of patients with dementia.

We build upon the existing literature by focusing on data collected in the wild that, while more challenging to analyse, also provides a richer and pervasive perspective on the subjects’ behaviour. Additionally, our aim is to automatically detect behavioural symptoms that may suggest early dementia. Finally, we want go beyond the analysis of a single participant and incorporate the monitoring of interactions within the home environment as a key element to understand the progression of the disease. 3

Methods 3.1

Data collection

This section introduces our data collection and modelling pipelines. Ethical approval was secured from the Carmarthen Research Ethics Committee and the Health Research Authority (HRA) to record sensor data from the homes of people who have received a diagnosis of dementia or have MCI for up to 12 months using the SPHERE [ 36, 35, 8 ] system. Informed consent was obtained to analyse this data to determine links between behavioural patterns, residents of the home and disease state. Participants are recruited from North Bristol NHS Trust, the Dementia Wellbeing Service in Bristol and the Research Institute for the Care of Older People (RICE) Centre in Bath. The SPHERE sensor system is installed into their homes [ 36, 35 ].

The key sensors that we use in the analysis of this work are accelerometers [ 11, 12, 34 ]. These are set record at a rate of 20 Hz. This data is broadcast to gateways by means of radio links. Six gateways are installed in every house. These simultaneously record accelerometer readings and the Received Signal Strength Intensity (RSSI).

In-home localisation is a key task in this work that leverages the RSSI data [ 20, 5, 19 ]. Labelled data are acquired from a ‘walkthrough’ script that is performed by the technician upon installation. Each room is visited while holding the wearables and RSSI and wearable data are simultaneously recorded. The real-time location of the technician is recorded with an annotation app.

To regress from sensor data to localisation and activity predictions, we extract basic features from windowed data (5 seconds length with overlap of 2•50 seconds following [ 33 ]). Features include: mean, standard deviation (std), max, min, diff (first-order difference operator) and the count of missing values.

We can complement labelled data by introducing domain knowledge into the analysis by suggesting reasonable locations based on calculable context. For example, if predicting ‘sleep’ at 3AM it is likely that the resident is in the bedroom. 3.2 3.2.1

Modelling considerations

Covariate Shift

We account for our expectations of covariate shift (due to disease progression) with Maximum Mean Discrepancy (MMD) [ 17 ]. In mathematical terms we assume that training and testing distributions differ, i.e. %CA ¹Gº < %C4 ¹Gº, but that conditional distributions remain the same %CA ¹HjGº = %C4 ¹HjGº. MMD offers an elegant approach to quantify and match distributional shift and we follow the methodology outlined by [ 17 ] in our work. 3.2.2

Semi-Supervised Learning

Since only a small portion of labelled data are available, we leverage supervised (L(! ) semi-supervised (L((! ) objectives to make efficient use of data. The supervised objective (L(! ) is the traditional conditional log-likelihood that one optimises for in Conditional Random Fields (CRF) [ 31 ]. The data and labels for this are obtained from the walkthrough that was introduced in the previous section. CRF model sequence probabilities as: ) % \0 ¹ yj ^º = / ¹ ^º 1 exp Õ qC ¹GC – HC – HC 1º

! C=1 where qC are the log-potentials and / ¹ ^º is the partition function. In our case the log-potentials decompose, to emission function and transition function, as follows: qC ¹GC – HC – HC 1º = D ¹GC – HC º g˜ ¹HC – HC 1º, where: g˜ ¹HC – HC 1º = g ¹HC – HC 1º– IHC =HC 1 –

U¹Cº ¡= n 0 otherwise where U¹Cº is the activity level of the participant at time C (calculated as the average absolute jerk of the tri-axial signals) and n 0 denotes a threshold, below which we assume there is no movement. This forbids location transitions unless a certain activity potential is surpassed.

The emission potentials are modelled with a 4-layer fullyconnected neural network, D ¹G– Hº = ¹GºH , i.e. the H-th element of ¹Gº. The semi-supervised objective (L((! ) constructed by iteratively using the most likely label sequence as temporary targets, i.e. y = arg maxy 2Y %¹ yj ^º [ 28 ]. 4

Findings

In this section we present findings relevant to sleeping disturbance, wandering and shadowing. We illustrate the analysis using 2 different houses that are inhabited by either a PwD, or by someone who has been founds to have MCI, and one partner. Data collection is ongoing, so these are preliminary findings and some houses have significantly more data than others. The analyses below result from localisation and activity level predictions that were trained and validated on the walkthrough data. 4.1

Shadowing

The mutual information (MI) between the location of the two residents in each house is measured to estimate shadowing, following a similar approach to that described in [ 32 ]. Additionally, in our analysis we stratify MI by time of the day in order to understand whether significant MI is confined to particular intervals, e.g. morning Fig.1.

Temporal lags were introduced in order to allow for short delays in shadowing, with no significant change, and are therefore not included in this work. This is likely due to the fact that the lag interval is significantly shorter than the analysis window. Looking at Fig.1, in the case of house B there is no trend of increasing/decreasing correlation. Even though in the case of house A, there is a slight increase over time, stronger conclusions will be drawn when more data arrives. (1) (2) 0.015 0.010 0.005 Our data collection is ongoing, and as it grows we will identify finer, more informative signals of disease state. Time lags are likely to become more relevant with finer and more targeted analysis that may be motivated by changes in behaviour as the disease progresses. The approach for modelling wandering is motivated by its antonym: non-wandering is typified by low activity levels and persistent location predictions. Thus we jointly compare the activity levels and the location complexity of residents in this section. Specifically, in Fig. 2 we track the total activity per room, as well as the total activity of the day, over the span of the data, and in Fig. 3 we track the complexity of the localisation predictions, derived from the Lempel-Ziv [ 21 ] complexity measure. It is clear in Fig. 2b that the activity levels of the patient are higher than that of the other participant in this house, and Fig. 3b gives similar results, although in Fig. 2a and Fig. 3a it is less obvious, for which there might be several reasons. One explanation could be that the data we have so far for house A spans only a short time period (less than 10 days). A second cause might be that the patient in house A only has MCI, whose daily life only gets affected to a mild degree compared to the participant with dementia in house B.

Additionally, Fig. 5 shows in detail the complexity of localisation predictions in different time intervals over the experiment span in house B. On every subplot, the number at the top-right is the average number of times the patient has a higher complexity than the partner. In this analysis, we only include house B, as the span is longer. In the case of house A these averages are not so much in favour of the patient - 58 segment as compared to 88 for house B. 4.3

Sleep Disturbance

In both houses, we observe that during the early hours of the day, 0-6 am, the participants have a higher activity levels, while the control/carers have a more steady, and at most times, lower activity levels. This might be an indication of the patients having a more uneasy sleep. As a number of studies suggest [ 25, 15 ], people have MCI or dementia will suffer from different levels of sleep disturbance. And a community-based study [ 16 ] indicates that nearly one-third of their subjects were reported as having been inappropriately active at night.

As can be seen from Fig. 2b, the line representing the activity levels in bedroom 2 of the participant with dementia is quite unstable compared to that of the other participant during 0-6 am. Although there is not a significant difference between activity levels in bedroom 1 of the two inhabitants during 0-6 am in Fig. 2a, it is because that the person with MCI got up earlier than 6 am, especially on July 23rd, 24th and 25th, and started walking in the house while the partner was sleeping, which may suggest the increased sleep disturbance in the person with cognitive disorder from another angle.

For reference, we also include a segment of the RSSI data in Fig. 4, from House A, for both participants, from the early hours of the day. It highlights the difference in sleeping quality, as observed through RSSI. The step changes are a result of rotations during sleep. It also shows how different sleeping positions can lead to missing values in certain gateways, e.g. kitchen_1 at 02:00 and at 03:15. 5

Conclusion and Future Work

This paper presents our preliminary results and shows the potential of detecting early symptoms of cognitive disorders automatically by utilizing data acquired from patients’ daily activities and signal processing methods equipped with machine learning techniques. We build a machine learning pipeline that considers the difference in training and testing distributions using MMD. We account for the sequential nature of data using a CRF, and use our domain knowledge to enhance our training data with data coming from the participants.

One of the limitations of this study is that our localisation precision can only reach room-level for now due to the configuration and the deployment of our sensor system. Finer-grained localisation will expose behavioural reactions to furniture placements across rooms.

Potential directions for improvements include taking into consideration environmental sensors, such as Passive Infrared and Video, through machine learning techniques such as multi-modal data fusion [ 9 ]. Moreover, we will enhance our training data using clinical data through our domain knowledge of activities of daily living.

The paper as it stands establishes baseline activity, sleep and colocalisation dependence that will be used after data collection completed. That we see signals here is promising for our future data collection, analysis and research.

Acknowledgements

This research was funded by CUBOID (UK MRC Momentum grant MC/PC/16029) and the SPHERE IRC (grant EP/K031910/1).

References

[1] Gregory D Abowd , Christopher G Atkeson, Aaron F Bobick, Irfan A Essa, Blair MacIntyre , Elizabeth D Mynatt , and Thad E Starner, ' Living laboratories: the future computing environments group at the georgia institute of technology' , in CHI'00 Extended Abstracts on Human Factors in Computing Systems , pp. 215 - 216 , ( 2000 ).

[2] Ahmad Akl , Jasper Snoek, and Alex Mihailidis, ' Unobtrusive detection of mild cognitive impairment in older adults through home monitoring' , IEEE journal of biomedical and health informatics , 21 ( 2 ), 339 - 348 , ( 2015 ).

[3] Ahmad Akl , Babak Taati, and Alex Mihailidis, ' Autonomous unobtrusive detection of mild cognitive impairment in older adults' , IEEE transactions on biomedical engineering , 62 ( 5 ), 1383 - 1394 , ( 2015 ).

[4] Barry Brumitt , Brian Meyers, John Krumm, Amanda Kern, and Steven Shafer, 'Easyliving: Technologies for intelligent environments' , in International Symposium on Handheld and Ubiquitous Computing , pp. 12 - 29 . Springer, ( 2000 ).

[5] Dallan Byrne , Michal Kozlowski, Raul Santos-Rodriguez, Robert Piechocki , and Ian Craddock, ' Residential wearable rssi and accelerometer measurements with detailed location annotations' , Scientific Data , 5 , ( August 2018 ).

[6] Diane J Cook, ' Health monitoring and assistance to support aging in place .', J. UCS , 12 ( 1 ), 15 - 29 , ( 2006 ).

[7] Ana Lígia Silva De Lima , Tim Hahn, Luc JW Evers, Nienke M De Vries , Eli Cohen, Michal Afek, Lauren Bataille, Margaret Daeschler, Kasper Claes, Babak Boroojerdi , et al., 'Feasibility of large-scale deployment of multiple wearable sensors in parkinson's disease' , PLoS One , 12 ( 12 ), ( 2017 ).

[8] Tom Diethe , Mike Holmes, Meelis Kull, Miquel Perello Nieto, Kacper Sokol, Hao Song, Emma Tonkin, Niall Twomey, and Peter Flach, ' Releasing ehealth analytics into the wild: Lessons learnt from the sphere project' , in Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pp. 243 - 252 , ( 2018 ).

[9] Tom Diethe , Niall Twomey, Meelis Kull, Peter Flach , and Ian Craddock, ' Probabilistic sensor fusion for ambient assisted living' , arXiv preprint arXiv:1702.01209 , ( 2017 ).

[10] Matthew D'Souza , Montserrat Ros , and Mohanraj Karunanithi, ' An indoor localisation and motion monitoring system to determine behavioural activity in dementia afflicted patients in aged care', ( 2012 ).

[11] Atis Elsts , Tilo Burghardt, Dallan Byrne, Massimo Camplani, Dima Damen, Xenofon Fafoutis, Sion Hannuna, William Harwin , Michael Holmes, Balazs Janko , et al., 'A guide to the sphere 100 homes study dataset' , arXiv preprint arXiv: 1805 . 11907 , ( 2018 ).

[12] Xenofon Fafoutis , Antonis Vafeas, Balazs Janko, R Simon Sherratt , James Pope, Atis Elsts, Evangelos Mellios, Geoffrey Hilton, George Oikonomou, Robert Piechocki , et al., ' Designing wearable sensing platforms for healthcare in a residential environment' , EAI Endorsed Transactions on Pervasive Health and Technology , 3 ( 12 ), ( 2017 ).

[13] Michael Grundman , Ronald C Petersen, Steven H Ferris , Ronald G Thomas, Paul S Aisen, David A Bennett, Norman L Foster, Clifford R Jack Jr , Douglas R Galasko, Rachelle Doody , et al., ' Mild cognitive impairment can be distinguished from alzheimer disease and normal aging for clinical trials' , Archives of neurology , 61 ( 1 ), 59 - 66 , ( 2004 ).

[14] Sumi Helal , William Mann , Hicham El-Zabadani, Jeffrey King, Youssef Kaddoura , and Erwin Jansen, ' The gator tech smart house: A programmable pervasive space' , Computer , 38 ( 3 ), 50 - 60 , ( 2005 ).

[15] Eva Hita-Yañez , Mercedes Atienza, Eulogio Gil-Neciga, and Jose L Cantero, ' Disturbed sleep patterns in elders with mild cognitive impairment: The role of memory decline and apoe Y 4 genotype' , Current Alzheimer Research , 9 ( 3 ), 290 - 297 , ( 2012 ).

[16] RA Hope and Christopher G Fairburn, 'The nature of wandering in dementia: A community-based study' , International journal of geriatric psychiatry , 5 ( 4 ), 239 - 245 , ( 1990 ).

[17] Jiayuan Huang , Arthur Gretton, Karsten Borgwardt, Bernhard Schölkopf, and Alex J Smola, ' Correcting sample selection bias by unlabeled data' , in Advances in neural information processing systems , pp. 601 - 608 , ( 2007 ).

[18] William D Kearns , Donna Algase , D Helen Moore , and Sadia Ahmed, ' Ultra wideband radio: A novel method for measuring wandering in persons with dementia' , Gerontechnology , 7 ( 1 ), 48 , ( 2008 ).

[19] Michal Kozlowski , Dallan Byrne, Raul Santos-Rodriguez, and Robert Piechocki, 'Data fusion for robust indoor localisation in digital health' , in 2018 IEEE Wireless Communications and Networking Conference Workshops (WCNCW) , pp. 302 - 307 , ( 2018 ).

[20] Michal Kozlowski , Raul Santos-Rodriguez, and Robert J. Piechocki , ' Sensor modalities and fusion for robust indoor localisation' , EAI Endorsed Transactions on Ambient Systems , 6 ( 18 ), (12 2019 ).

[21] Abraham Lempel and Jacob Ziv, ' On the complexity of finite sequences' , IEEE Transactions on information theory , 22 ( 1 ), 75 - 81 , ( 1976 ).

[22] Ting-Ying Li , Chao-Lin Wu , Yi-Wei Chien , Li-Chen Fu , Chi-Chun Chou , Chun-Chen Chou , and I-An Chen , ' A supporting system for quick dementia screening using pir motion sensor in smart home' , in 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC) , pp. 1369 - 1374 . IEEE, ( 2017 ).

[23] Mary L Lilly , Melinda Hermanns, Jan Beckstrand, Adam Booth, and Martin R Farlow, ' Illumination of shadowing behavior in individuals with alzheimer's disease: Proximity-seeking? as life becomes the “strange situation”' , The Internet Journal of Neurology , 13 ( 2 ), ( 2011 ).

[24] Constantine G Lyketsos , Maria C Carrillo, J Michael Ryan , Ara S Khachaturian, Paula Trzepacz, Joan Amatniek, Jesse Cedarbaum, Robert Brashear, and David S Miller. Neuropsychiatric symptoms in alzheimer's disease , 2011 .

[25] Bryce A Mander , Joseph R Winer , William J Jagust, and Matthew P Walker, ' Sleep: a novel mechanistic pathway, biomarker, and treatment target in the pathology of alzheimer's disease?' , Trends in neurosciences, 39 ( 8 ), 552 - 566 , ( 2016 ).

[26] Guy M McKhann , David S Knopman , Howard Chertkow, Bradley T Hyman, Clifford R Jack Jr , Claudia H Kawas , William E Klunk, Walter J Koroshetz, Jennifer J Manly, Richard Mayeux, et al., ' The diagnosis of dementia due to alzheimer's disease: recommendations from the national institute on aging-alzheimer's association workgroups on diagnostic guidelines for alzheimer's disease' , Alzheimer's & dementia, 7 ( 3 ), 263 - 269 , ( 2011 ).

[27] Alex J Mitchell and Mojtaba Shiri-Feshki, 'Rate of progression of mild cognitive impairment to dementia-meta-analysis of 41 robust inception cohort studies' , Acta Psychiatrica Scandinavica , 119 ( 4 ), 252 - 265 , ( 2009 ).

[28] Alexander M. Rush . Torch-struct: Deep structured prediction library , 2020 .

[29] Volker Schwarz , Alex Huber, and Michael Tuchler, ' Accuracy of a commercial uwb 3d location/tracking system and its impact on lt application scenarios' , in 2005 IEEE International Conference on Ultra-Wideband , pp. 599 - 603 . IEEE, ( 2005 ).

[30] Emma Stack , Veena Agarwal, Rachel King, Malcolm Burnett, Fatemeh Tahavori, Balazs Janko, William Harwin, Ann Ashburn, and Dorit Kunkel, ' Identifying balance impairments in people with parkinson's disease using video and wearable sensors' , Gait & posture, 62 , 321 - 326 , ( 2018 ).

[31] Charles Sutton , Andrew McCallum , et al., 'An introduction to conditional random fields' , Foundations and Trends® in Machine Learning , 4 ( 4 ), 267 - 373 , ( 2012 ).

[32] Niall Twomey , Tom Diethe, Ian Craddock , and Peter Flach, ' Unsupervised learning of sensor topologies for improving activity recognition in smart environments' , Neurocomputing, 234 , 93 - 106 , ( 2017 ).

[33] Niall Twomey , Tom Diethe, Xenofon Fafoutis, Atis Elsts, Ryan McConville , Peter Flach , and Ian Craddock, ' A comprehensive study of activity recognition using accelerometers' , in Informatics, volume 5 , p. 27 . Multidisciplinary Digital Publishing Institute, ( 2018 ).

[34] Antonis T Vafeas , Xenofon Fafoutis, Atis Elsts, Ian J Craddock, Md Israfil Biswas, Robert J Piechocki , and George Oikonomou, ' Wearable devices for digital health: The sphere wearable 3', in Embedded Wireless Systems and Networks (EWSN): On-Body Sensor Networks (OBSN 2020 ), ( 2020 ).

[35] Przemyslaw Woznowski , Alison Burrows, Tom Diethe, Xenofon Fafoutis, Jake Hall, Sion Hannuna, Massimo Camplani, Niall Twomey, Michal Kozlowski, Bo Tan , et al., 'Sphere: A sensor platform for healthcare in a residential environment' , in Designing, Developing, and Facilitating Smart Cities, 315 - 333 , Springer, ( 2017 ).

[36] Ni Zhu , Tom Diethe, Massimo Camplani, Lili Tao, Alison Burrows, Niall Twomey, Dritan Kaleshi, Majid Mirmehdi, Peter Flach , and Ian Craddock, ' Bridging e-health and the internet of things: The sphere project' , IEEE Intelligent Systems , 30 ( 4 ), 39 - 46 , ( 2015 ).