Using multi-sensor tracking data to analyze the mobility and activity behavior of older adults Hoda Allahbakhshi Robert Weibel University of Zurich University of Zurich Department of Geography Department of Geography Winterthurerstrasse 190 Winterthurerstrasse 190 8057 Zurich, Switzerland 8057 Zurich, Switzerland Hoda.Allahbakhshi@geo.uzh.ch Robert.Weibel@geo.uzh.ch The increase in the older adult population has been occurring at unprecedented and accelerating rates in recent decades.Healthy ageing as the process of maintaining the functional mobility is therefore important. In order to understand how functional mobility in daily life is associated with health in older adults, such behaviour needs to be studied in real-life conditions, which can be done using sensor-based ambulatory assessment methods. The aim of this study is to contribute to developing a full, individualized description of human mobility behavior considering different spatio-temporal patterns, and link such personal mobility profiles to psychological resources available to an individual. The participants are healthy older adults above 65 years old from MOASIS study, who will collect the data during 4 weeks of their everyday life. The multi-sensor data will be used for the movement analysis. Pattern recognition and classification algorithms are proposed methodologies to achieve the aim of this study. This paper is quite useful for understanding of individual movement through the use of new sensing devices. Keywords: movement analysis, ambulatory assessment, real-life, older adults 1 Introduction everyday concerns and activities and subjects them to an artificial environment in which nearly all contextual factors – Projections indicate that by 2050 the elderly population will for example physical features, goals, or other persons involved reach 2 billion people worldwide (Toledo & Barela, 2010) – are determined by the experimenter (Mehl & Conner, 2012). Along with these demographic changes, medical conditions In field settings (i.e. real-life contexts), in contrast, the associated with aging will represent a burden to society, for physical and social environment is substantially cluttered, example by an increase in demand for health services. Because people must choose for themselves which tasks to pursue and older adults demand more from the health service how to engage them and the option of changing the setting and infrastructure, efforts have been made to understand the factors tasks is usually available. All of these can, of course, alter the that contribute to healthy aging (Toledo & Barela, 2010). The results of the research (Mehl & Conner, 2012). In essence, we WHO, in its “World Report on Ageing and Health”, defines find the inverse of the laboratory situation, that is, a high healthy aging as the process of maintaining the “functional ecological validity of the results at the price of reduced internal ability” of individuals through a dynamic interplay between an validity. individual’s biological and physiological endowments, There is some flexibility in what counts as a method for abilities, skills, diseases, subjective evaluations, traits, studying daily life. Among the terms used for studying daily environments, and real-life activities (Sugawara & Nikaido, life we find, among others, the term ambulatory assessment 2014). (AA). Alternative labels for this methodology are ecological To describe and analyse functional mobility, then, one needs momentary assessment and experience sampling methodology to measure all mobility-related biological and physiological (Brose & Ebner-Priemer, 2015). endowments, mobility-related abilities, skills, diseases, The aim of this study is by utilizing ambulatory assessment subjective evaluations, traits, contexts, and activities, as methods to contribute to developing a full, individualized potentially these are all equally relevant parts of an individual’s description of human mobility behavior considering different functional mobility profile, uniquely characterizing an spatio-temporal patterns, and link such personal mobility individual. Research on some of these elements of a mobility profiles to psychological resources available to an individual. profile has been carried out in mobility laboratories or in To do so, we aim to address the following research questions: clinical and experimental studies. The advantages of laboratory experimentation have a price, however, as the laboratory setting by definition isolates research participants from their 1. What are the main movement behaviors expressed in Using GPS devices and accelerometers together provides the people’s movement using multi-sensor data? most complete information about human mobility in community environments. Although combined GPS and 2. What types of movement patterns do the individuals accelerometer technologies have been used successfully to show considering different temporal granularities gather detailed information about discrete bouts of outdoor (daily, weekdays, weekends, and particular days, activity (physical endeavors, as well as driving), the same weekly)? success has not been realized in studies that have attempted to monitor functional everyday human movement over an 3. To what extent does context/environment play a role extended period of time. These technologies, however, would in the human movement behavior at the micro and offer the potential to accurately monitor mobility patterns in macro level? And can we find patterns that match older adults (Webber & Porter, 2009). certain psychological trait. In order to study the movement behavior of moving objects, 2 Background it is important to understand what types of movement patterns can be identified from their movement (Dodge et al. 2008). The WHO in its “World Report on Ageing and Health” defines Among different movement pattern detection methods, healthy aging as the process of maintaining the “functional periodic pattern mining (PPM) can be used for discovering the ability” of individuals through a dynamic interplay between an intrinsic behavior of moving objects, compressing movement individual’s biological and physiological endowment data (Agrawal & Srikant, 1995), predicting future movements (Sugawara & Nikaido, 2014). Therefore, on the one hand, to of objects (Jeung et al. 2008), and detecting abnormal events. describe and analyse functional mobility, one needs to measure Mining periodic behaviors can bridge the gap between raw data all mobility-related biological and physiological endowments, and semantic understanding of the data (Li et al., 2010). and on the other hand, to achieve the ecologically valid It is difficult to study human mobility without considering its measurements needs considering real-life contexts, which can temporal nature. It has been shown that both the ordering of be realized using ambulatory assessment methods. Most visits and the timing of visits (Song et al. 2010) contains existing examples of ambulatory assessment fall into one of the information that can be used to build powerful predictors of two broad categories below: future behavior. Furthermore, human behavior is driven by daily and weekly routines ((Williams et al., 2012; Scellato et The first and most common category includes self-reports. al., 2010). Although this form of temporal structure is a rich Self-reports provide information that no one but the respondent source of information about individual behavior, there has been knows including goals, emotions, thoughts etc., which is why little work to examine the regularity in individual visiting many theories about human behavior and interventions focus patterns. Factors such as wealth, profession, lifestyle, and on them (Brose et al. 2015; Ebner-Priemer et al., 2013a; Ebner- health affect an individual’s routine, and therefore his or her Priemer et al., 2013b; Niermann et al., 2016). mobility patterns. This is likely to give rise to diversity in the population’s visiting patterns and regularity (Williams et al. The second and newer category includes more technically 2012). oriented methods for capturing diverse, non-self-reported The literatures show that there is still little research on using aspects of everyday experience, such as the auditory real-life datasets for movement analysis and compared to self- environment, physiological status, the physical location or reported ambulatory assessment using sensor-based proximity to particular other persons etc., all of which can be ambulatory assessment in mobility and activity analysis is provided by using different sensors including Bluetooth, RFID, limited. Considering temporal information in human GPS, accelerometer, heart rate sensors, audio sensors, etc. movement pattern is also a topic that requires further studies. These instruments provide extensively detailed data that can be used to examine the operation of social, psychological, and 3 Dataset physiological processes within their natural contexts (Verlaan et al., 2015; Zisko et al., 2015; Reichert et al., 2016). The Mobility, Activity and Social Interaction Study (MOASIS) collects individualized everyday-life health-related data in There is much research that shows that two of the most older adults. MOASIS started in August 2015 and ultimately frequent sensors used in movement analysis are GPS and aims to develop computational models to measure, analyze, and accelerometer (Kaghyan 2013; Spink et al. 2013). In addition improve health behaviors and health outcomes in the everyday to sensing different aspects of a person’s life (GPS = position, life of aging individuals (Bereuter et al., 2016). The mobile spatial activity; accelerometer = physical activity), each of sensor uTrail is used for the data collection, assuming no prior these two sensors also provides us information about different technical knowledge by the participants. uTrail, a tracker scales of movement. For example, by extracting information specifically developed for this study, measures the mobility from an accelerometer we may explore human activity at the (spatial activity) with GPS, physical activity with a 3-axis micro-scale (e.g. physical activity mode, body motion, number accelerometer and social interaction with a microphone using of steps, gesture change, intensity, duration, etc.), while by the electronically activated recorder (EAR) method (Mehl & analyzing GPS data we get to know about the macro-scale of Conner, 2012). human movement (e.g. point of interest, transportation modes, displacement, speed, etc.). The MOASIS initialization phase started in November 2015 including initial device testing and ethical approval. After that a first pilot study with 5 participants during 14 days took place The impact of context on behavior is fundamental. To in December 2015, focusing primarily on testing and understand behavior, one had to first understand what sorts of improving the sensor device. behavior the setting – its context – was likely to evoke. Thus, The second pilot study ran from March to April 2016 with 27 there is a need to identify regularities in the properties of participants during 30 days. Further testing and refinement of behavior setting (e.g. home, social activity places, medical the device, as well as the data collection protocol sampling offices or roadways) and the behavioral patterns that they evoke rates, and observation length were included in this stage. The (Mehl & Conner, 2012). main data collection with 150 participants during 30 days will take place in the first half of 2018. 4 Proposed methodology and discussion 5 References To address the research questions, the methodologies below are Agrawal, R., & Srikant, R. (1995) Mining sequential patterns. proposed: Proc. of the 11th Int. Conf. on Data Engineering, 3–14. Available from: http://doi.org/10.1016/j.jbi.2007.05.004 1. What are the main movement behaviors expressed in regular patterns of people’s movement using multi-sensor data Bereuter P., Fillekes M.P., W. R. (2016) Analyzing GPS and and how do they interrelate with each other? Accelerometer Data in the Study of the Mobility, Activity and Travel behavior and physical activity (PA) are two important Social Interaction of Older Adults. In Urska Demsar, Aidan movement behaviors in older adults’ daily routine. To figure Slingsby RW (ed) Visually-supported Computational out how variable or predictable older adults are in their Movement Analysis (VCMA 2016). routines, one way is to investigate their movement patterns by answering the questions of when, how and where they go Brose, A., & Ebner-Priemer, U. W. (2015) Ambulatory during the day (travel behavior) and what they do meanwhile Assessment in the Research on Aging: Contemporary and (physical activity). Having this, the association between Future Applications. Gerontology, 61(4), 372–380. Available movement patterns and the way it affects older adults’ physical from: http://doi.org/10.1159/000371707 and mental well-being can be distinguished. 2. What types of regular or irregular patterns do the De Montjoye, Y.-A., Hidalgo, C. A., Verleysen, M., & Blondel, individuals show considering different temporal granularities V. D. (2013) Unique in the Crowd: The privacy bounds of (daily, weekdays, weekends, and particular days, weekly)? human mobility. Scientific Reports, 3, 1376. Available from: In order to study the movement behaviors of moving objects, http://doi.org/10.1038/srep01376 it is important to understand what types of movement patterns can be identified from their movement. A considerable number Dodge, S., Weibel, R., & Lautenschütz, A.-K. (2008) Towards of works have shown that human mobility is regular, a taxonomy of movement patterns. Information Visualization, predictable and unique in both the temporal and spatial domains 7(April), 240–252. (De Montjoye et al., 2013; Kim et al. 2007; Wang et al. 2011). Observable regular movements among a few frequented Eagle, N., & Pentland, A. S. (2009) Eigenbehaviors: locations, such as home and work (Eagle & Pentland, 2009; Li identifying structure in routine. Behavioral Ecology and et al., 2010) embody the regularity and predictability of human Sociobiology, 63(7), 1057–1066. mobility. It is claimed that human trajectories show a high degree of Ebner-Priemer, U. W., Koudela, S., Mutz, G., & Kanning, M. temporal and spatial regularity, each individual being (2013a) Interactive multimodal ambulatory monitoring to characterized by a time independent characteristic travel investigate the association between physical activity and affect. distance and a significant probability to return to a few highly Frontiers in Psychology, 3, 596. frequented locations (Liang et al. 2012; González et al., 2008). Even older adults without cognitive problems feel safer and less Ebner-Priemer, U. W., Koudela, S., Mutz, G., & Kanning, M. anxious when they do not have to worry about ‘the unknown’ (2013b) Interactive multimodal ambulatory monitoring to or what is coming next (Lythgoe, 2016) in their mobility. It is investigate the association between physical activity and affect. easier to cope with memory and cognitive issues when as many Frontiers in Psychology, 3, 596. activities as possible are predictable. Even though a person with cognitive problems might not be aware of the routine or even González, M. C., Hidalgo, C. a., & Barabási, A.-L. (2008) of the time passing, having a routine helps them feel more Understanding individual human mobility patterns. Nature, grounded and secure (Lythgoe, 2016), and consequently 453(June), 779–782 maintain their mental health. The one way to address this research question is to apply human movement pattern Jeung, H., Liu, Q., Shen, H. T., & Zhou, X. (2008) A hybrid recognition methods and see what types of patterns could be prediction model for moving objects. In: Proceedings - derived from the dataset considering temporal information. International Conference on Data Engineering, IEEE, 70–79. 3. To what extent does context/environment play a role in the human movement behavior at the micro and macro level? Kaghyan, S., (2013) Accelerometer and GPS sensor And can we find patterns that match certain psychological combination based system for human activity recognition. In: traits? Computer Science and Information Technologies (CSIT), 2013, with those of the AcrAB-TolC system of Escherichia coli. 1–9. Antimicrobial Agents and Chemotherapy, 58(12), 7250–7257. Kim, M., & Kotz, D. (2007) Periodic properties of user Toledo, D. R., & Barela, J. A. (2010) Sensory and motor mobility and access-point popularity. Personal and Ubiquitous differences between young and older adults: somatosensory Computing, 11(6), 465–479. contribution to postural control. Brazilian Journal of Physical Therapy, 14(3), 267–275. Li, Z., Ding, B., Han, J., Kays, R., & Nye, P. (2010) Mining periodic behaviors for moving objects. In: Proceedings of the Verlaan, L., Bolink, S. A. A. N., Van Laarhoven, S. N., 16th ACM SIGKDD international conference on Knowledge Lipperts, M., Heyligers, I. C., Grimm, B., & Senden, R. (2015) discovery and data mining, New York, USA: ACM Press, 1099. Accelerometer-based physical activity monitoring in patients with knee osteoarthritis: Objective and ambulatory assessment Liang, Y., Zhou, X., Guo, B., & Yu, Z. (2012) Understanding of actual physical activity during daily life circumstances. Open the regularity and variability of human mobility from geo- Biomedical Engineering Journal, 9, 157. trajectory. In: Proceedings - 2012 IEEE/WIC/ACM International Conference on Web Intelligence, WI 2012, 409– Wang, D., Pedreschi, D., Song, C., Giannotti, F., & Barabasi, 414. A.-L. (2011) Human mobility, social ties, and link prediction. In: Proceedings of the 17th ACM SIGKDD international Lythgoe, M., DTN Home Care, “Why a Daily Routine Is conference on Knowledge discovery and data mining, New Important for the Elderly & Others with Dementia,” (2016) York, New York, USA: ACM Press, 2011, 1100-1108 Available from: https://dtnhomecare.wordpress.com/2016/04/01/why-a-daily- Webber, S. C., & Porter, M. M. (2009). Monitoring Mobility in routine-is-important-for-the-elderly-others-with-dementia/ , Older Adults Using Global Positioning System (GPS) Watches (Accessed 2017, Jan.) and Accelerometers: A Feasibility Study. Journal of Aging & Physical Activity, 17(4), 455–467. Mehl, M. R., & Conner, T. S. (2012) Handbook of research methods for studying daily life. Handbook of Research Williams, M. J., Whitaker, R. M., & Allen, S. M. Methods for Studying Daily Life, 1–4. Available from: (2012).Decentralised detection of periodic encounter http://libproxy2.usouthal.edu/login?url=http://search.ebscohos communities in opportunistic networks. Ad Hoc Networks, t.com/login.aspx?direct=true&db=psyh&AN=2012-05165- 10(8), 1544–1556. 000&site=ehost-live Williams, M. J., Whitaker, R. M., & Allen, S. M. (2012) Niermann, C. Y. N., Herrmann, C. Von Haaren, B., Van Kann, Measuring individual regularity in human visiting patterns. In: D., & Woll, A. (2016) Affect and subsequent physical activity: Proceedings - 2012 ASE/IEEE International Conference on An ambulatory assessment study examining the affect-activity Privacy, Security, Risk and Trust and 2012 ASE/IEEE association in a real-life context. Frontiers in Psychology, 7, International Conference on Social Computing, 677. SocialCom/PASSAT 2012, 117–122. Reichert, M., Tost, H., Reinhard, I., Zipf, A., Salize, H.-J., Zisko, N., Carlsen, T., Salvesen, Ø., Aspvik, N. P., Meyer-Lindenberg, A., & Ebner-Priemer, U. W. (2016) Ingebrigtsen, J. E., Wisløff, U., & Stensvold, D. (2015) New Within-subject associations between mood dimensions and relative intensity ambulatory accelerometer thresholds for non-exercise activity: An ambulatory assessment approach elderly men and women: The Generation 100 study. BMC using repeated real-time and objective data. Frontiers in Geriatrics, 15(1), 97 Psychology, 7, 918. Scellato, S., Musolesi, M., Mascolo, C., & Latora, V. (2010) On Non stationarity of Human Contact Networks. In: 2010 IEEE 30th International Conference on Distributed Computing Systems Workshops, 105–111. Song, C., Qu, Z., Blumm, N., & Barabasi, A.-L. (2010) Limits of Predictability in Human Mobility. Science, 327(5968), 1018–1021. Spink, A. et al., (2013) Animal behaviour analysis with GPS and 3D accelerometers. In: Precision Livestock Farming, 10- 12 September, 2013, Leuven, Belgium. 229–239. Sugawara, E., & Nikaido, H. (2014) Properties of AdeABC and AdeIJK efflux systems of Acinetobacter baumannii compared