Learning to Self-Manage by Intelligent Monitoring, Prediction and Intervention Nirmalie Wiratunga1 , David Corsar1 , Kyle Martin1 , Anjana Wijekoon1 , Eyad Elyan1 , Kay Cooper1 , Zina Ibrahim2,3 , Oya Celiktutan2 , Richard J. Dobson2,3 , Stephen McKenna4 , Jacqui Morris4 , Annalu Waller4 , Raed Abd-Alhammed5 , Rami Qahwaji5 , Ray Chaudhuri6 1 Robert Gordon University, Aberdeen, UK 2 King’s College London, London, UK 3 University College London, London, UK 4 University of Dundee, Dundee, UK 5 University of Bradford, Bradford, UK 6 King’s College Hospital, NHS Foundation Trust, UK {n.wiratunga, d.corsar, k.martin, a.wijekoon, e.elyan, k.cooper}@rgu.ac.uk, {zina.ibrahim, oya.celiktutan, richard.j.dobson}@kcl.ac.uk, {s.j.z.mckenna, j.y.morris, a.waller}@dundee.ac.uk, {r.a.a.abd, r.s.r.qahwaji}@bradford.ac.uk, ray.chaudhuri@nhs.net Abstract Although the number of individuals living with multiple morbidities is predicted to increase significantly over the Despite the growing prevalence of multimorbidi- coming years [Barnett et al., 2012], current self-management ties, current digital self-management approaches solutions prioritise single conditions. A 2018 study from the still prioritise single conditions. The future of out- UK National Institute of Health Research (NIHR) predicted of-hospital care requires researchers to expand their that two-thirds of people aged 65 and over will have multi- horizons; integrated assistive technologies should ple morbidities by 2035, and 17% with four or more condi- enable people to live their life well regardless of tions. One third of these people will have a mental illness their chronic conditions. Yet, many of the cur- (e.g. dementia or depression). Increased life expectancy for rent digital self-management technologies are not both men and women means people will spend a longer time equipped to handle this problem. In this position living with multiple morbidities, placing increased demand paper, we suggest the solution for these issues is on the healthcare system. a model-aware and data-agnostic platform formed Integrated assistive technologies promises a to enable peo- on the basis of a tailored self-management plan ple to live their life well regardless of their chronic condi- and three integral concepts - Monitoring (M) mul- tions. Advances in telecommunications and Artificial Intelli- tiple information sources to empower Predictions gence (AI) technologies paves the way for personalised vir- (P) and trigger intelligent Interventions (I). Here we tual health companions that provide a intelligently-on con- present our ideas for the formation of such a plat- nection between the patient and those providing their care. form, and its potential impact on quality of life for Such companions should be an intermediary, supporting pa- sufferers of chronic conditions. tients with confidently managing their condition(s), proac- tively engaging with health care professions only when nec- essary, reducing their workload and replacing the current re- 1 Introduction active patient-clinician interaction. This can be achieved by Chronic health conditions currently incur over 80% of all reasoning with data from automated observations of patient’s healthcare spending in the United Kingdom. Living with progress along their healthcare plan using real-time predic- one or more chronic illnesses almost certainly means major tions to trigger appropriate proactive interventions. Chronic changes in one’s life; the latter can be minimised with ef- patients have to live with their conditions 24/7, so it is natural fective self-management. Studies show that the capability to that their care should reflect that. self-manage (chronic) health conditions effectively promises This position paper presents our framework for multi- lower associated healthcare costs and more efficient use of morbidity virtual health companions, each tailored to the primary and secondary care [Wolff et al., 2002]. unique health needs of individuals, assisting them to take Copyright © 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 60 an assured, active role in managing their health.The frame- These interventions are commonly delivered face to face work is based on a configurable architecture comprising mul- by healthcare practitioners. However current studies indicate tiple reasoning components for Monitoring, Prediction and that healthcare time is extremely limited and of short dura- Intervention (MPI). This will provide a plug and play model tion. Without ongoing support, patient physical activity lev- enabling the bespoke integration of existing and yet-to-be- els decline as maintaining motivation is difficult [Morris et created devices, along with modes of reasoning as neces- al., 2012]. Innovative, person-centred strategies to monitor sary in a library of AI skills. For example, humanoid- and predict physical activity and exercise behaviours, to scan robot driven conversational dialogue systems, autonomous and anticipate environmental barriers to activity, and to pro- image and video analysis, analysis of real-time wearable vide social and motivation support are required. These must and implant-generated data, and advanced telecommunica- support evidence-based, personally tailored behaviour change tion networks (e.g. 5G) for remote interactions. strategies by monitoring and providing feedback on perfor- This paper is structured as follows. In Section 2 we discuss mance; provide virtual real-time social support for activity; related work within digital self-management. In Section 3 we provide feedback on physical performance and evaluation of present our concepts on requirements for a generic framework environmental barriers to physical activity. with emphasis on reasoning centered around Monitoring, Pre- diction and Intervention (MPI) components, and detail how 2.1 Case-based reasoning for self-management these can be expanded to cover multiple morbidities. In Sec- Previous work has demonstrated the effectiveness of apply- tion 4 we describe several technologies which we expect to be ing decision support and reasoning systems to the manage- key players in the future and explore their integration within ment of a specific chronic disease. For instance Case-based a data agnostic framework. Finally, in Section 5 we provide reasoning (CBR) which is an AI approach that solves new some conclusions. problems using specific knowledge extracted from previously solved problems, has been successfully used to incorporate 2 Related Work evidence-base practices. Here, reasoning is facilitated by a collection of cases, a unique set of past experiences stored Self-management is a set of approaches which aim to enable in a case base. However to the best of our knowledge CBR people living with long-term conditions to take control of has only been applied in self-management of single chronic their care and manage their own health. Assistive technology diseases. can support self-management on several levels: CBR has been applied to managing diabetes types 1 and 1. Problem-solving (e.g. coping with flare-ups or adapting 2, using records that provide details about periodical vis- plan activities); its with a physician in a case consisting of features that 2. Decision-making (e.g. when to seek support or help represent a problem (e.g. weight, blood glucose level), with decisions around positive behaviour changes like its solution (e.g. levels of insulin) and the outcome (e.g. improving diet, reducing alcohol consumption, quitting hyper/hypo(glycemia)) observed after applying the solu- smoking or increasing physical activity and social inter- tion [Marling et al., 2012; Montani et al., 2000]. More re- action); cent work [Chen et al., 2017], explored the management of diabetes type 1 to support monitoring of blood glucose levels 3. Resource utilisation (e.g. making best use of healthcare before, during and after exercises. Interventions recommend and other resources, including 3rd sector, peer-support, carbohydrate intake based on similar cases retrieved from the web-based resources and other sources of information or case base. In related work on self-management of low-back advice); pain (LBP) [Bach et al., 2016], CBR recommends care plans 4. Forming patient-healthcare provider relationships and from similar patients. Management involves a human activity encouraging patients to interact with their healthcare recognition (HAR) component to monitor the patient activity provider appropriately. This would ideally occur before using sensor data that is continuously polled from a wearable emergency or crisis situations arise to prevent decline in device. Patient reported monitoring is used by the SelfBACK health and/or hospital admission. system to manage exercise adherence. Monitoring allows the system to detect periods of low activity behaviour, at which 5. Action planning and self-tailoring (e.g. encouraging pa- point a notification is generated to nudge the user to be more tient participation in creating their own self-management active - the intervention. An important contribution of this plan (which might include physical activity, specific work is the integration of behaviour change techniques such exercises, relaxation) and tailoring it to their specific as goal setting to focus the expected level of activity. There- needs, improving patient’s knowledge of their condi- after comparison of expected and actual behaviours analyse tions. goal achievement. The goal of self-management is to encourage behaviour Evidence for self-management of patients with multimor- change in sufferers of chronic conditions. A systematic re- bidities is limited, despite the prevalence of co-occurring view of interventions to promote physical activity [Morris et conditions and its impact on patients and healthcare sys- al., 2014] illustrated that interventions involving behaviour tems [Smith et al., 2012]. Interventions tend to have mixed change strategies are more effective for sustaining longer- effects requiring careful design underpinned by evidence- term physically active lifestyles than time-limited interven- based practice. Personalisation is important to ensure that tions involving structured exercises alone. care plans are tailored to the needs of the individual. Al- 61 though there has been recent work on personalised learning 2014; van der Drift et al., 2014] and motivation coaching for using state-of-the-art learning architectures (e.g. matching healthy living, weight loss and exercise [Leme et al., 2019]. networks) more work is needed when applying them to in- To the best of our knowledge, the application of social robots dividuals with multimorbidities [Sani et al., 2018]. in supporting individuals suffering multiple conditions has re- mained an unexplored area. 2.2 Pervasive and ubiquitous self-management Computer vision can be used to detect, track, and re- Pervasive and ubiquitous AI enabled devices are arguably identify patients without the need for any specific sensors or best placed to continuously monitor a person’s adherence markers to be worn or carried. In their place, technical re- to self-management plans, make real-time predictions about quirements include the need for the computer to infer the lo- the likelihood of adherence and the impact of that. How- cation, pose and movement of the trainee (and other people in ever intervention requires a good understanding of human be- its vicinity). Person following [Honig et al., 2018] is a key ca- haviours and direct inspection by health providers, which al- pability for the machine to be able to observe and guide a pa- though valuable, cannot be scaled to large and diverse groups tient. For this purpose, unmanned aerial vehicles (commonly of people. Wearables, such as smart watches or phones, are referred to as drones) may present a flexible solution; rather the most common form of physical activity monitoring de- than fast-flying quad-copters, blimps may offer a more stable vices and sources of delivering digital interventions. These and safer platform [Yao et al., 2019]. Their use in related ap- are embedded with inertial measurement devices (e.g. ac- plications such as monitoring older people in care homes has celerometers or gyroscopes) that generate time-series data been suggested (but not yet developed) [Srisamosorn et al., which can be exploited for human activity recognition of am- 2016]. However, the computer vision systems used need to be bulatory activities, activities of daily living, gait analysis and made more robust and reliable before a flying socially-aware pose recognition [Sani et al., 2018; Reiss and Stricker, 2012; robot for monitoring patients could be deployed and trialled, Chavarriaga et al., 2013]. especially in less controlled environments outside the clinic. Although commercial wearable activity trackers such as A further potential solution in this field is Ambient As- Fit-Bit are increasingly being used to monitor levels of phys- sisted Living (AAL), which targets the use of multiple de- ical activity and provide feedback to users, their utility is lim- vices and sensors around the home to support personal health- ited. Accuracy in determining activity in people who walk at care monitoring. AAL offers an opportunity for non-intrusive slow ambulatory speeds in free living conditions is low [Fee- tracking of patient condition through smart home technol- han et al., 2018], and evidence of effects on physical activ- ogy [Forbes et al., 2019]. Though traditionally difficult to ity levels are uncertain [Lynch et al., 2018]. They have lim- apply this for accurate patient monitoring (particularly in ited interactive and personalisation options, due in part to a open areas), recent advancements demonstrate detailed activ- reliance upon text notifications as the intervention method. ity profiling can be gained from non-intrusive RFID chips sit- As a direct result, current wearable technologies struggle to uated in locations around an individual’s home [Oguntala et understand and address individual barriers to promoting be- al., 2019], even in large rooms [Obeidat et al., 2019]. Though haviour change in individuals with complex disabilities, pro- work has targeted AAL to support assisted living and fall pre- vide insufficient information to determine specific rehabilita- diction for the elderly [Massie et al., 2018], we are unaware tion activities (such as exercises), and are not adapted to peo- of any current AAL approaches which comprehensively sup- ple with communication impairments. Importantly, provision port multimorbidities in every age group. of real-time information on outdoor environmental hazards (e.g. stairways) and mental health issues (such as anxiety or lack of confidence) which are likely to impact physical activ- 3 Generic self-management framework ity behaviours in patients is limited. Another promising solution is to develop and employ so- Generic self-management frameworks need to be capable of cial robots that can be designed to offer psychological sup- covering a wide range of devices and conditions in order port. In this context, Socially Assistive Robotics (SAR) is a to support personalisation, if trained models are to move rapidly growing domain that aims to enhance psychological away from the current one-size-fits all systems driven by cen- well-being through human interactions with a robot. Robots tralised datasets. Further they must ensure patient privacy can be programmed to perceive and interpret human actions and data security despite the large volumes of data needed and nonverbal cues, and provide assistance both at the level for training machine learning models that will provision edge of goal setting and tracking and socio-emotional communi- computing devices and intelligent decision support systems. cation through personalised conversational dialogues. The Frameworks must also be flexible, able to react to changes usefulness of social robots in mental healthcare contexts has in the environment and adapt reasoning appropriately. We been investigated by a number of previous works [Rabbitt et argue the key to addressing these requirements is treating al., 2015]. However, most of the available therapeutic robotic self-management as an AI planning problem; where AI meth- platforms target supporting either children with special condi- ods support and recommend interventions centred around the tions such as autism or assisting elderly people in their daily likely achievement of goals and actions recorded in plans. To lives. In particular, robotic platforms for patient education achieve this we need constructs that can be configured to a and self-management interventions are still scarce. There are given self-management plan; and a generic architecture that a few lines of work focusing on self-management and aware- can support the reuse of these constructs to enable evidence- ness of type 1 diabetes in children [Kruijff-Korbayová et al., based community care. 62 Observe individual Compare Observed versus Reduce difference between Expected Observed and Expected ... Medical record ... ... responses therapy Social interaction Questionnaire Clinical Help analysis Smart phone Mood analysis Conversation Exercises angle of movement Drone Haptic feedback recognition Wearable device Gait analysis Notifications Educational Robot HAR material Monitor Predict Intervene Figure 1: Instance of a blueprint for stroke rehabilitation configured with two Monitor-Predict-Intervene (MPI) component paths. 3.1 Reasoning with the MPI Cycle ing treatment. In this image, a patient is managing their joint strengthening exercises using an exercise-MPI (consisting of We propose inclusion of three components: digital Moni- the components shaded orange), while managing their mood toring (M) to track patient condition(s) and underpin antic- through an emotion-MPI (shaded blue). The exercise-MPI ipatory Predictions (P) to trigger real-time Interventions (I) monitors bending of the joint via machine vision technology for additional support when it is required. We refer to the and a smart phone camera. Reasoning on the monitored data combination of these components as an MPI. Reasoning is allows the system to predict whether the exercises are being facilitated by a self-management plan consisting of guide- performed correctly (e.g. the angle of movement is satisfac- lines, health recommendations, patient goals, decisions and tory). The system can then intervene by actuating haptic feed- a trace of previous lessons learned. These lessons learned back through a patient’s wearable sensor, guiding the patient data supports tailoring of plans to an individual. Given a self- to perform the exercise correctly. Similarly, the emotion-MPI management plan (e.g. level and frequency of exercise, tar- monitors mood by reasoning on the patient’s responses to geted levels of anxiety), the MPI cycle monitors adherence questionnaires. If mood is predicted to be below the threshold by a combination of patient self-reported outcome measures determined by a clinician, an intervention can organise clini- (e.g. pain) and automated real-time monitoring (e.g. of phys- cal help before this evolves into depression. However if mood iological response) by a variety of existing and yet-to-be- is above this threshold, no intervention is necessary. The goal created devices (e.g. wearables, implants, in-home sensors, of each MPI is to Observe (O) the patient through monitor- drones, robots). Differences in observed and expected adher- ing to predict a comparison with the Expected (E) outcome ence, combined with environmental and personal factors can as established by their clinician. If the observed actions de- be used to predict likely trends and outcomes. Thereafter, au- viate sufficiently from the expected, then an intervention is tonomous and remotely supported proactive interventions by necessary to minimise the difference (min(δ(O, E))). health professionals are initiated and plans are adapted col- laboratively, replacing the current reactive patient-clinician interaction. 3.2 Reusing self-management plans An evidence-based approach to self-management is facili- We propose the idea of a blueprint through which an indi- tated by having access to previous plans and lessons learned vidual’s self-management plan for multimorbidities can be as well as reusing care-plans from similar patients that have formulated. Essentially a blueprint is a combination of one been successful in relation to outcome measures. The rea- or more MPIs and their relationships (e.g. contraindica- soning capability of the MPI increases with increasing adop- tions) which has been configured jointly with a clinician. As tion of the framework. As the richness of the evidence base an example, the aforementioned Figure 1 shows a patient’s grows (from initial guidelines to personalised plans) the im- blueprint for the self-management of stroke rehabilitation, as pact on the community and their common self-management it combines the exercise-MPI and the emotion-MPI. conditions will be improved. The reasoning necessary when combining multiple differ- Thus MPIs form the building blocks for a multi-faceted ent MPIs is an important area of research; it must ensure ad- self-management plan - a plan that can cater for multimor- herence to a patient’s collective self-management plans and bidities. For example, consider Figure 1, which features suggest interventions that are suitable for all the multimor- two MPIs created to support stroke rehabilitation. Stroke bidities, whilst maintaining knowledge of any contraindica- survivors commonly struggle with freedom of movement in tions between those conditions. Central to this is a knowl- their joints and have a tendency to develop depression dur- edge structure, an MPINet ontology, where known MPI re- 63 lationships can be recorded and used to form generalised model under the coordination of a central server acting as a blueprints. These can be refined by co-occurrences inferred curator, selecting which participating devices (i.e. the fed- from collected data. eration) to incorporate when training models. This form of A shared community of blueprints is formed from individ- learning is ideal for community health care ensuring privacy uals who share the same or a similar set of co-occurring con- by default, respecting data ownership, and maintaining lo- ditions. Configurations embedded in blueprints can then be cality of data (without centralising data) for application de- reused and adapted to suit new individuals joining that com- ployment at scale. An interesting direction of research here munity (see Figure 2). In this image, blueprints have been is to the use of data provenance records combined with met- extracted from members of the community who are similar rics evaluating quality and trust of individuals to influence to the new individual, where ϕ is a reuse function containing the global computational curator’s decisions on sampling of adaptation knowledge. The output is a blueprint configured MPIs on devices. With complex model architectures there is to the individual’s needs founded on evidence-based practice. also a need to share and describe the architectural properties This can be formalised as: so as to inform the curator about compatibility. This perhaps O0 , E 0 = ϕ[(O, E)1 , (O, E)2 , ...] (1) calls for a meta-language for architectural descriptors. Learn- ing from few labelled data is important for technology to op- where O the configuration for what is to be observed and E 0 0 erate at scale. It will be useful to extend the FL paradigm represents the modified expectations in this new blueprint. to evidence-based reasoning methods such as matching net- works [Vinyals et al., 2016] to enable few-shot learning while using a federated strategy. = ϕ This type of environment is potentially suitable for im- M P I M P I , M P I ,… plementing a multi-agent framework [Moreno A, 2003], whereby autonomous, adaptive and interactive software com- Figure 2: Reuse of blueprints from similar individuals. ponents, provide notions that specifically meet the MPI challenges to form the base of a robust and scalable self- Personalisation of blueprints is achieved through the rec- management infrastructure. A multi-agent framework will ommendation of contextually-relevant MPIs and customis- also enable the collection of patient-reported well-being in- ing general community blueprints. Similarities in self- formation (as done in [Ibrahim et al., 2015]), integrating management goals within a community can be used for per- them with sensor and device-generated data via API-based sonalisation and to anticipate known common complications real-time data collection and streaming platforms such as the of a condition, which can be extended to enable forecasting in-house build RADAR-base platform [Ranjan et al., forth- for healthcare service demand. Metric learning algorithms coming 2019]. The combination will yield an intelligent and that are suited for evidence-based reasoning lend well to real-time framework for schematised, secure and role-based learning personalised models on the basis of similarity com- data collection, harmonisation and integration based on uni- putations, and also lend themselves well to few-shot learning fied temporal intervals from all devices, sensors and individ- approaches. Recent advances in similarity driven attention uals. mechanisms currently use simplistic metrics, hence in future research we must establish how similarity on the basis of the 4 The Future of Digital Self-Management user contexts can be computed in a differentiating manner. Tackling self-management of multimorbidities requires a Configuration of blueprints shares many similarities with model-aware and data-agnostic machine learning platform - On-Line Case-Based Planning (OLCBP). Both require con- a modular digital health ecosystem, where multiple monitor- tinuous monitoring of expected versus observed and acting ing technologies enable comprehensive predictions that trig- upon deviations in a timely manner. Much like the snippets ger the most appropriate intervention strategy from a broad that exist within a plan [Ontanón et al., 2010], MPIs act as range of possibilities. With that in mind, we suggest the pro- the basic constituent pieces of a blueprint and are configured posed MPI architecture could answer many of the research with individual goals. Though failure to achieve individual challenges discussed for individual technologies in Section 2 MPI goals does not necessarily define the failure of a self- and greatly improve the self-management experience from a management plan, we suggest this is an early warning of the user perspective. However, there is also a need to consider need for additional care. Importantly recording such failures how this architecture could be practically utilised at scale and the follow-on adaptation of both the self-management while preserving patient privacy. goal and agreed activities are useful experiential content that are crucial when developing evidence-based practices for the 4.1 Wearables and Haptic Feedback community. We aim to develop a framework to allow wearable technolo- gies to provide real-time haptic feedback to support users ef- 3.3 Role of federated machine learning fectively perform rehabilitation exercises and physical activ- Ideally, MPI models will learn from the data generated by all ities. Haptic feedback uses digital cues to stimulate the feel- users of the system, i.e. in a distributed manner. This requires ing of touch in users. Whereas current wearable technolo- us to adopt ideas from Federated Learning (FL) [McMahan et gies are reliant on the user reacting to textual notifications al., 2017] and meta-learning. Specifically maintaining con- or messages, haptic feedback provides an opportunity to cor- trol of device data, where training involves a shared global rect execution of exercises on-the-fly. The result would be 64 a decreased reliance on the external technology of a smart which can be exacerbated by the freedom of movement af- phone and less expectation for users to continually refer to forded to a drone. Additionally, the anticipated skewed dis- their phone throughout an exercise session. tribution of the data will pose another challenge that needs There are two primary challenges to provisioning haptic to be addressed. In particular, having enough representative feedback remotely; (1) currently, haptic feedback for therapy training examples that captures the different scenarios of a is performed one-to-one, either with a healthcare professional specific case study or patient can potentially be very diffi- activating the necessary feedback while locally observing the cult, meaning that few-shot learning strategies are likely to be patient or the skeleton following a pre-programmed set of in- very relevant here. The benefits for patients go beyond sup- structions; and (2) current telecommunication networks do port with performing exercises and documenting their self- not have the throughput required to enable such feedback re- management progress; for example, people with movement motely at scale. We believe that the former research chal- difficulties are often anxious about walking in unfamiliar en- lenge could be answered through development of a learned vironments as they are unaware of any obstacles they may model which can provide activation of the necessary compo- face - this can lead them to withdrawing from social activities nents to actuate haptic feedback. The latter challenge may be and an increased feeling of loneliness. Being accompanied answered with effective advanced networking infrastructures, by a drone that advises them of upcoming obstacles and how such as 5G. they can be avoided, may reduce anxiety associated with such situations. 4.2 Robots and Conversational Dialogue The use of therapeutic robots to support community-dwelling 4.4 Ambient Assisted Living individuals participate in rehabilitation, fitness exercises, so- We aim to use Ambient Assisted Living (AAL) technology to cial engagement and outdoor physical activities are not only support non-intrusive monitoring of patients within their own a viable solution to the problem of shortage in care resources, home. Several interesting research challenges arise from this but also potentially address loneliness in the elderly. How- field. In particular, it will be interesting to see how the con- ever, current SAR systems are limited in their capability to of- stant monitoring of AAL devices allows the identification of fer truly personalised support. They do not take the user state self-management activities as they take place within aspects into consideration, beyond verbal and interaction logs, suffer of daily living. For example, if a patient has climbed the stairs from low engagement, and uni-directional information flow. 5 times in an afternoon as part of cleaning their home, this Expanding this to exchange more information between the action may cover some of the ambulatory physical activity user and the system through multiple modalities will enable a scheduled as part of managing their lower-back pain condi- more versatile and natural interaction, ensuring that the user tion. Understanding the situations in which daily living activ- maximally benefits from the system. In addition, it provides a ities can replace self-management exercises (or alternatively, means to build a richer user-context that can help to improve, the situations in which they cannot) is an intriguing avenue structure and personalise a robot’s behaviours and conversa- for exploration. Furthermore, there is the technical challenge tional dialogue. To address these issues, novel methods are of being able to seamlessly change monitoring devices that needed for semantically integrating various data sources re- feed into a single MPI e.g. as the patient transitions from lating to (1) mood evaluations based on visual cues (i.e., fa- one room to the next. A comprehensive self-management cial, bodily cues); (2) physical performance, psychological solution involving AAL devices should maximise comfort variables and environmental variables collected from wear- of the patient and ensure that they are able to relax within ables; and (3) adaptive activity programmes and behaviour their home without feeling like they are being continually change strategies in real-time. Integrated information can be observed. Developing an individual’s trust requires further then used to build effective reasoning and intervention mech- understanding of human machine interfaces by bringing to- anisms for the interactive robot companion that can support gether behavioural psychologists and computer scientists. citizens to lead active and social lives. 4.5 A Comprehensive Self-Management Solution 4.3 Autonomous Computer Vision The distributed MPI model comprises a number of devices Computer-vision models can provide visual feedback to sup- collaborating autonomously, and a substantial amount of real- port patients with rehabilitation activities. This involves the time data generated passively by sensors and devices or ac- analysis of large volumes of imagery data, recognition of tively reported by the users (e.g. well-being reports) for a sin- fine-grained human movement details and generation of vi- gle non-fragmented self-management solution. As such, the sual feedback interventions. Drones could offer a rich source MPI framework constitutes a decentralised, heterogeneous of information for on-going monitoring and observation of and open environment that operates on multiple computing patients in different contexts, including both indoor self- systems to manipulate, share and analyse strictly-confidential management activities and outdoor exercises. Utilising a vi- patient records. Moreover, to build a platform to continu- sual source that captures patient’s movement and behaviours ously inform patients using their own records, requires mech- from different angles provides a unique opportunity for learn- anisms to mine the ‘big’ and heterogeneous data for relevant ing and improving practices, but at the same time poses some knowledge, and learn the adapted and personalised recom- key challenges. These are due to the inherent vision prob- mendations and possible interventions in real-time to aid in lem, in particular, performing accurate detection, tracking self-management. These methods are underpinned by the and classification under different poses and light conditions, growing availability of advanced networking technology (e.g. 65 5G), where the faster connection speeds promise huge bene- [Chavarriaga et al., 2013] Ricardo Chavarriaga, Hesam fits to future healthcare systems. Such advances allow high- Sagha, Alberto Calatroni, Sundara Tejaswi Digumarti, intensity data processing to be supported by near real-time Gerhard Tröster, José del R Millán, and Daniel Roggen. decision-making. We believe these will play a crucial role in The opportunity challenge: A benchmark database for supporting at-home care (e.g. wearable sensors applications on-body sensor-based activity recognition. Pattern and online consultations) and remote treatments (e.g. digital Recognition Letters, 34(15):2033–2042, 2013. diagnosis). In particular, 5G will facilitate the transfer of ex- [Chen et al., 2017] Yoke Yie Chen, Nirmalie Wiratunga, pertise over a great distance in real-time using technologies Stewart Massie, Jenny M Hall, Kate Stephen, Amanda such as robotics and haptic feedback. Croall, Jacky MacMillan, Lesley Murray, Geoff Wilcock, To give an example, a stroke survivor patient may use AAL and Sandra M MacRury. Designing a personalised case- sensors for non-intrusive tracking of physical activity and in- based recommender system for mobile self-management teract with a SAR to self-report on their pain and mental of diabetes during exercise. In In Workshop proceedings condition while in the home. When journeying outside the of the 2nd Knowledge Discovery from Health Workshop at house, the patient initially uses a drone with computer vi- the International Joint Conference on AI, 2017. sion capabilities to monitor their journey, before transitioning to use of a wearable technology as their walking ability im- [Feehan et al., 2018] Lynne M Feehan, Jasmina Geldman, proves. The combination of monitoring technologies to em- Eric C Sayre, Chance Park, Allison M Ezzat, Ju Young power interventions is a comprehensive solution to their self- Yoo, Clayon B Hamilton, and Linda C Li. Accuracy management. We believe that not only will this greatly im- of fitbit devices: Systematic review and narrative syn- prove adherence to the various aspects of a self-management theses of quantitative data. JMIR mHealth and uHealth, plan, it will also have an irreplicable effect on patient quality 6(8):e10527, 2018. of life. Furthermore, integration with the MPI framework will [Forbes et al., 2019] Glenn Forbes, Stewart Massie, and Su- bring AI power to off-the-shelf hardware rather than building san Craw. Fall prediction using behavioural modelling expensive devices, thus encouraging socially-inclusive care. from sensor data in smart homes. Artificial Intelligence Review, pages 1–21, 2019. 5 Conclusion [Honig et al., 2018] S. S. Honig, T. Oron-Gilad, H. Zaichyk, V. Sarne-Fleischmann, S. Olatunji, and Y. Edan. To- In conclusion, in this position paper we have discussed the ward socially aware person-following robots. IEEE need for an affordable comprehensive self-management solu- Transactions on Cognitive and Developmental Systems, tion to improve quality of life for patients living with multiple 10(4):936–954, Dec 2018. morbidities. We have presented our ideas towards the cre- ation of MPI components - constituent pieces of wider-scale [Ibrahim et al., 2015] Zina Ibrahim, Lorena Fernández de la framework which allows a configurable and personalised self- Cruz, Argyris Stringaris, Robert Goodman, Michael Luck, management plan for individual patients, while encouraging and Richard Dobson. A multi-agent platform for automat- reuse plans from within a community of similar patients and ing the collection of patient-provided clinical feedback. In morbidities. The platform and MPIs potentially offer sev- Proceedings of the 2015 International Conference on Au- eral interesting avenues of research, not only around how new tonomous Agents and Multiagent Systems, page 831–839, and existing technologies can be integrated, but also around 2015. areas such as comparing the similarity of MPIs and under- [Kruijff-Korbayová et al., 2014] I. Kruijff-Korbayová, standing the ramifications of using them for reuse and adap- E. Oleari, I. Baroni, B. Kiefer, M. C. Zelati, C. Pozzi, tation. Most importantly, we believe that a wide-scale inte- and A. Sanna. Effects of off-activity talk in human-robot grated framework of devices is necessary for supporting pa- interaction with diabetic children. In The 23rd IEEE In- tients with confidently managing their condition(s), improv- ternational Symposium on Robot and Human Interactive ing their quality of life in the future. Communication, pages 649–654, Aug 2014. [Leme et al., 2019] Bruno Leme, Masakazu Hirokawa, References Hideki Kadone, and Kenji Suzuki. A socially assistive [Bach et al., 2016] Kerstin Bach, Tomasz Szczepanski, Ag- mobile platform for weight-support in gait training. nar Aamodt, Odd Erik Gundersen, and Paul Jarle Mork. International Journal of Social Robotics, Apr 2019. Case representation and similarity assessment in the self- [Lynch et al., 2018] Elizabeth A Lynch, Taryn M Jones, back decision support system. In Ashok Goel, M Belén Dawn B Simpson, Natalie A Fini, Suzanne S Kuys, Karen Dı́az-Agudo, and Thomas Roth-Berghofer, editors, Case- Borschmann, Sharon Kramer, Liam Johnson, Michele L Based Reasoning Research and Development, pages 32– Callisaya, Niruthikha Mahendran, et al. Activity monitors 46, Cham, 2016. Springer International Publishing. for increasing physical activity in adult stroke survivors. [Barnett et al., 2012] Karen Barnett, Stewart W Mercer, Cochrane Database of Systematic Reviews, (7), 2018. Michael Norbury, Graham Watt, Sally Wyke, and Bruce [Marling et al., 2012] Cindy Marling, Matthew Wiley, Raz- Guthrie. Epidemiology of multimorbidity and implica- van Bunescu, Jay Shubrook, and Frank Schwartz. Emerg- tions for health care, research, and medical education: a ing applications for intelligent diabetes management. AI cross-sectional study. The Lancet, 380(9836):37–43, 2012. Magazine, 33(2):67–67, 2012. 66 [Massie et al., 2018] Stewart Massie, Glenn Forbes, Susan beeck, Sebastian Boettcher, Pauline Conde, Richard Dob- Craw, Lucy Fraser, and Graeme Hamilton. Fitsense: Em- son, and Amos Folarin. RADAR-base: An open source ploying multi-modal sensors in smart homes to predict mhealth platform for collecting, monitoring and analyzing falls. In International Conference on Case-Based Reason- data using sensors, wearables, and mobile devices. Journal ing, pages 249–263. Springer, 2018. of Medical Informatics Research mHealth and uHealth, [McMahan et al., 2017] H. B. McMahan, E. Moore, Ram- forthcoming, 2019. age, S. D., Hampson, and B. A. y Arcas. Communication- [Reiss and Stricker, 2012] Attila Reiss and Didier Stricker. efficient learning of deep networks from decentralized Introducing a new benchmarked dataset for activity moni- data. In Proceedings of the 20th International Conference toring. In Wearable Computers (ISWC), 2012 16th Inter- on Artificial Intelligence and Statistics, pages 1273 – 1282, national Symposium on, pages 108–109. IEEE, 2012. 2017. [Sani et al., 2018] Sadiq Sani, Nirmalie Wiratunga, Stew- [Montani et al., 2000] Stefania Montani, Riccardo Bellazzi, art Massie, and Kay Cooper. Personalised human activ- Luigi Portinale, Giuseppe d’Annunzio, Stefano Fiocchi, ity recognition using matching networks. In Michael T. and Mario Stefanelli. Diabetic patients management ex- Cox, Peter Funk, and Shahina Begum, editors, Case-Based ploiting case-based reasoning techniques. Computer meth- Reasoning Research and Development, pages 339–353. ods and programs in biomedicine, 62(3):205–218, 2000. Springer International Publishing, 2018. [Moreno A, 2003] Garbay C. Moreno A. Software agents [Smith et al., 2012] Susan M Smith, Hassan Soubhi, Martin in health care. Artificial intelligence in medicine, Fortin, Catherine Hudon, and Tom O’Dowd. Managing 27(3):229–232, 2003. patients with multimorbidity: systematic review of inter- ventions in primary care and community settings. BMJ, [Morris et al., 2012] Jacqui Morris, Tracey Oliver, Thilo 345, 2012. Kroll, and Steve MacGillivray. The importance of psy- chological and social factors in influencing the uptake and [Srisamosorn et al., 2016] Veerachart Srisamosorn, Noriaki maintenance of physical activity after stroke: a structured Kuwahara, Atsushi Yamashita, Taiki Ogata, and Jun Ota. review of the empirical literature. Stroke research and Design of face tracking system using environmental cam- treatment, 2012, 2012. eras and flying robot for evaluation of health care. In Dig- ital Human Modeling: Applications in Health, Safety, Er- [Morris et al., 2014] Jacqui H Morris, Stephen gonomics and Risk Management, LNCS. Springer, 2016. MacGillivray, and Sarah Mcfarlane. Interventions to promote long-term participation in physical activity after [van der Drift et al., 2014] Esther J.G. van der Drift, stroke: a systematic review of the literature. Archives Robbert-Jan Beun, Rosemarijn Looije, Olivier A. Blan- of physical medicine and rehabilitation, 95(5):956–967, son Henkemans, and Mark A. Neerincx. A remote 2014. social robot to motivate and support diabetic children in keeping a diary. In Proceedings of the 2014 ACM/IEEE [Obeidat et al., 2019] Huthaifa Obeidat, Ali AlAbdullah, International Conference on Human-robot Interaction, Nazar Ali, Rameez Asif, Omar Obeidat, Mohammed Bin- HRI ’14, pages 463–470, New York, NY, USA, 2014. Melha, Wafa Shauib, Raed Abd-Alhameed, and Peter Ex- ACM. cell. Local average signal strength estimation for indoor multipath propagation. Accepted to IEEE Access, April [Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Tim- 2019. othy Lillicrap, and Daan Wierstra. Matching networks for one shot learning. In Advances in neural information pro- [Oguntala et al., 2019] George A. Oguntala, Raed A. Abd- cessing systems, pages 3630–3638, 2016. Alhameed, James M. Noras, Yim-Fun Hu, Eya N. [Wolff et al., 2002] Jennifer L. Wolff, Barbara Starfield, and Nnabuike, Nazar Ali, Issa T. Elfergani, and Jonathan Ro- driguez. Smartwall: Novel rfid-enabled ambient human Gerard Anderson. Prevalence, Expenditures, and Com- activity recognition using machine learning for unobtru- plications of Multiple Chronic Conditions in the Elderly. sive health monitoring. Accepted to IEEE Access, May JAMA Internal Medicine, 162(20):2269–2276, 11 2002. 2019. [Yao et al., 2019] Ning-shi Yao, Qiu-yang Tao, Wei-yu Liu, [Ontanón et al., 2010] Santi Ontanón, Kinshuk Mishra, Zhen Liu, Ye Tian, Pei-yu Wang, Timothy Li, and Fumin Zhang. Autonomous flying blimp interaction with human Neha Sugandh, and Ashwin Ram. On-line case-based in an indoor space. Frontiers of Information Technology & planning. Computational Intelligence, 26(1):84–119, Electronic Engineering, 20(1):45–59, Jan 2019. 2010. [Rabbitt et al., 2015] Sarah M. Rabbitt, Alan E. Kazdin, and Brian Scassellati. Integrating socially assistive robotics into mental healthcare interventions: Applications and rec- ommendations for expanded use. Clinical Psychology Re- view, 35:35 – 46, 2015. [Ranjan et al., forthcoming 2019] Yatharth Ranjan, Zulqar- nain Rashid, Callum Stewart, Mark Begale, Denny Ver- 67