=Paper=
{{Paper
|id=Vol-2429/paper10
|storemode=property
|title=Learning to Self-Manage by Intelligent Monitoring, Prediction and Intervention
|pdfUrl=https://ceur-ws.org/Vol-2429/paper10.pdf
|volume=Vol-2429
|authors=Nirmalie Wiratunga,David Corsar,Kyle Martin,Anjana Wijekoon,Eyad Elyan,Kay Cooper,Zina Ibrahim,Oya Celikutan,Richard Dobson,Stephen McKenna,Jacqui Morris,Annalu Waller,Raed Abd-Alhammed,Rami Qahwaji,Ray Chaudhuri
|dblpUrl=https://dblp.org/rec/conf/ijcai/WiratungaCMWEIC19
}}
==Learning to Self-Manage by Intelligent Monitoring, Prediction and Intervention==
https://ceur-ws.org/Vol-2429/paper10.pdf
Learning to Self-Manage by Intelligent Monitoring, Prediction and Intervention
Nirmalie Wiratunga1 , David Corsar1 , Kyle Martin1 , Anjana Wijekoon1 , Eyad Elyan1 , Kay
Cooper1 ,
Zina Ibrahim2,3 , Oya Celiktutan2 , Richard J. Dobson2,3 ,
Stephen McKenna4 , Jacqui Morris4 , Annalu Waller4 ,
Raed Abd-Alhammed5 , Rami Qahwaji5 ,
Ray Chaudhuri6
1
Robert Gordon University, Aberdeen, UK
2
King’s College London, London, UK
3
University College London, London, UK
4
University of Dundee, Dundee, UK
5
University of Bradford, Bradford, UK
6
King’s College Hospital, NHS Foundation Trust, UK
{n.wiratunga, d.corsar, k.martin, a.wijekoon, e.elyan, k.cooper}@rgu.ac.uk,
{zina.ibrahim, oya.celiktutan, richard.j.dobson}@kcl.ac.uk,
{s.j.z.mckenna, j.y.morris, a.waller}@dundee.ac.uk,
{r.a.a.abd, r.s.r.qahwaji}@bradford.ac.uk,
ray.chaudhuri@nhs.net
Abstract Although the number of individuals living with multiple
morbidities is predicted to increase significantly over the
Despite the growing prevalence of multimorbidi- coming years [Barnett et al., 2012], current self-management
ties, current digital self-management approaches solutions prioritise single conditions. A 2018 study from the
still prioritise single conditions. The future of out- UK National Institute of Health Research (NIHR) predicted
of-hospital care requires researchers to expand their that two-thirds of people aged 65 and over will have multi-
horizons; integrated assistive technologies should ple morbidities by 2035, and 17% with four or more condi-
enable people to live their life well regardless of tions. One third of these people will have a mental illness
their chronic conditions. Yet, many of the cur- (e.g. dementia or depression). Increased life expectancy for
rent digital self-management technologies are not both men and women means people will spend a longer time
equipped to handle this problem. In this position living with multiple morbidities, placing increased demand
paper, we suggest the solution for these issues is on the healthcare system.
a model-aware and data-agnostic platform formed Integrated assistive technologies promises a to enable peo-
on the basis of a tailored self-management plan ple to live their life well regardless of their chronic condi-
and three integral concepts - Monitoring (M) mul- tions. Advances in telecommunications and Artificial Intelli-
tiple information sources to empower Predictions gence (AI) technologies paves the way for personalised vir-
(P) and trigger intelligent Interventions (I). Here we tual health companions that provide a intelligently-on con-
present our ideas for the formation of such a plat- nection between the patient and those providing their care.
form, and its potential impact on quality of life for Such companions should be an intermediary, supporting pa-
sufferers of chronic conditions. tients with confidently managing their condition(s), proac-
tively engaging with health care professions only when nec-
essary, reducing their workload and replacing the current re-
1 Introduction active patient-clinician interaction. This can be achieved by
Chronic health conditions currently incur over 80% of all reasoning with data from automated observations of patient’s
healthcare spending in the United Kingdom. Living with progress along their healthcare plan using real-time predic-
one or more chronic illnesses almost certainly means major tions to trigger appropriate proactive interventions. Chronic
changes in one’s life; the latter can be minimised with ef- patients have to live with their conditions 24/7, so it is natural
fective self-management. Studies show that the capability to that their care should reflect that.
self-manage (chronic) health conditions effectively promises This position paper presents our framework for multi-
lower associated healthcare costs and more efficient use of morbidity virtual health companions, each tailored to the
primary and secondary care [Wolff et al., 2002]. unique health needs of individuals, assisting them to take
Copyright © 2019 for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
60
an assured, active role in managing their health.The frame- These interventions are commonly delivered face to face
work is based on a configurable architecture comprising mul- by healthcare practitioners. However current studies indicate
tiple reasoning components for Monitoring, Prediction and that healthcare time is extremely limited and of short dura-
Intervention (MPI). This will provide a plug and play model tion. Without ongoing support, patient physical activity lev-
enabling the bespoke integration of existing and yet-to-be- els decline as maintaining motivation is difficult [Morris et
created devices, along with modes of reasoning as neces- al., 2012]. Innovative, person-centred strategies to monitor
sary in a library of AI skills. For example, humanoid- and predict physical activity and exercise behaviours, to scan
robot driven conversational dialogue systems, autonomous and anticipate environmental barriers to activity, and to pro-
image and video analysis, analysis of real-time wearable vide social and motivation support are required. These must
and implant-generated data, and advanced telecommunica- support evidence-based, personally tailored behaviour change
tion networks (e.g. 5G) for remote interactions. strategies by monitoring and providing feedback on perfor-
This paper is structured as follows. In Section 2 we discuss mance; provide virtual real-time social support for activity;
related work within digital self-management. In Section 3 we provide feedback on physical performance and evaluation of
present our concepts on requirements for a generic framework environmental barriers to physical activity.
with emphasis on reasoning centered around Monitoring, Pre-
diction and Intervention (MPI) components, and detail how 2.1 Case-based reasoning for self-management
these can be expanded to cover multiple morbidities. In Sec- Previous work has demonstrated the effectiveness of apply-
tion 4 we describe several technologies which we expect to be ing decision support and reasoning systems to the manage-
key players in the future and explore their integration within ment of a specific chronic disease. For instance Case-based
a data agnostic framework. Finally, in Section 5 we provide reasoning (CBR) which is an AI approach that solves new
some conclusions. problems using specific knowledge extracted from previously
solved problems, has been successfully used to incorporate
2 Related Work evidence-base practices. Here, reasoning is facilitated by a
collection of cases, a unique set of past experiences stored
Self-management is a set of approaches which aim to enable in a case base. However to the best of our knowledge CBR
people living with long-term conditions to take control of has only been applied in self-management of single chronic
their care and manage their own health. Assistive technology diseases.
can support self-management on several levels:
CBR has been applied to managing diabetes types 1 and
1. Problem-solving (e.g. coping with flare-ups or adapting 2, using records that provide details about periodical vis-
plan activities); its with a physician in a case consisting of features that
2. Decision-making (e.g. when to seek support or help represent a problem (e.g. weight, blood glucose level),
with decisions around positive behaviour changes like its solution (e.g. levels of insulin) and the outcome (e.g.
improving diet, reducing alcohol consumption, quitting hyper/hypo(glycemia)) observed after applying the solu-
smoking or increasing physical activity and social inter- tion [Marling et al., 2012; Montani et al., 2000]. More re-
action); cent work [Chen et al., 2017], explored the management of
diabetes type 1 to support monitoring of blood glucose levels
3. Resource utilisation (e.g. making best use of healthcare before, during and after exercises. Interventions recommend
and other resources, including 3rd sector, peer-support, carbohydrate intake based on similar cases retrieved from the
web-based resources and other sources of information or case base. In related work on self-management of low-back
advice); pain (LBP) [Bach et al., 2016], CBR recommends care plans
4. Forming patient-healthcare provider relationships and from similar patients. Management involves a human activity
encouraging patients to interact with their healthcare recognition (HAR) component to monitor the patient activity
provider appropriately. This would ideally occur before using sensor data that is continuously polled from a wearable
emergency or crisis situations arise to prevent decline in device. Patient reported monitoring is used by the SelfBACK
health and/or hospital admission. system to manage exercise adherence. Monitoring allows the
system to detect periods of low activity behaviour, at which
5. Action planning and self-tailoring (e.g. encouraging pa- point a notification is generated to nudge the user to be more
tient participation in creating their own self-management active - the intervention. An important contribution of this
plan (which might include physical activity, specific work is the integration of behaviour change techniques such
exercises, relaxation) and tailoring it to their specific as goal setting to focus the expected level of activity. There-
needs, improving patient’s knowledge of their condi- after comparison of expected and actual behaviours analyse
tions. goal achievement.
The goal of self-management is to encourage behaviour Evidence for self-management of patients with multimor-
change in sufferers of chronic conditions. A systematic re- bidities is limited, despite the prevalence of co-occurring
view of interventions to promote physical activity [Morris et conditions and its impact on patients and healthcare sys-
al., 2014] illustrated that interventions involving behaviour tems [Smith et al., 2012]. Interventions tend to have mixed
change strategies are more effective for sustaining longer- effects requiring careful design underpinned by evidence-
term physically active lifestyles than time-limited interven- based practice. Personalisation is important to ensure that
tions involving structured exercises alone. care plans are tailored to the needs of the individual. Al-
61
though there has been recent work on personalised learning 2014; van der Drift et al., 2014] and motivation coaching for
using state-of-the-art learning architectures (e.g. matching healthy living, weight loss and exercise [Leme et al., 2019].
networks) more work is needed when applying them to in- To the best of our knowledge, the application of social robots
dividuals with multimorbidities [Sani et al., 2018]. in supporting individuals suffering multiple conditions has re-
mained an unexplored area.
2.2 Pervasive and ubiquitous self-management Computer vision can be used to detect, track, and re-
Pervasive and ubiquitous AI enabled devices are arguably identify patients without the need for any specific sensors or
best placed to continuously monitor a person’s adherence markers to be worn or carried. In their place, technical re-
to self-management plans, make real-time predictions about quirements include the need for the computer to infer the lo-
the likelihood of adherence and the impact of that. How- cation, pose and movement of the trainee (and other people in
ever intervention requires a good understanding of human be- its vicinity). Person following [Honig et al., 2018] is a key ca-
haviours and direct inspection by health providers, which al- pability for the machine to be able to observe and guide a pa-
though valuable, cannot be scaled to large and diverse groups tient. For this purpose, unmanned aerial vehicles (commonly
of people. Wearables, such as smart watches or phones, are referred to as drones) may present a flexible solution; rather
the most common form of physical activity monitoring de- than fast-flying quad-copters, blimps may offer a more stable
vices and sources of delivering digital interventions. These and safer platform [Yao et al., 2019]. Their use in related ap-
are embedded with inertial measurement devices (e.g. ac- plications such as monitoring older people in care homes has
celerometers or gyroscopes) that generate time-series data been suggested (but not yet developed) [Srisamosorn et al.,
which can be exploited for human activity recognition of am- 2016]. However, the computer vision systems used need to be
bulatory activities, activities of daily living, gait analysis and made more robust and reliable before a flying socially-aware
pose recognition [Sani et al., 2018; Reiss and Stricker, 2012; robot for monitoring patients could be deployed and trialled,
Chavarriaga et al., 2013]. especially in less controlled environments outside the clinic.
Although commercial wearable activity trackers such as A further potential solution in this field is Ambient As-
Fit-Bit are increasingly being used to monitor levels of phys- sisted Living (AAL), which targets the use of multiple de-
ical activity and provide feedback to users, their utility is lim- vices and sensors around the home to support personal health-
ited. Accuracy in determining activity in people who walk at care monitoring. AAL offers an opportunity for non-intrusive
slow ambulatory speeds in free living conditions is low [Fee- tracking of patient condition through smart home technol-
han et al., 2018], and evidence of effects on physical activ- ogy [Forbes et al., 2019]. Though traditionally difficult to
ity levels are uncertain [Lynch et al., 2018]. They have lim- apply this for accurate patient monitoring (particularly in
ited interactive and personalisation options, due in part to a open areas), recent advancements demonstrate detailed activ-
reliance upon text notifications as the intervention method. ity profiling can be gained from non-intrusive RFID chips sit-
As a direct result, current wearable technologies struggle to uated in locations around an individual’s home [Oguntala et
understand and address individual barriers to promoting be- al., 2019], even in large rooms [Obeidat et al., 2019]. Though
haviour change in individuals with complex disabilities, pro- work has targeted AAL to support assisted living and fall pre-
vide insufficient information to determine specific rehabilita- diction for the elderly [Massie et al., 2018], we are unaware
tion activities (such as exercises), and are not adapted to peo- of any current AAL approaches which comprehensively sup-
ple with communication impairments. Importantly, provision port multimorbidities in every age group.
of real-time information on outdoor environmental hazards
(e.g. stairways) and mental health issues (such as anxiety or
lack of confidence) which are likely to impact physical activ- 3 Generic self-management framework
ity behaviours in patients is limited.
Another promising solution is to develop and employ so- Generic self-management frameworks need to be capable of
cial robots that can be designed to offer psychological sup- covering a wide range of devices and conditions in order
port. In this context, Socially Assistive Robotics (SAR) is a to support personalisation, if trained models are to move
rapidly growing domain that aims to enhance psychological away from the current one-size-fits all systems driven by cen-
well-being through human interactions with a robot. Robots tralised datasets. Further they must ensure patient privacy
can be programmed to perceive and interpret human actions and data security despite the large volumes of data needed
and nonverbal cues, and provide assistance both at the level for training machine learning models that will provision edge
of goal setting and tracking and socio-emotional communi- computing devices and intelligent decision support systems.
cation through personalised conversational dialogues. The Frameworks must also be flexible, able to react to changes
usefulness of social robots in mental healthcare contexts has in the environment and adapt reasoning appropriately. We
been investigated by a number of previous works [Rabbitt et argue the key to addressing these requirements is treating
al., 2015]. However, most of the available therapeutic robotic self-management as an AI planning problem; where AI meth-
platforms target supporting either children with special condi- ods support and recommend interventions centred around the
tions such as autism or assisting elderly people in their daily likely achievement of goals and actions recorded in plans. To
lives. In particular, robotic platforms for patient education achieve this we need constructs that can be configured to a
and self-management interventions are still scarce. There are given self-management plan; and a generic architecture that
a few lines of work focusing on self-management and aware- can support the reuse of these constructs to enable evidence-
ness of type 1 diabetes in children [Kruijff-Korbayová et al., based community care.
62
Observe individual Compare Observed versus Reduce difference between
Expected Observed and Expected
...
Medical record
...
...
responses therapy
Social interaction
Questionnaire Clinical Help
analysis
Smart phone Mood analysis Conversation
Exercises angle of movement
Drone Haptic feedback
recognition
Wearable device Gait analysis Notifications
Educational
Robot HAR
material
Monitor Predict Intervene
Figure 1: Instance of a blueprint for stroke rehabilitation configured with two Monitor-Predict-Intervene (MPI) component paths.
3.1 Reasoning with the MPI Cycle ing treatment. In this image, a patient is managing their joint
strengthening exercises using an exercise-MPI (consisting of
We propose inclusion of three components: digital Moni-
the components shaded orange), while managing their mood
toring (M) to track patient condition(s) and underpin antic-
through an emotion-MPI (shaded blue). The exercise-MPI
ipatory Predictions (P) to trigger real-time Interventions (I)
monitors bending of the joint via machine vision technology
for additional support when it is required. We refer to the
and a smart phone camera. Reasoning on the monitored data
combination of these components as an MPI. Reasoning is
allows the system to predict whether the exercises are being
facilitated by a self-management plan consisting of guide-
performed correctly (e.g. the angle of movement is satisfac-
lines, health recommendations, patient goals, decisions and
tory). The system can then intervene by actuating haptic feed-
a trace of previous lessons learned. These lessons learned
back through a patient’s wearable sensor, guiding the patient
data supports tailoring of plans to an individual. Given a self-
to perform the exercise correctly. Similarly, the emotion-MPI
management plan (e.g. level and frequency of exercise, tar-
monitors mood by reasoning on the patient’s responses to
geted levels of anxiety), the MPI cycle monitors adherence
questionnaires. If mood is predicted to be below the threshold
by a combination of patient self-reported outcome measures
determined by a clinician, an intervention can organise clini-
(e.g. pain) and automated real-time monitoring (e.g. of phys-
cal help before this evolves into depression. However if mood
iological response) by a variety of existing and yet-to-be-
is above this threshold, no intervention is necessary. The goal
created devices (e.g. wearables, implants, in-home sensors,
of each MPI is to Observe (O) the patient through monitor-
drones, robots). Differences in observed and expected adher-
ing to predict a comparison with the Expected (E) outcome
ence, combined with environmental and personal factors can
as established by their clinician. If the observed actions de-
be used to predict likely trends and outcomes. Thereafter, au-
viate sufficiently from the expected, then an intervention is
tonomous and remotely supported proactive interventions by
necessary to minimise the difference (min(δ(O, E))).
health professionals are initiated and plans are adapted col-
laboratively, replacing the current reactive patient-clinician
interaction. 3.2 Reusing self-management plans
An evidence-based approach to self-management is facili- We propose the idea of a blueprint through which an indi-
tated by having access to previous plans and lessons learned vidual’s self-management plan for multimorbidities can be
as well as reusing care-plans from similar patients that have formulated. Essentially a blueprint is a combination of one
been successful in relation to outcome measures. The rea- or more MPIs and their relationships (e.g. contraindica-
soning capability of the MPI increases with increasing adop- tions) which has been configured jointly with a clinician. As
tion of the framework. As the richness of the evidence base an example, the aforementioned Figure 1 shows a patient’s
grows (from initial guidelines to personalised plans) the im- blueprint for the self-management of stroke rehabilitation, as
pact on the community and their common self-management it combines the exercise-MPI and the emotion-MPI.
conditions will be improved. The reasoning necessary when combining multiple differ-
Thus MPIs form the building blocks for a multi-faceted ent MPIs is an important area of research; it must ensure ad-
self-management plan - a plan that can cater for multimor- herence to a patient’s collective self-management plans and
bidities. For example, consider Figure 1, which features suggest interventions that are suitable for all the multimor-
two MPIs created to support stroke rehabilitation. Stroke bidities, whilst maintaining knowledge of any contraindica-
survivors commonly struggle with freedom of movement in tions between those conditions. Central to this is a knowl-
their joints and have a tendency to develop depression dur- edge structure, an MPINet ontology, where known MPI re-
63
lationships can be recorded and used to form generalised model under the coordination of a central server acting as a
blueprints. These can be refined by co-occurrences inferred curator, selecting which participating devices (i.e. the fed-
from collected data. eration) to incorporate when training models. This form of
A shared community of blueprints is formed from individ- learning is ideal for community health care ensuring privacy
uals who share the same or a similar set of co-occurring con- by default, respecting data ownership, and maintaining lo-
ditions. Configurations embedded in blueprints can then be cality of data (without centralising data) for application de-
reused and adapted to suit new individuals joining that com- ployment at scale. An interesting direction of research here
munity (see Figure 2). In this image, blueprints have been is to the use of data provenance records combined with met-
extracted from members of the community who are similar rics evaluating quality and trust of individuals to influence
to the new individual, where ϕ is a reuse function containing the global computational curator’s decisions on sampling of
adaptation knowledge. The output is a blueprint configured MPIs on devices. With complex model architectures there is
to the individual’s needs founded on evidence-based practice. also a need to share and describe the architectural properties
This can be formalised as: so as to inform the curator about compatibility. This perhaps
O0 , E 0 = ϕ[(O, E)1 , (O, E)2 , ...] (1) calls for a meta-language for architectural descriptors. Learn-
ing from few labelled data is important for technology to op-
where O the configuration for what is to be observed and E 0
0
erate at scale. It will be useful to extend the FL paradigm
represents the modified expectations in this new blueprint. to evidence-based reasoning methods such as matching net-
works [Vinyals et al., 2016] to enable few-shot learning while
using a federated strategy.
= ϕ This type of environment is potentially suitable for im-
M P I M P I
, M P I
,… plementing a multi-agent framework [Moreno A, 2003],
whereby autonomous, adaptive and interactive software com-
Figure 2: Reuse of blueprints from similar individuals. ponents, provide notions that specifically meet the MPI
challenges to form the base of a robust and scalable self-
Personalisation of blueprints is achieved through the rec- management infrastructure. A multi-agent framework will
ommendation of contextually-relevant MPIs and customis- also enable the collection of patient-reported well-being in-
ing general community blueprints. Similarities in self- formation (as done in [Ibrahim et al., 2015]), integrating
management goals within a community can be used for per- them with sensor and device-generated data via API-based
sonalisation and to anticipate known common complications real-time data collection and streaming platforms such as the
of a condition, which can be extended to enable forecasting in-house build RADAR-base platform [Ranjan et al., forth-
for healthcare service demand. Metric learning algorithms coming 2019]. The combination will yield an intelligent and
that are suited for evidence-based reasoning lend well to real-time framework for schematised, secure and role-based
learning personalised models on the basis of similarity com- data collection, harmonisation and integration based on uni-
putations, and also lend themselves well to few-shot learning fied temporal intervals from all devices, sensors and individ-
approaches. Recent advances in similarity driven attention uals.
mechanisms currently use simplistic metrics, hence in future
research we must establish how similarity on the basis of the 4 The Future of Digital Self-Management
user contexts can be computed in a differentiating manner. Tackling self-management of multimorbidities requires a
Configuration of blueprints shares many similarities with model-aware and data-agnostic machine learning platform -
On-Line Case-Based Planning (OLCBP). Both require con- a modular digital health ecosystem, where multiple monitor-
tinuous monitoring of expected versus observed and acting ing technologies enable comprehensive predictions that trig-
upon deviations in a timely manner. Much like the snippets ger the most appropriate intervention strategy from a broad
that exist within a plan [Ontanón et al., 2010], MPIs act as range of possibilities. With that in mind, we suggest the pro-
the basic constituent pieces of a blueprint and are configured posed MPI architecture could answer many of the research
with individual goals. Though failure to achieve individual challenges discussed for individual technologies in Section 2
MPI goals does not necessarily define the failure of a self- and greatly improve the self-management experience from a
management plan, we suggest this is an early warning of the user perspective. However, there is also a need to consider
need for additional care. Importantly recording such failures how this architecture could be practically utilised at scale
and the follow-on adaptation of both the self-management while preserving patient privacy.
goal and agreed activities are useful experiential content that
are crucial when developing evidence-based practices for the 4.1 Wearables and Haptic Feedback
community. We aim to develop a framework to allow wearable technolo-
gies to provide real-time haptic feedback to support users ef-
3.3 Role of federated machine learning fectively perform rehabilitation exercises and physical activ-
Ideally, MPI models will learn from the data generated by all ities. Haptic feedback uses digital cues to stimulate the feel-
users of the system, i.e. in a distributed manner. This requires ing of touch in users. Whereas current wearable technolo-
us to adopt ideas from Federated Learning (FL) [McMahan et gies are reliant on the user reacting to textual notifications
al., 2017] and meta-learning. Specifically maintaining con- or messages, haptic feedback provides an opportunity to cor-
trol of device data, where training involves a shared global rect execution of exercises on-the-fly. The result would be
64
a decreased reliance on the external technology of a smart which can be exacerbated by the freedom of movement af-
phone and less expectation for users to continually refer to forded to a drone. Additionally, the anticipated skewed dis-
their phone throughout an exercise session. tribution of the data will pose another challenge that needs
There are two primary challenges to provisioning haptic to be addressed. In particular, having enough representative
feedback remotely; (1) currently, haptic feedback for therapy training examples that captures the different scenarios of a
is performed one-to-one, either with a healthcare professional specific case study or patient can potentially be very diffi-
activating the necessary feedback while locally observing the cult, meaning that few-shot learning strategies are likely to be
patient or the skeleton following a pre-programmed set of in- very relevant here. The benefits for patients go beyond sup-
structions; and (2) current telecommunication networks do port with performing exercises and documenting their self-
not have the throughput required to enable such feedback re- management progress; for example, people with movement
motely at scale. We believe that the former research chal- difficulties are often anxious about walking in unfamiliar en-
lenge could be answered through development of a learned vironments as they are unaware of any obstacles they may
model which can provide activation of the necessary compo- face - this can lead them to withdrawing from social activities
nents to actuate haptic feedback. The latter challenge may be and an increased feeling of loneliness. Being accompanied
answered with effective advanced networking infrastructures, by a drone that advises them of upcoming obstacles and how
such as 5G. they can be avoided, may reduce anxiety associated with such
situations.
4.2 Robots and Conversational Dialogue
The use of therapeutic robots to support community-dwelling 4.4 Ambient Assisted Living
individuals participate in rehabilitation, fitness exercises, so- We aim to use Ambient Assisted Living (AAL) technology to
cial engagement and outdoor physical activities are not only support non-intrusive monitoring of patients within their own
a viable solution to the problem of shortage in care resources, home. Several interesting research challenges arise from this
but also potentially address loneliness in the elderly. How- field. In particular, it will be interesting to see how the con-
ever, current SAR systems are limited in their capability to of- stant monitoring of AAL devices allows the identification of
fer truly personalised support. They do not take the user state self-management activities as they take place within aspects
into consideration, beyond verbal and interaction logs, suffer of daily living. For example, if a patient has climbed the stairs
from low engagement, and uni-directional information flow. 5 times in an afternoon as part of cleaning their home, this
Expanding this to exchange more information between the action may cover some of the ambulatory physical activity
user and the system through multiple modalities will enable a scheduled as part of managing their lower-back pain condi-
more versatile and natural interaction, ensuring that the user tion. Understanding the situations in which daily living activ-
maximally benefits from the system. In addition, it provides a ities can replace self-management exercises (or alternatively,
means to build a richer user-context that can help to improve, the situations in which they cannot) is an intriguing avenue
structure and personalise a robot’s behaviours and conversa- for exploration. Furthermore, there is the technical challenge
tional dialogue. To address these issues, novel methods are of being able to seamlessly change monitoring devices that
needed for semantically integrating various data sources re- feed into a single MPI e.g. as the patient transitions from
lating to (1) mood evaluations based on visual cues (i.e., fa- one room to the next. A comprehensive self-management
cial, bodily cues); (2) physical performance, psychological solution involving AAL devices should maximise comfort
variables and environmental variables collected from wear- of the patient and ensure that they are able to relax within
ables; and (3) adaptive activity programmes and behaviour their home without feeling like they are being continually
change strategies in real-time. Integrated information can be observed. Developing an individual’s trust requires further
then used to build effective reasoning and intervention mech- understanding of human machine interfaces by bringing to-
anisms for the interactive robot companion that can support gether behavioural psychologists and computer scientists.
citizens to lead active and social lives.
4.5 A Comprehensive Self-Management Solution
4.3 Autonomous Computer Vision The distributed MPI model comprises a number of devices
Computer-vision models can provide visual feedback to sup- collaborating autonomously, and a substantial amount of real-
port patients with rehabilitation activities. This involves the time data generated passively by sensors and devices or ac-
analysis of large volumes of imagery data, recognition of tively reported by the users (e.g. well-being reports) for a sin-
fine-grained human movement details and generation of vi- gle non-fragmented self-management solution. As such, the
sual feedback interventions. Drones could offer a rich source MPI framework constitutes a decentralised, heterogeneous
of information for on-going monitoring and observation of and open environment that operates on multiple computing
patients in different contexts, including both indoor self- systems to manipulate, share and analyse strictly-confidential
management activities and outdoor exercises. Utilising a vi- patient records. Moreover, to build a platform to continu-
sual source that captures patient’s movement and behaviours ously inform patients using their own records, requires mech-
from different angles provides a unique opportunity for learn- anisms to mine the ‘big’ and heterogeneous data for relevant
ing and improving practices, but at the same time poses some knowledge, and learn the adapted and personalised recom-
key challenges. These are due to the inherent vision prob- mendations and possible interventions in real-time to aid in
lem, in particular, performing accurate detection, tracking self-management. These methods are underpinned by the
and classification under different poses and light conditions, growing availability of advanced networking technology (e.g.
65
5G), where the faster connection speeds promise huge bene- [Chavarriaga et al., 2013] Ricardo Chavarriaga, Hesam
fits to future healthcare systems. Such advances allow high- Sagha, Alberto Calatroni, Sundara Tejaswi Digumarti,
intensity data processing to be supported by near real-time Gerhard Tröster, José del R Millán, and Daniel Roggen.
decision-making. We believe these will play a crucial role in The opportunity challenge: A benchmark database for
supporting at-home care (e.g. wearable sensors applications on-body sensor-based activity recognition. Pattern
and online consultations) and remote treatments (e.g. digital Recognition Letters, 34(15):2033–2042, 2013.
diagnosis). In particular, 5G will facilitate the transfer of ex- [Chen et al., 2017] Yoke Yie Chen, Nirmalie Wiratunga,
pertise over a great distance in real-time using technologies Stewart Massie, Jenny M Hall, Kate Stephen, Amanda
such as robotics and haptic feedback. Croall, Jacky MacMillan, Lesley Murray, Geoff Wilcock,
To give an example, a stroke survivor patient may use AAL and Sandra M MacRury. Designing a personalised case-
sensors for non-intrusive tracking of physical activity and in- based recommender system for mobile self-management
teract with a SAR to self-report on their pain and mental of diabetes during exercise. In In Workshop proceedings
condition while in the home. When journeying outside the of the 2nd Knowledge Discovery from Health Workshop at
house, the patient initially uses a drone with computer vi- the International Joint Conference on AI, 2017.
sion capabilities to monitor their journey, before transitioning
to use of a wearable technology as their walking ability im- [Feehan et al., 2018] Lynne M Feehan, Jasmina Geldman,
proves. The combination of monitoring technologies to em- Eric C Sayre, Chance Park, Allison M Ezzat, Ju Young
power interventions is a comprehensive solution to their self- Yoo, Clayon B Hamilton, and Linda C Li. Accuracy
management. We believe that not only will this greatly im- of fitbit devices: Systematic review and narrative syn-
prove adherence to the various aspects of a self-management theses of quantitative data. JMIR mHealth and uHealth,
plan, it will also have an irreplicable effect on patient quality 6(8):e10527, 2018.
of life. Furthermore, integration with the MPI framework will [Forbes et al., 2019] Glenn Forbes, Stewart Massie, and Su-
bring AI power to off-the-shelf hardware rather than building san Craw. Fall prediction using behavioural modelling
expensive devices, thus encouraging socially-inclusive care. from sensor data in smart homes. Artificial Intelligence
Review, pages 1–21, 2019.
5 Conclusion [Honig et al., 2018] S. S. Honig, T. Oron-Gilad, H. Zaichyk,
V. Sarne-Fleischmann, S. Olatunji, and Y. Edan. To-
In conclusion, in this position paper we have discussed the
ward socially aware person-following robots. IEEE
need for an affordable comprehensive self-management solu-
Transactions on Cognitive and Developmental Systems,
tion to improve quality of life for patients living with multiple
10(4):936–954, Dec 2018.
morbidities. We have presented our ideas towards the cre-
ation of MPI components - constituent pieces of wider-scale [Ibrahim et al., 2015] Zina Ibrahim, Lorena Fernández de la
framework which allows a configurable and personalised self- Cruz, Argyris Stringaris, Robert Goodman, Michael Luck,
management plan for individual patients, while encouraging and Richard Dobson. A multi-agent platform for automat-
reuse plans from within a community of similar patients and ing the collection of patient-provided clinical feedback. In
morbidities. The platform and MPIs potentially offer sev- Proceedings of the 2015 International Conference on Au-
eral interesting avenues of research, not only around how new tonomous Agents and Multiagent Systems, page 831–839,
and existing technologies can be integrated, but also around 2015.
areas such as comparing the similarity of MPIs and under- [Kruijff-Korbayová et al., 2014] I. Kruijff-Korbayová,
standing the ramifications of using them for reuse and adap- E. Oleari, I. Baroni, B. Kiefer, M. C. Zelati, C. Pozzi,
tation. Most importantly, we believe that a wide-scale inte- and A. Sanna. Effects of off-activity talk in human-robot
grated framework of devices is necessary for supporting pa- interaction with diabetic children. In The 23rd IEEE In-
tients with confidently managing their condition(s), improv- ternational Symposium on Robot and Human Interactive
ing their quality of life in the future. Communication, pages 649–654, Aug 2014.
[Leme et al., 2019] Bruno Leme, Masakazu Hirokawa,
References Hideki Kadone, and Kenji Suzuki. A socially assistive
[Bach et al., 2016] Kerstin Bach, Tomasz Szczepanski, Ag- mobile platform for weight-support in gait training.
nar Aamodt, Odd Erik Gundersen, and Paul Jarle Mork. International Journal of Social Robotics, Apr 2019.
Case representation and similarity assessment in the self- [Lynch et al., 2018] Elizabeth A Lynch, Taryn M Jones,
back decision support system. In Ashok Goel, M Belén Dawn B Simpson, Natalie A Fini, Suzanne S Kuys, Karen
Dı́az-Agudo, and Thomas Roth-Berghofer, editors, Case- Borschmann, Sharon Kramer, Liam Johnson, Michele L
Based Reasoning Research and Development, pages 32– Callisaya, Niruthikha Mahendran, et al. Activity monitors
46, Cham, 2016. Springer International Publishing. for increasing physical activity in adult stroke survivors.
[Barnett et al., 2012] Karen Barnett, Stewart W Mercer, Cochrane Database of Systematic Reviews, (7), 2018.
Michael Norbury, Graham Watt, Sally Wyke, and Bruce [Marling et al., 2012] Cindy Marling, Matthew Wiley, Raz-
Guthrie. Epidemiology of multimorbidity and implica- van Bunescu, Jay Shubrook, and Frank Schwartz. Emerg-
tions for health care, research, and medical education: a ing applications for intelligent diabetes management. AI
cross-sectional study. The Lancet, 380(9836):37–43, 2012. Magazine, 33(2):67–67, 2012.
66
[Massie et al., 2018] Stewart Massie, Glenn Forbes, Susan beeck, Sebastian Boettcher, Pauline Conde, Richard Dob-
Craw, Lucy Fraser, and Graeme Hamilton. Fitsense: Em- son, and Amos Folarin. RADAR-base: An open source
ploying multi-modal sensors in smart homes to predict mhealth platform for collecting, monitoring and analyzing
falls. In International Conference on Case-Based Reason- data using sensors, wearables, and mobile devices. Journal
ing, pages 249–263. Springer, 2018. of Medical Informatics Research mHealth and uHealth,
[McMahan et al., 2017] H. B. McMahan, E. Moore, Ram- forthcoming, 2019.
age, S. D., Hampson, and B. A. y Arcas. Communication- [Reiss and Stricker, 2012] Attila Reiss and Didier Stricker.
efficient learning of deep networks from decentralized Introducing a new benchmarked dataset for activity moni-
data. In Proceedings of the 20th International Conference toring. In Wearable Computers (ISWC), 2012 16th Inter-
on Artificial Intelligence and Statistics, pages 1273 – 1282, national Symposium on, pages 108–109. IEEE, 2012.
2017. [Sani et al., 2018] Sadiq Sani, Nirmalie Wiratunga, Stew-
[Montani et al., 2000] Stefania Montani, Riccardo Bellazzi, art Massie, and Kay Cooper. Personalised human activ-
Luigi Portinale, Giuseppe d’Annunzio, Stefano Fiocchi, ity recognition using matching networks. In Michael T.
and Mario Stefanelli. Diabetic patients management ex- Cox, Peter Funk, and Shahina Begum, editors, Case-Based
ploiting case-based reasoning techniques. Computer meth- Reasoning Research and Development, pages 339–353.
ods and programs in biomedicine, 62(3):205–218, 2000. Springer International Publishing, 2018.
[Moreno A, 2003] Garbay C. Moreno A. Software agents [Smith et al., 2012] Susan M Smith, Hassan Soubhi, Martin
in health care. Artificial intelligence in medicine, Fortin, Catherine Hudon, and Tom O’Dowd. Managing
27(3):229–232, 2003. patients with multimorbidity: systematic review of inter-
ventions in primary care and community settings. BMJ,
[Morris et al., 2012] Jacqui Morris, Tracey Oliver, Thilo
345, 2012.
Kroll, and Steve MacGillivray. The importance of psy-
chological and social factors in influencing the uptake and [Srisamosorn et al., 2016] Veerachart Srisamosorn, Noriaki
maintenance of physical activity after stroke: a structured Kuwahara, Atsushi Yamashita, Taiki Ogata, and Jun Ota.
review of the empirical literature. Stroke research and Design of face tracking system using environmental cam-
treatment, 2012, 2012. eras and flying robot for evaluation of health care. In Dig-
ital Human Modeling: Applications in Health, Safety, Er-
[Morris et al., 2014] Jacqui H Morris, Stephen gonomics and Risk Management, LNCS. Springer, 2016.
MacGillivray, and Sarah Mcfarlane. Interventions to
promote long-term participation in physical activity after [van der Drift et al., 2014] Esther J.G. van der Drift,
stroke: a systematic review of the literature. Archives Robbert-Jan Beun, Rosemarijn Looije, Olivier A. Blan-
of physical medicine and rehabilitation, 95(5):956–967, son Henkemans, and Mark A. Neerincx. A remote
2014. social robot to motivate and support diabetic children in
keeping a diary. In Proceedings of the 2014 ACM/IEEE
[Obeidat et al., 2019] Huthaifa Obeidat, Ali AlAbdullah, International Conference on Human-robot Interaction,
Nazar Ali, Rameez Asif, Omar Obeidat, Mohammed Bin- HRI ’14, pages 463–470, New York, NY, USA, 2014.
Melha, Wafa Shauib, Raed Abd-Alhameed, and Peter Ex- ACM.
cell. Local average signal strength estimation for indoor
multipath propagation. Accepted to IEEE Access, April [Vinyals et al., 2016] Oriol Vinyals, Charles Blundell, Tim-
2019. othy Lillicrap, and Daan Wierstra. Matching networks for
one shot learning. In Advances in neural information pro-
[Oguntala et al., 2019] George A. Oguntala, Raed A. Abd- cessing systems, pages 3630–3638, 2016.
Alhameed, James M. Noras, Yim-Fun Hu, Eya N.
[Wolff et al., 2002] Jennifer L. Wolff, Barbara Starfield, and
Nnabuike, Nazar Ali, Issa T. Elfergani, and Jonathan Ro-
driguez. Smartwall: Novel rfid-enabled ambient human Gerard Anderson. Prevalence, Expenditures, and Com-
activity recognition using machine learning for unobtru- plications of Multiple Chronic Conditions in the Elderly.
sive health monitoring. Accepted to IEEE Access, May JAMA Internal Medicine, 162(20):2269–2276, 11 2002.
2019. [Yao et al., 2019] Ning-shi Yao, Qiu-yang Tao, Wei-yu Liu,
[Ontanón et al., 2010] Santi Ontanón, Kinshuk Mishra, Zhen Liu, Ye Tian, Pei-yu Wang, Timothy Li, and Fumin
Zhang. Autonomous flying blimp interaction with human
Neha Sugandh, and Ashwin Ram. On-line case-based
in an indoor space. Frontiers of Information Technology &
planning. Computational Intelligence, 26(1):84–119,
Electronic Engineering, 20(1):45–59, Jan 2019.
2010.
[Rabbitt et al., 2015] Sarah M. Rabbitt, Alan E. Kazdin, and
Brian Scassellati. Integrating socially assistive robotics
into mental healthcare interventions: Applications and rec-
ommendations for expanded use. Clinical Psychology Re-
view, 35:35 – 46, 2015.
[Ranjan et al., forthcoming 2019] Yatharth Ranjan, Zulqar-
nain Rashid, Callum Stewart, Mark Begale, Denny Ver-
67