=Paper=
{{Paper
|id=Vol-2804/paper5
|storemode=property
|title=A Recommender System for Behavioral Change in 60-70-year-old Adults
|pdfUrl=https://ceur-ws.org/Vol-2804/paper5.pdf
|volume=Vol-2804
|authors=Pierpaolo Palumbo,Luca Cattelani,Federica Fusco,Mirjam Pijnappels,Lorenzo Chiari,Federico Chesani,Sabato Mellone
|dblpUrl=https://dblp.org/rec/conf/aiia/PalumboCFPCCM20
}}
==A Recommender System for Behavioral Change in 60-70-year-old Adults==
https://ceur-ws.org/Vol-2804/paper5.pdf
A recommender system for behavioral change in 60-70-year-old
adults
Pierpaolo Palumboa, Luca Cattelania,b, Federica Fuscoc, Mirjam Pijnappelsd, Lorenzo Chiaria,
Federico Chesania, and Sabato Mellonea
a
University of Bologna, Viale del Risorgimento, 2, 40136, Bologna, Italy
b
Tampere University, Arvo Ylpön katu 34, 33520 Tampere, Finland
c
Yoox Net-A-Porter Group S.p.A., Via Nerio Nannetti, 1, 400069, Zola Predosa, Italy
d
Vrije Universiteit Amsterdam, Department of Human Movement Sciences, van der Boechorststraat 7, 1081BT
Amsterdam, The Netherlands
Abstract
Early old age (60-70 years old) is a particular period of life when possible habit modifications
may occur, often related to job retirement. While taking up a more sedentary lifestyle may be
pernicious for health, changing behavior by introducing simple exercises within daily life
routines can effectively prevent age-related functional decline.
This article presents the Profiling Tool, a system that provides 60-70-year-old adults with
personalized recommendations to integrate simple activities, promoting balance, strength, and
physical activity into their daily life. Its first implementation has been designed on information
from literature, data from previously available longitudinal datasets, and experts' opinions. It
has been deployed within a randomized controlled trial. Strategies for its update are based on
model-based reinforcement learning approaches.
Keywords 1
Ageing, functional decline, prevention, recommender system, behavioral change
1. Introduction
Population aging is one of the major issues of our present world. Developing preventive
interventions is one of the keys to tackling this issue, and Artificial Intelligence (AI) can enable these
interventions and make them more effective and efficient.
Early old age (60-70 years old) is thought to be the right window of opportunity for prevention. In
this period of life, habit modifications may take place, often related to job retirement. While taking up
a more sedentary lifestyle may be pernicious for health, changing behavior by introducing simple
exercises within daily life routines can effectively prevent age-related functional decline.
Different mobile applications for healthy lifestyle promotion have been developed using behavioral
change theories, and some of them have been tested within randomized controlled trials [1]–[3].
However, no design principle for using users' data to issue optimal recommendations has been ideated
and put in place. Within this work, we present the Profiling Tool, a tool developed within the PreventIT
project [4] that provides personalized recommendations to 60-70-year-old adults on strength, balance,
and physical activities to integrate into daily life routines. Its ideation and design are based on
psychological theories and techniques of behavioral change [5] and AI solutions for recommender
systems [6].
Italian Workshop on Artificial Intelligence for an Ageing Society (AIxAS 2020), November 25–27, 2020, Anywhere
EMAIL: pierpaolo.palumbo@unibo.it (P.P.); luca.cattelani@unibo.it (L.C.); fedef.89@hotmail.it (F.F.); m.pijnappels@vu.nl (M.P.);
lorenzo.chiari@unibo.it (L.C.); federico.chesani@unibo.it (F.C.); sabato.mellone@unibo.it (S.M.)
ORCID: 0000-0002-4438-1787 (P.P.); 0000-0003-4852-2310 (L.C.); 0000-0001-8416-2602 (M.P.); 0000-0002-2318-4370 (L.C.); 0000-
0003-1664-9632 (F.C.); 0000-0001-7688-0188 (S.M.).
©️ 2020 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
In the following sections, we overview the PreventIT project and IT ecosystem and describe the
Profiling Tool, including its modeling, its first implementation and deployment within a randomized
control trial, and strategies for its update.
2. PreventIT and the iPAS
PreventIT stands for 'Early risk detection and prevention in aging people by self-administered ICT-
supported assessment and a behavioral change intervention, delivered by use of smartphones and
smartwatches.' It is a European Horizon 2020 project carried out from January 2016 to March 2019 [4].
The project aimed to develop a proof-of-concept, unobtrusive mobile health system based on a
personalized behavior change intervention on balance, strength, and physical activity. The intervention
is designed for young older adults (adults between 60 and 70 years old) to prevent accelerated functional
decline at an older age.
The PreventIT ICT based Personalized Activity System (iPAS) is a mobile health system delivering
the intervention on smartphones and smartwatches. It includes a smartphone and smartwatch app as
frontend and a risk model for functional decline [7], [8], the eLiFE intervention program, a Profiling
Tool for personalizing the intervention, and a behavior change theories-based motivational strategy
running on a cloud-based backend (Figure 1).
Figure 1. The PreventIT mobile health system's architecture, including risk screening for functional
decline, profiling for personalizing the intervention, the eLiFE intervention program with balance,
strength, physical activity integrated into daily life, and individual feedback on behavior aimed at
increasing motivation for behavior change. A smartphone and a smartwatch are used to monitor
behavior, deliver the intervention, and give individualized feedback on behavior.
The PreventIT intervention program is based on the Lifestyle-integrated Exercise (LiFE) approach
[9]. In LiFE, rather than using a prescribed set of exercises, activities are performed whenever the
opportunity arises during the day. The LiFE approach allows personalizing and integrating exercise in
daily life, and it was found to significantly reduce falls, improve physical function, decrease disability
and improve adherence, compared with a traditional exercise program and a sham intervention [10]. In
PreventIT, the original LiFE was adapted (aLiFE, adapted LiFE, [11]) to the needs of 60-70-year-old
adults to make activities challenging and complex enough for a younger target population. The
integration of the aLiFE program into the PreventIT iPAS is named eLiFE (enhanced LiFE, [12]).
Since the LiFE program relies on users embedding balance, strength, and physical activities into
their everyday life, it can only be successful if they change their behavior. The original LiFE concept
is underpinned by the behavioral change concepts of habit formation, self-efficacy, skills training, and
outcomes gained. The motivational strategy in PreventIT is based on the extension of the behavioral
change framework supporting the intervention [5].
3. Profiling Tool
The Profiling Tool is a tool for personalized recommendations on activities to be integrated into
seniors' daily life routines.
The Profiling Tool takes as input an individual's health state, a list of potential activities and
difficulty levels, an estimate of their expected impact on the individual's health state, and contextual
information, including the individual's preferences for the activities. On this knowledge basis, the
Profiling Tool provides recommendations to the individual on which activities best fit their needs and
the appropriate difficulty level for each activity.
There are 21 types of activities in the eLiFE program with up to four difficulty levels for each
activity, grouped into three domains:
1. Strength domain: squatting, lunging, walking on toes, walking on heels, stair climbing, sit-to-
stand, move legs sideways, tighten muscles;
2. Balance domain: tandem stand, one-leg stand, tandem walk, side-to-side leaning, forward-
backward leaning, stepping over objects, stepping and changing direction, square stepping and
hopping, square jumping;
3. Physical activity domain: walk longer, walk faster, sit less, break-up sitting.
These same three domains describe the individual's health state.
An expected benefit is calculated for every single eLiFE activity on the specific user profile.
Recommendations are provided to the individual, accompanied by motivational messages, designed
according to theoretical constructs of behavioral change (e.g., the Health Action Process Approach) [5].
Each day the subject selects a list of activities he/she will perform during the day and confirms the
actually-performed activities at the end of the day. After every six months, the subject is assessed for
his/her health state (Figure 2A). All this information about the interactions between the Profiling Tool
and the individual and their effects is recorded by the iPAS and used by the Profiling Tool for its update.
In the following, we give a modeling description of the Profiling Tool and its interactions with the
user, describe the implementation of its first version within the PreventIT project, and present an
updating strategy.
4. Models
To appropriately design the Profiling Tool, including its recommendation policy and updating
strategy, we characterize the interactions between the Profiling Tool and the user in terms of two
models, describing the preferences of the individual for the activities and the benefit of these activities
on the health state respectively.
4.1. Preference model
We define the preference model as the model that describes the activities that an individual with
specific characteristics would perform when given personalized recommendations.
We use the subscript 𝑡 to indicate the 𝑡-th six-month time period and the subscript 𝑡 ∙ 𝑖 to indicate
the 𝑖-th day of the 𝑡-th six-month period.
We call 𝑥𝑡 the vector of subject's features – including their health state, 𝑑𝑡 = 𝑑(𝑥𝑡 ) the personalized
′
recommendations issued by the Profiling Tool, and 𝑧𝑡∙𝑖 = (𝑧𝑡∙𝑖,1 , 𝑧𝑡∙𝑖,2 , … , 𝑧𝑡∙𝑖,𝐾 ) the vector expressing
the 𝐾 = 21 activities performed by the individual. In particular, 𝑧𝑡∙𝑖,𝑘 is the number of times the subject
has performed activity 𝑘 during day 𝑡 ∙ 𝑖. We call 𝑧𝑡 the vector expressing the number of times the
individual has performed each activity during six months
179 (1)
𝑧𝑡 = ∑ 𝑧𝑡∙𝑖
𝑖=0
Thus, the preference model that relates the cumulative selections 𝑧𝑡 with the suggestions 𝑑𝑡 can be
expressed as
𝑝(𝑧𝑡 |𝑥𝑡 , 𝑑𝑡 ) (2)
Within the first version of the Profiling Tool, recommendations 𝑑𝑡 were given in the form of an
ordered list of potential activities, sorted according to their expected benefit on the individual's health
state. Other choices are also possible to express more quantitatively the strength of recommendation for
each activity. For example, 𝑑𝑡 = (𝑑𝑡,1 , 𝑑𝑡,2 , … , 𝑑𝑡,𝐾 ) could be a vector of such degrees of
recommendation for each activity, under constrains
𝑑𝑡,𝑘 ≥ 0 (3)
𝐾
∑ 𝑑𝑡,𝑘 = 1 (4)
𝑘=1
A simple parametric form for the preference model (2) is
𝑧𝑡 = 𝛿0 + 𝛿1′ 𝑑𝑡 + 𝜀𝑧,𝑡 (5)
′
where 𝛿0 encodes personal preferences, 𝛿1 𝑑𝑡 encodes the influence of the recommendations, and
𝜀𝑧,𝑡 is an error term.
For recommendations that vary every day, the preference model can be expressed by
𝑝(𝑧𝑡∙𝑖 |𝑥𝑡 , 𝑑𝑡∙𝑖 ) (6)
and the linear model (5) could be replaced by a logistic or Poisson model over 𝑧𝑡∙𝑖,𝑘 . We note that
the features of the individual 𝑥𝑡 do not change every day, as the health state is assessed once every six
months.
Other forms for the preference model can be borrowed by the rich literature on choice modeling,
and random utility theory [13], and models can easily be tested on data, as quantities 𝑥, 𝑑, and 𝑧 are all
observed and recorded in the iPAS system.
4.2. Health effect model
We define the health effect model as the model that describes the future health state 𝑥𝑡+1 , based on
the current health state 𝑥𝑡 and the activities 𝑧𝑡 performed by the individual
𝑝(𝑥𝑡+1 |𝑥𝑡 , 𝑧𝑡 ) (7)
Within the feature vector 𝑥, one variable 𝑦 can be chosen as the primary outcome. A simple
parametric form of the health effect model restricted to this outcome is
𝑦𝑡+1 = 𝑦𝑡 + 𝛼 + 𝛽 ′ 𝑧𝑡 + 𝛾 ′ 𝑥𝑡 + 𝑧𝑡′ 𝜃𝑥𝑡 + 𝜀𝑦,𝑡 (8)
where 𝛼, 𝛽, and 𝛾 are vector parameters, 𝜃 is a matrix parameter, and 𝜀𝑦,𝑡 is an error term.
According to this model for the outcome, replacing 𝑧𝑡 with a vector having 1 in the 𝑘-th component
and zero otherwise, we get the expected health benefit on the outcome of one unit of activity 𝑘 as
𝛽𝑘 + ∑ 𝜃𝑘𝑗 𝑥𝑗 (9)
𝑗
where 𝛽𝑘 is the 𝑘-th component of vector 𝛽 and 𝜃𝑘𝑗 is the entry in position (𝑘, 𝑗) of matrix 𝜃.
Figure 2: Activity selection and health effects. Panel A: The green rectangle represents a six-month
cycle. Panel B: direct acyclic graph (DAG) [14] for the Bayesian network of health states 𝑥𝑡 and 𝑥𝑡+1 ,
recommendation 𝑑𝑡 , and performed activities 𝑧𝑡 . It encodes the conditional independence between
𝑥𝑡+1 and 𝑑𝑡 , given 𝑥𝑡 and 𝑧𝑡 .
As it is reasonable, we assume that the future health state 𝑥𝑡+1 is independent of the recommendation
𝑑𝑡 , conditional on the current health state 𝑥𝑡 and the performed activities 𝑧𝑡 (Figure 2B)
𝑥𝑡+1 ⊥ 𝑑𝑡 | 𝑥𝑡 , 𝑧𝑡 (10)
Under this assumption, the transition probability 𝑝(𝑥𝑡+1 |𝑥𝑡 , 𝑑𝑡 ) can be expressed as
𝑝(𝑥𝑡+1 |𝑥𝑡 , 𝑑𝑡 ) = ∫ 𝑝(𝑥𝑡+1 |𝑥𝑡 , 𝑧𝑡 ) 𝑝(𝑧𝑡 |𝑥𝑡 , 𝑑𝑡 )𝑑𝑧𝑡 (11)
𝑧
where we recognize the product of the preference and health effect models within the integral.
5. The first version of the Profiling Tool
The first version of the Profiling Tool was developed on knowledge from the literature, data from
population studies on aging, and opinions from experts. It was tested in a feasibility randomized
controlled trial (RCT) within the PreventIT study.
5.1. Design
This version for activity recommendation was based on four rules.
First, the feature vector at baseline 𝑥0 was the three-score individual profile
𝑥0 = (𝑠1 , 𝑠2 , 𝑠3 ) (12)
each score 𝑠𝑖 ranging from 0 to 5 and expressing the prioritization of exercise on balance, strength,
and physical activity domains. Each 𝑠𝑗 was derived comparing measures of physical performance
against cut-offs derived from the literature [15]–[19] and data of 60-70-year-old individuals pooled
from three longitudinal studies on aging (ActiFE Ulm [20], InCHIANTI [21], LASA [22]). More in
particular, for each domain, we considered two-to-three variables and created categories on these
variables using cut-off values found in the literature. After applying these categories on the pooled
cohort, if a prevalence of at least 10% was found in each category, the cut-off was retained valid.
Otherwise, the cut-off was derived from the tertiles of the variable on the pooled cohort. Table 1 reports
cut-offs, scores, and summary statistics on participants of the PreventIT study.
Table 1
Prevalence of categories of the individual profile of the Profiling Tool version 1 in participants in the
feasibility RCT (n=189)
Assessment Cut-off scores Profiling Prevalence Prevalence
(males/females) score in males in females
(n=90) (n=99)
Balance
Tandem stance Unable or 0-9.99 s 2 24 (26.7%) 34 (34.3%)
(eyes open) Able to hold for ≥10s 0 66 (73.3%) 65 (65.7%)
Tandem stance Unable or 0-9.99 s 2 76 (84.4%) 80 (80.8%)
(eyes closed) Able to hold for ≥10s 0 14 (15.6%) 19 (19.2%)
One leg stance Unable or 0-9.99 s 1 38 (42.2%) 41 (41.4%)
(eyes open) Able to hold for ≥10s 0 52 (57.8%) 58 (58.6%)
𝑠1: total score Balance Range 0-5 2 (2-4) 2 (2-4)
median (IQR)
Strength
Handgrip strength ≤40.0 kg/ ≤23.0 kg 2 23 (25.8%) 25 (25.3%)
(max. of one hand) 40.0-47.0 kg/ 23.1-28.0kg 1 41 (46.1%) 35 (35.4%)
>47.0 kg/ >28.0 kg 0 25 (28.1%) 39 (39.4%)
Chair stand test Unable or ≥13 s/ ≥14.1 s 3 14 (15.6%) 10 (10.2%)
(five times) 10.8-12.9 s/ 11.5-14.0 s 2 18 (20%) 19 (19.4%)
≤10.7 s/ ≤11.4 s 0 58 (64.4%) 69 (70.4%)
𝑠2: total score Range 0-5 2 (1-3) 1 (0-3)
Strength
median (IQR)
Physical activity
Gait speed <1.0 m/s 2 1 (1.1%) 3 (3%)
≥1.0 m/s 0 89 (98.9%) 96 (97%)
Moderate/vigorous <150 minutes 1 30 (33.7%) 30 (31.2%)
activity per week ≥150 minutes 0 59 (66.3%) 66 (68.8%)
Step count <7,499 steps/day 2 10 (11.2%) 6 (6.2%)
7,500-9,999 steps/day 1 13 (14.6%) 17 (17.7%)
≥ 10,000 steps/day 0 66 (74.2%) 73 (76%)
𝑠3: total score Physical Range 0-5 1 (0-1) 1 (0-1)
activity
median (IQR)
Total score user 5 (4-7) 5 (3-7)
profile, median (IQR)
Values are n (%) unless otherwise indicated.
Second, suggested activities were taken from a list of 21 activities, grouped according to three
domains. Each activity was made of up to five difficulty levels, for a total of 89 exercises. The expected
health impact of each activity was estimated from equation (9). In particular, the offsets 𝛽𝑘 were set to
zero, and matrix 𝜃 for the impact of each activity on each domain was filled by expert judgments with
scores from 0 to 5.
Third, activities marked as not pleasant by the individual were dropped off the list of
recommendations for the following days.
Fourth, for each suggested activity, its starting difficulty level was determined based on the
individual's abilities assessed at the beginning of using the Profiling Tool by a trainer. The individual
could decide at any time to downgrade the difficulty level of an activity, but they needed to train long
enough to upgrade it.
Resulting recommendations 𝑑0 = 𝑑(𝑥0 ) were given in the form of a list of activities, sorted in
descending order according to their expected health benefit.
The activities 𝑧𝑡∙𝑖 performed each day were registered by the iPAS system, integrating feedback
provided by the individual at the end of the day and recordings from global positioning system (GPS)
and inertial measurement units (IMU) sensors embedded in the mobile phone.
A demo of this first version of the Profiling Tool is available on the Internet
(http://taxonomy.disi.unibo.it/TaskRecommenderDemo/) [23].
5.2. Deployment
The Profiling Tool was tested within the three-arm PreventIT feasibility RCT (n=180) on three
clinical centers in Trondheim, Stuttgart, and Amsterdam. One arm was assigned to the iPAS system
and the Profiling Tool (eLiFE), one was given a booklet with recommendations by a trainer on activities
to integrate into daily life (aLiFE). At the same time, participants of the control group were provided
general physical activity recommendations. The primary outcome 𝑦 was taken to be the Late-Life
Function and Disability Instrument (LLFDI) [24], [25]. A detailed description of the trial protocol is
available at [4].
The scoring system for the individual profile showed to be appropriate in stratifying the target
population on domains of balance and strength, whereas, in the physical activity domain, too few
participants (< 10%) fell on the lowest categories defined on gait speed and step count (Table 1).
On the participants of the eLiFE intervention arm that used the Profiling Tool (n=50), we evaluated
with the iPAS system whether the ranking that was suggested by the Profiling Tool 𝑑(𝑥0 ) was actually
selected by the participants. In Table 2, it can be seen that there is not a clear association between the
ranking of activities by the Profiling Tool and the actual choice of participants from the 21 activities.
Activities ranked higher by the Profiling Tool, such as 'Square stepping and hopping' and 'Square
jumping,' were not more frequently selected by participants to incorporate in their intervention regime.
The only activities that showed a significant association (p<0.05) were 'Stepping over objects,' 'Stepping
and changing direction,' and 'Lunging,' but there is not a clear pattern in the data to explain these
associations.
The first evidence also shows that changes in health outcomes were modest over the RCT
participants, making health effect models challenging to fit (data not shown).
Table 2
Frequency of activities ranked in the top 7 with Profiling Tool version 1 and that were actually selected
by eLiFE participants (n=50).
Activities Most frequent In top 7 based Actually selected Chi-square test
ranking profiling on profiling tool by participants ranking vs.
tool selected p-value
Domain 1: Balance
Tandem stand 21 - 25 (50.0%) 0.848
One leg stand 10 - 34 (68.0%) 0.083
Tandem walk 11 10 (20.0%) 24 (48.0%) 0.153
Side-to-Side leaning 7 13 (26.0%) 9 (14.0%) 0.609
Forwards and backwards leaning 3 23 (46.0%) 11 (22.0%) 0.685
Stepping over objects 8 2 (4.0%) 5 (10.0%) 0.009
Stepping and changing direction 9 9 (18.0%) 8 (16.0%) 0.039
Square stepping and hopping 2 40 (80.0%) 3 (6.0%) 0.869
Square jumping 1 48 (96.0%) 3 (6.0%) 0.060
Domain 2: Strength
Squatting 15 8 (16.0%) 26 (52.0%) 0.251
Lunging 4 41 (82.0%) 27 (54.0%) 0.023
Walking on toes 5 39 (78.0%) 22 (44.0%) 0.247
Walking on heels 6 37 (74.0%) 11 (22.0%) 0.214
Stair climbing 17 24 (48.0%) 29 (58.0%) 0.064
Sit to stand 13 32 (64.0%) 26 (52.0%) 0.150
Move leg sideways 20 11 (22.0%) 10 (20.0%) 0.072
Tighten muscles 21 2 (4.0%) 11 (22.0%) 0.104
Domain 3: Physical activity
Walk longer 18 - 9 (18.0%) 0.948
Walk faster 15 2 (4.0%) 5 (10.0%) 0.927
Sit less 19 - 17 (34.0%) 0.498
Break up sitting 12 9 (18.0%) 24 (48.0%) 0.387
Values are n (%).
6. Updating strategy
Data collected from the iPAS (either in the above-mentioned feasibility RCT or its continuous usage)
may refine the Profiling Tool and make it more effective. To this end, we model the interactions
between the Profiling Tool and the subject with a Markov Decision Process (MDP) [26]
𝑀𝐷𝑃 = {𝑇, 𝑋, 𝔇, 𝑝(𝑥𝑡+1 |𝑥𝑡 , 𝑑𝑡 ), 𝑟(𝑥𝑡 , 𝑑𝑡 )} (13)
where:
• 𝑇 is an ordered set of time points;
• 𝑋 is the set of features characterizing the individuals;
• 𝔇 is the set of recommendations that the Profiling Tool can issue;
• 𝑝(𝑥𝑡+1 |𝑥𝑡 , 𝑑𝑡 ) is the transition probability between state 𝑥𝑡 ∈ 𝑋 at time 𝑡 ∈ 𝑇 to state 𝑥𝑡+1
∈ 𝑋 at time 𝑡 + 1 ∈ 𝑇, when the Profiling Tool has issued the recommendation 𝑑𝑡 ∈ 𝔇;
• 𝑟(𝑥𝑡 , 𝑑𝑡 ) is the reward of being in the state 𝑥𝑡 and issuing recommendation 𝑑𝑡 .
We assume that issuing different recommendations has the same cost and thus the reward 𝑟(𝑥𝑡 , 𝑑𝑡 )
is a function of the sole health state 𝑥𝑡+1 . In particular, we pose a reward equal to the primary outcome:
𝑟(𝑥𝑡 , 𝑑𝑡 ) = 𝑦𝑡+1 . (14)
Considering to use the data collected during the PreventIT feasibility trial to develop a second
version of the Profiling Tool (Figure 3A), the Markov decision problem is defined over only one period
(𝑇 = {0,1}) and is stated as follow
max 𝐸[𝑟(𝑥1 )|𝑥1 , 𝑑1 ] = max 𝐸[𝑦2 |𝑥1 , 𝑑1 ] (15)
𝑑1 :𝑋 →𝔇 𝑑1 :𝑋 → 𝔇
= max ∫ 𝐸[𝑦2 |𝑥1 , 𝑑1 , 𝑧1 ]𝑝(𝑧1 |𝑥1 , 𝑑1 )𝑑𝑧1
𝑑1 :𝑋 →𝔇
𝑧
= max ∫ 𝐸[𝑦2 |𝑥1 , 𝑧1 ]𝑝(𝑧1 |𝑥1 , 𝑑1 )𝑑𝑧1
𝑑1 :𝑋 → 𝔇
𝑧
Using the linear outcome model for 𝐸[𝑦2 |𝑥1 , 𝑧1 ] as in equation (8), and the preference selection
model in equations (3-5), the problem (15) becomes:
maximize
𝑑1 ′ 𝛿1 (𝛽 + 𝜃𝑥1 ) (16)
subject to
𝑑1,𝑘 ≥ 0 ∀𝑘 (17)
𝐾
∑ 𝑑1,𝑘 = 1 (18)
𝑘=1
Figure 3: Update of the Profiling Tool (PT). Panel A. Quantities in black are those already collected with
the first experimentation of the PT in PreventIT; quantities in grey are those relative to a second
version of the PT. Panel B. Iterative updating strategy of the PT's preference and outcome models, in
the case of iterative deployment. The inner green rectangle represents a six-month cycle with a time
unit equal to one day, while the outer blue rectangle represents a cycle over repetitions of six-month
cycles (i=0:179).
Given 𝑥1 and having estimated the parameters 𝛿1 , 𝛽, and 𝜃 from the data, the problem (16-18) is a
simple linear program in the canonical form. Calling 𝑎 the vector 𝛿1 (𝛽 + 𝜃𝑥1 ), and provided that 𝑎 has
at least one positive component, the problem is solved by the sparse vector 𝑑∗ = (𝑑𝑘∗ ), so that 𝑑𝑘∗ = 1
for 𝑘 = 𝑎𝑟𝑔 max 𝑎𝑘 , and 𝑑𝑗∗ = 0 for all others 𝑗 ≠ 𝑘. We note that replacing constraint (3) with one
𝑘
over the L2 norm of 𝑑𝑡 , makes the solution non-sparse.
Model parameters (e.g. 𝛿0 , 𝛿1 , 𝛼, 𝛽, …) could be derived for a) the whole population or sets of users,
b) in a subject-specific manner, or c) combining both approaches with mixed-effect models. We judged
that data from the PreventIT feasibility trial are insufficient to estimate all model parameters with
appropriate precision and robustness. Hence, model fitting can proceed according to Bayesian
estimation using parameter values of the first version to construct prior parameter distributions.
Otherwise, data-driven recommendations can be combined heuristically with recommendations coming
from the first version.
Figure 3B further shows a schema for updating the Profiling Tool beyond the second version, upon
consecutive deployments over a time horizon 𝑇. The Profiling Tool is foreseen to evolve as more data
accrue and update the preference and health effect models. The preference model can be updated every
day since performed activities 𝑧𝑡∙𝑖 are recorded daily, while the health effect model is updated with a
six-month periodicity.
Focusing on the slower update periodicity and following the conceptual framework usually
employed with MDPs, we define a cumulative reward
𝑇 𝑇
𝑅(𝑥0 ) = 𝐸 [∑ 𝑟(𝑥𝑡 , 𝑎𝑡 ) | 𝑥0 ] = 𝐸 [∑ 𝑦𝑡 | 𝑥0 ]
(19)
𝑡=0 𝑡=1
Considering the recommendation function
𝑑𝑡 : 𝑋 → 𝔇
(20)
that possibly changes with time as the Profiling Tool is updated, we aim to find a recommendation
policy
𝜋 = (𝑑0 , 𝑑1 , … , 𝑑𝑇 )
(21)
that maximizes 𝑅(𝑥0 ).
Upon knowledge of the preference and health effect models, the transition probability is known and
the Markov decision problem to find the optimal policy 𝜋 ∗ can be solved with linear programming
techniques (e.g., backward induction, value iteration, or policy iteration algorithms). However, in the
more general case, both the transition probability and the recommendation policy have to be learned on
data, as long as they accrue. Reinforcement learning heuristics serve this case [27], balancing the
tradeoff between exploiting the likely most effective recommendations and exploring others'
effectiveness.
7. Discussion
We have presented the Profiling Tool's design and first deployment, a recommender system for
behavioral change of 60-70-year-old adults.
Its design was inspired by and based on psychological theories and techniques of behavioral change
[5] and AI solutions for recommender systems [6]. Its first version was designed on information from
the literature, data from cohorts of epidemiological studies on aging, and experts' opinions. The
mathematical models that describe its interactions with the user serve to analyze its functioning and
plan updating strategies as more data get available. To the best of our knowledge, their employment is
new in the applicative field of mobile applications for prevention.
Analyses from its first deployment within the PreventIT feasibility RCT have provided insights.
First of all, the individual profile scoring was shown to be satisfactory, distinguishing distribution of
scores on the domains of balance and strength, but not on the physical activity domain, in our cohort of
people aged 60-70 years old.
Secondly, recommendations are only loosely associated with actually-selected activities. In the
PreventIT feasibility RCT, the intervention regime was put together by the participants themselves, in
consultation with the trainer. This might have affected the decisions of participants and could have
overruled the ranking by the Profiling Tool. Another possible cause behind this lack of correspondence
between recommendations and user selection of activities may lie in the form the recommendations
were provided. More specifically, recommendations were ordered list of activities without any
indication of the strength of recommendation associated with each activity. For future developments,
we could test whether recommendations become more convincing by expressing the strength of
recommendation more quantitatively or by presenting a limited number (e.g., only the top 7 rankings)
of activities. It is further suggested to explore different strategies for planning the intervention regime
and sending motivational messages accompanying the recommendations.
Preliminary analyses have also shown that health changes could be small over six months for a
highly functional target population, making health effect models challenging to estimate. This issue
could be solved by deploying the tool on a population which is broader and more heterogeneous.
8. Acknowledgments
This study has been partly funded by the European Commission under the project 'PreventIT' (2016–
2018, grant number 689238) responding to the Horizon 2020, Personalised Health and Care call PHC-
21: Advancing active and healthy aging with ICT: Early risk detection and intervention.
9. References
[1] S. Mehra et al., "Translating behavior change principles into a blended exercise intervention for
older adults: Design study," J. Med. Internet Res., vol. 20, no. 5, May 2018, doi:
10.2196/resprot.9244.
[2] A. C. King et al., "Effects of Three Motivationally Targeted Mobile Device Applications on
Initial Physical Activity and Sedentary Behavior Change in Midlife and Older Adults: A
Randomized Trial," PLoS One, vol. 11, no. 6, p. e0156370, Jun. 2016, doi:
10.1371/journal.pone.0156370.
[3] L. Paul et al., "Increasing physical activity in older adults using STARFISH, an interactive
smartphone application (app); a pilot study," J. Rehabil. Assist. Technol. Eng., vol. 4, p.
205566831769623, Jan. 2017, doi: 10.1177/2055668317696236.
[4] K. Taraldsen et al., "Protocol for the PreventIT feasibility randomised controlled trial of a
lifestyle-integrated exercise intervention in young older adults.," BMJ Open, vol. 9, no. 3, p.
e023526, Mar. 2019, doi: 10.1136/bmjopen-2018-023526.
[5] E. Boulton et al., "Implementing behaviour change theory and techniques to increase physical
activity and prevent functional decline among adults aged 61–70: The PreventIT project,"
Progress in Cardiovascular Diseases, vol. 62, no. 2. W.B. Saunders, pp. 147–156, 01-Mar-
2019, doi: 10.1016/j.pcad.2019.01.003.
[6] F. Ricci, L. Rokach, and B. Shapira, Eds., Recommender systems handbook, Second Edi. New
York: Springer, 2015.
[7] N. H. Jonkman et al., "Predicting Trajectories of Functional Decline in 60- to 70-Year-Old
People," Gerontology, vol. 64, no. 3, pp. 212–221, Mar. 2018, doi: 10.1159/000485135.
[8] N. H. Jonkman et al., "Development of a clinical prediction model for the onset of functional
decline in people aged 65-75 years: Pooled analysis of four European cohort studies," BMC
Geriatr., vol. 19, no. 1, Jun. 2019, doi: 10.1186/s12877-019-1192-1.
[9] L. Clemson et al., "LiFE Pilot Study: A randomised trial of balance and strength training
embedded in daily life activity to reduce falls in older adults.," Aust. Occup. Ther. J., vol. 57,
no. 1, pp. 42–50, Feb. 2010, doi: 10.1111/j.1440-1630.2009.00848.x.
[10] L. Clemson et al., "Integration of balance and strength training into daily life activity to reduce
rate of falls in older people (the LiFE study): randomised parallel trial.," BMJ, vol. 345, no.
aug07_1, p. e4547, Jan. 2012, doi: 10.1136/bmj.e4547.
[11] M. Schwenk et al., "The adapted lifestyle-integrated functional exercise program for preventing
functional decline in young seniors: Development and initial evaluation," Gerontology, vol. 65,
no. 4, pp. 362–374, Jul. 2019, doi: 10.1159/000499962.
[12] J. L. Helbostad et al., "Mobile health applications to promote active and healthy ageing," Sensors
(Switzerland), vol. 17, no. 3. MDPI AG, 18-Mar-2017, doi: 10.3390/s17030622.
[13] H. A. Soufiani, D. C. Parkes, and L. Xia, "Random utility theory for social choice," Adv. Neural
Inf. Process. Syst., vol. 1, no. 1, pp. 126–134, 2012.
[14] K. J. Rothman, S. Greenland, and T. L. Lash, Modern epidemiology, Third edit. Lippincott
Williams and Wilkins, 2008.
[15] J. M. Guralnik, L. Ferrucci, E. M. Simonsick, M. E. Salive, and R. B. Wallace, "Lower-extremity
function in persons over the age of 70 years as a predictor of subsequent disability.," N. Engl. J.
Med., vol. 332, no. 9, pp. 556–61, Mar. 1995, doi: 10.1056/NEJM199503023320902.
[16] F. Lauretani et al., "Age-associated changes in skeletal muscles and their effect on mobility: An
operational diagnosis of sarcopenia," J. Appl. Physiol., vol. 95, no. 5, pp. 1851–1860, 2003, doi:
10.1152/japplphysiol.00246.2003.
[17] M. Cesari et al., "Prognostic value of usual gait speed in well-functioning older people - Results
from the health, aging and body composition study," J. Am. Geriatr. Soc., vol. 53, no. 10, pp.
1675–1680, Oct. 2005, doi: 10.1111/j.1532-5415.2005.53501.x.
[18] World Health Organization, Global Recommendations on Physical Activity for Health. Geneva,
Switzerland, 2010.
[19] C. Tudor-Locke and D. R. Bassett, "How Many Steps/Day Are Enough? Preliminary Pedometer
Indices for Public Health," Sports Medicine, vol. 34, no. 1. Sports Med, pp. 1–8, 2004, doi:
10.2165/00007256-200434010-00001.
[20] M. D. Denkinger et al., "Accelerometer-based physical activity in a large observational cohort-
-study protocol and design of the activity and function of the elderly in Ulm (ActiFE Ulm)
study.," BMC Geriatr., vol. 10, no. 1, p. 50, Jan. 2010, doi: 10.1186/1471-2318-10-50.
[21] L. Ferrucci et al., "Subsystems contributing to the decline in ability to walk: bridging the gap
between epidemiology and geriatric practice in the InCHIANTI study.," J. Am. Geriatr. Soc.,
vol. 48, no. 12, pp. 1618–25, Dec. 2000.
[22] M. Huisman et al., "Cohort profile: the Longitudinal Aging Study Amsterdam.," Int. J.
Epidemiol., vol. 40, no. 4, pp. 868–76, Aug. 2011, doi: 10.1093/ije/dyq219.
[23] University of Bologna - AI Group, “PreventIT Profiling Tool,” 2017. [Online]. Available:
http://taxonomy.disi.unibo.it/TaskRecommenderDemo/. [Accessed: 14-Sep-2020].
[24] A. M. Jette et al., "Late life function and disability instrument: I. Development and evaluation
of the disability component.," J. Gerontol. A. Biol. Sci. Med. Sci., vol. 57, no. 4, pp. M209-16,
Apr. 2002.
[25] S. M. Haley et al., "Late Life Function and Disability Instrument: II. Development and
evaluation of the function component.," J. Gerontol. A. Biol. Sci. Med. Sci., vol. 57, no. 4, pp.
M217-22, Apr. 2002.
[26] M. L. Puterman, Markov Decision Processes. Discrete Stochastic Dynamic Programming.
Hoboken, New Jersey: John Wiley & Sons, Inc., 2005.
[27] R. S. Sutton and A. G. Barto, Reinforcement learning. An introduction, Second ed. Cambridge,
Massachusetts, USA: MIT Press, 2018.