Early old age (60-70 years old) is a particular period of life when possible habit modifications may occur, often related to job retirement. While taking up a more sedentary lifestyle may be pernicious for health, changing behavior by introducing simple exercises within daily life routines can effectively prevent age-related functional decline. This article presents the Profiling Tool, a system that provides 60-70-year-old adults with personalized recommendations to integrate simple activities, promoting balance, strength, and physical activity into their daily life. Its first implementation has been designed on information from literature, data from previously available longitudinal datasets, and experts' opinions. It has been deployed within a randomized controlled trial. Strategies for its update are based on model-based reinforcement learning approaches. Ageing, functional decline, prevention, recommender system, behavioral change Population aging is one of the major issues of our present world. Developing preventive interventions is one of the keys to tackling this issue, and Artificial Intelligence (AI) can enable these interventions and make them more effective and efficient.
2020 Copyright for this paper by its authors.
In the following sections, we overview the PreventIT project and IT ecosystem and describe the Profiling Tool, including its modeling, its first implementation and deployment within a randomized control trial, and strategies for its update.
PreventIT stands for 'Early risk detection and prevention in aging people by self-administered ICTsupported assessment and a behavioral change intervention, delivered by use of smartphones and smartwatches.' It is a European Horizon 2020 project carried out from January 2016 to March 2019 [4]. The project aimed to develop a proof-of-concept, unobtrusive mobile health system based on a personalized behavior change intervention on balance, strength, and physical activity. The intervention is designed for young older adults (adults between 60 and 70 years old) to prevent accelerated functional decline at an older age.
The PreventIT ICT based Personalized Activity System (iPAS) is a mobile health system delivering the intervention on smartphones and smartwatches. It includes a smartphone and smartwatch app as frontend and a risk model for functional decline [7], [8], the eLiFE intervention program, a Profiling Tool for personalizing the intervention, and a behavior change theories-based motivational strategy running on a cloud-based backend (Figure 1).
The PreventIT intervention program is based on the Lifestyle-integrated Exercise (LiFE) approach [9]. In LiFE, rather than using a prescribed set of exercises, activities are performed whenever the opportunity arises during the day. The LiFE approach allows personalizing and integrating exercise in daily life, and it was found to significantly reduce falls, improve physical function, decrease disability and improve adherence, compared with a traditional exercise program and a sham intervention [10]. In PreventIT, the original LiFE was adapted (aLiFE, adapted LiFE, [11]) to the needs of 60-70-year-old adults to make activities challenging and complex enough for a younger target population. The integration of the aLiFE program into the PreventIT iPAS is named eLiFE (enhanced LiFE, [12]).
Since the LiFE program relies on users embedding balance, strength, and physical activities into their everyday life, it can only be successful if they change their behavior. The original LiFE concept is underpinned by the behavioral change concepts of habit formation, self-efficacy, skills training, and outcomes gained. The motivational strategy in PreventIT is based on the extension of the behavioral change framework supporting the intervention [5].
The Profiling Tool is a tool for personalized recommendations on activities to be integrated into seniors' daily life routines.
The Profiling Tool takes as input an individual's health state, a list of potential activities and difficulty levels, an estimate of their expected impact on the individual's health state, and contextual information, including the individual's preferences for the activities. On this knowledge basis, the Profiling Tool provides recommendations to the individual on which activities best fit their needs and the appropriate difficulty level for each activity.
There are 21 types of activities in the eLiFE program with up to four difficulty levels for each activity, grouped into three domains: 1. Strength domain: squatting, lunging, walking on toes, walking on heels, stair climbing, sit-tostand, move legs sideways, tighten muscles; 2. Balance domain: tandem stand, one-leg stand, tandem walk, side-to-side leaning, forwardbackward leaning, stepping over objects, stepping and changing direction, square stepping and hopping, square jumping; 3. Physical activity domain: walk longer, walk faster, sit less, break-up sitting.
These same three domains describe the individual's health state.
An expected benefit is calculated for every single eLiFE activity on the specific user profile. Recommendations are provided to the individual, accompanied by motivational messages, designed according to theoretical constructs of behavioral change (e.g., the Health Action Process Approach) [5].
Each day the subject selects a list of activities he/she will perform during the day and confirms the actually-performed activities at the end of the day. After every six months, the subject is assessed for his/her health state (Figure 2A). All this information about the interactions between the Profiling Tool and the individual and their effects is recorded by the iPAS and used by the Profiling Tool for its update.
In the following, we give a modeling description of the Profiling Tool and its interactions with the user, describe the implementation of its first version within the PreventIT project, and present an updating strategy.
To appropriately design the Profiling Tool, including its recommendation policy and updating strategy, we characterize the interactions between the Profiling Tool and the user in terms of two models, describing the preferences of the individual for the activities and the benefit of these activities on the health state respectively. 4.1.
We define the preference model as the model that describes the activities that an individual with specific characteristics would perform when given personalized recommendations.
We use the subscript to indicate the -th six-month time period and the subscript ∙ to indicate the -th day of the -th six-month period. individual has performed each activity during six months
We call the vector of subject's features – including their health state, = ( ) the personalized recommendations issued by the Profiling Tool, and ∙ = ( ∙ ,1, ∙ ,2, … , ∙ , )′ the vector expressing the
= 21 activities performed by the individual. In particular, ∙ , is the number of times the subject has performed activity during day ∙ . We call the vector expressing the number of times the = ∑ ∙ 179 ( | , ) , ≥ 0 ∑ , = 1 =1 = 0 + 1′ + ,
( ∙ | , ∙ ) A simple parametric form for the preference model (2) is (1) (2) (3) (4) (5) (6) (7) (8) , is an error term.
where 0 encodes personal preferences, 1′ encodes the influence of the recommendations, and For recommendations that vary every day, the preference model can be expressed by months.
and the linear model (5) could be replaced by a logistic or Poisson model over ∙ , . We note that the features of the individual do not change every day, as the health state is assessed once every six
Other forms for the preference model can be borrowed by the rich literature on choice modeling, and random utility theory [13], and models can easily be tested on data, as quantities , , and are all observed and recorded in the iPAS system. 4.2.
expressed as
Thus, the preference model that relates the cumulative selections with the suggestions can be Within the first version of the Profiling Tool, recommendations were given in the form of an ordered list of potential activities, sorted according to their expected benefit on the individual's health state. Other choices are also possible to express more quantitatively the strength of recommendation for each activity. For example, = ( ,1, ,2, … , , ) could be a vector of such degrees of recommendation for each activity, under constrains
We define the health effect model as the model that describes the future health state +1, based on the current health state and the activities performed by the individual
( +1| , ) Within the feature vector , one variable
can be chosen as the primary outcome. A simple parametric form of the health effect model restricted to this outcome is +1 = + + ′ + ′ + ′ + , where , , and are vector parameters, is a matrix parameter, and , is an error term.
According to this model for the outcome, replacing with a vector having 1 in the -th component and zero otherwise, we get the expected health benefit on the outcome of one unit of activity as + ∑
where is the -th component of vector and is the entry in position ( , ) of matrix . (9) (10) (11) cycle. Panel B: direct acyclic graph (DAG) [14] for the Bayesian network of health states and +1, recommendation , and performed activities . It encodes the conditional independence between +1 and , given and .
As it is reasonable, we assume that the future health state +1 is independent of the recommendation , conditional on the current health state and the performed activities (Figure 2B) Under this assumption, the transition probability ( +1| , ) can be expressed as +1 ⊥ | , ( +1| , ) = ∫ ( +1| , ) ( | , ) where we recognize the product of the preference and health effect models within the integral.
The first version of the Profiling Tool was developed on knowledge from the literature, data from population studies on aging, and opinions from experts. It was tested in a feasibility randomized controlled trial (RCT) within the PreventIT study. 5.1.
This version for activity recommendation was based on four rules.
First, the feature vector at baseline 0 was the three-score individual profile
0 = ( 1, 2, 3) (12) each score ranging from 0 to 5 and expressing the prioritization of exercise on balance, strength, and physical activity domains. Each was derived comparing measures of physical performance against cut-offs derived from the literature [15]–[19] and data of 60-70-year-old individuals pooled from three longitudinal studies on aging (ActiFE Ulm [20], InCHIANTI [21], LASA [22]). More in particular, for each domain, we considered two-to-three variables and created categories on these variables using cut-off values found in the literature. After applying these categories on the pooled cohort, if a prevalence of at least 10% was found in each category, the cut-off was retained valid. Otherwise, the cut-off was derived from the tertiles of the variable on the pooled cohort. Table 1 reports cut-offs, scores, and summary statistics on participants of the PreventIT study.
Second, suggested activities were taken from a list of 21 activities, grouped according to three domains. Each activity was made of up to five difficulty levels, for a total of 89 exercises. The expected health impact of each activity was estimated from equation (9). In particular, the offsets were set to zero, and matrix for the impact of each activity on each domain was filled by expert judgments with scores from 0 to 5.
Third, activities marked as not pleasant by the individual were dropped off the list of recommendations for the following days.
Fourth, for each suggested activity, its starting difficulty level was determined based on the individual's abilities assessed at the beginning of using the Profiling Tool by a trainer. The individual could decide at any time to downgrade the difficulty level of an activity, but they needed to train long enough to upgrade it.
Resulting recommendations 0 = ( 0) were given in the form of a list of activities, sorted in descending order according to their expected health benefit.
The activities ∙ performed each day were registered by the iPAS system, integrating feedback provided by the individual at the end of the day and recordings from global positioning system (GPS) and inertial measurement units (IMU) sensors embedded in the mobile phone.
A demo of this first version of the Profiling Tool is available on the Internet (http://taxonomy.disi.unibo.it/TaskRecommenderDemo/) [23]. 5.2.
The Profiling Tool was tested within the three-arm PreventIT feasibility RCT (n=180) on three clinical centers in Trondheim, Stuttgart, and Amsterdam. One arm was assigned to the iPAS system and the Profiling Tool (eLiFE), one was given a booklet with recommendations by a trainer on activities to integrate into daily life (aLiFE). At the same time, participants of the control group were provided general physical activity recommendations. The primary outcome was taken to be the Late-Life Function and Disability Instrument (LLFDI) [24], [25]. A detailed description of the trial protocol is available at [4].
The scoring system for the individual profile showed to be appropriate in stratifying the target population on domains of balance and strength, whereas, in the physical activity domain, too few participants (< 10%) fell on the lowest categories defined on gait speed and step count (Table 1).
On the participants of the eLiFE intervention arm that used the Profiling Tool (n=50), we evaluated with the iPAS system whether the ranking that was suggested by the Profiling Tool ( 0) was actually selected by the participants. In Table 2, it can be seen that there is not a clear association between the ranking of activities by the Profiling Tool and the actual choice of participants from the 21 activities. Activities ranked higher by the Profiling Tool, such as 'Square stepping and hopping' and 'Square jumping,' were not more frequently selected by participants to incorporate in their intervention regime. The only activities that showed a significant association (p<0.05) were 'Stepping over objects,' 'Stepping and changing direction,' and 'Lunging,' but there is not a clear pattern in the data to explain these associations.
The first evidence also shows that changes in health outcomes were modest over the RCT participants, making health effect models challenging to fit (data not shown). Frequency of activities ranked in the top 7 with Profiling Tool version 1 and that were actually selected by eLiFE participants (n=50).
Activities
Most frequent In top 7 based Actually selected Chi-square test ranking profiling on profiling tool by participants ranking vs.
tool selected p-value Domain 1: Balance
Tandem stand One leg stand
Tandem walk
Side-to-Side leaning Forwards and backwards leaning
Stepping over objects Stepping and changing direction
Square stepping and hopping
Square jumping Domain 2: Strength
Squatting
Lunging Walking on toes Walking on heels
Stair climbing
Sit to stand Move leg sideways
Tighten muscles Domain 3: Physical activity
Walk longer Walk faster
Sit less
Break up sitting
Values are n (%). 6. Updating strategy is an ordered set of time points; is the set of features characterizing the individuals; is the set of recommendations that the Profiling Tool can issue; ( +1| , ) is the transition probability between state ∈ at time ∈ to state +1 ∈ at time + 1 ∈ , when the Profiling Tool has issued the recommendation ∈ ; • ( , ) is the reward of being in the state and issuing recommendation .
We assume that issuing different recommendations has the same cost and thus the reward ( , ) is a function of the sole health state +1. In particular, we pose a reward equal to the primary outcome: ( , ) = +1. (14)
Considering to use the data collected during the PreventIT feasibility trial to develop a second version of the Profiling Tool (Figure 3A), the Markov decision problem is defined over only one period ( = {0,1}) and is stated as follow model in equations (3-5), the problem (15) becomes:
Using the linear outcome model for [ 2| 1, 1] as in equation (8), and the preference selection maximize subject to 1′ 1( + 1) 1, ≥ 0 ∀ ∑ 1, = 1 =1 (15) (16) (17) (18) the first experimentation of the PT in PreventIT; quantities in grey are those relative to a second version of the PT. Panel B. Iterative updating strategy of the PT's preference and outcome models, in the case of iterative deployment. The inner green rectangle represents a six-month cycle with a time unit equal to one day, while the outer blue rectangle represents a cycle over repetitions of six-month cycles (i=0:179).
Given 1 and having estimated the parameters 1, , and from the data, the problem (16-18) is a simple linear program in the canonical form. Calling the vector 1( + 1), and provided that has at least one positive component, the problem is solved by the sparse vector ∗ = ( ∗ ), so that ∗ = 1 for =
max , and ∗ = 0 for all others ≠ . We note that replacing constraint (3) with one over the L2 norm of , makes the solution non-sparse.
Model parameters (e.g. 0, 1, , , …) could be derived for a) the whole population or sets of users, b) in a subject-specific manner, or c) combining both approaches with mixed-effect models. We judged that data from the PreventIT feasibility trial are insufficient to estimate all model parameters with appropriate precision and robustness. Hence, model fitting can proceed according to Bayesian estimation using parameter values of the first version to construct prior parameter distributions. Otherwise, data-driven recommendations can be combined heuristically with recommendations coming from the first version. six-month periodicity. consecutive deployments over a time horizon . The Profiling Tool is foreseen to evolve as more data accrue and update the preference and health effect models. The preference model can be updated every day since performed activities ∙ are recorded daily, while the health effect model is updated with a
Focusing on the slower update periodicity and following the conceptual framework usually employed with MDPs, we define a cumulative reward
( 0) = [∑ ( , ) | 0] = [∑ | 0] Considering the recommendation function policy that possibly changes with time as the Profiling Tool is updated, we aim to find a recommendation that maximizes ( 0).
Upon knowledge of the preference and health effect models, the transition probability is known and the Markov decision problem to find the optimal policy ∗ can be solved with linear programming techniques (e.g., backward induction, value iteration, or policy iteration algorithms). However, in the more general case, both the transition probability and the recommendation policy have to be learned on data, as long as they accrue. Reinforcement learning heuristics serve this case [27], balancing the tradeoff between exploiting the likely
most effective recommendations and exploring others' effectiveness.
We have presented the Profiling Tool's design and first deployment, a recommender system for behavioral change of 60-70-year-old adults.
Its design was inspired by and based on psychological theories and techniques of behavioral change [5] and AI solutions for recommender systems [6]. Its first version was designed on information from the literature, data from cohorts of epidemiological studies on aging, and experts' opinions. The mathematical models that describe its interactions with the user serve to analyze its functioning and plan updating strategies as more data get available. To the best of our knowledge, their employment is new in the applicative field of mobile applications for prevention.
Analyses from its first deployment within the PreventIT feasibility RCT have provided insights. First of all, the individual profile scoring was shown to be satisfactory, distinguishing distribution of scores on the domains of balance and strength, but not on the physical activity domain, in our cohort of people aged 60-70 years old.
Secondly, recommendations are only loosely associated with actually-selected activities. In the PreventIT feasibility RCT, the intervention regime was put together by the participants themselves, in consultation with the trainer. This might have affected the decisions of participants and could have overruled the ranking by the Profiling Tool. Another possible cause behind this lack of correspondence between recommendations and user selection of activities may lie in the form the recommendations were provided. More specifically, recommendations were ordered list of activities without any indication of the strength of recommendation associated with each activity. For future developments, we could test whether recommendations become more convincing by expressing the strength of recommendation more quantitatively or by presenting a limited number (e.g., only the top 7 rankings) of activities. It is further suggested to explore different strategies for planning the intervention regime and sending motivational messages accompanying the recommendations.
Preliminary analyses have also shown that health changes could be small over six months for a highly functional target population, making health effect models challenging to estimate. This issue could be solved by deploying the tool on a population which is broader and more heterogeneous.
This study has been partly funded by the European Commission under the project 'PreventIT' (2016– 2018, grant number 689238) responding to the Horizon 2020, Personalised Health and Care call PHC21: Advancing active and healthy aging with ICT: Early risk detection and intervention. 9. References
S. Mehra et al., "Translating behavior change principles into a blended exercise intervention for older adults: Design study," J. Med. Internet Res., vol. 20, no. 5, May 2018, doi: 10.2196/resprot.9244.
A. C. King et al., "Effects of Three Motivationally Targeted Mobile Device Applications on Initial Physical Activity and Sedentary Behavior Change in Midlife and Older Adults: A Randomized Trial," PLoS One, vol. 11, no. 6, p. e0156370, Jun. 2016, doi: 10.1371/journal.pone.0156370.
L. Paul et al., "Increasing physical activity in older adults using STARFISH, an interactive smartphone application (app); a pilot study," J. Rehabil. Assist. Technol. Eng., vol. 4, p. 205566831769623, Jan. 2017, doi: 10.1177/2055668317696236.
K. Taraldsen et al., "Protocol for the PreventIT feasibility randomised controlled trial of a lifestyle-integrated exercise intervention in young older adults.," BMJ Open, vol. 9, no. 3, p. e023526, Mar. 2019, doi: 10.1136/bmjopen-2018-023526.
E. Boulton et al., "Implementing behaviour change theory and techniques to increase physical activity and prevent functional decline among adults aged 61–70: The PreventIT project," Progress in Cardiovascular Diseases, vol. 62, no. 2. W.B. Saunders, pp. 147–156, 01-Mar2019, doi: 10.1016/j.pcad.2019.01.003.
F. Ricci, L. Rokach, and B. Shapira, Eds., Recommender systems handbook, Second Edi. New York: Springer, 2015.
N. H. Jonkman et al., "Predicting Trajectories of Functional Decline in 60- to 70-Year-Old People," Gerontology, vol. 64, no. 3, pp. 212–221, Mar. 2018, doi: 10.1159/000485135. N. H. Jonkman et al., "Development of a clinical prediction model for the onset of functional decline in people aged 65-75 years: Pooled analysis of four European cohort studies," BMC Geriatr., vol. 19, no. 1, Jun. 2019, doi: 10.1186/s12877-019-1192-1.
L. Clemson et al., "LiFE Pilot Study: A randomised trial of balance and strength training embedded in daily life activity to reduce falls in older adults.," Aust. Occup. Ther. J., vol. 57, no. 1, pp. 42–50, Feb. 2010, doi: 10.1111/j.1440-1630.2009.00848.x.
L. Clemson et al., "Integration of balance and strength training into daily life activity to reduce rate of falls in older people (the LiFE study): randomised parallel trial.," BMJ, vol. 345, no. aug07_1, p. e4547, Jan. 2012, doi: 10.1136/bmj.e4547.
M. Schwenk et al., "The adapted lifestyle-integrated functional exercise program for preventing functional decline in young seniors: Development and initial evaluation," Gerontology, vol. 65, no. 4, pp. 362–374, Jul. 2019, doi: 10.1159/000499962.
J. L. Helbostad et al., "Mobile health applications to promote active and healthy ageing," Sensors (Switzerland), vol. 17, no. 3. MDPI AG, 18-Mar-2017, doi: 10.3390/s17030622.
H. A. Soufiani, D. C. Parkes, and L. Xia, "Random utility theory for social choice," Adv. Neural Inf. Process. Syst., vol. 1, no. 1, pp. 126–134, 2012.
K. J. Rothman, S. Greenland, and T. L. Lash, Modern epidemiology, Third edit. Lippincott Williams and Wilkins, 2008.
J. M. Guralnik, L. Ferrucci, E. M. Simonsick, M. E. Salive, and R. B. Wallace, "Lower-extremity function in persons over the age of 70 years as a predictor of subsequent disability.," N. Engl. J. Med., vol. 332, no. 9, pp. 556–61, Mar. 1995, doi: 10.1056/NEJM199503023320902. F. Lauretani et al., "Age-associated changes in skeletal muscles and their effect on mobility: An operational diagnosis of sarcopenia," J. Appl. Physiol., vol. 95, no. 5, pp. 1851–1860, 2003, doi: 10.1152/japplphysiol.00246.2003.
M. Cesari et al., "Prognostic value of usual gait speed in well-functioning older people - Results from the health, aging and body composition study," J. Am. Geriatr. Soc., vol. 53, no. 10, pp. 1675–1680, Oct. 2005, doi: 10.1111/j.1532-5415.2005.53501.x.
World Health Organization, Global Recommendations on Physical Activity for Health. Geneva, Switzerland, 2010.
C. Tudor-Locke and D. R. Bassett, "How Many Steps/Day Are Enough? Preliminary Pedometer Indices for Public Health," Sports Medicine, vol. 34, no. 1. Sports Med, pp. 1–8, 2004, doi: 10.2165/00007256-200434010-00001.
M. D. Denkinger et al., "Accelerometer-based physical activity in a large observational cohort-study protocol and design of the activity and function of the elderly in Ulm (ActiFE Ulm) study.," BMC Geriatr., vol. 10, no. 1, p. 50, Jan. 2010, doi: 10.1186/1471-2318-10-50. L. Ferrucci et al., "Subsystems contributing to the decline in ability to walk: bridging the gap between epidemiology and geriatric practice in the InCHIANTI study.," J. Am. Geriatr. Soc., vol. 48, no. 12, pp. 1618–25, Dec. 2000.
M. Huisman et al., "Cohort profile: the Longitudinal Aging Study Amsterdam.," Int. J. Epidemiol., vol. 40, no. 4, pp. 868–76, Aug. 2011, doi: 10.1093/ije/dyq219.
University of Bologna - AI Group, “PreventIT Profiling Tool,” 2017. [Online]. Available: http://taxonomy.disi.unibo.it/TaskRecommenderDemo/. [Accessed: 14-Sep-2020]. A. M. Jette et al., "Late life function and disability instrument: I. Development and evaluation of the disability component.," J. Gerontol. A. Biol. Sci. Med. Sci., vol. 57, no. 4, pp. M209-16, Apr. 2002.
S. M. Haley et al., "Late Life Function and Disability Instrument: II. Development and evaluation of the function component.," J. Gerontol. A. Biol. Sci. Med. Sci., vol. 57, no. 4, pp. M217-22, Apr. 2002.
M. L. Puterman, Markov Decision Processes. Discrete Stochastic Dynamic Programming. Hoboken, New Jersey: John Wiley & Sons, Inc., 2005.
R. S. Sutton and A. G. Barto, Reinforcement learning. An introduction, Second ed. Cambridge, Massachusetts, USA: MIT Press, 2018.