A challenge in promoting physical activity with recommender systems lies in balancing repeat recommendations to create habits, with exploration to prevent boredom. This study aims to identify the key variables influencing users' decision-making between the two. Through the analysis of data from an eight-week Micro-Randomized Trial conducted via a mobile health app, using random forest variable importance measures and SHAP analyses, we identified factors afecting these decisions. Our findings reveal that participants were more likely to explore new activities during the first two weeks of the intervention, in the afternoons and evenings, on Sundays, and when activities involved specific locations or workouts. These findings provide valuable insights into the transition from exploration to repetition, contributing to more efective recommender systems for physical activity promotion.
Insuficient physical activity (PA) is associated with adverse health outcomes [
As these physically inactive people might not have established healthy habits yet, the RS should
repeat some activities to create habits, because repetition makes the behavior more automatic [
In addition to their recommended minimum of 150 minutes of moderate-to-vigorous PA (MVPA)
per week, the World Health Organization (WHO) also recommends a mix of aerobic and
musclestrengthening activities [
This study focuses on identifying the key variables that impact users’ decision-making process to choose either an exploration or repetition PA item, arguing that future RSs can integrate these variables in their own decision-making processes as well. The research question is:
RQ: Which variables afect users’ decision to choose an exploration or repetition PA recommendation, and in which conditions are they more likely to choose one over the other?
By analyzing data from an eight-week Micro-Randomized Trial (MRT) conducted with a mobile health
(mHealth) app, we aim to uncover the most important factors, determined with Random Forests’ (RFs)
variable importances [
CEUR
ceur-ws.org
learning technique used for classification or regression to predict an outcome variable based on several
input variables by aggregating the results from multiple decision trees. RFs are widely used for data
exploration and understanding using variable importance measures [
This paper merges two study waves: one that ran from October 2023 until January 2024 (started in autumn), and one from March until June 2024 (spring). Our previous research was solely conducted on the data from the study starting in autumn [9]. To increase participation and dataset, we replicated the study in spring and combined both study’s data. In a previous paper [10], we already conducted a first analysis on the subjective perceptions and preferences measured with star rating of the repetition vs. exploration PA recommendations. The current paper dives deeper in the decision-making of the users using a set of factors that might influence their decision, contributing to the design of more efective RS algorithms in the PA domain.
The rest of the paper is organized as follows. Section 2 covers previous work on RFs in the domain of PA. The methods are discussed in Sect. 3. Next, the results and their discussion are elaborated in Sect. 4, followed by the conclusion in Sect. 5.
Previous work with RFs in the domain of PA mainly focused on the classification and recognition of the activity using sensors, such as accelerometers [11, 12, 13]. In those RFs, the input data consists of the data collected from the sensors and the predicted outcome variable is the detected type of PA. In our study, the input data is a combination of manual information (e.g., the self-reported motivation of the user at that time) and automatic information about the context (e.g., the weather).
Other studies used PA as an input variable in the RF to predict a certain outcome variable. In [14], body mass index and depressive symptoms are predicted with sedentary time and MVPA. Another study investigated severity of menopausal symptoms with an RF regression and found that PA level was in the top four of variable importances [15]. In [16], COVID-19 death rates were estimated with 29 socioeconomic and health-related factors with an RF.
PA amount was also used as outcome variable in several studies. Meeting the guidelines for suficient aerobic PA in a target group of adults with autism was assessed in [17]. Another study found that greenery in the streets impacts the duration of light intensity PA the highest for older adults [18]. In [19], an RF algorithm performed best to predict the probability of achieving a daily steps threshold. In our study, the predicted outcome class is whether the user decides to engage in an exploration or repetition item, thus resulting in a binary classification.
Previous work has also investigated the exploration and repetition decision in next basket recommendations [20] and sequential recommendations [21]. While these are mainly focused on e-commerce and grocery shopping, our study is situated in the health domain and PA promotion with mHealth interventions.
An Android app was created to display the personalized PA recommendations as shown in Fig. 1. Our PA dataset is based on activities from the compendium of physical activities [22], for which we distinguish between workout and location PA types. In our dataset, we also integrated general PA tips from the Belgian website for health (www.gezondleven.be), which contains small-efort activities people can integrate in all four PA situations of their daily life (free time, during work, household task, active transport). The resulting dataset contains 237 PA items, connected to the corresponding PA type and situation. For the content-based RS algorithm, all PA items were manually extended with 24 binary attributes to describe their content. For a full implementation description of content-based RS algorithm, we recommend reading our previous paper [23]. The exact same RS algorithm was used in this study, with an adjustment of the dataset and output list that now shows six recommendations.
A list of six items was chosen because six combinations can be made in our MRT study design: 2 from the outcome variable (repetition vs. exploration) x 3 from the PA type (general vs. location vs. workout). A repetition item refers to a personalized recommendation for an activity that the user has previously submitted during the study, while an exploration item is a personalized recommendation for an activity that the user has not yet submitted. By randomly positioning each of the six combination in the list for every participant at every delivery time, the micro-randomization of this study is achieved [24] and position bias efects are prevented. As a result, the user can freely choose between a repetition or exploration item of all three PA types at any time.
We recruited healthy adults (between 18 and 65 years old), who have less than 150 min of MVPA per week, via the Sona Platform and Facebook groups for paid studies of Ghent University. For both study waves (starting in autumn and spring), participants were asked to use the app for at least eight consecutive weeks, after which they receive 30 EUR. To prevent false submissions about their PA behavior, participants were informed that they do not receive more money for submitting more PAs, but only qualify for the incentive if they actively use the app for eight weeks and complete all questionnaires. At the start and after eight weeks, they answer the pre-test and post-test questionnaire, respectively, both containing the European Health Interview Survey - Physical Activity Questionnaire (EHIS-PAQ) to measure weekly PA [25] and a question about their age group (18 – 44 or 45 – 65 years). The study received ethical approval from the Ethical Committee (www.ugent.be/pp/en/research/ec) on August 22, 2023 (reference number: 2023-061A).
An RF classifier is tuned with Randomized Search and trained using scikit-learn (scikit-learn.org/)
version 1.5.1. Table 1 provides an overview of all 15 input variables of the RF model. The predicted
output of the RF is whether the user engaged in a repetition or exploration item (binary classification).
SHAP provides an explanation for this output by assigning feature importance values to each variable for
a specific prediction [
Of the 62 participants who started the study (100% in the age group of 18 - 44 years), 34 continued for at least eight weeks. Throughout the study, the participants submitted 457 recommended items, of which the amounts are categorized in Table 2.
Firstly, the hyperparameters of the model were tuned with Randomized Search with 5-fold cross validation and a max depth of 6 to prevent overfitting. The resulting optimal hyperparameters are: n estimators = 493, min samples split = 10, min samples leaf = 10, max features = sqrt, max depth = 6, bootstrap = True, resulting in a train accuracy of .764 and test accuracy of .663. The corresponding feature importances are shown in Fig. 2, in which the time in study showed the highest relevance.
Figure 3 shows the SHAP dependence plots that visualize the relation between the six input variables with the highest feature importances and their corresponding SHAP value. We added a color legend to show possible interaction efects with days in study, as this variable ranked highest on the variable importances.
The first dependence plot shows that days in the beginning of the study are associated with a higher prediction for an exploration item (above the y=0 line). The threshold at which the model switches input variable for the RF
T: day in study
T: hour of day
W: tem perature S: free time/work/household/transport
T: day of week P: general/location/workout M: m otivation before PA
T: season C: alone/with buddy
W: clouds T: weekend/week day
L: indoor/outdoor
SC: steps W: clear W: rain 0.00 category 0.05 0.10 0.30
0.35 0.15 0.20 0.25 relative feature importance from predicting exploration to repetition lies around day 15 in the study.
The switch from exploration to repetition after two weeks was also found in our previous study,
which showed a significant interaction efect on the star rating feedback after this two-week mark for
general PAs [10]. In this previous study [10], we referred to the lower efort and complexity of general
PAs as the reason for the quicker shift to repetition, based on [
This pattern could also be attributed to people’s circadian rhythm and the distinction between morning chronotypes who rise and peak early in the day, and evening chronotypes who experience arousal in the afternoon or evening [ 27]. However, as we did not collect the chronotype of our participants, nor an exact age or occupation, we cannot derive a clear explanation from this.
The third plot suggests a larger prediction for exploration when the outdoor temperature is around 10 degrees Celsius, with two threshold values around 5 and 15 degrees. According to [28], the first and last warm days of the year may motivate people for more PA, which could explain why they are more open to exploration in this temperature range, unlike higher temperatures above 20 degrees.
High variable importance was also found for PA type and situation, of which the dependence plots are shown at the bottom of Fig. 3. The higher association with repetition in free-time, household, and general contexts can be due to established habits in these contexts. For example, exploration could be less preferred in household tasks because people already have their own routine for these.
Exploration was mostly associated with activities for transportation, during work, at a location, and as a workout. The results of our previous analysis [10] showed that exploration for general PAs was only preferred in the first two weeks of the study, after which higher star ratings were given to repetitions. This can explain why general PAs are mostly associated with repetition prediction in the SHAP dependence plot. Similarly, the star rating was consistently higher for exploration of location and workout PAs in the LMM [10], which corresponds to the conclusions of this SHAP analysis.
As 94% of the active transport and 75% of during work submits were an exploration, as displayed in Table 2, this explains the higher association with exploration. Nonetheless, submits in the situations of active transport and during work are limited, thwarting reliable analyses for these situations. 4.1.5. Day of week The fith plot shows that exploration predictions are more associated with Sundays, likely because more people have time of then, allowing more time for exploration. As Saturdays are not associated with more exploration, this can explain the lower variable importance of the “weekend/week day” variable.
SHAP value for exploration submit prediction 0.20 0.f1re5etime 20 40 60 80 100 120 Tim e: day in study 0 6 12 18 Tim e: hour of day 4.1.6. Season The season in which the participant started the study (autumn vs. spring) ranked seventh in the variable importances. We did not create a dependence plot for this variable, but conducted an LMM analysis with this variable to investigate the efect on the amount of weekly MVPA.
To investigate the increase in weekly PA, we compare the total MVPA measured by the EHIS-PAQ (Q4*Q5 + Q7) in the pre-test and post-test questionnaires [25]. An LMM was fitted with the time (pre-test vs. post-test), the season in which the participant started (autumn vs. spring), and their interaction as fixed efects [ 26, 29]. To account for possible variations between users, the user ID was considered as a random efect in a random coeficient model with a random slope [ 26].
We found a significant interaction efect, of which the mean fixed predicted values are shown in Fig. 4, illustrating that the group of participants who started the study in spring had a higher increase in weekly MVPA, on average (F (1,46.094)=4.079, p=.049). While a general increase in MVPA in both study groups was expected [30, 31], the stronger increase in the spring group can be explained because PA is higher in warmer months [28].
mean xed predicted values of MVPA (minutes per week) 250 200 150 100 50 0 started study in:
spring pre-test (week 0) autumn post-test (week 8)
As only 34 participants finished the full eight-week study, a large amount of data of the later weeks are missing. Additionally, the study was not conducted year-round or across diferent climate zones, limiting reliable conclusions about the impact of outdoor temperature. Given the limited submissions for transportation and work PAs, we suggest future research to include longer studies with a larger participant pool.
Our results show that the variables company, location, and motivation had lower importance. However, this company variable could be extended with the presence of a human trainer, as their supervision and guided workout plans, often containing repetition of activities, increase engagement in trainings delivered via mobile apps [32].
Nonetheless, our results indicate that time-related factors primarily drive the decision to explore new activities. We suggest integrating these time-related variables in future RSs for PA promotion. Although we identified specific conditions for exploration, we propose to tailor these conditions to the user. For example, an RS could learn at what moments a user prefers to explore a new activity. Additional information could be integrated in the system, such as the user’s chronotype to take into account moments at which the user is most active [27].
Two eight-week user studies with a total of 62 physically inactive participants (<150 minutes MVPA/week) were conducted to investigate factors influencing people’s decision to either repeat or explore a PA recommendation. In the MRT, repetition and exploration PAs were provided to the user in an mHealth app at random positions, allowing participants to freely choose between the options at each delivery time.
The RF and SHAP approach identified key factors and conditions influencing the likelihood of exploring a new activity: in the first two weeks of the mHealth intervention, in afternoons and evenings, on Sundays, and for activities at a location or as a workout. Other factors scored lower on the variable importances of the decision between an exploration or repetition item, such as the season, company, motivation, and location. However, we did find a significant higher increase in weekly MVPA for the group that started the study in spring, suggesting that the season might not largely influence the exploration/repetition decision, but does afect the amount of PA.
By defining moments and contexts at which people are more open to explore new activities, this study contributes to future PA recommenders to balance between repeating favorite activities and introducing new ones.
This research received funding from the Flemish Government under the “Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen” programme. [9] I. Coppens, T. De Pessemier, L. Martens, Repeating my workouts or exploring new activities? a longitudinal micro-randomized user study for physical activity recommender systems, in: Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization, UMAP Adjunct ’24, Association for Computing Machinery, New York, NY, USA, 2024, p. 176–182.
URL: https://doi.org/10.1145/3631700.3664867. doi:10.1145/3631700.3664867. [10] I. Coppens, T. De Pessemier, L. Martens, Balancing habit repetition and new activity exploration: A longitudinal micro-randomized trial in physical activity recommendations, in: Proceedings of the 18th ACM Conference on Recommender Systems, RecSys ’24, Association for Computing Machinery, New York, NY, USA, 2024, p. 1147–1151. URL: https://doi.org/10.1145/3640457.3691715. doi:10.1145/3640457.3691715. [11] K. Lee, M.-P. Kwan, Physical activity classification in free-living conditions using smartphone accelerometer data and exploration of predicted results, Computers, Environment and Urban Systems 67 (2018) 124–131. URL: https://linkinghub.elsevier.com/retrieve/pii/S0198971517301977. doi:10.1016/j.compenvurbsys.2017.09.012. [12] M. T. Mardini, C. Bai, A. A. Wanigatunga, S. Saldana, R. Casanova, T. M. Manini, Age Diferences in Estimating Physical Activity by Wrist Accelerometry Using Machine Learning, Sensors 21 (2021) 3352. URL: https://www.mdpi.com/1424-8220/21/10/3352. doi:10.3390/s21103352. [13] Z. Junjie, C. Shenghao, X. Jie, Y. Hua, Random forest-based physical activities recognition by usingwearable sensors, Industria Textila 73 (2022) 27–33. URL: http://revistaindustriatextila.ro/ images/2022/1/004%20ZHANG%20JUNJIE%20Industria%20Textila%201_2022.pdf. doi:10.35530/ IT.073.01.20215. [14] G. E. Duncan, A. R. Avery, S. Tsang, N. F. Watson, B. D. Williams, E. Turkheimer, The pillars of health: influence of multiple lifestyle behaviors on body mass index and depressive symptoms in adult twins, BMC Public Health 22 (2022) 1487. URL: https://bmcpublichealth.biomedcentral.com/ articles/10.1186/s12889-022-13901-7. doi:10.1186/s12889- 022- 13901- 7. [15] S. Wu, Y. Shi, Q. Zhao, K. Men, The relationship between physical activity and the severity of menopausal symptoms: a cross-sectional study, BMC Women’s Health 23 (2023) 212. URL: https://bmcwomenshealth.biomedcentral.com/articles/10.1186/s12905-023-02347-7. doi:10.1186/ s12905- 023- 02347- 7. [16] G. Grekousis, Z. Feng, I. Marakakis, Y. Lu, R. Wang, Ranking the importance of demographic, socioeconomic, and underlying health factors on US COVID-19 deaths: A geographical random forest approach, Health & Place 74 (2022) 102744. URL: https://linkinghub.elsevier.com/retrieve/ pii/S1353829222000053. doi:10.1016/j.healthplace.2022.102744. [17] S. Healy, B. Brewer, E. Hoopes, A. Paller, S. Mayberry, J. Maguire, J. Daly, P. Laxton, F. Patterson, Identifying the most proximal multi-level factors associated with meeting each of the 24-h movement behavior recommendations in a sample of autistic adults, Disability and Health Journal 15 (2022) 101367. URL: https://linkinghub.elsevier.com/retrieve/pii/S1936657422001170. doi:10.1016/j.dhjo.2022.101367. [18] P. Zang, H. Qiu, F. Xian, L. Yang, Y. Qiu, H. Guo, Nonlinear Efects of the Built Environment on Light Physical Activity among Older Adults: The Case of Lanzhou, China, International Journal of Environmental Research and Public Health 19 (2022) 8848. URL: https://www.mdpi.com/1660-4601/ 19/14/8848. doi:10.3390/ijerph19148848. [19] T. B. Dijkhuis, F. J. Blaauw, M. W. Van Ittersum, H. Velthuijsen, M. Aiello, Personalized Physical Activity Coaching: A Machine Learning Approach, Sensors 18 (2018) 623. URL: https://www.mdpi. com/1424-8220/18/2/623. doi:10.3390/s18020623. [20] M. Li, S. Jullien, M. Ariannezhad, M. de Rijke, A next basket recommendation reality check, ACM
Trans. Inf. Syst. 41 (2023). URL: https://doi.org/10.1145/3587153. doi:10.1145/3587153. [21] M. Li, A. Vardasbi, A. Yates, M. De Rijke, Repetition and Exploration in Sequential Recommendation, in: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, Taipei Taiwan, 2023, pp. 2532–2541. URL: https://dl.acm.org/doi/ 10.1145/3539618.3591914. doi:10.1145/3539618.3591914. [22] B. E. Ainsworth, W. L. Haskell, S. D. Herrmann, N. Meckes, D. R. Bassett, C. Tudor-Locke, J. L. Greer,