Persuasive health technologies in the sport domain focus mostly on motivating and supporting people in reaching an active lifestyle. In this paper, we exploit a real-world dataset made available by a commercial persuasive ecosystem called u4fit . u4fit allows coaches to create tailored workout plans and to constantly monitor and support their sportsmen remotely. Occasional sportsmen often and suddenly abandon their workout routines, without giving any prior notice to the coach, frequently because of a decline in motivation. In this paper, we tackle this issue by developing an approach able to spot users' behavioral changes and predict if one will soon stop exercising. These predictions can be further elaborated and provided as a recommendation to the user's coach, to let her get in touch with the sportsmen and prevent such a situation. Experiments, validated through standard accuracy metrics, revealed that behavioral changes in training patterns represent one of the main markers that lead sportsmen to abandon.
• Information systems → Mobile information processing systems; Data mining;
HealthRecSys’17, August 2017, Como, Italy © 2017 Copyright for the individual papers remains with the authors. Copying permitted for private and academic purposes. This volume is published and copyrighted by its editors. 1
Persuasive technologies in the eHealth domain are an emerging and
promising research field. eHealth persuasive technologies (eHPT)
are designed to help people change their habits to overcome their
frictions to healthier behaviors [
The rise of mobile technologies and the spread of Internet access
across the globe favored the development of eHPT. In 2010, we
started investigating on the efects of the gamification on users’
motivation to exercise [
One important issue that coaches face while following people is that sometimes they stop exercising, without giving any prior notice to them. This aspect is critical, given that a sedentary behavior can negatively afect several social, physical, and mental aspects of the individual, leading, especially on elder people, to chronic diseases.
There are obviously unpredictable cases of users who stop training (e.g., injuries), but there are also behavioral changes in the way users workout that might help predicting such a situation. The automatic detection of motivation decline is still an open issue that, if addressed, will allow coaches (or professionals in general) to act proactively in order to prevent this scenario.
Recommender systems (RS) can help supporting decisions in
health environments. As highlighted in [
In this paper, we aim at predicting when a sportsman will stop exercising by analyzing and inferring behavioral patterns from past performances recorded through u4fit. To this end, we train a classifier using as class the fact that the current performance led or not to a subsequent workout within x days (where x is a parameter that sets the granularity with which a coach monitors the sportsmen she follows). Therefore, our objective is to predict if a sportsman will stop exercising within x days. If this happens, 1www.u4fit.com. Please note that the coaches marketplace is visible only by setting the Italian language on the platform. we are able to promptly recommend the sportsman to his coach, in order for her to intervene and prevent him to give up training.
The analysis and the results presented in this paper take into account users’ workouts made by means of u4fit for a timespan of approximately two years and a half. The proposed method can greatly help human coaches to intervene in order to reduce the amount of people going to abandon an active lifestyle with the obvious positive outcomes for their health.
The contributions of this paper can be summarized as follows: • this is the first time in the literature of health RS in which, given the past exercising behavior of a sportsman, he is recommended to his professional (the coach), to trigger her reaction and motivate him not to stop; • we validated our proposal on a real-world dataset made up of approximately two years and a half of data, by comparing diferent classifiers on standard accuracy metrics; • our solution can be embedded in real-world persuasive eHealth systems, thus finding practical and efective applications.
The rest of the paper is structured as follows: Section 2 describes the collected dataset; Section 3 illustrates the features we modeled and the details of the classifiers; Section 4 presents the experimental framework; Section 5 contains conclusions and future work. 2
The dataset employed for the analysis consists of a subset of workouts collected by means of the u4fit platform. For each workout we considered the following aggregate statistics: (1) covered distance (in meters); (2) workout duration (in seconds); (3) rest time (in seconds); (4) average speed (in km/h); (5) burnt calories; (6) time elapsed since the previous workout (in hours). From the total number of available workouts we excluded those whose aggregated statistics are neither related to running nor to the beginner/amateur runner2. In particular, we considered only those matching the following constraints: (i) 0.5 km ≤ distance ≤ 43 km, (ii) workout duration ≤ 5 hours, (iii) rest time ≤ 1 hour , (iv) averaдe speed ≤ 16 km/h, (vii) burnt calories ≤ 3000. The final sample contains 65315 workouts, performed by u4fit users from January 1, 2015 to May 10, 2017. 3
This section will illustrate the features we modeled and will introduce the classifiers used in this study. 3.1
The values that were measured for each workout have been used to model six sets of features, given as input to the classifiers: ABS. Each feature contains the absolute value recorded during the workout, presented in the previous list (e.g., the distance feature of a workout contains the number of meters covered by the sportsman). In addition to the features of the current 2We are deliberately not disclosing the total number of workouts before the cleanup given it is a sensitive commercial information. workout, we considered also a new feature representing the number of workouts performed by the user previous to the current one, which allows us to contextualize the current performance of the sportsman to his expertise; INC. Each feature is represented as the increment with respect to the previous workout performed by the user. For example, let distanceC represent the distance covered in the current workout and distanceP the one associated to the previous workout. The covered distance feature for the INC set is represented as: (distanceC − distanceP )/distanceP ; MIN, AVG, MAX. Each feature contains the minimum [average, maximum] of the user’s historical training data; In addition to the features that consider the current workout and perform a matching with the history of a user, we modeled an additional set, named Weekly Load (WL), which measures the impact of the previous week’s workouts on the current performance. This is achieved by summing the values of the previous seven days for a user, considering the following three features: (i) covered distance, (ii) workout duration, and (iii) burnt calories.
In conclusion, each workout is represented by 34 features.
The class used to train a classifier was binary and it was 1 if the user worked out in the next x days and 0 otherwise. 3.2
In our study, we evaluated and compared the performances of four
among the most efective classifiers at the state of the art [
Random Forest (RF) is a meta-estimator of the family of the
ensemble methods [
Ada Boost (AB) is another ensemble method that, in order to
produce the final prediction, combines the predictions from a
sequence of weak learners (i.e., models that are only slightly better
than random guessing, such as small decision trees) that are fit on
repeatedly modified versions of the data [
Extra Trees (ET) is also an ensemble method. Similarly to Random
Forest, it uses a random subset of candidate features while splitting
a tree node; however, instead of looking for the most discriminative
thresholds, thresholds are drawn at random for each candidate
feature and the best of these randomly-generated thresholds is
picked as the splitting rule [
Multi-Layer Perceptrons (MLP) is a neural network that, given a
set of features and a target, can learn a non-linear function
approximator for classification or regression. It is diferent from logistic
regression, in that between the input and the output layer there
can be one or more non-linear layers, called hidden layers [
This section will present the experimental setup and strategy, the evaluation metrics, and the obtained results. 4.1
The experimental framework exploits the Python scikit-learn 0.17.1 library3 for all the classifiers. The experiments were executed on 3http://scikit-learn.org/0.17/ an Intel Core i7 2630QM processor equipped with 8GB RAM. Each classification was repeated 10 times with a 10-fold cross-validation.
In order to avoid class imbalance, we tested several undersampling and oversampling strategies on the training set, but the most accurate one was random undersampling. It is worth noting, however, that with x = 1, classes are naturally balanced.
We performed three sets of experiments: (1) Classifiers comparison. We compared the classifiers, by running them on all the modeled features, to evaluate the most efective one for x = 1, 3, 7. (2) Feature sets importance evaluation. For each set of features, we evaluated its importance by measuring the feature_importances_ parameter of the RandomForest class in scikit-learn. (3) Evaluation of the classifier with less features. Considering the most performing classifier in the first experiment, we removed one by one the least important set of features. This allows us to evaluate if any of them represents noise in the classification process (i.e., not only a set is less relevant, but it also contains misleading information for our task) or if, even though it is less relevant, each set of features is essential to improve the classification accuracy. 4.2
In the evaluation phase, we selected a series of metrics that, going beyond simple accuracy, are suitable to assess the performance of a binary classifier that operates with imbalanced classes (i.e., when one class is much bigger than the other, as in our case).
Accuracy, which measures the fraction of all instances that are correctly classified, can be defined as: (T P +T N )/(P +N ). T P are the true positives (i.e., instances of the positive class that are correctly labeled as positive by a classifier), T N are the true negatives (i.e., instances of the negative class that are correctly labeled as negative by a classifier), P are the real positive instances and N are the real negative ones. This metric can be misleading when the two classes are highly imbalanced (a classifier that always predicts the majority class would have a high accuracy). For this reason, we also measured some other metrics that are more reliable in our scenario.
Recall measures a classifier’s completeness and is defined as: T P /P .
Precision is a measure of a classifier’s exactness and is defined as: T P /(T P + F P ).
In our case-study, we considered recall to be more relevant than precision; indeed, motivating a user who does not need it (false positive) is a lesser evil for a trainer, compared to failing to motivate a user who, on the contrary, needs to be motivated (false negative). Considered this, we decided to measure the F2-score, which, like other forms of F-measures, is a metric that considers both recall and precision, although weighing the former higher than the latter:
Precision · Recall
F 2 = 5 · 4 · Precision + Recall
It is important to notice that neither the recall nor the precision
(and, consequently, not even the F2-score) take the true negative
rate into account and this is a problem when dealing with highly
imbalanced classes [
is T N /N . It ranges between -1 and 1, where 1 represents a perfect
prediction, 0 no better than random prediction, and -1 indicates
total disagreement between prediction and observation. As [
Experimental Results 4.3.1 Classifiers comparison. Tables 1, 2, and 3 compare the capability of the classifiers to predict if a sportsman will stop working out within the next 1, 3, or 7 days. Figures show that the classifiers are efective for all the metrics we measured. Indeed, for the metrics whose values are between 0 and 1 (i.e., all except Informdness), the values are high; Informedness, whose values are between -1 and 1, returns values close to 0.5, thus in line with the other metrics. This means that, in 70% or more of the cases, we can correctly predict if a sportsman will stop training by monitoring his past performance; therefore, we can assume that the remaining 30% is partially related to unpredictable events for a classifier, such as injuries or personal problems not related to sports. Therefore, by tracking behavioral changes, we can produce efective recommendations for the coaches in most of the cases.
It is also worth noting that the best results are obtained with x = 1, i.e., when classes are already balanced. This happens because, when x grows, we remove information with the undersampling, in order to balance the classes. Moreover, sportsmen in our platform tend to work out with a high frequency, so this explains why our
WL diferent sets of features (1: ABS+AVG+INC+WL+MAX+MIN, ABS+AVG+INC, 5: ABS+AVG, 6: ABS, 7: AVG). rent workout (ABS) and the average historical values (AVG) clearly have more impact in the classification process, although no set of features is much more important than the others.
4.3.3
Evaluation of the classifier with less features. Thanks to the previous experiment, we can rank the sets of features by importance and remove them one by one, to see how the efectiveness of the Random Forest classifier is actually afected by them. The results reported in Table 5 show that the MAX, MIN, and WL sets do not help improve the efectiveness of the classifier and that the MIN set actually represents a small form of noise (indeed, by removing it, the recall increases of 0.1). However, if we remove the other, more important sets of features, the classifier’s efectiveness decreases noticeably. 5
In this paper, we presented an efective approach to predict if a sportsman will soon stop training. These predictions can be used to recommend the sportsman to his coach, who can motivate him not to stop.
We modeled 34 features, by considering the performance of a sportsman during a workout and by correlating it with his past performances. The results showed that behavioral changes can be efectively tracked by monitoring the workout performances, and that nor the recent history of the user, nor the best or worst performance, add information that can be exploited by a classifier.
In future work we will integrate our RS in the u4fit platform to actually investigate the impact of the recommendations. Moreover, we will also exploit chats between sportsmen and trainers to try to improve the classification accuracy and to investigate if there exist other markers not considered by the presented study.
The authors would like to thank Alessio Demurtas, Marika Cappai, and Stefano Amico for their contribution in this research work.
This work is partially funded by Regione Sardegna under project NOMAD (Next generation Open Mobile Apps Development), through PIA - Pacchetti Integrati di Agevolazione “Industria Artigianato e