Doing exercises regularly and scienti cally can bring better health for people and improve sports performance for athletes. Many investigations have carried on to build necessary models that utilize data collected from people to predict sports performance. Nevertheless, most researches use data directly related to the moment when exercises happen to build prediction models, even though more data related to people's daily activities also impact on sports performance. Thanks to lifelogging, we can now have more data reports not only on people's exercises but also on people's daily activities (both mental and physical aspects). Unfortunately, nding out which data attributes correlate to the changing of sports performance and leveraging these correlated attributes to build a precise prediction model is not a trivial problem. In this paper, we introduce the solution that utilized the predictive power score of lifelog data attributes during a long time to predict an athlete who trained for a sporting event. We evaluate our solution using the dataset and evaluation metric given by imageCLEFlifelog task 2: sports performance lifelog.
Having long-term and regular exercise can bring many bene ts to people's health and daily activities [1]. This argument is also valid for athletes who want to improve their sports performance [2]. Hence, if we can monitor training periods and other factors that could impact both the mental and physical aspects of a person, we can predict that person's sports performance.
In [3], the authors used SVM with particle swarm optimization to tackle the problem of athlete performance prediction. They used that dataset of 500 records of 100 meters running for training and testing their model. The comparison between the proposed method and the linear regression and neural networks con rmed the better performance. This method's vital contribution is to apply chaotic theory on the historical data of the athletes to discover the hidden rules towards improving the productivity of prediction models. Unfortunately, the description of the dataset was not clari ed enough. Hence, the re-producing of this method could be di cult.
In [4], the authors introduced the problem of post hoc analysis (i.e., a process to analyze the athlete's performance after the performed sports activity) using arti cial intelligence. The historical data of the athlete's performance was analyzed, mostly using heart rate, to automatically make up the time de cit on the running competition by using a di erential evolution (DE) algorithm. Unfortunately, the model did not concern the environmental conditions (e.g., weather, altitude, topography, humidity) that logically in uences the athlete's performance.
In [5], the authors utilized computational intelligence and visualization to analyze heart rate and GPS data to better understand cycling and tness physical activities. The authors discovered the positive correlation between heart rate and altitude gradient, the negative correlation between heart rate and speed, and the correlation between The mean heart rate change delay and changes in the altitude gradients associated with cycling up and down.
In [6], the authors introduced a new algorithm based on the behavior of
micro-bats for association rule mining (BatMiner) to explore the athlete's
characteristics that have the most signi cant positive impact on performance. Based
on the results, an athlete can practice alone without the appearance of his/her
coach. There were two kinds of data sources: (
The imageCLEFlifelog 2020 (task sports performance lifelog) [7] provides such a lifelog dataset and raises the exciting challenge to predict the change in running time and weight of a person between the beginning and end of training periods. The challenge here, in our opinion, is to nd out which data attributes correlate to the changing of sports performance, and leveraging these correlated attributes to build a precise prediction model is not a trivial problem. In this paper, we introduce the solution that utilized the predictive power score of lifelog data attributes during a long time to predict an athlete who trained for a sporting event.
The paper is organized as follows: section 2 introduces our methodology, section 3 reports our results and discussions, and section 4 concludes our contribution. 2
In this section, we describe the dataset given by the organizers and our approach used to predict the athlete's expected performance. 2.1
We have to process a multi-modal dataset collected from Fitbit versa 2, PMSYS, and Google form [8]. This dataset consists of various interval observed attributes such as minute-observed attributes (e.g., calories burned, heart rate, step), dayobserved attributes (e.g., weight, meals), and event-observed attributes (e.g., sleep, activity).
First, we synchronized di erent intervals of time into one interval of time for further processing. To do that, we summarized minute-observed attributes following the time length of event-observed attributes and day-observed attributes. For instance, we calculated the total calories burned for each activity and total calories consumed per day.
Then, we normalized the unit of attributes sharing the same meaning to one basic unit. For example, we converted the 'duration' attribute from millisecond to second and active action attributes (e.g., lightly active, very active) from minutes to seconds. Besides, we used one-hot encoding with category attributes such as 'meal.'
Next, we dealt with missing values by lling them with previous value if they are not in the rst order of time and replacing the rst time-order values with their following values. Especially with non-value attributes, we lled with the average value of these attributes from all participants. We also detected and deleted outliers to decrease their impact on the nal results.
Finally, we generated a new attribute representing time per kilometer of running activity in exercise data using speed attributes. 2.2
As mentioned in the previous section, we use a predictive power score (PPS) to
nd a correlation coe cient among dataset's attributes. We are utilizing PPS
to imply the (hidden) correlations among attributes so that the following things
can be detected and summarized (
After building the PPS matrix from cleaned data, we remove all attributes that do not have any relation to running time and weight. Besides, we also ignore attributes that can be predicted by another attribute to reduce the complexity of feature sets. Moreover, we keep pairs of mutually predictive attributes to maintain a strong correlation. Finally, we come up with a set of attributes denoted in Table 1, 2, and 3
M
C weight x x x x x x x x x time-series glasses x x x x x x x x x auxiliary very active x x x x x x x x x auxiliary lightly active x x x x x x x x x auxiliary sedentary x x x x x x x x x auxiliary calories x x x x x x x x x auxiliary distance x x x x x x x x x auxiliary steps x x x x x x x x x auxiliary heart rate x x x x x x x x x auxiliary fatigue x x x x x x x x x auxiliary mood x x x x x x x x x auxiliary readiness x x x x x x x x x auxiliary sleep duration h x x x x x x x x x auxiliary sleep quality x x x x x x x x x auxiliary soreness x x x x x x x x x auxiliary stress x x x x x x x x x auxiliary breakfast x x x x x x x x x auxiliary lunch x x x x x x x x x auxiliary dinner x x x x x x x x x auxiliary evening x x x x x x x x x auxiliary e ciency x x x x x x x x x auxiliary end time x x x x x x x x x auxiliary overall score x x x x x x x x x auxiliary composition score x x x x x x x x x auxiliary revitalization score x x x x x x x x x auxiliary duration score x x x x x x x x x auxiliary resting heart rate x x x x x x x x x auxiliary restlessness x x x x x x x x x auxiliary deep seconds x x x x x x x x x auxiliary deep thirty day avg seconds x x x x x x x x x auxiliary 2.3
We consider data attributes that reported exercise activities (e.g., running, jogging) as time-series data because athletes are in the training time prepared for sports events. It means that their exercises repeat regularly and seasonally. We also consider the rest of the data attributes as auxiliary data such as age, gender, and height.
As discussed in previous sections, two main directions of research in an athlete's performance prediction topic have investigated. The rst one considers only
e t ibu ls r e t d ta o -e M n O data collected during the exercise and ignore other data even these data probably correlate to the athlete's performance. The second one concern all correlated and related data. Followed these directions, we design two types of models. The rst one, called the univariate time-series model, utilizes one attribute. The second one uses a set of attributes.
First, we build two baseline methods: (
Then, we build three CNN models: (
Next, we create three LSTM-like models: (
Finally, we build the GRU model (Fig. 5). In this section, we describe how to utilize our models with selected data attributes to predict the change in running speed and weight since the beginning of the reporting period to the end of the reporting period.
We use the dataset and evaluation metric provided by the imageCLEFlifelog organizers [9]. The readers could refer to the paper written by the organizers for more details [7]. For short, the evaluation metric is de ned as follow: "For the evaluation of the tasks the main ranking will be based on whether there is a correct positive or negative change (a point per correct) - and if there is a draw, the di erence between the predicted and actual change will be evaluated and used to rank the task participants."3.
The organizers de ne three subtasks: "(
We train di erent prediction models for predicting the change of running time, both for a univariate attribute (i.e., time) and a set of attributes. We use three di erent numbers of input time steps, which are 3, 5, and 7. Table 1 denotes which attributes are used for training which models.
After training these models, we evaluate them and select the ten best models with the lowest validation loss as the o cial models for this subtask. Table 4 shows information of these models during training and validating stages. 3.2
Like the rst subtask, we build two types of models that use one or a set of attributes. These models have the same architecture as the models of the rst subtask. However, we use four di erent input time steps: 7, 14, 21, and 30.
After training these models, we evaluate them and select the ten best models with the lowest validation loss as the o cial models for this subtask. Table 5 shows information of these models during training and validating stages.
For subtask 3 "Predict the change in weight from the beginning of February to the end of the reporting period in kilos (1 decimal) using the images ", we use the app, namely Calorie Mama4, to approximately calculate the calories from meal/food images. Then, we add this attribute to the set of current attributes. Then, we run as subtask 2.
The table 6 shows our results evaluated by the organizers. We apply ten di erent models with the best accuracy ltered at the training stage, as described in Table 4 and 5, to run at the testing stage (i.e., the model's IDs expressed in the table are the same with the run's IDs denoted in table 6). As we can see in the table 6, our models cannot reach the optimal stage where both accuracy and the absolute di erence can be optimal at the same time. For example, with subtask 1 (i.e., predict the change in running time), run 8 got the best accuracy (i.e., 1) while 10 received the smallest absolute di erence (i.e., 96). With subtask 2 (i.e., predict the change in weight without image data), run 9 reached the best accuracy (i.e., 0.9), while 6 and 8 gained the smallest absolute di erence (i.e., 4 https://www.caloriemama.ai/CalorieMama 11). With subtask 3 (i.e., predict the change in weight with image data), the results look more stable than other subtasks; at run 8, the accuracy got the maximum (e.g., 1) while at run 10 the smallest absolute di erence was received (i.e., 1). Nevertheless, we can accept one model that can balance between the accuracy and the absolute di erence, according to the purpose of users.
The table 7 illustrates the results for three subtasks conducted by organizers, namely baseline. Regarding the rst subtask, our result is far better than the baseline on both metrics. For instance, except run 5, the rest of our runs have a higher accuracy score than the best accuracy of baseline from 0.2 to 0.4 points, while run 10 and run 8 have absolute di erences fewer 100 points than the best one of baseline. Besides, the baseline cannot precisely perform on both accuracy and absolute di erence categories at the same time. Considering the second subtask, although our best run for absolute di erence (run 1) has a slightly lower score than the baseline does, about 3 points, its accuracy is twice better than that of the baseline (0.4). When coming to the third subtasks, our result has double accuracy (run 8) and half absolute di erence (run 10) compared to the baseline. 3.3
After we gain an insight into the given data, we nd that people subjectively provide plenty of information such as stress, fatigue, mood, and sleep score. It means that these data are likely to be irrelevant to what we want to predict and irrelevant among each person who provided the data. This subjective data could lead to the unstable accuracy of our models.
Moreover, the data collected from tbit has many noises, making it more di cult to generalize models. Furthermore, the amount of data for each participant is limited and inconsistent. For instance, there are approximately twenty running activities for participants 1, 2, and 4, especially only three of these activities for participants 3 and 5. Another illustration of this is that the interval of day-observer attributes is not equal, or some participants like 12 lack much information such as sleep data. These things also prevent our models from reaching optimal accuracy.
Additionally, regarding the time running prediction task, the given data does not have the direct attribute representing time running. However, there is an initial 5km run time for each participant. We nd that this initial time is randomly extracted from each participant's running activities by dividing the distance attribute by speed attribute. Although the initial running time is claimed to belong to 5km running time, the distance attribute shows the much shorter run. Meanwhile, we are informed that data are collected from 16 people who train for a 5km run, almost running activities have done less than 5km. Moreover, despite the requirement of predicting the di erence between seconds used per km (kilometer speed) from the initial run to the run at the end of the reporting period, there is no information to indicate which run or day is at the end of the reporting period for each participant. To cope with these problems, we have to apply some ad-hoc methods to preprocess data that prevent us from generalizing our models. 4
We introduced the solution for predicting an athlete's performance during the training period using neural networks and predictive power score. The predictive power score supports us in enhancing the quality of attributes/features sets towards improving the accuracy of prediction models. We built di erent prediction models and tested with various parameters and hyperparameters to nd the best one. The gained results are auspicious. We will compare our solution with others and thoroughly consider the predictive power score of data attributes to discover hidden patterns useful for improving the accuracy of prediction models.
This research is conducted under the Collaborative Research Agreement between National Institute of Information and Communications Technology and University of Science, Vietnam National University at Ho-Chi-Minh City. 6. I. Fister, I. Fister Jr, and D. Fister, \Batminer for identifying the characteristics of athletes in training," in Computational intelligence in sports. Springer, 2019, pp. 201{221. 7. V.-T. Ninh, T.-K. Le, L. Zhou, L. Piras, M. Riegler, P. l Halvorsen, M.-T. Tran, M. Lux, C. Gurrin, and D.-T. Dang-Nguyen, \Overview of ImageCLEF Lifelog 2020:Lifelog Moment Retrieval and Sport Performance Lifelog," in CLEF2020 Working Notes, ser. CEUR Workshop Proceedings. Thessaloniki, Greece: CEURWS.org <http://ceur-ws.org>, September 22-25 2020. 8. V. Thambawita, S. A. Hicks, H. Borgli, H. K. Stensland, D. Jha, M. K. Svensen, S.A. Pettersen, D. Johansen, H. D. Johansen, S. D. Pettersen et al., \Pmdata: a sports logging dataset," in Proceedings of the 11th ACM Multimedia Systems Conference, 2020, pp. 231{236. 9. B. Ionescu, H. Muller, R. Peteri, A. B. Abacha, V. Datla, S. A. Hasan, D. DemnerFushman, S. Kozlovski, V. Liauchuk, Y. D. Cid, V. Kovalev, O. Pelka, C. M. Friedrich, A. G. S. de Herrera, V.-T. Ninh, T.-K. Le, L. Zhou, L. Piras, M. Riegler, P. l Halvorsen, M.-T. Tran, M. Lux, C. Gurrin, D.-T. Dang-Nguyen, J. Chamberlain, A. Clark, A. Campello, D. Fichou, R. Berari, P. Brie, M. Dogariu, L. D. Stefan, and M. G. Constantin, \Overview of the ImageCLEF 2020: Multimedia retrieval in lifelogging, medical, nature, and internet applications," in Experimental IR Meets Multilinguality, Multimodality, and Interaction, ser. Proceedings of the 11th International Conference of the CLEF Association (CLEF 2020), vol. 12260. Thessaloniki, Greece: LNCS Lecture Notes in Computer Science, Springer, September 22-25 2020.