Considerations in Feedback and Periodization for the Multimodal Learning Experience of Running via Wearable Devices Fernando P. Cardenas Hernandez 1 and Jan Schneider 1 1 Information Center for Education, DIPF Leibniz Institute for Research and Information in Education, Rostocker Straße 6, 60323 Frankfurt am Main, Germany Abstract For the integral learning/training of a psychomotor activity such as running, it is necessary to target not only the physical aspects but also the technical and mental aspects that make it up, an alternative to solve this issue is through the understanding and consideration of feedback and periodization, which are elements that constitute and influence transcendentally and differently each of the three aspects involved. That is why, in this paper, some of the definitions and classifications commonly used in the study of psychomotor skills are first cited. Likewise, the use of wearable devices is proposed as a multimodal and main technology to analyze both elements since they have provided positive results in the same field of research. Subsequently, important considerations are discussed which could serve as a starting point or a conceptual reference for the correct application and analysis of feedback and periodization in each aspect of running and possibly other similar psychomotor skills. Keywords 1 Feedback, Periodization, Multimodal, Adaptation, Sensors, Augmented and Virtual Reality 1. Introduction Running is an activity that most healthy humans after 18 to 24 months of age can do naturally. However, running fast and long distances requires a decent amount of training as well as training the many aspects that compose it such as the physical, technical, and mental aspects. First of all, training the physical aspect allows runners to increase their strength, and flexibility [1]. Likewise, training the technical aspect allows runners to improve their efficiency of movement, helping them to run faster and longer distances while reducing the risk of injury [2]. Finally, training the mental aspect allows runners to maintain motivation, concentration, mental toughness, regulated cognitive load, strategy, and decision-making during training and competition [3,4,5]. To methodologically train and improve these aspects, any intended intervention must appraise feedback and periodization. We argue that a Multimodal Learning Experiences (MLX) approach, which exploits the use of sensor and immersive technologies in authentic practice settings, can be used to support the feedback and periodization for the physical, technical, and mental aspects of running. Therefore, in this workshop paper, we present relevant considerations that serve as starting points for MLX research concerning the feedback and periodization of running. MILeS 22: Proceedings of the second international workshop on Multimodal Immersive Learning Systems, September 13, 2022, Toulouse, France cardenas@dipf.de (F. P. Cardenas Hernandez); Schneider.Jan@dipf.de (J. Schneider) 0000-0002-0592-163X (F. P. Cardenas Hernandez); 0000-0001-8578-6409 (J. Schneider) ©️ 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) 2. Background MLX can be defined as the digital enhancement of any learning activity, which occurs in an authentic learning setting and is analyzed with more than two modalities. It is based on three basic principles: the use of sensors, authentic practice, and immersive and ubiquitous technologies. MLX has shown to support the practice of different skills, such as the simulation of medical techniques with the cardiopulmonary resuscitation tutor [6], the development of calligraphy skills with the sensor-based calligraphy trainer [7], and the practice of nonverbal communication skills for presentations with the presentation trainer [8]. Therefore, we consider it a proper approach to support the feedback and periodization of running. 2.1. Feedback Feedback is information that people receive about their performance to identify which responses are effective or ineffective for the purpose of acquiring new skills or improving future performance [9, 10]. Based on the origin or source of the information received by the person, feedback can be categorized into: a) intrinsic or non-augmented feedback: the information is of a sensory-perceptive type which is originated by the person through diverse sensory processes such as vision, audition and touch. b) extrinsic or augmented feedback (AF): the supplemental information comes from a source extrinsic to the person, i.e. an outside source provides additional information [11]. AF feedback has several dimensions. In terms of the content it conveys, it can be classified in two different categories. The first one is knowledge of result where the feedback information is about the performance outcome i.e. goal achievements. (e.g. success/failure, scores, and dimensions). The second one is knowledge of performance where the feedback information is related to performance nature and its descriptive patterns, i.e. execution quality (e.g. assessment of limb positions) [12, 13]. A second dimension of the AF concerns its timing and frequency. In terms of timing, AF can be concurrent or real time feedback when the information is presented to the learner during the actual performance. If the presentation of information is without interruptions, then the concurrent feedback is called immediate or continuous concurrent feedback. On the other hand, if the presentation is with interruptions, it is referred to as intermittent or discontinuous concurrent feedback. In case the feedback provides the information after the learner completes the task or performance, it is called terminal feedback [12, 14]. Considering its frequency variable, the AF is divided into faded feedback which is characterized by a decrease in frequency over time and increasing feedback whose frequency increases over time [10]. Another important dimension of AF considers the affective aspects of the presented information. In this respect, the feedback can be positive or negative. Positive feedback delivers encouraging data to the learner or focuses mainly on successful trials, whereas, negative feedback does the opposite [15]. Other dimensions of feedback based on different principles are: a) bandwidth feedback: the information is presented only if the learner’s performance falls outside of an acceptable range of performance [16]; b) self-controlled feedback: learners choose “when” (sometimes also “what” and “how”) to receive the feedback [17]; c) subliminal or metaphorical feedback: the information is given by metaphors/analogies as instructions; d) social comparative or normative feedback: the information provided gives a reference on the global or average performance within a group which serves as a standard for the learners [18]. Finally, based on the feedback modalities, if the feedback relies on a single modality, it is called unimodal feedback; in the event that the feedback uses numerous modalities, it is named multimodal feedback. Unimodal feedback is commonly classified as visual, auditory, and haptic [19]. 2.2. Periodization The concept of periodization refers to a frequently used strategy for the planning and organization of events associated with physical adaptation to chronologically enhance the performance of athletes at specific time points without endangering their health. To handle the training process management, periodization fragments the entire training season into three interrelated periodic time units called cycles [20]: a) Macrocycle is the longest cycle and corresponds to a single training season dedicated to successfully achieving the athletes’ long- term goal. b) Mesocycle is the medium-term cycle resulting from the dissection of the macrocycle into subunits that are usually months or weeks. A mesocycle is the minimum period of time to produce a relative adaptation [21]. c) Microcycle is the shortest cycle of the periodization, normally calculated in days or weeks. A set of microcycles constitutes a mesocycle. Additionally, although they are not defined as cycles, daily training sessions or even subunits of these training sessions are the basic constituents of microcycles [20]. It is worthy to say that periodization is referred to training the physical aspect and it is normally planned by coaches, however, the application of periodization in the training of technical and mental aspects is not properly defined or systematized. 3. MLX Considerations of Feedback and Periodization 3.1. Logistical Considerations An MLX system designed to support the development of psychomotor skills, such as running, has to provide useful feedback to the learners. This means that the feedback must be AF. However, constantly providing feedback to learners might lead to a dependency on AF, and hence hinder the development of the skills. Therefore, it is important to consider mechanisms for fading out the feedback as learners develop their skills so that the original AF eventually becomes intrinsic feedback. In other words, the MLX system should help learners to become independent from external sources. Regarding the type of information contained in the feedback, it is suggested that the knowledge of performance feedback is more appropriate to target the technical aspect since it is expected to influence the running form through the qualitative assessment of the movements and positions of body parts. Knowledge of result feedback should be used mainly in the physical aspect because it is proposed that the information transmitted is related to the results from kinematic parameters (e.g. speed and acceleration). The mental aspect could also use knowledge of result feedback, through the use of scales or scores linked to the decisions or actions taken by the learner in training or interventions created in the virtual/extended world. Besides, considering the efficacy demonstrated in the learning process in previous studies [15,18], the use of positive feedback and normative feedback is advisable in each of the aspects of running. Speaking of periodization, as it is not well defined in the mental and technical aspects, we concentrated and proposed some starting points for their study. To develop a mental and technical periodization that makes use of Augmented and Virtual Reality (AR/VR) glasses, the cybersickness produced by these devices is considered, that is why "virtual" interventions of no more than 10 minutes are recommended [22]. Virtual interventions aimed at influencing the mental aspect should be performed around 6 hours before the training of the physical or technical aspects to avoid cognitive overload caused by an excess of continuous information; in that way the athlete could gradually assimilate the information and make it part of their inherent or internal knowledge similar to intrinsic feedback. On the other hand, it is proposed that virtual interventions aimed at influencing the technical aspect should be done separately from physical and mental training. They could also be done before or during the physical aspect warm-up, as long as the intervention only contains an instruction or command. Finally, to elaborate the periodization of the mental and technical aspects, we consider elaborate interventions that are aligned with the periodization of macro, meso, and micro cycles. 3.2. Technological Considerations Not all sensors and immersive devices are appropriate to be used in all aspects of running, given that they can indirectly modify the results of training or interventions; considering this, the most appropriate devices for each aspect must first be described. The application of AR/VR glasses restricts training to activities that are static or not very dynamic because these devices impede the natural movement of people. In the particular case of running, it involves rapid and dynamic body movements that in turn generate perspiration that can affect AR/VR glasses, thus, the use of this technology to provide feedback or periodization while running is not recommended. However, the mental and technical aspects can be addressed if static or non-very-dynamic activities or interventions are properly designed with the restrictions imposed by AR/VR glasses; for example, virtual/augmented scenarios can be created to induce “virtual experience”. According to running coaches, the physical aspect of running should be done outdoors as much as possible. Even when treadmills can be used to train it, its use should only be as a complementary tool but not as a main learning method since treadmills modify in a certain way the natural running gait of people. For this reason, static cameras are not suitable for MLX running systems; sensors and actuators analyzing and providing feedback while training the physical aspects of running should be able to follow the learner. Therefore, the use of wearable accelerometers and/or smartwatches along with smartphones and their respective embedded microphones and speakers represent the most viable alternatives for providing feedback outdoors in this regard. Besides, these devices can also be used to give feedback on the technical aspect. In this way, about the feedback modality, it is proposed that the physical aspect should mainly use auditory feedback (earphones), the feedback on the mental aspect should be primarily visual (AR/VR lenses), and the technical aspect can be favored by both auditory and visual feedback depending on the type of intervention designed. 4. Conclusions and Future Research This workshop paper aims to make readers aware of the relevance of feedback and periodization for the development of an MLX system designed to support the development of running skills. It provides a synthesis of our research in the field that shows some logistical and technological considerations needed to conduct further research on this topic. Figure 1 shows some of these considerations. Some questions that can lead future research on this topic are the following: ● Q1: How to integrate various types of feedback or modalities to correctly train each aspect? ● Q2: How can we evaluate the efficacy of the proposed periodization and feedback in technical and mental elements? ● Q3: Is it possible to target physical, technical and mental aspects in parallel? If so, how? ● Q4: How to develop a periodization plan or protocol based on the results obtained from virtual and non-virtual interventions? ● Q5: What particular factors should be considered to elaborate a periodization for each aspect? Figure. 1. Logistical and technological considerations for the development of the MLX system. 5. References [1] Ferreira, M. L., Sherrington, C., Smith, K., Carswell, P., Bell, R., Bell, M., Nascimento, D. P., Máximo Pereira, L. S., & Vardon, P. (2012). Physical activity improves strength, balance and endurance in adults aged 40–65 years: a systematic review. Journal of Physiotherapy, 58(3), 145– 156. https://doi.org/10.1016/S1836-9553(12)70105-4 [2] Gabbett, T. J. (2016). The training—injury prevention paradox: should athletes be training smarter and harder? British Journal of Sports Medicine, 50(5), 273–280. https://doi.org/10.1136/bjsports- 2015-095788 [3] Raab, M. (2003). Decision making in sports: Influence of complexity on implicit and explicit learning. International Journal of Sport and Exercise Psychology, 1(4), 406–433. https://doi.org/10.1080/1612197X.2003.9671728 [4] Chueh, T.-Y., Huang, C.-J., Hsieh, S.-S., Chen, K.-F., Chang, Y.-K., & Hung, T.-M. (2017). Sports training enhances visuo-spatial cognition regardless of open-closed typology. PeerJ, 5, e3336. https://doi.org/10.7717/peerj.3336 [5] Stroth, S., Hille, K., Spitzer, M., & Reinhardt, R. (2009). Aerobic endurance exercise benefits memory and affect in young adults. Neuropsychological Rehabilitation, 19(2), 223–243. https://doi.org/10.1080/09602010802091183 [6] di Mitri, D., Schneider, J., Trebing, K., Sopka, S., Specht, M., & Drachsler, H. (2020). Real-Time Multimodal Feedback with the CPR Tutor (pp. 141–152). https://doi.org/10.1007/978-3-030- 52237-7_12 [7] Limbu, B. H., Jarodzka, H., Klemke, R., & Specht, M. (2019). Can You Ink While You Blink? Assessing Mental Effort in a Sensor-Based Calligraphy Trainer. Sensors, 19(14), 3244. https://doi.org/10.3390/s19143244 [8] Schneider, J., Börner, D., van Rosmalen, P., & Specht, M. (2015). Presentation Trainer, your Public Speaking Multimodal Coach. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, 539–546. https://doi.org/10.1145/2818346.2830603 [9] Walsh, C. M., Ling, S. C., Wang, C. S., & Carnahan, H. (2009). Concurrent Versus Terminal Feedback: It May Be Better to Wait. Academic Medicine, 84(Supplement), S54–S57. https://doi.org/10.1097/ACM.0b013e3181b38daf [10] Goodman, J. S., & Wood, R. E. (2009). Faded Versus Increasing Feedback, Task Variability Trajectories, and Transfer of Training. Human Performance, 22(1), 64–85. https://doi.org/10.1080/08959280802541013 [11] van Vliet, P. M., & Wulf, G. (2006). Extrinsic feedback for motor learning after stroke: What is the evidence? Disability and Rehabilitation, 28(13–14), 831–840. https://doi.org/10.1080/09638280500534937 [12] Wulf, G., Shea, C. H., & Matschiner, S. (1998). Frequent Feedback Enhances Complex Motor Skill Learning. Journal of Motor Behavior, 30(2), 180–192. https://doi.org/10.1080/00222899809601335 [13] WÄLCHLI, M., RUFFIEUX, J., BOURQUIN, Y., KELLER, M., & TAUBE, W. (2016). Maximizing Performance. Medicine & Science in Sports & Exercise, 48(4), 714–719. https://doi.org/10.1249/MSS.0000000000000818 [14] Schmidt, R. A., & Wulf, G. (1997). Continuous Concurrent Feedback Degrades Skill Learning: Implications for Training and Simulation. Human Factors: The Journal of the Human Factors and Ergonomics Society, 39(4), 509–525. https://doi.org/10.1518/001872097778667979 [15] Stoate, I., Wulf, G., & Lewthwaite, R. (2012). Enhanced expectancies improve movement efficiency in runners. Journal of Sports Sciences, 30(8), 815–823. https://doi.org/10.1080/02640414.2012.671533 [16] Sadowski, J., Mastalerz, A., & Niznikowski, T. (2013). Benefits of Bandwidth Feedback in Learning a Complex Gymnastic Skill. Journal of Human Kinetics, 37(1), 183–193. https://doi.org/10.2478/hukin-2013-0039 [17] Sanli, E. A., Patterson, J. T., Bray, S. R., & Lee, T. D. (2013). Understanding Self-Controlled Motor Learning Protocols through the Self-Determination Theory. Frontiers in Psychology, 3. https://doi.org/10.3389/fpsyg.2012.00611 [18] Neighbors, C., Rodriguez, L. M., Rinker, D. v., Gonzales, R. G., Agana, M., Tackett, J. L., & Foster, D. W. (2015). Efficacy of personalized normative feedback as a brief intervention for college student gambling: A randomized controlled trial. Journal of Consulting and Clinical Psychology, 83(3), 500–511. https://doi.org/10.1037/a0039125 [19] Sigrist, R., Rauter, G., Riener, R., & Wolf, P. (2013). Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review. Psychonomic Bulletin & Review, 20(1), 21– 53. https://doi.org/10.3758/s13423-012-0333-8 [20] Naclerio Ayllón, F., Moody, J., & Chapman, M. (2013). Applied periodization: a methodological approach. Journal of Human Sport and Exercise, 8(2), 350–366. https://doi.org/10.4100/jhse.2012.82.04 [21] Zatsiorsky VM, Kraemer JW. Sciences and practice of strength training (2º ed.): Human Kinetics. 2006 [22] Palmisano, S., Allison, R. S., & Kim, J. (2020). Cybersickness in Head-Mounted Displays Is Caused by Differences in the User’s Virtual and Physical Head Pose. Frontiers in Virtual Reality, 1. https://doi.org/10.3389/frvir.2020.587698