Supporting Sportspeople in Gaining Bodily Insights Through Reflective Feedback Bettina Eska1 , Jakob Karolus1 1 LMU Munich, Germany Abstract With the proliferation of affordable sensing devices, more and more sportspeople are able to monitor their exercise sessions and gain valuable insights into their exercise form and exertion. Yet, manufactures seldom employ intrinsic motivation of their users as a motivational factor and rather rely on external elements such as gamification. Ultimately users might fail to see the potential benefits of their exercises and rather blindly strive for completing the next app challenge, lacking active reflection about their exercise form. We argue that this aspect is quintessential in acquiring genuine proficiency for a given exercise, yet current sports technology is seldom designed to encourage active reflection from users. In this position paper, we depict how designing for reflective feedback, leveraging mobile and wearable sensing devices, provides users with the means to actively reflect on their exercise form. We envision that this — already emerging form of feedback — will allow users to gain deeper bodily insights and facilitate an inherent understanding of the meaning and purpose of their physical activity. Based on existing research works, we highlight the potential of this approach to generalize well over a diverse set of physical activities and outline future research directions. Keywords reflective feedback, HCI for sports, physical activity, body awareness 1. Introduction Technological advances have had a big impact on how we perform sports activities, from simple tracking apps [1] for an evening run to highly sophisticated sensing devices in competitive sports [2], allowing us to monitor our activities more closely and more objectively. In theory, this would allow users to gain a deeper understanding of their body and abilities. Rather, manufactures often need to employ external motivation, e.g. through gamification [1], to successfully market their products. As such, sports technology often limits itself to tailor-made products that serve one specific purpose, seldom tasking users for active reflection on their bodily abilities. We argue that this approach of providing niche-only assistance systems that dictate users what to do may pose the inherent risk of limiting lasting training effects [3]. Especially since novel sensing technologies do have the potential to provide augmentation to our own senses and increase the user’s own awareness of their body [4]. Here, we have identified an upcoming trend in interactive feedback systems for physical activities that make use of active user reflection to NTSPORT’22: New Trends in HCI and Sports Workshop at MobileHCI’22, October 1, 2022 $ bettina.eska@ifi.lmu.de (B. Eska); jakob.karolus@ifi.lmu.de (J. Karolus)  0000-0002-0954-1394 (B. Eska); 0000-0002-0698-7470 (J. Karolus) © 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) facilitate bodily insights and allowing users to gain a better understanding of their bodies [5, 6, 7]. Research has shown that being aware of the effectiveness of the exercises [8] and gaining an understanding of one’s body physiology [9] helps to increase wellbeing and performance [10], hence fostering motivation [11]. We believe that such an approach has the potential to deliver intrinsic motivation for users, thus enabling consistent improvements among a multitude of sports domains. Several aspects make this a challenging endeavor, such as the user’s affinity to specific learning methods and the potentially limited applicability across different types of physical activity. As HCI researchers, we usually do not have the luxury of extensive knowledge about the human anatomy. Thus, we should play to our strength and focus on creating systems that can enable users during their sports practice by means of technology without interfering with their traditional learning method. In other words, we should aim to provide a seamless extension to their current training regime and create new insights through the use of sensing technology. As such, the provided feedback should not dictate the training routine, but rather provide the means for users to gain new insights about their bodily abilities. We envision this emerging paradigm of allowing for active reflection during physical activities as quintessential in creating long-lasting bodily insights for users, facilitating the consolidation of exercise forms at their own pace. In this position paper, we take a holistic look at interactive feedback systems in sports and derive tangible design dimensions for future research directions. Ultimately, we identified reflective feedback as a sweet-spot, (1) providing the user with the means to gain bodily insights and thus potentially increasing intrinsic motivation for physical activity and (2) enabling means for generalization across domains of physical activity. 2. Background The field of HCI has already investigated different design and modalities in interactive feedback systems for physical activity, drawing extensively from theoretical models behind motor learning and consolidation of movement forms. In this section, we present a short introduction of these models and respective learning methods. Finally, we give an overview on commonly used facets for feedback design for physical activity. Motor Learning And Skill Acquisition In the motor learning process, we develop new skills in different stages, gaining more experience through practice and consolidating it over time. Different theories, such as Doyon et al.’s model of motor skill learning [12] and Sweller’s cognitive workload theory [13], describe the stages in the learning process and the influence of cognitive factors. Throughout the motor learning process, the knowledge transfers from short-term memory to long-term memory. We have yet to completely understand how our bodies create motor memory [14], allowing us to effortlessly perform highly complex movements given adequate training. Nevertheless, research on motor learning has shown with retention tests that practice and feedback have a decisive influence on the consolidation of motor skills [12, 15]. It remains the teacher’s task to support the learner with exercises that correspond to the current learning stage and to correct mistakes using individual methods. Considering the underlying processes in motor learning, it is essential to support the consolidation during this process and to gain deeper bodily insights, for example through active reflection on one’s own exercise form. Consequently, different approaches exist for teaching and learning motor skills. Common learning approaches include learning by imitation or observation of an expert performing the exercises [16]. Another approach is learning from descriptions of the expected execution, as applied by Velloso et al. [17]. Independent of the exercise, it requires the knowledge about the correct execution of movement to avoid injuries [18]. Using dancing as an example for motor learning, Villa et al. [19] derived design requirements for wearable feedback systems from literature and expert interviews. Among others, they highlight the importance of personalized feedback for the learner. In most motor learning contexts, individual feedback is expensive or not available because the students outnumber the teacher [20]. As a step towards an assistive wearable feedback system for motor learning, Villa et al. [19] showed that dancers prefer implicit feedback, e.g., on specific body parts to know where the movement was incorrect. For the instruction of a new movement, however, they preferred explicit explanations. Consequently, the question arises of how to design feedback that provides the means for sportspeople to gain deeper bodily insights. A Take on Existing Feedback Design in HCI for Sports Reflecting on one’s own performance is an essential part of the learning process and can be supported by corresponding feedback design. Schön [21] distinguishes between reflection- in-action and reflection-on-action, where the former happens while performing a task and the latter is retrospective. Systems which support the user with reflection-in-action, visualize the effect of a certain task to encourage reflecting on the performance. Likewise, inducing an internal or external focus of attention affects how the learner reflects on their performance and self-assessment. According to Shea and Wulf [22], an internal focus requires the learner to direct the attention to the learned movement itself. In contrast, the external focus makes the learner reflect on the effect of the movement. The feedback design in related research can be categorized into many different facets, but mainly deals with what to show and when to show. For example (cf. when to show), Raheb et al. [23] distinguish between tools that provide feedback continuously while performing the exercises or discretely at specific timestamps. A further distinction is whether the feedback is provided in real-time in the situation [24] or as post-hoc feedback afterward [25]. Further, Wulf and Shea [26] discovered the beneficial effect of concurrent feedback in early learning stages by reducing cognitive overload. Moreover, it leads to improved performance and learning with reduced information to process [27]. Related work has identified the granularity of feedback, cf. what to show, i.d., the level of data aggregation as a vital factor for proper reflection. This aspect is especially important when considering the social context of users, implying a certain necessities when designing for multiple user groups. An amateur might require light and curated feedback (see Figure 1b), while an expert — on the way to perfect their exercise form — might prefer to consult the data in its original form (see Figure 1a). While raw data (and derivatives) can provide deeper insights for experts, they are unsuited for laypeople as interpretation is difficult. On the other side of the spectrum are feedback Optimum Avoid Optimum Avoid Biceps Deltoid Biceps Deltoid (a) Epoch-averaged raw data. (b) Simplified representation. Figure 1: Visual feedback for a fitness exercise (biceps curl), showing activation of two muscles (biceps, deltoid) [6]. Experts prefer the visualization on the left giving them more insights into how to perfect their exercise form, while amateurs favor the simpler representation on the right. systems that exclusively provide domain-specific feedback, drawing extensively from prior knowledge of the task domain1 . This type of feedback is usually limited to address exactly one specific aspect of the physical activity and recommend a course of action [24]. A compromise are generic representations of collected sensor data, such as reducing the temporal fidelity and simplifying the visualization, hence allowing the user to focus better and providing them with a clear goal [6]. Although this comes at the cost of the need of some additional domain knowledge, it usually poses a good compromise between the need for elaborated classification algorithms (high domain knowledge) and overloading the user with complex signals. We envision this representation as an incentive for users to make use of their own domain knowledge while the system provides the means for them to improve their movement form. 3. Towards Reflective Feedback Based on insights from related work, we propose a categorization as shown in Figure 2. On one axis, we consider the system’s level of feedback granularity, ranging from domain-specific to generic feedback, as outlined above. On the other axis, we evaluated the required effort by the users to interpret the feedback and make sense of it (the amount of active reflection needed). At one extreme, the system interprets the data and tells the user what to do and how to correct it. On the other end, the user must interpret the provided feedback himself and draw conclusions from it. We populate the categorization with example works to highlight the benefits and caveats of specific feedback designs. Further, we discuss a possible extension to this space to outline future research directions. An example for highly domain-specific feedback with no need for active interpretation from the user can be found in Footstriker [24] and Saltate! [28]. Both systems actively correct their users, e.g., through EMS-triggered activation of muscles [24] and emphasizing incorrect beat timings [28]. 1 Such as the type of sport and involved movements. Need for Interpretation Reflective Feedback high [7] [6] [30] [17] [29] [28] [24] low [25] Granularity of Feedback domain-specific generic Figure 2: Categorization of feedback systems. The y-axis shows the need for interpretation by the user. The x-axis shows the granularity of feedback. The red shaded area depicts the sweet spot for reflective feedback. In contrast, Turmo Vidal et al. [7]’s example BodyLights (see Figure 3) can be placed on the opposite side of our categorization, i.d., on the generic feedback side with a higher need for interpretation. BodyLights uses generic visual feedback through laser projections to support error identification in exercises for strength training but requires users to identify erroneous positions themselves and how to correct them. Through this external focus of attention, the users learn to acquire the understanding of the correct technique and how to execute it. Figure 3: Turmo Vidal et al. [7] system BodyLights using generic visual feedback through laser projection to support error identification as an example of reflective feedback. Further works can be placed along a corridor between the two aforementioned works. Park and Lee [29]’s work provides subtle correction feedback to the user through color-coding during snowboarding, whereas Subletee [30] employs multi-modal feedback for posture correction in golf play. Both works focus on active reflection and can thus be placed in the upper right quadrant of our categorization. Throughout our investigation, we have identified this area as a sweet spot for level of feedback granularity and the required effort by the user to interpret it. Systems within this sweet spot enable users to gain body awareness and further their ability to transfer the knowledge across domains of physical activity while keeping the cognitive load at a reasonable level. In this work, we introduce reflective feedback as a definition for feedback in this sweet spot that: 1. leverages sensor data in the form of an generic representation as feedback to the user 2. encourages active reflection by the user requiring them to connect their performance to the observed feedback 4. Encouraging Genuine Proficiency for Physical Activities Designing feedback to increase body awareness is a balancing act between cognitive load and a more profound learning effect by understanding the meaning of the feedback and gaining the knowledge to transfer it across domains. In this position paper, we highlighted existing approaches for feedback design in HCI for sports. Drawing from our categorization scheme, we derived a working definition for reflective feedback to facilitate future research directions. While highly customized feedback systems support fast improvements of exercise proficiency, we advocate that purely domain-specific feedback suffers from potential ceiling effects, leaving the user without an adequate base knowledge of their exercise form and bodily capabilities to built genuine proficiency. Contrarily, a more generic approach to feedback design requires only little knowledge about the exercise domain allowing it to potentially generalize well across multiple domains. Here, we envision reflective feedback as a means to allow sportspeople to gain deeper bodily insights when performing physical activities. This form of feedback can potentially be challenging for novice sportspeople that are still unaware about their exercise form. An alternative for novice users to reduce the effort to understand generic feedback and to reduce the risk of misinterpretation can be a cascaded feedback solution. Even though reflective feedback has a steeper learning curve [10], a cascaded feedback solution can reduce this complexity. For example, by starting with explicit explanations and gradually fading into reflective feedback when the user gained experience about the physical activity, their own body and how to interpret the given feedback. While initially requiring a higher cognitive load from the user – the signal needs to be interpreted — reflective feedback potentially enables more profound learning methods leading to increased retention of movement forms in the long run. Actively engaging with one’s exercise form is a key aspect to create motor memory and consolidate new motor skills [12]. Here, reflective feedback provides an additional sensory channel for users to reflect on their exercise form, understand its benefits. While this approach inherently addresses a user’s needs for autonomy [11], it remains a challenge how to design feedback system that adequately address competence — providing an optimal challenge for self-improvement — and relatedness, as in, combining intrinsic motivation about one’s own wellbeing with extrinsic motivational factors, such as sharing one’s fitness advances [11]. Yet, current commercial systems mostly rely on external motivational factors, such as gam- ification. The apple watch with its close-your-rings feature [1] is a good example for this approach. It encourages the user to perform a certain amount of physical activity every day to live healthier. This is reinforced by the competitive aspect of sharing the activity with friends. Such systems fail to create a deeper understanding of inherent benefits associated with physical activity and consequently cannot support the transition from extrinsic to intrinsic motivational factors. Here, we envision reflective feedback as a means for users to make personal benefits for their physical health and wellbeing more apparent. Consequently, employing a reflective feedback approach allows designers more freedom, but also shifts the focus towards the need to create engaging systems that encourage personal growth in users [31]. A priori, the benefits of advancing the base understanding of one’s own bodily capabilities might not be apparent, thus leading to sparse motivation. It remains a challenge how we can design systems that incite a user’s intrinsic motivation for physical activity. 5. Conclusion In this position paper, we presented a categorization for feedback designs in HCI for sports and identified a sweet spot for level of feedback granularity and the required effort by the user to interpret it. Further, we introduced a working definition for reflective feedback that encourages active reflection, thus, enabling the user to gain deeper bodily awareness and potentially facilitates intrinsic motivation. We believe that our definition and categorization outlines future research direction for the realization of sensor-based feedback system in HCI for sports. References [1] Apple watch. close your rings., 2022. URL: https://www.apple.com/watch/ close-your-rings/. [2] Athos Coaching System, 2022. URL: https://shop.liveathos.com/. [3] J. Karolus, Proficiency-aware systems: designing for user skill and expertise, Ph.D. the- sis, Ludwig Maximilian University of Munich, Germany, 2021. URL: https://edoc.ub. uni-muenchen.de/28900/. [4] D. Svanæs, Interaction design for and with the lived body: Some implications of merleau- ponty’s phenomenology, ACM Transactions on Computer-Human Interaction 20 (2013). URL: https://doi.org/10.1145/2442106.2442114. doi:10.1145/2442106.2442114. [5] E. P. Baumer, V. Khovanskaya, M. Matthews, L. Reynolds, V. Schwanda Sosik, G. Gay, Reviewing reflection: on the use of reflection in interactive system design, in: Proceedings of the 2014 conference on Designing interactive systems, 2014, pp. 93–102. [6] J. Karolus, F. Bachmann, T. Kosch, A. Schmidt, P. W. Woźniak, Facilitating bodily insights using electromyography-based biofeedback during physical activity, in: Proceedings of the 23rd international conference on mobile human-computer interaction, MobileHCI ’21, Association for Computing Machinery, New York, NY, USA, 2021. URL: https://doi.org/ 10.1145/3447526.3472027. doi:10.1145/3447526.3472027, number of pages: 15 Place: Toulouse & Virtual, France tex.articleno: 14. [7] L. Turmo Vidal, H. Zhu, A. Riego-Delgado, Bodylights: Open-ended augmented feedback to support training towards a correct exercise execution, in: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, pp. 1–14. [8] I. Wellard, Body-reflexive pleasures: exploring bodily experiences within the context of sport and physical activity, Sport, Education and Society 17 (2012) 21–33. [9] J. McCarthy, P. Wright, Putting ‘felt-life’at the centre of human–computer interaction (hci), Cognition, technology & work 7 (2005) 262–271. [10] J. Andres, m. schraefel, A. Tabor, E. B. Hekler, The body as starting point: Applying inside body knowledge for inbodied design, in: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, CHI EA ’19, Association for Computing Machinery, New York, NY, USA, 2019, p. 1–8. URL: https://doi.org/10.1145/3290607.3299023. doi:10. 1145/3290607.3299023. [11] R. M. Ryan, E. L. Deci, Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being., American psychologist 55 (2000) 68. Publisher: American Psychological Association. [12] J. Doyon, P. Bellec, R. Amsel, V. Penhune, O. Monchi, J. Carrier, S. Lehéricy, H. Benali, Contributions of the basal ganglia and functionally related brain structures to motor learning, Behavioural brain research 199 (2009) 61–75. doi:10.1016/j.bbr.2008.11. 012. [13] J. Sweller, Cognitive load during problem solving: Effects on learning, Cogn. Sci. 12 (1988) 257–285. [14] J. W. Krakauer, R. Shadmehr, Consolidation of motor memory, Trends in Neurosciences 29 (2006) 58–64. doi:10.1016/j.tins.2005.10.003. [15] R. Sigrist, G. Rauter, R. Riener, P. Wolf, Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review, Psychonomic bulletin & review 20 (2013) 21–53. doi:10.3758/s13423-012-0333-8. [16] S. Vogt, R. Thomaschke, From visuo-motor interactions to imitation learning: behavioural and brain imaging studies, Journal of sports sciences 25 (2007) 497–517. [17] E. Velloso, A. Bulling, H. Gellersen, W. Ugulino, H. Fuks, Qualitative activity recognition of weight lifting exercises, in: Proceedings of the 4th Augmented Human International Conference, 2013, pp. 116–123. [18] N. J. Hodges, A. M. Williams (Eds.), Skill Acquisition in Sport: Research, theory and practice, third edition ed., Routledge, Third Edition. | New York : Routledge, 2019. | “First edition published by Routledge 2004”–T.p. verso. | Previous edition: 2012., 2019. URL: https://www.taylorfrancis.com/books/e/9781351189750. doi:10.4324/9781351189750. [19] S. Villa, J. Niess, B. Eska, A. Schmidt, T.-K. Machulla, Assisting motor skill transfer for dance students using wearable feedback, in: 2021 International Symposium on Wearable Computers, Association for Computing Machinery, New York, NY, USA, 2021, p. 38–42. URL: https://doi.org/10.1145/3460421.3478817. [20] Q. Zhou, C. C. Chua, J. Knibbe, J. Goncalves, E. Velloso, Dance and choreography in hci: A two-decade retrospective, in: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021, pp. 1–14. [21] D. Schon, The reflective practicioner: How professionals think in action. london (1983). [22] C. H. Shea, G. Wulf, Enhancing motor learning through external-focus instruc- tions and feedback, Human Movement Science 18 (1999) 553–571. URL: https:// www.sciencedirect.com/science/article/pii/S0167945799000317. doi:https://doi.org/ 10.1016/S0167-9457(99)00031-7. [23] K. E. Raheb, M. Stergiou, A. Katifori, Y. Ioannidis, Dance interactive learning systems: A study on interaction workflow and teaching approaches, ACM Computing Surveys (CSUR) 52 (2019) 1–37. [24] M. Hassan, F. Daiber, F. Wiehr, F. Kosmalla, A. Krüger, Footstriker: An ems-based foot strike assistant for running, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1 (2017) 1–18. [25] A. Dias Pereira dos Santos, K. Yacef, R. Martinez-Maldonado, Let’s dance: how to build a user model for dance students using wearable technology, in: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, 2017, pp. 183–191. doi:10.1145/3079628.3079673. [26] G. Wulf, C. H. Shea, Principles derived from the study of simple skills do not generalize to complex skill learning, Psychonomic bulletin & review 9 (2002) 185–211. [27] G. Wulf, Attention and motor skill learning, Human Kinetics, 2007. [28] D. Drobny, M. Weiss, J. Borchers, Saltate! a sensor-based system to support dance beginners, in: D. R. Olsen, R. B. Arthur, K. Hinckley, M. R. Morris, S. Hudson, S. Greenberg (Eds.), Proceedings of the 27th international conference extended abstracts on Human factors in computing systems - CHI EA ’09, ACM Press, New York, New York, USA, 2009, pp. 3943–3948. doi:10.1145/1520340.1520598. [29] H. K. Park, W. Lee, Motion echo snowboard: enhancing body movement perception in sport via visually augmented feedback, in: Proceedings of the 2016 ACM Conference on Designing Interactive Systems, 2016, pp. 192–203. [30] M. P. Woźniak, J. Dominiak, M. Pieprzowski, P. Ładoński, K. Grudzień, L. Lischke, A. Ro- manowski, P. W. Woźniak, Subtletee: Augmenting posture awareness for beginner golfers, Proc. ACM Hum.-Comput. Interact. 4 (2020). URL: https://doi.org/10.1145/3427332. doi:10.1145/3427332. [31] Y. Rogers, Moving on from Weiser’s Vision of Calm Computing: Engaging UbiComp Experiences, in: P. Dourish, A. Friday (Eds.), UbiComp 2006: Ubiquitous Computing, Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 2006, pp. 404–421. doi:10. 1007/11853565_24.