With the proliferation of afordable sensing devices, more and more sportspeople are able to monitor their exercise sessions and gain valuable insights into their exercise form and exertion. Yet, manufactures seldom employ intrinsic motivation of their users as a motivational factor and rather rely on external elements such as gamification. Ultimately users might fail to see the potential benefits of their exercises and rather blindly strive for completing the next app challenge, lacking active reflection about their exercise form. We argue that this aspect is quintessential in acquiring genuine proficiency for a given exercise, yet current sports technology is seldom designed to encourage active reflection from users. In this position paper, we depict how designing for reflective feedback, leveraging mobile and wearable sensing devices, provides users with the means to actively reflect on their exercise form. We envision that this - already emerging form of feedback - will allow users to gain deeper bodily insights and facilitate an inherent understanding of the meaning and purpose of their physical activity. Based on existing research works, we highlight the potential of this approach to generalize well over a diverse set of physical activities and outline future research directions.
Technological advances have had a big impact on how we perform sports activities, from simple
tracking apps [
We argue that this approach of providing niche-only assistance systems that dictate users
what to do may pose the inherent risk of limiting lasting training efects [
Several aspects make this a challenging endeavor, such as the user’s afinity to specific learning methods and the potentially limited applicability across diferent types of physical activity. As HCI researchers, we usually do not have the luxury of extensive knowledge about the human anatomy. Thus, we should play to our strength and focus on creating systems that can enable users during their sports practice by means of technology without interfering with their traditional learning method. In other words, we should aim to provide a seamless extension to their current training regime and create new insights through the use of sensing technology. As such, the provided feedback should not dictate the training routine, but rather provide the means for users to gain new insights about their bodily abilities.
We envision this emerging paradigm of allowing for active reflection during physical activities as quintessential in creating long-lasting bodily insights for users, facilitating the consolidation of exercise forms at their own pace. In this position paper, we take a holistic look at interactive feedback systems in sports and derive tangible design dimensions for future research directions. Ultimately, we identified reflective feedback as a sweet-spot, (1) providing the user with the means to gain bodily insights and thus potentially increasing intrinsic motivation for physical activity and (2) enabling means for generalization across domains of physical activity.
The field of HCI has already investigated diferent design and modalities in interactive feedback systems for physical activity, drawing extensively from theoretical models behind motor learning and consolidation of movement forms. In this section, we present a short introduction of these models and respective learning methods. Finally, we give an overview on commonly used facets for feedback design for physical activity.
In the motor learning process, we develop new skills in diferent stages, gaining more experience through practice and consolidating it over time. Diferent theories, such as Doyon et al.’s model of motor skill learning [12] and Sweller’s cognitive workload theory [13], describe the stages in the learning process and the influence of cognitive factors. Throughout the motor learning process, the knowledge transfers from short-term memory to long-term memory. We have yet to completely understand how our bodies create motor memory [14], allowing us to efortlessly perform highly complex movements given adequate training. Nevertheless, research on motor learning has shown with retention tests that practice and feedback have a decisive influence on the consolidation of motor skills [12, 15]. It remains the teacher’s task to support the learner with exercises that correspond to the current learning stage and to correct mistakes using individual methods. Considering the underlying processes in motor learning, it is essential to support the consolidation during this process and to gain deeper bodily insights, for example through active reflection on one’s own exercise form.
Consequently, diferent approaches exist for teaching and learning motor skills. Common learning approaches include learning by imitation or observation of an expert performing the exercises [16]. Another approach is learning from descriptions of the expected execution, as applied by Velloso et al. [17]. Independent of the exercise, it requires the knowledge about the correct execution of movement to avoid injuries [18].
Using dancing as an example for motor learning, Villa et al. [19] derived design requirements for wearable feedback systems from literature and expert interviews. Among others, they highlight the importance of personalized feedback for the learner. In most motor learning contexts, individual feedback is expensive or not available because the students outnumber the teacher [20]. As a step towards an assistive wearable feedback system for motor learning, Villa et al. [19] showed that dancers prefer implicit feedback, e.g., on specific body parts to know where the movement was incorrect. For the instruction of a new movement, however, they preferred explicit explanations. Consequently, the question arises of how to design feedback that provides the means for sportspeople to gain deeper bodily insights.
Reflecting on one’s own performance is an essential part of the learning process and can be supported by corresponding feedback design. Schön [21] distinguishes between reflectionin-action and reflection-on-action, where the former happens while performing a task and the latter is retrospective. Systems which support the user with reflection-in-action, visualize the efect of a certain task to encourage reflecting on the performance. Likewise, inducing an internal or external focus of attention afects how the learner reflects on their performance and self-assessment. According to Shea and Wulf [22], an internal focus requires the learner to direct the attention to the learned movement itself. In contrast, the external focus makes the learner reflect on the efect of the movement.
The feedback design in related research can be categorized into many diferent facets, but mainly deals with what to show and when to show. For example (cf. when to show), Raheb et al. [23] distinguish between tools that provide feedback continuously while performing the exercises or discretely at specific timestamps. A further distinction is whether the feedback is provided in real-time in the situation [24] or as post-hoc feedback afterward [25]. Further, Wulf and Shea [26] discovered the beneficial efect of concurrent feedback in early learning stages by reducing cognitive overload. Moreover, it leads to improved performance and learning with reduced information to process [27].
Related work has identified the granularity of feedback, cf. what to show, i.d., the level of data aggregation as a vital factor for proper reflection. This aspect is especially important when considering the social context of users, implying a certain necessities when designing for multiple user groups. An amateur might require light and curated feedback (see Figure 1b), while an expert — on the way to perfect their exercise form — might prefer to consult the data in its original form (see Figure 1a).
While raw data (and derivatives) can provide deeper insights for experts, they are unsuited for laypeople as interpretation is dificult. On the other side of the spectrum are feedback Biceps
Deltoid
Biceps
Deltoid (a) Epoch-averaged raw data.
(b) Simplified representation.
systems that exclusively provide domain-specific feedback, drawing extensively from prior
knowledge of the task domain1. This type of feedback is usually limited to address exactly one
specific aspect of the physical activity and recommend a course of action [ 24]. A compromise
are generic representations of collected sensor data, such as reducing the temporal fidelity and
simplifying the visualization, hence allowing the user to focus better and providing them with a
clear goal [
Based on insights from related work, we propose a categorization as shown in Figure 2. On one axis, we consider the system’s level of feedback granularity, ranging from domain-specific to generic feedback, as outlined above. On the other axis, we evaluated the required efort by the users to interpret the feedback and make sense of it (the amount of active reflection needed). At one extreme, the system interprets the data and tells the user what to do and how to correct it. On the other end, the user must interpret the provided feedback himself and draw conclusions from it. We populate the categorization with example works to highlight the benefits and caveats of specific feedback designs. Further, we discuss a possible extension to this space to outline future research directions.
An example for highly domain-specific feedback with no need for active interpretation from the user can be found in Footstriker [24] and Saltate! [28]. Both systems actively correct their users, e.g., through EMS-triggered activation of muscles [24] and emphasizing incorrect beat timings [28]. 1Such as the type of sport and involved movements.
Need for Interpretation high low
In contrast, Turmo Vidal et al. [7]’s example BodyLights (see Figure 3) can be placed on the opposite side of our categorization, i.d., on the generic feedback side with a higher need for interpretation. BodyLights uses generic visual feedback through laser projections to support error identification in exercises for strength training but requires users to identify erroneous positions themselves and how to correct them. Through this external focus of attention, the users learn to acquire the understanding of the correct technique and how to execute it.
Further works can be placed along a corridor between the two aforementioned works. Park and Lee [29]’s work provides subtle correction feedback to the user through color-coding during snowboarding, whereas Subletee [30] employs multi-modal feedback for posture correction in golf play. Both works focus on active reflection and can thus be placed in the upper right quadrant of our categorization.
Throughout our investigation, we have identified this area as a sweet spot for level of feedback granularity and the required efort by the user to interpret it. Systems within this sweet spot enable users to gain body awareness and further their ability to transfer the knowledge across domains of physical activity while keeping the cognitive load at a reasonable level.
In this work, we introduce reflective feedback as a definition for feedback in this sweet spot that: 1. leverages sensor data in the form of an generic representation as feedback to the user 2. encourages active reflection by the user requiring them to connect their performance to the observed feedback
Designing feedback to increase body awareness is a balancing act between cognitive load and a more profound learning efect by understanding the meaning of the feedback and gaining the knowledge to transfer it across domains. In this position paper, we highlighted existing approaches for feedback design in HCI for sports. Drawing from our categorization scheme, we derived a working definition for reflective feedback to facilitate future research directions.
While highly customized feedback systems support fast improvements of exercise proficiency, we advocate that purely domain-specific feedback sufers from potential ceiling efects, leaving the user without an adequate base knowledge of their exercise form and bodily capabilities to built genuine proficiency. Contrarily, a more generic approach to feedback design requires only little knowledge about the exercise domain allowing it to potentially generalize well across multiple domains. Here, we envision reflective feedback as a means to allow sportspeople to gain deeper bodily insights when performing physical activities.
This form of feedback can potentially be challenging for novice sportspeople that are still unaware about their exercise form. An alternative for novice users to reduce the efort to understand generic feedback and to reduce the risk of misinterpretation can be a cascaded feedback solution. Even though reflective feedback has a steeper learning curve [ 10], a cascaded feedback solution can reduce this complexity. For example, by starting with explicit explanations and gradually fading into reflective feedback when the user gained experience about the physical activity, their own body and how to interpret the given feedback.
While initially requiring a higher cognitive load from the user – the signal needs to be interpreted — reflective feedback potentially enables more profound learning methods leading to increased retention of movement forms in the long run. Actively engaging with one’s exercise form is a key aspect to create motor memory and consolidate new motor skills [12]. Here, reflective feedback provides an additional sensory channel for users to reflect on their exercise form, understand its benefits. While this approach inherently addresses a user’s needs for autonomy [11], it remains a challenge how to design feedback system that adequately address competence — providing an optimal challenge for self-improvement — and relatedness, as in, combining intrinsic motivation about one’s own wellbeing with extrinsic motivational factors, such as sharing one’s fitness advances [11].
Yet, current commercial systems mostly rely on external motivational factors, such as
gamification. The apple watch with its close-your-rings feature [
Consequently, employing a reflective feedback approach allows designers more freedom, but also shifts the focus towards the need to create engaging systems that encourage personal growth in users [31]. A priori, the benefits of advancing the base understanding of one’s own bodily capabilities might not be apparent, thus leading to sparse motivation. It remains a challenge how we can design systems that incite a user’s intrinsic motivation for physical activity.
In this position paper, we presented a categorization for feedback designs in HCI for sports and identified a sweet spot for level of feedback granularity and the required efort by the user to interpret it. Further, we introduced a working definition for reflective feedback that encourages active reflection, thus, enabling the user to gain deeper bodily awareness and potentially facilitates intrinsic motivation. We believe that our definition and categorization outlines future research direction for the realization of sensor-based feedback system in HCI for sports. [7] L. Turmo Vidal, H. Zhu, A. Riego-Delgado, Bodylights: Open-ended augmented feedback to support training towards a correct exercise execution, in: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 2020, pp. 1–14. [8] I. Wellard, Body-reflexive pleasures: exploring bodily experiences within the context of sport and physical activity, Sport, Education and Society 17 (2012) 21–33. [9] J. McCarthy, P. Wright, Putting ‘felt-life’at the centre of human–computer interaction (hci), Cognition, technology & work 7 (2005) 262–271. [10] J. Andres, m. schraefel, A. Tabor, E. B. Hekler, The body as starting point: Applying inside body knowledge for inbodied design, in: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, CHI EA ’19, Association for Computing Machinery, New York, NY, USA, 2019, p. 1–8. URL: https://doi.org/10.1145/3290607.3299023. doi:10. 1145/3290607.3299023. [11] R. M. Ryan, E. L. Deci, Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being., American psychologist 55 (2000) 68. Publisher: American Psychological Association. [12] J. Doyon, P. Bellec, R. Amsel, V. Penhune, O. Monchi, J. Carrier, S. Lehéricy, H. Benali, Contributions of the basal ganglia and functionally related brain structures to motor learning, Behavioural brain research 199 (2009) 61–75. doi:10.1016/j.bbr.2008.11. 012. [13] J. Sweller, Cognitive load during problem solving: Efects on learning, Cogn. Sci. 12 (1988) 257–285. [14] J. W. Krakauer, R. Shadmehr, Consolidation of motor memory, Trends in Neurosciences 29 (2006) 58–64. doi:10.1016/j.tins.2005.10.003. [15] R. Sigrist, G. Rauter, R. Riener, P. Wolf, Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review, Psychonomic bulletin & review 20 (2013) 21–53. doi:10.3758/s13423-012-0333-8. [16] S. Vogt, R. Thomaschke, From visuo-motor interactions to imitation learning: behavioural and brain imaging studies, Journal of sports sciences 25 (2007) 497–517. [17] E. Velloso, A. Bulling, H. Gellersen, W. Ugulino, H. Fuks, Qualitative activity recognition of weight lifting exercises, in: Proceedings of the 4th Augmented Human International Conference, 2013, pp. 116–123. [18] N. J. Hodges, A. M. Williams (Eds.), Skill Acquisition in Sport: Research, theory and practice, third edition ed., Routledge, Third Edition. | New York : Routledge, 2019. | “First edition published by Routledge 2004”–T.p. verso. | Previous edition: 2012., 2019. URL: https://www.taylorfrancis.com/books/e/9781351189750. doi:10.4324/9781351189750. [19] S. Villa, J. Niess, B. Eska, A. Schmidt, T.-K. Machulla, Assisting motor skill transfer for dance students using wearable feedback, in: 2021 International Symposium on Wearable Computers, Association for Computing Machinery, New York, NY, USA, 2021, p. 38–42.
URL: https://doi.org/10.1145/3460421.3478817. [20] Q. Zhou, C. C. Chua, J. Knibbe, J. Goncalves, E. Velloso, Dance and choreography in hci: A two-decade retrospective, in: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 2021, pp. 1–14. [21] D. Schon, The reflective practicioner: How professionals think in action. london (1983). [22] C. H. Shea, G. Wulf, Enhancing motor learning through external-focus instructions and feedback, Human Movement Science 18 (1999) 553–571. URL: https:// www.sciencedirect.com/science/article/pii/S0167945799000317. doi:https://doi.org/ 10.1016/S0167-9457(99)00031-7. [23] K. E. Raheb, M. Stergiou, A. Katifori, Y. Ioannidis, Dance interactive learning systems: A study on interaction workflow and teaching approaches, ACM Computing Surveys (CSUR) 52 (2019) 1–37. [24] M. Hassan, F. Daiber, F. Wiehr, F. Kosmalla, A. Krüger, Footstriker: An ems-based foot strike assistant for running, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1 (2017) 1–18. [25] A. Dias Pereira dos Santos, K. Yacef, R. Martinez-Maldonado, Let’s dance: how to build a user model for dance students using wearable technology, in: Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization, 2017, pp. 183–191. doi:10.1145/3079628.3079673. [26] G. Wulf, C. H. Shea, Principles derived from the study of simple skills do not generalize to complex skill learning, Psychonomic bulletin & review 9 (2002) 185–211. [27] G. Wulf, Attention and motor skill learning, Human Kinetics, 2007. [28] D. Drobny, M. Weiss, J. Borchers, Saltate! a sensor-based system to support dance beginners, in: D. R. Olsen, R. B. Arthur, K. Hinckley, M. R. Morris, S. Hudson, S. Greenberg (Eds.), Proceedings of the 27th international conference extended abstracts on Human factors in computing systems - CHI EA ’09, ACM Press, New York, New York, USA, 2009, pp. 3943–3948. doi:10.1145/1520340.1520598. [29] H. K. Park, W. Lee, Motion echo snowboard: enhancing body movement perception in sport via visually augmented feedback, in: Proceedings of the 2016 ACM Conference on Designing Interactive Systems, 2016, pp. 192–203. [30] M. P. Woźniak, J. Dominiak, M. Pieprzowski, P. Ładoński, K. Grudzień, L. Lischke, A. Romanowski, P. W. Woźniak, Subtletee: Augmenting posture awareness for beginner golfers, Proc. ACM Hum.-Comput. Interact. 4 (2020). URL: https://doi.org/10.1145/3427332. doi:10.1145/3427332. [31] Y. Rogers, Moving on from Weiser’s Vision of Calm Computing: Engaging UbiComp Experiences, in: P. Dourish, A. Friday (Eds.), UbiComp 2006: Ubiquitous Computing, Lecture Notes in Computer Science, Springer, Berlin, Heidelberg, 2006, pp. 404–421. doi:10. 1007/11853565_24.