Among the most recent Digital Twin applications, there is Digital Twin Coaching, where a model of an athlete is used to improve the efectiveness, eficiency, and safety of sports training. With my research project, I intend to study this tool and its application to Practical Shooting, a modern shooting sport strongly reliant on traditional training methods. In this paper, I describe my training system, the development progress, and some hypothesis for future improvements.
The application of a Digital Twin (DT) in the field of sport training is known as Digital Twin Coaching
[
Practical Shooting is a sport discipline in which athletes are tasked to engage targets in nonstandardized shooting stages, where a varying number of targets can be placed along a shooting ifeld. A final score called hit-factor is calculated as the sum of the scores on the targets, minus penalties and misses, divided by the time elapsed between the start signal and the last shot.
Shooting fields are usually equipped with barriers that delimit the area in which the athletes can move and partially occlude their sight. Athletes also have some limitations, such as the starting position and weapon configuration, but can otherwise freely decide their trajectory, the sequence of targets, and where to engage them from, making the performance dependent on their strategy. For instance, capable athletes are known to time the reloading operations while moving between shooting positions to reduce the overall time.
Despite being the most recent and innovative among shooting sports, Practical Shooting is usually performed in a simple context with limited technological support and a direct confrontation between the athlete and the trainer. A shooting timer to track shot timings and a camera to record the stage are the only widespread technological aids.
The project’s overall objective is the development of an innovative training system aimed at monitoring athletes’ movements and tracking their training progression in a Digital Twin model, for generating customised training insights. In doing so, I applied techniques of user-centred interaction design to meet the needs and expectations of athletes and trainers.
The term Digital Twin Coaching designates sport applications where a Digital Twin of the athlete is used
to enhance the efectiveness and safety of the training process [
Among the considered scientific sources, no relevant studies concerning Practical Shooting or shooting sports in general have been found. Sport coaching in broader terms was used as reference for this research.
One of the most common tasks performed in Digital Twin Coaching is to monitor the athletes. In
those sports that require the execution of specific movements, the athlete’s gesture can be recorded and
compared with an optimal version to highlight errors [
Among several possibilities, monitoring and generating insights and suggestions are the most suitable uses for this specific discipline.
During the research, I also identified a set of requirements that are usually agreed on but not always
pursued [
The design process revolved around requirements established during the initial literature review and during user research.
Literature review demonstrated the potentialities of Digital Twin Coaching with several possible strategies to improve the training process, guided the sensors selection by showing which quantities are most useful and reporting potentials and limitations of diferent measurement technologies, and ifnally highlighted some open issues in this field, some general for DTs, some specific for Human DTs.
This information was contextualised with the specific necessities of Practical Shooting during a user research, during which I could observe the training process and have unstructured interviews with certified trainers and with trainees. For example, the common task to compare the athlete’s gesture with an optimal gesture is not applicable to this discipline because, for the variability of the shooting stages, it is not possible to define an ideal gesture.
In the development of a Digital Twin-based training system suitable for practical shooting, the following objectives were set: • To monitor the athletes during their activity, employing advanced instruments to create a complete reconstruction of the subject’s pose. By fusing data from diferent sources and by employing a biomechanical model, the movements of the whole body can be reconstructed and the gesture can be represented or animated with greater detail than through a simple recording. • To provide a calendar of past activities, and for each activity, link an analysis and a graphical representation. • To provide an overview of the status of an athlete to the trainer. The Digital Twin data comprises the training data, the physicality of the athlete, their performance in relation to diferent shooting stages, and additional information about the athlete’s daily habits. • To generate training suggestions for the trainer. The system is meant to employ the shooting stage data and the Digital Twin data to generate suggestion classes as if an expert trainer reviewed the stage. Said suggestions are meant to be evaluated and validated by the trainer, who can decide to relay them to the athlete, to modify them, or to reject them.
Training field
Athlete
Inertial Measurement Units Single Channel ECG
Microphone Trainer manual input
Stereoscopic
camera manual input
On-field computing unit Wi-Fi router
Database
Follows an overview of the training system that is being developed, and of the design methodology. Thematically, the system can be described as follows: an instrumented shooting field is provided, where athletes can perform their training while being monitored by a combination of fixed and wearable sensors. The collected data is pre-processed locally and transferred to a database where it is stored for future access. A Digital Twin model of the athlete is used to reconstruct the complete athlete’s kinematics and compute further analytics of the shooting stage. The enhanced data is then processed by a suggestion model that infers suggestion classes. The feedback loop of the training system is completed with two Android applications that function as user interface to access the data. The trainer’s application is also the tool that manages the starting of a training session and allows the insertion of manual data, comprising the stage scores from the coach and additional daily habits from the athletes.
A set of measurement instruments was selected to monitor the athletes during their activity. The selection was driven by the literature, for which quantities and instruments are typically useful in similar studies, including their potentialities and limitations, and by confrontation with the field experts. The instruments were selected to collect significant and qualitative data, with the least possible interference with the gesture of the athletes and with the natural execution of the training.
The final setup included IMUs on wrists, belt, and weapon(s), a single-channel ECG on a chest belt, and an on-field stereoscopic camera for three-dimensional tracking of the athletes. Figure 1 depicts the instruments and their placement.
The shooting field was not only equipped with measurement instruments, but also with tools necessary to manage the flux of operation and the data. Athletes were fitted with a Raspberry PI to communicate with the Bluetooth wearable instruments and mitigate the problem of connection robustness, which is typical for a Human Digital Twin, with a custom-built shooting timer to allow direct acquisition by the Raspberry Pi and with a power bank.
Finally, the shooting field was equipped with a computing unit to control and acquire the camera, to send commands to the athlete’s Raspberry Pi and collect and pack the data, and with a wireless router to create a local network and a connection to the database.
The training system revolves around creating a digital copy of the athlete that represents its current state in terms of training performance. In my system, the model comprises a biomechanical skeletal model of the athlete, historical kinematic, physiological, and performance data.
The Digital Twin comprises classification algorithms to extract gesture information from the Inertial Measurement Units data, to provide insights into the time distribution of the diferent shooting phases and on the strategy eficiency.
Sensor fusion algorithms are being investigated to improve the quality of the camera body tracking by means of the IMUs’ accelerations and angular velocities, to improve the accuracy, reduce noise, and potentially compensate for the occlusions that inevitably happen when recording images from a single point of view.
The core of the Digital Twin is the biomechanical model that represents the athlete’s skeletal system.
The model is implemented in the OpenSim environment, comprising information on the dimensions,
mass, and inertia of the most significant body links, their joints, their degrees of freedom, and their
movement limitations. The model was obtained by modifying a fairly complete model available in
literature [
Through this model, I both extract additional information not directly measured and provide a graphical representation of the athlete’s movements.
In order to autonomously generate suggestions for the athlete, the system includes a Neural Networkbased model that relates the execution data to a set of suggestion classes.
At the time of writing this paper, the model has not been implemented yet, but the implementation strategy is as follows.
Expert trainers, registered in the Italian federation of this discipline, are involved during the on-field shooting sessions to assist the stages of expert athletes. The trainers are asked to provide feedback, corrections, and suggestions as they would give to the athlete during a standard training session. With the supervision of the trainers, the suggestions are clustered in a set of possible classes. I hypothesize that more than one classification will coexist in parallel in relation to diferent aspects of the discipline, but their nature is not known at the moment and will be derived from the experimental activity.
A classification algorithm, possibly a neural network or a family of such, will be trained with the shooting data as input, and the augmented data generated by the athlete’s Digital Twin, and with the suggestion class as label, with the intention of identifying significant relations.
The quality of the suggestions is to be validated by the trainers, who, during usage, will be able to evaluate, accept, modify, or reject the suggestions.
A set of possible interfaces was hypothesized to function as a so-called feedback system between the target users and the training system. In particular, given the necessity to access the data on the shooting ifeld, an Android tablet was selected as a viable medium.
Two diferent applications were developed for athletes and trainers, given the diferent necessities of the two types of users. For both, the target users were involved in the design process, both with an initial questionnaire aimed at understanding which features to implement, and with a final usability and user experience test, during which the users performed a set of tasks under monitoring. Researchers observed the users and recorded the success rate, the elapsed time, the number of errors, and the number of taps for each task. Users were also tasked to compile a final questionnaire structure according to the standard System Usability Scale (SUS) [11], Computer System Usability Questionnaire (CSUQ) [11], and User Experience Questionnaire (UEQ) [12] scales. User experience and usability were evaluated positively across all tests as reported in table 1 The applications were initially developed before the DT was available, so some of the functions were iflled with hypothesized data, and will need to be integrated once the system is completed. Among those, the most innovative function is a 3D navigable reconstruction of the athletic gesture, augmented with the computed analytics.
I also hypothesized, as a further development, to provide a realistic, immersive, and navigable reconstruction of the scene, with annexed augmented reality capabilities on a VR headset.
The training system described in this paper is being developed as the author’s Ph.D. project, currently in its third and final year of development. At the time of writing this contribution, a literature research and a user research posed a set of requirements and objectives, the instrumented shooting field, meant to collect training data, has been designed for the context of this sport and implemented, and two tablet applications were developed with the involvement of the field experts.
The current efort is in terminating the implementation of the Digital Twin of the athlete, and in particular, in implementing the suggestions model.
The project will be completed by tying the output of the Digital Twin with the tablet applications, in order for the overall system to be tested and validated in terms of prediction accuracy, user experience, and usability.
Thanks to Prof. Barbara Rita Barricelli and Prof. Federico Cerutti of the University of Brescia for the support as supervisors in my Ph.D. research project.
The author(s) have not employed any Generative AI tools. [11] J. Lewis, Measuring perceived usability: The csuq, sus, and umux, International Journal of
Human-Computer Interaction 34 (2018) 1–9. doi:10.1080/10447318.2017.1418805. [12] M. Schrepp, A. Hinderks, J. Thomaschewski, Applying the user experience questionnaire (ueq) in diferent evaluation scenarios, in: A. Marcus (Ed.), Design, User Experience, and Usability. Theories, Methods, and Tools for Designing the User Experience, Springer International Publishing, Cham, 2014, pp. 383–392. doi:10.1007/978-3-319-07668-3_37.