Roland Klemke, Khaleel Asyraaf Mat Sanusi, Daniel Majonica, Anja Richert, Valerie Varney, Tobias Keller, Jan Schneider, Daniele Di Mitri, George-Petru Ciordas-Hertel, Fernando P. Cardenas-Hernandez, Gianluca Romano, Milos Kravc k, Benjamin Paa en, Ralf Klamma, Michal Slupczynski, Stefanie Klatt, Mai Geisen, Tobias Baumgartner, & Nina Riedl: Proceedings of the 1st International Workshop on Multimodal Multimodal Immersive Learning Systems. At the Sixteenth European Conference on Technology Enhanced Learning: Technology-Enhanced Learning for a Free, Safe, and Sustainable WorldO. nline, Bozen-Bolzano, Italy, September 20-24, 2021, CEUR-WS.org/Vol-2979, ISSN 1613-0073. © 2021 for the individual papers by the papers' authors. Copying is permitted for private and academic purposes. Re-publication of material from this volume requires permission by the copyright owners.
Address of rst editor: Khaleel Asyraaf Mat Sanusi Cologne Game Lab - Cologne University of Applied Sciences (TH Koln) Schanzenstr. 28, 51063 Koln, Germany ks@colognegamelab.de
Advances in the related elds of wearable sensors, virtual/augmented reality, and arti cial intelligence make it possible to connect these technologies into integrated learning solutions. The intersection of these elds of emerging technologies is an area of many opportunities for innovative learning systems, but likewise a eld of fuzzy expectations. With this workshop, we contribute to the systematic organisation of the eld and to the advancement of solutions.
The MILeS (Multimodal Immersive Learning Systems) 2021 workshop was organised in the context of the German BMBF-funded research project MILKIPSY (Multimodal Immersive Learning with Arti cial Intelligence for Psychomotor Skills), which aims to develop AI-supported, data-intensive, multimodal, immersive learning environments for the independent learning of psychomotor skills. This leads to a cross-domain approach that makes it possible to record the activities of experts in a multimodal manner and to use these recordings as blueprints for learners. With the help of AI-based analysis, learning progress is to be supported by automated error detection and automatically generated, individual feedback. This creates holistic, innovative environments for cultivating psychomotor skills, in which personalized AI support enables individual learning processes based on complex data analyzes.
With the interdisciplinary workshop, we brought together experts and practitioners from technology-enhanced learning and educational application of technologies to collect ideas, requirements, best practices, and example cases in the intersection of Arti cial Intelligence, Multimodal Systems, Immersive Systems, and their application into actual education. With this context in mind, in this rst edition of the International Workshop on Multimodal Immersive Learning Systems (MILeS 2021), we have compiled eight research studies that go from early stages of development to present empirical studies where novel experimental designs, theoretical contributions, and practical demonstrations. MILeS 2021 took place on September 21st, 2021, and was run virtually in conjunction with the Sixteenth European Conference on Technology Enhanced Learning (EC-TEL 2021). Following are the core topics for this workshop: { Mobile, wearable, and pervasive technologies { Sensors, sensor networks, and Internet of Things { Augmented reality, virtual reality, and mixed reality { Arti cial intelligence
The website of the workshop can be found here https://milki-psy.de/milesworkshop/.
A peer-reviewed process was carried out to select the workshop papers. At least three members of the Program Committee with expertise in the area reviewed each paper. As a result, the following eight submissions were accepted, which discuss ideas and progress on several interesting topics: { Paa en et al. sketched a machine learning approach as an early signi cant step into providing feedback to the learners in the domains of running and human-robotic interaction. The authors evaluated how movements can be compared to highlight the variations between the student and expert movements. { Cardenas et al. proposed the existing Multimodal Learning Analytics Pipeline to be applied in the domains of running and human-robotic interaction through the process of data collection, storage, annotation, preparation, and exploitation. Suitable sensors that can be potentially applied in the two application cases are also suggested. { Slupczynski et al. proposed a cloud-based architecture as a basis for an AI-based multimodal training and learning environment. The article discusses some related work on service-based architectures for machine learning, blockchain, and learning analytics. { Quin et al. introduced a prototype that utilizes holographic technologies to foster natural interactions between teachers and students in an online classroom setting. The paper described the system architecture of the prototype, two di erent 3D modeling techniques, and students' experience. { Keller et al. presented an approach on how to develop a design framework for an augmented reality-based training system for the acquisition of psychomotor skills, more speci cally how to teach workers to properly collaborate with a robot (i.e., human-robot interaction) and how to keep them motivated along the way. This is done by exploring the results of three interdisciplinary workshops. { Dikken et al. proposed a framework for a handwriting learning system that compares a learner trace to an expert trace via dynamic time warping and then performs an error classi cation based on the remaining deviation compared to the expert. { Mat Sanusi et al. presented two student prototypes in di erent application areas related to multimodal learning of psychomotor skills in immersive environments. The prototypes are presented in detail and possible future optimisation processes within the framework of MILKY-PSY are discussed in order to ultimately determine the feasibility and advantages of psychomotor learning with AI-based on both use cases. { Geisen et al. proposed a study design to enable subjects to optimize their psychomotor training performance with real-time feedback in an immersive training environment. In the context of this paper, a squat exercise was selected. The authors aim to compare di erent feedback methods that can be given in real-time and identify the most suitable feedback for motion learning and optimization.
MILeS 2021 workshop aimed at gathering new insights around the use of Arti cial Intelligence, Multimodal Systems, and Immersive Systems for education and learning leveraging multimodal data sources. With the broad spectrum of submissions received, the rst edition of the MILeS workshop showed an interesting perspective on the eld of multimodal immersive learning and highlights the relevance of this emerging multidisciplinary eld of research, that connects expertise in computer science, arti cial intelligence, human-computer interaction, sensor-based systems, educational science, technology-enhanced learning, and game design. We expect that the rst edition of the MILeS workshop sets the foundation for a continued and growing series of workshops and publications in the eld.
The MILeS 2021 chairs would like to thank the authors for their submissions and the EC-TEL workshop chairs for their advice and guidance during the MILeS workshop. The MILeS chairs also served as Program Committee that reviewed high-quality reviews for the received submissions.
The following project has supported the organisation of the MILeS 2021 workshop: Multimodal Immersive Learning with Arti cial Intelligence for Psychomotor Skills (MILKI-PSY) funded under the grant number: 16DHB4013 by the Federal Ministry of Education and Research (BMBF).
German Research Center for Arti cial Intelligence (DFKI)
Benjamin Paa en, Milos Kravc k . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Fernando P. Cardenas-Hernandez, Gianluca Romano, Hendrik Drachsler . 15
Michal Slupczynski, Ralf Klamma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
Tristan Quin, Bibeg Limbu, Michel Beerens, Marcus Specht . . . . . . . . . . . . . . . 29
Tobias Keller, Valerie Varney, Anja Richert . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
Olivier Dikken, Bibeg Limbu, Marcus Specht . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
Khaleel Asyraaf Mat Sanusi, Daniel Majonica, Lukas Kunz, Roland Klemke 53
Mai Geisen, Tobias Baumgartner, Nina Riedl, Stefanie Klatt . . . . . . . . . . . . . 59