Capturing learners' interaction in and with immersive environments is challenging. Interactions relevant to being tracked must be identified, data specifications must be derived, and data collection must be implemented. This does not even include the analysis at the end of the process. Integrating Learning Analytics in immersive applications is often either just part of a small-scope research prototype or flufed up subsequently on already existing systems. In this paper, we describe a diferent approach: Having the opportunity to implement an authoring tool for immersive experiences from scratch, we integrated all required components into the system from the early conceptual stages on. Additionally, we went a step further, as our tool is intended to be used by educators and learners alike. This allows us to gather data and hopefully gain insights into the creation and the experience of immersive learning environments. In this paper, we describe the development process as well as the reasoning behind our data collection decisions on those two levels and elaborate on the insights we envision to thereby enable.
Schools in Germany are increasingly being digitized, driven by the German Digital Pact and additional funding from federal, state, and local governments, as well as school authorities. In recent years, a significant amount of hardware has been introduced into classrooms, including devices that go beyond traditional interaction methods like mouse (or touchpad) and keyboard. The availability of class sets of tablets is no longer uncommon, and VR-compatible hardware is also beginning to appear in some schools or is available for lending from local media centers.
However, there is a lack of accessible tools for creating content tailored to these media. In augmented
and virtual reality, the challenge is that content creation usually involves significant technical barriers,
whether in model development or capturing recordings. Inflexible applications are a software-related
challenge [
The following presents an authoring tool for educators to create interactive and educationally valuable learning media. The web-based tool does not only enable teachers and students to develop their own immersive and interactive learning content for the classroom, but it’s also created with learning analytics directly from scratch. The central medium consists of 360° recordings (photos and videos), which can now be produced relatively easily using action cameras, smartphone gimbals, or (M. Ehlenz); https://elearn.rwth-aachen.de/heinemann (B. Heinemann)
CEUR
ceur-ws.org drones. This open-source tool allows users to process these recordings into interactive tours that can be explored in a browser, on a tablet, or in virtual reality. Additionally, the software ofers features for collaborative editing and the creation of virtual escape rooms. It is designed to track learning progress through integrated learning analytics and ofers the opportunity to investigate collaboration and gamification efects. Due to the collaborative nature of the tool, it is also feasible to use the tool itself to foster constructive learning opportunities in which the students learn by creating their own immersive environments. Therefore, the analytics features are implemented on two distinct levels: The experience analytics allow tour creators and educators to gain insights into the exploration of the virtual environment, such as interaction with hotspots or the times individual users spent on specific scenes; the creation level, on the other hand, allows educators to monitor the work of learners building own tours: How is the division of labor within the group, are there specific tasks and roles or how big are the individual shares of work divided.
Albeit this paper describes an authoring tool for immersive learning experiences, we refrain from reiterating the benefits of immersive learning in this related work section as those can be considered suficiently known within the audience of the LAVR workshop and instead want to focus on the areas specific to our work. This is, on the one hand, the area of authoring tools for immersive experiences and, on the other hand, the challenge of setting a technical frame for sustainable data collection in line with FAIR principles.
The idea of providing authoring tools for (learning) environments in virtual reality is not new. There have already been some interesting developments and approaches. There are numerous products and service providers in the commercial sector, but these are often unafordable for educational institutions. 1; 3D Vista, one-of license for 499 €/device or VR Easy. Therefore, the focus of the research was on freely available solutions. The research included both free platforms from major providers and open-source tools.
On the one hand, there are projects like Google VR (discontinued in 2019) [3] and Mozilla Hubs (discontinued in 2024)[4], which fundamentally allow the creation of immersive content without programming skills. However, these are questionable from a data protection perspective and are not sustainable due to their tight coupling with a single provider, as demonstrated by the discontinuation of the mentioned products.
In contrast, these issues do not arise in the open-source world, but other challenges become apparent. There are very powerful tools such as Blender [5] (usually in combination with Unity or similar), which can be used to create entire immersive worlds. However, these require solid prior knowledge and have a steep learning curve. An example of a more accessible tool is Marzipano [6], which allows the creation of interactive tours with 360° media. It is very user-friendly, and the created content can be used on many devices, such as VR headsets (including Cardboard), tablets, and PCs, making it suitable for various scenarios.
However, Marzipano has a critical drawback, particularly regarding the sustainable creation of educational media. Tours created with the tool must be uploaded to a private server after export. Additionally, re-importing exported results is not supported, significantly limiting the use of its products as an Open Educational Resource (OER).
Learning Analytics in immersive contexts like VR has long crossed the boundaries of traditional learning analytics. While classic e-learning environments like LMSs heavily depend on log-files, Learning 1Example: [2], paid plans for more than 8 users, $50/month (retrieved April 8, 2024) Analytics in VR faces similar challenges as other advances in Multi-Modal Learning Analytics [7]. There is a need for standards and specifications, especially regarding the recent attention the community pays to FAIR principles2. Collected data is not collected for the sole purpose of one-shot studies and buried in archives afterwards, but meant to be revisited, combined, reused and extended. In-house ”standards” used in individual institutions are not up to that task, and many other older specifications like SCORM have a hard time mapping non-LMS learning contexts. Of the currently most popular specifications, IMS Caliper and Experience API (xAPI), the latter is more suitable for immersive learning and has recently even been promoted into standard status by IEEE [9]. xAPI has been successfully used for data collection before, suficiently abstracted and specified with context-specific meta-data definitions and incorporated into analytics toolkits, so it is a suficiently safe choice as foundation for the data collection within the experience [10, 11, 12]. Furthermore, existing Learning Record Stores and Analytics Toolkits/Dashboards reduce the efort that needs to be put into data warehousing and visualization.
Another strong argument for xAPI is its versatility. Beyond being able to record eforts in nontraditional use cases like VR, it is perfectly fine to capture interaction in the authoring tool itself. So by using xAPI, all interaction within the system, even when semantically on two completely distinct levels, we can vastly enhance maintainability and encourage the implementation of reusable components for the visualization of the collected data on the integrated dashboards.
The open-source tool presented here aims to close this gap in available authoring tools and go further. 19squared is freely available3 and actively developed. The following sections provide an overview of the technology stack, current features, and ongoing developments. Initial case studies in interdisciplinary contexts have already been conducted but are not described in detail here.
Sustainability, usability and openness are the prioritized criteria for the chosen technologies. In addition, broad interoperability is sought. For these reasons, web technologies (browser-based) were chosen for both the product level of the interactive tours and the authoring tool itself, using open-source libraries wherever feasible and sensible.
The application’s core is the 360° view, which can be used across platforms via WebVR [13]. As a fallback, the application can be operated on PCs with a mouse or touch display, although the immersion is noticeably greater on other devices. Gyroscope sensors are utilized on mobile devices, allowing users to look around by moving their devices. Active head movement in a VR headset provides an even more immersive experience. The use of A-Frame as a rendering framework for WebVR enables the integration of interactive hotspots and ofers further development opportunities, such as embedding 3D objects.
The interface for creating and editing interactive excursions is deliberately kept simple. Complex web frameworks are currently avoided. Only jQuery as a JavaScript convenience solution and Bootstrap as a (S)CSS framework are included. On the server side, the tool is also based on open technologies. Python is used with the Flask framework, the database is connected via SQLAlchemy, and MariaDB is currently used in a bundled container. Docker files are provided to facilitate deployment in a container environment, which is currently the preferred way of usage. Collaboration features are implemented with Socket.IO, using a Redis Adapter with bundled Redis instance to maintain scalability and allow concurrent request handling. Asset delivery is handled by a separate container from the system’s business logic, and a load-balancing reverse proxy ensures the interplay between all components. Diferent docker-compose files provide suitable settings for either a Linux server environment or development on either Windows, Linux, or MacOS machines. The deployment process is reasonably 2FAIR stands for Findable, Accessible, Interoperable and Reusable [8] 3Repository at https://git.rwth-aachen.de/medialab/interactive360vr, public instance at https://19squared.de well documented, and a small-scale evaluation with users familiar with Linux-based servers but not with 19squared showed that an average admin can deploy the system within 15 minutes on a current vanilla minimal Debian system. The system has been load-tested with Locust4, and we currently estimate that a moderately sized server (8 cores, 16 GB RAM) can handle up to 1.500 concurrent users given suficient bandwidth. Compatibility with professional content delivery networks has been tested 5 and documented, as well as Zero-Trust approaches like Cloudflare Tunnels for serverless deployment 6. Video tutorials for various deployment approaches have been produced or are currently in the process of production to minimize the threshold for educational institutions without dedicated IT staf.
19squared is a continuously evolving product, tool, and subject of teaching and research. It is already in productive use and has been utilized in several projects across all stages of teacher training as well as in schools and higher education. At the time of submission, four student projects are focused on further development of the features. The described state reflects version 2.6.12, with the expectation that even more features will already be tested and available by the time of the conference.
When launching the web application, users can log in or create a user account, which they can then use to create new tours. Figure 1 shows a screenshot of the scene editing view of the authoring tool. On the left side, uploaded media can be found; in the center is the 360° view, and on the right side, there is an overview of the already-created scenes and associated hotspots. These can be freely placed in the scene and display additional text, images, videos, or audio. It is also possible to link scenes via corresponding hotspots, creating complete excursions. A public share link can be generated anytime, allowing access without a user account. Additionally, created tours can be exported, imported by other users or into other instances, edited further, and distributed as OER. Shared tours can be copied and edited, allowing teachers to create a base of media pages and hotspot content and distributing it to the students in-platform to let them finish the details and editing content. The recent updates brought additional features that added significant educational value with more interactive elements: quizzes and combination hotspots can be used to check learning progress, but combined with the recently added conditional settings for hotspots, virtual excursions are more than just virtual walkabouts. Cleverly used, users can create immersive digital escape rooms in line with the current trend of edu-breakouts. The current two interactive hotspot types are considered proof-of-concept for such features and act 4https://locust.io/ 5Tested with Cloudflare on a free plan, be advised that there is a hard upload limit of 100 MB per file in this setup 6I.e. for safe execution on a teacher’s workstation for school usage. as a blueprint for additional types like pair-wise matching, point-and-click riddles, or more complex implementations. Based on which hotspots have been visited or successfully completed, other hotspots will show up or get unlocked.
To facilitate collaborative learning, 19squared implements some features that make it unique in its ifeld: Virtual tours can be shared both read-only or with editing privileges. The latter allows groups of students to co-construct the immersive environments in real-time, even remotely, on separate devices. Added materials will show up immediately, as will any changes in the virtual environment; all assets and scenes can be annotated within the editor, and a tour-specific group chat is available within the user interface.
Learning Analytics is a broad field and can address, as such, the needs of many diferent stakeholders. Currently, the implementation in 19squared focuses on two of them: Tour creators and educators, which in turn will consequently (hopefully) benefit the learners as well. In this section, we will first describe the technical details of data collection, then specify the collected data, give a glimpse of the current visualizations, and will conclude with our long-term vision of actionable results.
Implementing an authoring tool for immersive learning environments from scratch opens a unique opportunity for integrating Learning Analytics. Especially the fundamental decision to create a tool that can be used for co-constructive knowledge building7 by targeting learners with the experiences and the authoring tool alike facilitates new perspectives onto the learning process. Thus, the decision was made to implement xAPI-based data collection for both the creation and exploration of virtual tours in parallel.
Still, as this project is based within the European Union, taking into account privacy and data protection is not only morally mandatory but also required by law. Thus, and with deployment in the specially protected environments of classrooms in mind, special measures had to be taken to satisfy those needs.
In consequence, users can always decide to opt-out of data collection with every login. Experiencing created environments is possible without logging in, in that case the session is logged using a specifically generated pseudonym not traceable to an individual being. Those are the defaults used for the public instances, as it is provided by the university and as such users declare themselves implicitly in agreement with this data being used for research purposes.
Beyond the public instance, users are free to configure their setup to their specific needs, from forced collection of everything to no logging at all. This is done via the .env file during deployment, so it is completely in the hands of the deployer. We are currently testing bundling a pre-configured lightweight LRS in the docker package to make Learning Analytics in school-scope deployments more accessible.
Practice showed that collecting xAPI statements for all interactions is quite challenging: while it is (somewhat) trivial for tour editing, which is essentially a classic point & click and drag & drop web application, the tour experience posed more of a challenge. The interaction with the website is event-driven, and the keyboard and mouse are pretty easy to handle. In contrast, the immersive experience opened potentially relevant continuous data streams from HMDs or gyro sensors within the tablets. Currently, we record significant changes in this data as ”rotated” events, which, therefore, constitute the majority of data points within our LRS. One option to solve this might be to increase the threshold triggering the change event; for now, we decided on a filter option on our dashboards to exclude this type of event. 7Co-constructive learning analytics development is needed for pedagogically meaningful use of learning analytics [14]
There has been a lot of work done on making data collection for Learning Analytics in Virtual Reality FAIR, and part of those eforts is the metadata definitions for xAPI data collection the Learning Technologies Research Group, which are hosted within the registry provided by that same institution8 [10]. To follow the Open Science spirit here and encourage the reusability of collected data, the definitions from the already defined VR context have been applied where suitable. Everything not yet covered by pre-existing definitions within the registry has been added in its own context to provide a foundation for further work on authoring tools based on 360-degree media. Figure 2 shows an overview of those new definitions.
Until now, the focus in regards of Learning Analytics was primarily on the end of data collection, as the project is currently mainly driven by the evident need for an open source authoring tool and not embedded within the context of any (funded) research project. Thus, there are currently no formally ifxated research questions that drive analysis or visualization of collected data. Instead, we consider our overall research and development goal to provide a toolset enabling educators from all domains to utilize the collected data for their specific purposes. So the (meta) research question could be phrased as ”What are essential requirements to facilitate interdisciplinary research using Learning Analytics in an immersive authoring tool?”.
In consequence, we try to map all interactions within the system to datasets within the Learning Record Store, but the actual integrated visualization of those datasets is to be considered a proof-ofconcept without special focus.
Nevertheless, we implemented two dashboards of similar look and feel for the two layers of learning analytics, respectively. In both cases, we added the option to filter for individual users or tours, constrain the timeframe, or show only specific device types. In both versions, there is a single form of activity that dominates the records: In experiencing, it is the rotation resulting from users looking around the virtual scene; in authoring, it is accessing media pages. Thus, both are usually excluded from the visualization (as they currently generate only little added value) and can be included by checking a dedicated box. Additionally, we found it pretty helpful in the exploration of the recorded data to have a side-by-side view of the individual dashboards with diferent filter settings. So, instead of juggling two separate browser windows, we implemented an ”interactive comparison” version for both modalities as seen in Figure 3, providing the same dashboard side-by-side with synchronized scrolling.
The LA component is currently part of the standard deployment and can be easily configured on deployment. A problem with the present implementation is the data polling strategy ”poll everything, iflter locally”. This strategy principally helps keep the interface responsive when applying filters, but it does not really scale for large deployments and will be adjusted soon.
Furthermore, the current implementation is used on the administrative/researcher level, as there is no distinction between users and groups. The long-term goal is to make Learning Analytics available to the teacher with restrictions to their associated students. The groups are already part of the user rights management and integrated into the system, though not yet applied in the data visualization end.
Not having a research context yet has not kept us from envisioning interesting research questions we would like to investigate in the future. At best, they lead to actionable results, but they should at least provide some additional insights into learning within immersive environments.
First, we focused on possible research questions regarding the collaborative creation process where students build an interactive virtual environment together, as those questions might be of immediate interest to the accompanying teacher. So one of the leading questions whenever group work or CSCL is involved is: How is the division of labor? Is it scene-based (individuals annotate a scene each with all relevant hotspots and information), or is it role-based (e.g., one individual connects scenes and creates paths while another is responsible for annotation with textual hotspots)? It might also be interesting when work is done: Deadline-driven last minute in bulk or evenly distributed across the task period? Do students work synchronously or asynchronously? Which features are actually used, is there any annotation done, is the tour chat used, and in which order is tour creation done? Especially the latter questions also pose a viable research interest regarding the continuous work on the tool itself and help guide the development team in the next steps.
Concerning the actual tour experience, the tour creators (mostly educators) might question whether their created learning experience works as intended: Have all relevant materials been found? How long are the gaps between individual hotspot interactions? How long do learners remain in specific scenes? How often do switches occur? Are some hotspots revisited? Is audio/video completely consumed or prematurely stopped? How many attempts do learners need to solve integrated riddles? Which paths do learners take when exploring the virtual environment if multiple alternatives are given?
Actually, a third layer of Learning Analytics is integrated into the current project, though not specific to this tool. We use another xAPI aggregator to monitor the use of GitLab features in this repository, looking into how developers of this project interact with features, wiki, branches, and merges, but this is a diferent tale for a diferent paper.
Despite the advancements and potential of 19squared, several open challenges remain, both technical and organizational, that need to be addressed to ensure the long-term success and impact of the tool.
The development of 19squared serves a broad range of stakeholders, including educators, students, and researchers, each with diferent expectations and needs. One of the ongoing challenges is harmonizing these needs and ensuring seamless interplay between these groups. For instance, learning analytics operates at multiple levels, from tracking user engagement during virtual tour exploration to monitoring collaborative eforts in content creation. Ensuring these analytics provide actionable insights for all stakeholders—particularly teachers—remains a priority.
Long-term usability and scalability are critical challenges, particularly as 19squared transitions from its current prototype state to widespread adoption. While quotas are currently employed to manage resources, concepts like streaming servers or distributed databases for media delivery are under development to support larger-scale production operations.
Moreover, some functionalities will likely remain within the domain of commercial providers, including a multi-user simultaneous exploration of VR environments or the inclusion of 3D-modeled environments or 6-DoF applications. The provision of native apps for specific systems is also not currently planned.
Managing data responsibly, particularly in educational contexts, is another ongoing challenge. While group management is already possible, the next step involves implementing consent management features that allow teachers to oversee fine-grained data collection. This includes enabling teachers to create student accounts without requiring email addresses—a frequent request.
The long-term vision involves granting teachers access to their own data and the data of their associated students while ensuring anonymity and privacy in aggregated outputs. This would include retrospective revocation of consent to give users full control of their data, aligning with the principle of ”data ownership” and fostering trust among users.
Perhaps the most significant challenge is building a sustainable community around 19squared. While feedback from early adopters has been overwhelmingly positive, emphasizing the need for an opensource tool tailored to education, achieving critical mass remains an issue. A community of contributors, educators, and researchers is essential to sustain development, gather new ideas, and ensure the tool remains useful and relevant.
Thanks to all our students, who contributed to the development and evaluation of 19squared. Thank you for your enthusiasm and hard work, which have significantly advanced the potential of 360° material and virtual reality in education.