Game Learning Analytics (GLA) collects game player interaction data that can be analyzed to offer insights into player behavior and support stakeholders such as developers and educators in understanding the learning process. However, current GLA methods are usually complex and costly to implement, with proprietary data formats and software, and too technology-dependent, limiting their applicability in real scenarios. To address these issues, we created SIMVA (SIMple VAlidator), an open-source, standards-based platform designed to simplify GLA applied to serious games. SIMVA uses the IEEE xAPI standard to collect and store data, facilitating player tracking and basic analysis/visualization. Our approach also allows for easy integration of various data storage systems (i.e. Learning Record Store) and different trackers, offering full data ownership to stakeholders. SIMVA also enables and simplifies game validation processes, supporting different experimental designs (e.g., control groups, A/B testing) and integrates pre-post questionnaires linked to player analytics data via anonymized codes. This integration supports the analysis of both quantitative and qualitative data, enhancing the assessment of the educational impact of games. Additionally, SIMVA is designed to simplify compliance with EU GDPR data privacy requirements. Overall, SIMVA offers a comprehensive open-code platform for both GLA and serious game validation. SIMVA will contribute to improving game effectiveness and to provide credible evidence of serious games educational value.
Games are complex software systems designed to entertain, typically by integrating elements such as rules, goals, reward mechanisms, and challenges within a responsive and interactive framework (usually supported by a narrative). When these components are effectively combined through thoughtful design, they create engaging experiences for players[1]. Many of these elements have also proven effective in educational contexts, giving rise to the concept of serious games[2].
In education, those serious games have shown significant potential as tools that motivate learners [3], provide experiential learning with immediate feedback[4], and encourage active engagement in the learning process[5]. However, despite their proven benefits, the use of games in the classroom remains limited. This is largely due to several challenges, including difficulties in assessing student performance within games, technical issues related to deployment and classroom hardware, limited digital skills among teachers, and a lack of conclusive evidence on the effectiveness of educational games [6], [7]. Many of these barriers can be addressed with appropriate tools that better support teachers, enable real-time classroom monitoring and provide reliable assessment methods [8].
00009-0003-2520-507X (J. Santilario Berthilier); 0000-0001-8701-7582 (A. Calvo Morata); 0000-0001-6595-5690 (I. Martinez Ortiz); 0000-0002-5094-4523 (M. Spaniol); 0000-0002-8200-6216 (B. Fernandez Manjon)
One promising approach to supporting the use of serious games in the classroom is the application of Game Learning Analytics (GLA) [9]. GLA involves collecting and analyzing data on players’ interactions within a game to gain insights into both learner behavior and the actual learning process. This information can be used by different stakeholders to improve the learning experience, optimize educational content, evaluate game effectiveness, and assess overall learning outcomes. By using interaction data, GLA enables more informed instructional decisions, and supports teachers by providing actionable data to assess student progress and address learning challenges in real time. Additionally, it contributes to the continuous improvement of both teaching strategies and game design, it allows developers and researchers to validate educational game design and effectiveness.
Despite all the proven advantages of applying analytics, Game Learning Analytics (GLA) remains a complex, costly, and often fragile process that poses significant implementation challenges for many game developers. Consequently, most educational games are still evaluated using traditional methods such as interviews and self-reported questionnaires. Furthermore, existing GLA solutions are typically ad hoc and proprietary, limiting their adaptability and reusability across different games and educational contexts. To address these challenges, we propose SIMVA, a standards-based, opensource platform that provides a suite of tools to simplify the application of GLA, as well as the design, deployment, and validation of serious games through experimental methodologies.
Our open-source platform, SIMVA (SIMple VAlidator), is designed for third-party deployment on local servers to collect and analyze player interaction data. It is a complex software infrastructure, but the idea is that it can be easily installed from our GitHub repository2 using a Docker-based infrastructure and with a turn-key approach (see Figure 1).
SIMVA integrates a set of software components to streamline the application of Game Learning Analytics (GLA), including: a reverse proxy (Traefik), a Single Sign-On service (Keycloak), a survey manager (LimeSurvey), a data storage system for player interactions (MinIO), a trace processing engine (Kafka), a URL shortener (Shlink), our main web application (Simva), a trace reallocator (Simva Trace Allocator) and a basic data analysis and visualization tool (T-Mon). We also provide trackers (in Unity and JavaScript) to simplify the data collection by communicating in-game interaction data and where data move from or to SIMVA. By capturing detailed player interactions and combining them with qualitative data from surveys, SIMVA enables a comprehensive assessment of learning outcomes and player experience[10].
Figure 1 shows a diagram of the infrastructure and summarizes how each of the clients that can be accessed by users works.
Dashboards (green color): SIMVA provides students, teachers, researchers and developers with a visualization system (TMON) to explore and analyze the data generated by the players. Through this framework (1), it is possible to visualize the list of available activities (2) and obtain information and visualizations that allow stakeholders to understand the data collected in each learning activity (3), but only if the stakeholder is owner of this activity and study data (researcher, teacher or developer) or is the user who generated the data (student). 2 https://github.com/e-ucm/simva-infra/ Figure 1 : Simva Infrastructure and flow of data.
Game clients (blue color): SIMVA provides telemetry libraries (trackers) that games can integrate to record player actions. This makes it possible to establish a direct integration between the game and the platform (1), to trigger surveys before and after the gameplay session (2,4) and to send interaction traces while students play (3). This data generated by the surveys and games will be displayed on the dashboards. Only students with studies permissions via Keycloak can send data from the game to SIMVA. In this way, valuable data is collected to assess student performance and to evaluate the effectiveness of serious games. While SIMVA specializes in integration with serious games, it can also be integrated with other types of activities, whether digital or not.
SIMVA client (purple color): Designed for teachers and researchers, this client facilitates the creation and management of game sessions. It allows organizing class groups and generate tokens by Keycloak (1) allowing anonymization of user information at source. Users can obtain the token lists for participants (2), access to the results of the activities carried out (4) and generate short links that simplify the distribution and access to the games in the classroom (5). In addition, it offers functionality to configure pre- and post-game surveys (3). Pre- and post-questionnaires evaluations together with the interaction data strengthen the validation of educational games.
The following sections describe in more detail what each of these clients allows.
SIMVA simplifies the validation of serious games [11] because it supports various experimental designs, including control groups and A/B testing. It also enables integrated data collection from multiple sources, such as gameplay, pre- and post-questionnaires or other types of educational resources such as articles, online presentations or videos with lessons. All data is linked to a player through a unique (anonymized) learner token, ensuring consistent and integrated tracking across activities.
Gameplay activity, which is the core of the experiment, where students interact with the serious game. While they play, SIMVA collects data on their in-game actions. Teachers and researchers are able to monitor the learning process using an analytic panel and store the data securely for later analysis - either in MinIO, as users backup or both, depending on the configuration shown in Figure 2. By setting the Game Uri parameter, the system knows where to redirect students and use this url to track and associate player data correctly. The two main types of activities, but not the only ones, are those related to surveys and games.
The Survey Activity component encapsulates LimeSurvey3, a free and open-source software tool for creating and running online surveys. LimeSurvey strimelines tasks such as questionnaire deployment, participant management, status monitoring, and automated player response collection. To ensure flexibility and efficiency in survey management, the system provides multiple methods for survey preparation:
Sometimes teachers or researchers want to reuse a survey that they have already designed and tested. This approach ensures consistency across experiments and allows comparison between studies. SIMVA supports this by allowing direct selection of existing surveys, either by knowing the unique identifier (ID) of the survey, by choosing from a list or by duplicating the existing test for another class, copying also the survey as shown in Figure 3. 3 https://www.limesurvey.org/
In some cases, stakeholders need to create new surveys specifically designed for the current study or context. This approach enables the inclusion of customized questions that align with the learning goal of the experiments.
When surveys have been developed outside of the system, using a standardized template or prior survey, they can be imported directly. Those pre-prepared survey designs imply less work for the stakeholders, save time and ensure the carefully validated instrument can be reused across different experimental settings.
Furthermore, all data collection is conducted anonymously to ensure compliance with European data privacy regulations, such as the EU GDPR [12], [13]. This is achieved through the use of anonymized tokens generated by the SIMVA platform (as shown in Figure 4), which are distributed to students via printed token reports.
A tracker is the software component that deals with the communication of the player interactions (traces) to a permanent external storage to enable analysis of behavior and learning outcomes. Two trackers are available: JSTracker, implemented for JavaScript, and Xasu, designed for use with Unity.
Both trackers support and implement xAPI standard so they can be used by a game programmer for sending out traces without requiring knowing the xAPI standard. xAPI is an international IEEE standard that provides a JSON format to track user actions. More specifically, the trackers support and implement an application profile of xAPI specially designed for representing relevant events in serious games [14].
Adopting the xAPI standard for representing interactions, along with a flexible platform for building custom components, greatly enhances the reusability of Game Learning Analytics (GLA) data [15]. It allows stakeholders to create, reuse, or modify system components while retaining full ownership of their data. For example, various trackers can be reused across projects, and new trackers can be developed to support different game platforms.
Xasu4 is a Unity plugin that includes a tracker capable of storing player interaction data either locally or on a server. It offers straightforward tracking functionality and flexible tools for configuring the collection of xAPI traces in a robust manner. Xasu can also send backups of interaction data to the server. To connect Xasu to a specific research study, users can download theXasu Config from the SIMVA platform, represented as a blue dot cube in the gameplay activity (Figure 5). This configuration file provides the necessary settings to Xasu and automatically launches the game with the correct setup.
JsTracker5 is a tracker designed for JavaScript-based web games. When integrated into a game, it sends player interaction data to the server and can also create a backup of this tracking data. To 4 https://github.com/e-ucm/xasu/ 5 https://github.com/e-ucm/js-tracker/ streamline the experimental research process, the game’s URL can be added to SIMVA, allowing the scheduler to launch the game at the appropriate stage of the experiment (e.g., once the user has completed the pregame survey).
If the trace Storage option has been activated in GamePlay activity shown in Figure 6, the data is stored in our database Minio as a json file - one json trace per line- accessible and downloadable by the stakeholders in the SIMVA web study page. If the backup was activated, there is an extra copy of that information to make the process more resilient to possible connectivity problems (this is quite frequent in actual school environments where we test our games).
To improve interoperability, SIMVA allows for using a standard xAPI data store that is usually known as a Learning Record Store (LRS) in xAPI vocabulary. From the game the data can also be sent to an existing LRS like Veracity6 or ADL LRS7.
Storing in-game user traces using the xAPI application profile for serious games (xAPI-SG) streamlines data analysis [16]. T-Mon8 is a lightweight data analysis and visualization tool specifically designed to process data in the xAPI-SG format[17]. It can load xAPI-SG statements from a JSON file (either a single file or one JSON trace per line) or directly from SIMVA storage via a connected user account. T-Mon then analyzes the data and generates a default set of visualizations, offering a quick overview and facilitating the creation of charts to represent game-based learning data. Originally developed as a Jupyter Notebook9, T-Mon is currently being migrated to a web-based 6 https://lrs.io/home/ 7 https://lrs.adlnet.gov/ 8 https://github.com/e-ucm/t-mon/ 9 https://jupyter.org/ Python application using Dash10, to improve performance and streamline report generation, particularly for large datasets.
Additionally, users have the option to download the data and perform custom analyses or create their visualizations outside of SIMVA using their preferred tools.
SIMVA significantly streamlines the process of validating serious games, offering an integrated, standards-based platform that reduces the technical and methodological barriers often faced by researchers and developers. Traditionally, validating educational games requires complex coordination across data collection, participant management, assessment, and analysis— usually relying on fragmented or proprietary tools. SIMVA addresses this by providing a unified, open-source infrastructure that supports the entire research workflow: from deploying controlled experiments and capturing in-game behavior, to integrating survey data and generating actionable insights through built-in analytics tools like T-Mon.
By simplifying these processes, SIMVA helps educators, researchers, and developers to conduct serious games with rigorous, reproducible studies with minimal setup, while maintaining full ownership and control of their data. It allows for the creation of well-structured experimental designs, automated complex data collection, and standardized trace formats (xAPI-SG) making comparing results across studies and contexts easier.
Therefore, SIMVA lowers the barrier to entry for serious game validation, accelerates the production of reliable evidence, and, in the long run, can help to promote the wider adoption of serious games as credible, data-driven tools for learning, training, and skills development.
This work has been partially funded by the Spanish Ministry of Education
During the preparation of this work, the author(s) used OpenAI ChatGPT-4 and DeepL in order to: Grammar and spelling check. After using these tool(s)/service(s), the author(s) reviewed and edited the content as needed and take(s) full responsibility for the publication’s content.
10 https://dash.plotly.com/