Learning Analytics Dashboards (LADs) are tools designed to visualize data on student behavior, performance, and engagement to support learning and teaching. Whilst their adoption has increased significantly in recent years, users often struggle to interpret the visual information they provide. To address this, recent research has begun incorporating Data Storytelling elements to enhance understanding. However, generating such narratives at scale remains a complex challenge. In this paper we present the work we have carried out to explore the use of Generative AI to facilitate the scalable creation of data-driven narratives. We present findings from three distinct studies and discuss the conclusions drawn from them.
Learning analytics dashboards have gained popularity as tools to provide teachers with insights
into the learning process of their students [
Building on a classic dashboard that visualizes various types of charts related to individual
students or groups[
RQ3: How significantlydoes providing the course curriculum design improve the alignment of AI-generated narratives with the educational objectives of teachers?
We have conducted three studies to analyze the use of Generative AI for enhancing dashboards with data storytelling features. Next the description and main results of each are presented.
The objective of the first study[
The study followed a four round process as summarized in Table 1. For each of the rounds a prompt generation/refinement step was carried out followed by an output analysis step.
The main objective was the test of different general prompts using as source data only the chart shown on the dashboard. The ouput was analysised to check whether the output contained errors or hallucinations.
It was detected that the GenAI generated hallucinations because it was not able to correctly identify the information contained in radar plots or bar plots when the charts included more than 20 items.
The main objective of this round was to eliminate the hallucinations.
To this aim, the prompt and the chart were enriched with text data regarding the assessment information of items. With this, hallucinations were eliminated.
The objective of this round was to work with the general prompt to improve the output in terms of clarity and pedagogical usability.
The generated narratives tended to be excessively long and included information on diverse aspects, some of which were not always relevant to the teachers.
The objective of this round was to generate and refine distinct prompts tailored to the diverse needs of teachers.
The results were considered satisfactory according to the participant teachers, and the prompt generation process was concluded.
As a result of this process, three distinct prompts were developed, each addressing a specific aspect of chart analysis and allowing each teacher to access directly the information he or she needs:
Interpretation of the data shown in the chart, both assessment item by assessment item and overall.
Based on these prompts, each chart of the dashboard was enhanced with three buttons, allowing to access these narrative elements directly (as illustrated in Figure 1).
The second study involved 15 teachers and aimed to evaluate the pedagogical usefulness and relevance of the enhanced dashboard obtained after the first study. Participating teachers were provided with charts generated from real student data and GenAI generated narratives for the three aspects identified in the previous study. After, they filled in a questionnaire to evaluate the narratives.
Overall, the results were very positive. According to the collected data, the narratives proved valuable in helping teachers understand and interpret the charts. The attitude of teachers towards the use of the enriched system was very positive.
One of the detected problems is that the narratives introduced sentences such as “This suggests a relatively solid knowledge in this topic” or “It would be useful for him to strengthen his knowledge in analysis and design”. Since topic information was not provided at any point, the GenAI relied on the names given to the assessment items – such as “analysis and design exam” – to infer the underlying subject matter, which may introduce ambiguity or inconsistency.
Therefore, one of the main findings from this second study was the importance of linking the assessment items displayed in the charts to the specific topics or competences they target, in order to make the narratives more useful and better aligned with the pedagogical goals of the teachers.
This aspect was specifically addressed in the third and final study presented in this paper.
The main objective of this last study [
The first step for this study was the definition of an ontology to formalize domain and student data (see Figure 2). The defined ontology formalizes the domain model for the curriculum structure including its competences, learning units, learning materials, and other related elements, as well as the student model with the student-related information. The defined ontology was after used to populate prompt templates with course curriculum design information and assessment related data from the students.
A validation study was conducted with two teachers and data from 60 students in a course to assess the alignment of the generated narratives with the pedagogical intentions of the teachers. We selected a sample of the students and for each of them the narratives were generated providing the system with the prompt template enriched with the student assessment data (see data example in Figure 3) which included also the information regarding the course design.
Figure 3: Extract of the assessment data of one of the students.
Once the narratives were generated, the participating teachers evaluated their level of alignment with the generated narratives. Preliminary results show that, when the information about the assessment results for the students is complete, the conclusions and recommendations generated by the Generative AI were well aligned with the pedagogical goals of teachers and provided actionable recommendations for teachers. Even if the number of participants was very small, the results of this last study were highly promising. 3. Conclusions and Future Work This paper has presented the process followed to generate a dashboard enhanced with Generative AI driven data storytelling features. Three studies have been conducted to design, refine and validate this proposal.
The results obtained thus far are very positive. The results of the first two studies have shown that the use of the generated narratives can contribute to the enhancement of the visual literacy of teachers (RQ1). Also, the attitude of teachers towards the use of the Generative AI (RQ2) has been very positive. However, the study of this aspect must continue including users from more diverse disciplinary backgrounds, as all the participating teachers had technical profile, which may have influenced both their interaction with the system and their attitude towards it.
The results of the last study where the curriculum design was provided have been very positive, showing, in general, a strong alignment between the generated narratives and the pedagogical objectives of teachers (RQ3). Only the situation in which there is data missing (i.e. because students have not completed some of the assessment items) generates narratives with a lower alignment and therefore, should be analyzed and improved in future studies.
Building on these encouraging results, our next step will focus on extending the study to teachers with different backgrounds. We also plan to continue working to facilitate the definition of the course curriculum design and its relation to the learning assessment items, as the context inclusion in the prompts generates more detailed and aligned narratives. Our objective is to implement this in Moodle, one of the most widely adopted learning management system in higher education.
Finally, it is important to address the ethical concerns associated with the use of GenAI in generating educational recommendations. It is important to take in mind that this work is intented to be supplementary, and not replace the teachers’ essential role in the decision-making process [10].
This work was partially funded by the Department of Education, Universities and Research of the Basque Government (ADIAN, IT-1437-22) and grant RED2022-134284-T.
During the preparation of this work, the author(s) used Perplexity AI in order to: Grammar and spelling check. After using these tool(s), the author(s) reviewed and edited the content as needed and take full responsibility for the publication’s content. [10] T. K. F. Chiu, «The impact of Generative AI (GenAI) on practices, policies and research direction in education: a case of ChatGPT and Midjourney», Interactive Learning Environments, vol. 32, n.o 10, pp. 6187-6203, nov. 2024, doi: 10.1080/10494820.2023.2253861.