Introduction & Background

Feedback is an important process of learning in higher education. Yet, monitoring how students engage with, interact with and act on feedback (traceability) is challenging [1]. Learning Analytics (LA) is a potential solution to enhance the effectiveness of feedback by closing the feedback loop through capturing and analysing students' interactions with feedback. LA derives insights from educational data and delivers them to learners through student-facing LA dashboards to facilitate the feedback process [2]. However, developing effective LA dashboards is often constrained by students' lack of expertise with data and visualisation literacy to appropriately interpret the insights from the dashboard [3,4]. Based on this challenge, Winstone [1] emphasised that the use of simple analytics in student-facing analytics dashboards is a safer choice to better influence students' actions based on feedback. But, the use of simple visualisation may not capture the full impact of feedback due to its limitation in tracking students' consistent strengths, repeated mistakes, or areas of confusion.

Data Storytelling (DS) is a promising technique that can assist students with varying levels of data literacy in their interpretation of visualisations by combining data, visual, and narrative elements in order to guide non-data experts through a data story that conveys the main insights [5]. As noted by Bach et al. [6], there are many ways that visual and narrative elements can be applied in order to tell an effective data story. For instance, a title that clarifies the main insights of the visualisation (Explanatory titles) [7] and the addition of textual elements to explain particular data points (Annotations) can improve the reader's understanding of the visualisation by adding context [5]. Moreover, the use of pre-attentive attributes (e.g., colour and contrast) can also direct the attention of the audience towards particular data points or regions of the visualisation (Highlighting important data points) [5]. Lastly, removing superfluous information or visual noise (e.g., grid lines and labels) that distracts from the main data story can improve the clarity of the visualisation (Decluttering) [7].

In designing an LA dashboard supported with DS principles, it is important to co-design with students to capture how they want to view the dashboard in a way that feels authentic to them [1]. In LA literature, the co-design approach has been implemented to engage and support educational stakeholders by ensuring that the presentation of LA aligns with their goals [8]. By doing so, it enables us as LA designers to identify ways to communicate insights from data effectively [9] to different groups of students. DS is goal oriented in the sense that data stories and visualisations need to be crafted with a specific purpose [10]. Therefore, the co-design approach is appropriate as it informs designers as to the DS features and chart types that align best with the goals of the students.

This study presents the preliminary findings of designing a DS-enhanced dashboard with students for an LA solution, PolyFeed, which designed and developed to capture and analyse students' interactions with feedback. PolyFeed was designed to enhance feedback process through annotate feedback, create action plans and visualise interactions with feedback functionalities. We defined Feedback Analytics (FA) as the capture, analysis, and data-based presentations of students' interactions with feedback. This preliminary study aims to build upon existing LA literature by exploring how students from different educational disciplines: i) perceive effective visualisations depicting FA; and ii) choose DS principles to best support effective interpretation of student-facing FA dashboards. Thus, this study is based on the following research questions.

• RQ1: What are the effective ways to visualise FA from students' points of view? • RQ2: How did students use DS principles to design an FA dashboard?

Methodology

This study is inspired by the design sprint methodology [11] and was conducted via a two-hour design sprint covering five key elements -Understand, Diverge, Decide, Prototype, and Test.

Thirteen higher education students from seven different faculties in a large Australian university volunteered for this design session. They were grouped into three groups based on faculty, degree program, and current year of study. The participants were introduced to the features of PolyFeed and the data collected through the tool. The dataset included quantitative and qualitative data. The quantitative data included assessment marks for each assignment, and students' ratings on received feedback. The qualitative data comprised labeled feedback extracts ('Strengths, ' 'Weaknesses, ' 'Action Points, ' 'Confusion, ' and 'Other. '), personal notes, to-do lists with deadlines, and further clarifications obtained using generative AI agent (e.g., ChatGPT). Using the provided dataset, students brainstormed questions they wanted to answer using the FA dashboard (Understand). Students individually sketched data visualisations they perceived as suitable for the selected questions (Diverge) referring to two key resources (e.g., Financial Times Visual Vocabulary1 , and Power BI Visual Vocabulary2 ) and collectively prioritised through group discussion and voting (Decide). In the prototype phase, students created a paper-based prototype that included these prioritised visualisations. Each group then further improved their initial design by incorporating data storytelling principles -Explanatory Title, Annotation, Highlighting important data points, and Decluttering- [5] introduced during the workshop (Prototype). Each of these principles were introduced with an explanation and a scenario based example. Lastly, each group presented their dashboard design to other groups and collected peer feedback (Test).

This paper presents the preliminary findings of the prototype phase. The co-design sessions was recorded and transcribed according to each step: Ideation, Exploration, Initial Design, DS-Enhanced Design, and Retrospective. The transcripts were then analysed alongside the artefacts created by students. In this preliminary analysis we analysed the dashboard design with and without DS elements. In finding answers for RQ1 we conducted a group discussion where we inductively identified the topics and different types of visualisations students included into their dashboards. Subsequently, in finding answers for RQ2 we compared the visualisations in the initial design with DS incorporated design to identify the DS principles students used in designing their dashboard.

Preliminary Results

RQ1: What are the effective ways to visualise FA?

Students primarily wanted to include four key aspects of FA in the dashboard: (1) an overview of strengths and weaknesses identified in their feedback, (2) an overview of to-do lists based on feedback, (3) mark comparisons with the class, and (4) individual performance within the course. In visualising these aspects, the groups utilised a variety of different data visualisations. For instance, group one used a pie chart while group two used a word cloud (Figure 1 -a) to visualise the overview of strengths and weaknesses (unit-specific and across units or assignments). In addition, to visualise the overview of to-do lists, groups one and three used a non-traditional data visualisation such as a checklist that supports prioritisation and records the completion of the to-do list (Figure 1 -b). In visualising the mark comparisons, group two used a Gaussian

RQ2: How did students use DS principles to design an FA dashboard?

The most common DS principle that student groups incorporated into their DS-enhanced design was highlighting important data points. Specifically, students would leverage pre-attentive attributes such as colour or size to emphasise what they believed to be salient information in their dashboard. As shown in Figure 1, group two used green for strengths and red for weakness in their word cloud (Figure 1 -f), while group one included red and green data points in the line graphs to highlight highs and lows in the individual performance (Figure 1 -i). Explanatory titles were also leveraged by groups two and three to their mark distribution graph (e.g., Gaussian curve, bar graph with distribution bars) (Figure 1 -h). Annotations and decluttering were less commonly observed in the final designs. Group three designed a 'decluttered' prototype by removing the bars showing individual students' marks and instead including a tick into the mark distribution to indicate a particular students' position in the class (Figure 1 -h).

Discussion and Conclusion

This study sought to examine how to effectively tell data stories relating to feedback by giving the opportunity to higher education students to co-design a FA dashboard with DS features.

In response to RQ1, What are the effective ways to visualise FA from students' points of view?, we found that students expected to see an overview of strengths and weaknesses and action plans (i.e., to-do lists), mark distribution across the class, and the trends in their performance. In visualising these aspects, students selected both common (e.g., pie charts and line charts) and uncommon data visualisations (e.g., radar graphs and custom visualisations). It was suggested by Knaflic [5] that a preference for familiar data visualisations may result from a perception that less conventional visualisations are too complicated. However, as students also selected uncommon data visualisations from the supplied reference sheet, this indicates that students may be receptive to the presence of a wider range of visualisations within FA dashboards, as long as they are appropriate to communicate the insights [1]. Therefore, in designing effective visualisations of FA, LA designers should be conscious of the diverse range of visualisations that can be employed to reach a wider audience while also ensuring that the intended message of the visualisations is being clearly conveyed. Doing so, we can ensure that we appropriately balance the presentations of the FA dashboard meets its' objective while catering to the needs of different student groups (e.g., disciplines, and performance). Moreover, we can provide different types of visualisations for one aspect (e.g., presenting common strengths and weaknesses through word clouds, lollipop graphs, and grouped bars) and students can select the most effective visualisation for them.

In response to RQ2, How did students use DS principles to design an FA dashboard?, we found that students primarily used two DS principles: Highlighting important data points and Explanatory titles. Some groups used these principles individually to improve the graphs, while others combined them. Knaflic [5] emphasised that incorporating pre-attentive attributes (e.g., colour and contrast) sparingly to textual content can improve the text. In this study, the students leveraged colour to emphasise the key point within their explanatory title. Students also combined multiple pre-attentive elements when applying the DS principle of highlighting important data points. For instance, they used colour and size (Figure 1-f) or colours and shapes (Figure 1-i) together. However, as the purpose of pre-attentive attributes are to assist in processing salient information in milliseconds [5], the inclusion of multiple pre-attentive attributes on the dashboard interface may obscure each other and make it difficult to distinguish the key insights within a short time. In addition, we observed that the students rarely used the DS principles Annotation and Decluttering. A possible reason for this could be because of the qualitative nature of the dataset (i.e., the PolyFeed dataset contains feedback extractions, labels, to-do lists and explanations from Generative AI agent) and the lesser complexity of the quantitative data. There are a wide range of data visualisations for quantitative data -yet, utilising them to visualise qualitative data is challenging because they are used for a purpose other than the original intention of the visualisation. In addition, the DS principles used in this study are useful to emphasise the key points of quantitative data. However, in the context of qualitative data, we believe that the effective use of DS should be further explored.

In conclusion, the co-design approach enabled further understanding into what students believe are effective ways to visualise FA using DS principles. The findings of this study have provided insights into how Polyfeed designers can successfully incorporate DS principles in the next phase of the FA dashboard's development. However, as DS principles are generally applicable to quantitative data, further investigations are necessary to see how DS can best support FA given the predominantly qualitative nature of FA datasets.