This paper discusses the introduction of personalized explorable visual narratives into the Technology Enhanced Learning domain, which were used by 233 students as part of their undergraduate degree program during two successive academic years. The research proposes evaluating the usage of visual narratives to indicate positive behavioral change in student engagement levels. It highlights the impact that the personalized explorable visual narratives had on the students with poor levels of engagement with course content, by instilling behavioral change through visual narrative interactions, which ultimately motivated them to improve their engagement levels.
Engagement and
As Technology Enhanced Learning (TEL) environments are
continuously growing in popularity, the challenge of addressing
poor student engagement with course content when using these
technologies needs to be tackled. One way to addressing poor
student engagement when using TEL environments is to provide
learners with visual narratives that can be navigated and explored
to help them make sense of their personal course related activity
data. Information Visualization (IV) provides an effective means to
comprehend data and supports pattern discoveries [
Typically, adaptive learning systems evaluate students’
understanding and uptake of course content [
The VisEN framework supports the construction and consumption of visual narratives. Authors (educators with an understanding of student data) use VisEN to build narratives that are automatically complemented with appropriate visualizations. The framework enables authors to connect to tabular data sources, including databases and uploading data from the file system (CSV and MS Excel formats are supported). It supports the construction of individual segments of visual narratives through drag and drop operations and automatically suggest sequences to the narrative by adjusting and mapping query terms to build related views. These related views can be used as sequences within the visual narrative. Student logged data from the AMAS PLE was used by educators to construct a visual narrative, which was then dynamically personalized to individual students and made available to them during the course.
The visual narratives communicated to each student his/her current
engagement with the course content, times spent on activities and
resources used [
Visual narratives were made available to the 233 (two groups of learners across two academic years: 2013-2014 and 2014-2015), studying the Information Management and Data Engineering course at using the AMAS PLE. The visual narratives provided a message individualized to each student informing him/her how he/she was engaging with course content, resources used and time spent on activities. The visual narratives also provided students with peer comparisons. The evaluation included a study that assessed the impact that the visual narratives had on behavioral change of students with poor levels of course engagement. It focused on usage of visual narratives rather than understanding and uptake of course content to assess behavioral change.
The AMAS PLE sent regular notifications to students informing them of their level of engagement with their course content. The notifications were categorized by level of engagement: bad, poor, above average, good and excellent. The study analyzed all the students that received an engagement notification in the bad and poor categories at any point during the course. It then grouped students that improved their engagement (went from bad or poor to above average or higher categories) and identified these as ‘improved learners’. It also grouped the students that received an engagement notification in the bad and poor categories but did not show the same level of improvement (i.e. remained at the same engagement level or went from bad to poor) and identified these as ‘unimproved students’. It compared the visual narrative usage of both of these student groups to determine if the visual narrative had any impact on the improvement of the former group. From the 233 students that participated in the course over the two academic years, 97 learners were identified as ‘improved students’ and 98 learners were identified as ‘unimproved students’. The remaining 38 students constantly received an above average or higher engagement notification throughout the course and hence were not included in this study.
The study found that the ‘improved students’ showed a very large increase in usage of their visual narratives during the period when their engagement improved. 80% of these learners showed a minimum of a four-fold increase in their visual narrative interactions during this period and constantly revisited their visual narratives as they worked through their activities. The visual narrative usage patterns of these students during the period of improved engagement was analyzed. The analysis showed that the students at the start would navigate through the entire visual narrative and view the explorations and then as they worked through their course content, they viewed specific elements of the visual narrative. The study also found that 72% of these students executed half of their total visual narrative interactions on the first day after reading the bad or poor engagement notification, indicating that they were drawn to their visual narratives to understand a bad or poor engagement notification. A post-course questionnaire was completed by the students during both academic years. One of the statements in the questionnaire addressed behavioral change: The visualizations motivated me to engage with the course. The responses (using a five point Likert scale) showed that 71% of the ‘improved students’ either strongly agreed or agreed with this statement.
For the ‘unimproved learners’, the result of the study highlighted that the vast majority of these students had a small difference in the usage of their visual narrative during both engagement periods. One third of these students did not view their visual narratives during both engagement periods and the visual narrative usage for the majority of this group of learners remained constantly low. However, 15% of these students did show a four-fold increase in visual narratives usage during the subsequent period but importantly their engagement improved from bad to poor. It is important to note that visual narrative usage was distinct from course engagement. Interacting the visual narratives did not impact course engagement. In addition, students needed to navigate to a specific section of the PLE in order to view their visual narratives, which was optional. Hence high course engagement did not necessarily equate to high visual narrative usage and poor course engagement also did not equate to low visual narrative usage.
This paper discussed a study focusing on usage of visual narratives produced by the VisEN framework and provided to 233 students using the AMAS PLE as part of their degree course in Trinity College Dublin. The framework introduced personalized explorable visual narratives into the TEL domain to promote student course engagement through behavioral change instilled by the usage of their visual narratives. The evaluation highlighted that when students’ course engagement was poor, their visual narrative usage was low. However, when visual narrative usage was high, the course engagement improved. In addition, the majority of students that improved their course engagement highlighted that the visual narratives motivated them to do so. These findings clearly showed that repeated visual narrative usage instilled a positive behavioral change in learners, which promoted course engagement.
This research is supported by the ADAPT Centre for Digital Content Technology under the SFI Research Centre’s Program (Grant 13/RC/2106) and is co-funded under the European Regional Development Fund.