Although many technological advances have been done in the last decades, textbooks in their traditional form are still the primary knowledge source for students' instruction around the world. With the aim of addressing this gap, we developed an online reading system that allows students to easily track their own progress on course mandatory readings and quizzes, as well as compare themselves with their peers through a mirrored icicle plot visualization. Preliminary results about the hypothesized e ects of the social visualization in students behavior/performance in two classroom studies is presented, as well as their qualitative feedback about the system.
Many innovations have been introduced to digital version of traditional
textbooks throughout the last years. Among other novel additions to e-textbooks, we
can list concept mapping activities [
In our own work, we explored an idea of extending electronic textbooks with
social navigation [
In our past attempt to implement social comparison in a textbook context
[
Reading Mirror visualizes student own reading progress in the context of a hierarchical textbook visualization, which follow the following structure: lecture ! book chapter ! section ! subsection (see Fig. ??). Each unit of reading is represented by a rectangle. The larger the height of a rectangle, the larger number of pages the chapter, section, or subsection has. This visual variable allows students to see at a glance which lectures are more heavy in terms of reading material. To display current reading progress, we colored rectangles representing readings units with di erent shades of blue. The color re ects the fraction of already read pages in a unit. If a rectangle is white it means that the student has not read any related page. Otherwise, the darker its blue shade, the more pages the student has visited in that speci c unit.
The reading progress visualization is combined with visualization of student performance on quizzes. To show quiz performance, next to the progress icicle plot we added a small bar graph which re ects success rate on answering the quiz associated with each section (see Fig. 1). Here the red portion of the bar shows the proportion of incorrect answers while the green portion shows the fraction of correct answers on that quiz. 2.2
The design of the social comparison part of the Reading Mirror was motivated
by recent ndings in information visualization research, which indicated that
correlation tasks (i.e. detecting if two data distributions were similar or not) are
better supported when presented with mirrored graphs [
Considering this, the visualization component that allows learners to compare their own work with the rest of the class was designed as a mirrored version of the individual progress icicle plot (see 2 in Fig. 2). Here, the left side of the visualization shows the aggregation of the class behavior and the right side shows personal progress.
Students could interact with the visualization by either clicking a section, which takes them to the rst page belonging to the corresponding lecture reading, or by mouseovering the rectangles which shows the values re ecting students/class reading progress and quiz answering performance (see 3 in Fig. 2). 3
To evaluate the Reading Mirror system, we performed a sequence of classroom studies in three di erent courses: graduate courses on Information Retrieval and Database Management and an undergraduate course Introduction to ObjectOriented Programming. We used open textbooks and licensed proprietary material as readings in the system and prepared a full set of quizzes for each book. Approximately 200 students have used the system over several semesters.
In this paper we review some data collected during our studies in a graduatelevel Information Retrieval class over Spring and Fall 2018 terms. In this class, students had to use the platform weekly to review the readings related to the upcoming lecture. In addition, for each lecture they had to answer a series of short quizzes associated with the sections they had to read.
The course had 11 topics/lectures supported by the reading system. In the analysis below we do not consider the rst lecture activity logs because learners during that time period could still add/drop the course so usage data is very sparse. A total of 80 students used the system actively every week in both terms (39 in Spring 2018 and 41 in Fall 2018). In order to measure the e ect that the social comparison component of the progress visualization had on the students, we explored two di erent setups: 1. Spring 2018 : In this semester, we turned on and introduced the social comparison features between Lectures 7 and 8. We expected that this design will help us to compare student behavior (reading progress/quizzes' performance) between the rst seven and the last three lectures. 2. Fall 2018 : Here, we incorporated the Social Comparison at the beginning of the course. In this case, we were interested to study the e ect of Social Comparison in a long-term setup. It also allowed us to more reliably assess the e ect of social comparison using the earlier semester as a baseline. 4 4.1
With the aim of quantifying the in uence of the social comparison component, we calculated the average success rate of students before and after it was enabled in the reading system in Spring 2018, and throughout the whole term in the case of Fall 2018 (see Fig. 2). We expected that the presence of social comparison will encourage students to work harder reaching higher success level in their quiz answers.
In the Spring 2018 semester, we found no di erence in success rate on answering quizzes with and without social comparison visualization. During the rst weeks it was 0.63 (SD=0.08), while after its inclusion its value was 0.63 (SD=0.05). We hypothesize that it could happen because the change was introduced too late in the term, so the students did not have enough time to get a stronger in uence from having access to others performance.
Indeed, in the following term (Fall 2018) where social comparison was offered from the beginning, we observed that students success rate was gradually increasing. While the overall pro le of success was similar (likely re ecting varying di culty of di erent chapters), starting from Lecture 3, student performance in the social comparison condition was always higher than the in the condition with individual progress tracking only (Fig. 3).
One of the reason that could explain this di erence is that the students need time to get used to social comparison and . We can see in Fig. 3 that after the rst three topics Fall 2018's students started to di erentiate from Spring 2018's students. This coincides with the fact that in Spring 2018, with only three weeks of social comparison, we could not see any expected change in their behavior (i.e. increase in their previous average performance on quizzes). Notwithstanding the above, one could hypothesize that the noticeable average di erent in success rate between both terms could be explained by having better performer students in Fall 2018, but we found that there were no signi cant di erences in average grades' performance in both terms (M Spring2018 = 88.0, SD Spring2018 = 4:6 and M F all2018=88.8, SD F all2018=7.3). After using the system throughout the term, students were asked to ll out a survey in order to get their opinion about the main features of the online reading system (see Fig. 4). The survey included questions related to: { How important was for them accessing to information about their own progress/performance. { How important was for them to have access to the average progress/performance of the rest of the class. { How well the visualization supported the goals of re ecting own and others performance. { How the introduction of the Social Comparison in uenced their behavior in the system (only Spring 2018 case).
As the Fig. 4) shows, student opinion about the key system features was highly positive. It was interesting to observe that the students were slightly more positive about the value of tracking own progress than the ability to see the class progress. They also considered the ability to compare quiz progress slightly higher than reading progress. Also, more than a half of the students felt that they altered their behavior due to the social comparison features.
Furthermore, we allowed students to give their opinion about how the system could be improved. Some of the comments they gave were the following: \The hierarchy bar may not be a good way to visualize the reading sections especially when the sections have too many pages that can't be properly visualize". Indeed, while the Reading Mirror visualization shows the \big picture" of comparative behavior, the ability to zoom on a speci c chapter or section might be important for better comparison. We plan to add this ability in future versions. 5
In this paper, we introduced the Reading Mirror interface, which enhances student textbook reading with progress tracking, social navigation, and social comparison features. Early results of two classroom studies demonstrated that the students' perception of the system is very positive. However, by comparing students performance on answering quizzes in both studies we can hypothesize that social comparison might take some weeks to have an in uence in students.
In our future work, we plan extending the social visualization to include estimations of students knowledge inferred from their reading behavior and performance on quizzes. On the other hand, we are working on using the knowledge modeling to recommend most relevant external learning resources. Ultimately, we would like to combine both visualization and recommendation approaches in order to explain why an speci c external learning content was suggested given their current level of knowledge.