=Paper=
{{Paper
|id=Vol-2384/paper03
|storemode=property
|title=Reading Mirror: Social Navigation and Social Comparison for Electronic Textbooks
|pdfUrl=https://ceur-ws.org/Vol-2384/paper03.pdf
|volume=Vol-2384
|authors=Jordan Barria-Pineda,Peter Brusilovsky,Daqing He
|dblpUrl=https://dblp.org/rec/conf/aied/Barria-PinedaBH19
}}
==Reading Mirror: Social Navigation and Social Comparison for Electronic Textbooks==
https://ceur-ws.org/Vol-2384/paper03.pdf
Reading Mirror: Social Navigation and Social
Comparison for Electronic Textbooks
Jordan Barria-Pineda, Peter Brusilovsky, and Daqing He
School of Computing and Information, University of Pittsburgh, PA 15260, USA
jab464,peterb,dah44@pitt.edu
Abstract. Although many technological advances have been done in
the last decades, textbooks in their traditional form are still the primary
knowledge source for students’ instruction around the world. With the
aim of addressing this gap, we developed an online reading system that
allows students to easily track their own progress on course mandatory
readings and quizzes, as well as compare themselves with their peers
through a mirrored icicle plot visualization. Preliminary results about
the hypothesized effects of the social visualization in students behav-
ior/performance in two classroom studies is presented, as well as their
qualitative feedback about the system.
Keywords: electronic textbook · social visualization · social navigation
1 Introduction
Many innovations have been introduced to digital version of traditional text-
books throughout the last years. Among other novel additions to e-textbooks, we
can list concept mapping activities [14], automatic recommendation of relevant
external content (such videos [11], Wikipedia articles [1], etc.), social annota-
tions [15], and embedded interactive learning activities [5]. All these efforts try
to leverage the development of new technologies with the aim of modernizing
the textbook as an educational resource.
In our own work, we explored an idea of extending electronic textbooks with
social navigation [7], a technology that uses behavior of past users to guide future
readers [3, 6]. Our results demonstrated that social navigation helps students to
focus on most important pages and increases their reading engagement. In this
paper we present some early results of our most recent work, which extends so-
cial navigation with a social comparison through an advanced reading support
interface “Reading Mirror”. This extension was motivated by our studies of so-
cial comparison interfaces in educational contexts [10, 4], which demonstrated
that social comparison (SC) features could act as a motivator for students en-
gagement and enable stronger students to act as guidance for weaker students.
In the following sections, we introduce the Reading Mirror interface and report
preliminary results of its classroom evaluation.
2 J. Barria-Pineda et al.
2 Reading Mirror
Reading Mirror is a textbook reading support interface with progress tracking
and social comparison features. It attempts to integrate features of social navi-
gation support, open social learner modeling, and social comparison [6, 3, 10, 9,
8] into a hierarchical structure of a typical textbook. Our main challenge was to
design a visualization, which allows students to track their own reading perfor-
mance and compare it with class performance while using space efficiently.
In our past attempt to implement social comparison in a textbook context
[9] we used a sunburst visualization, which offered an expressive approach for
tracking student progress, but consumed a considerable amount of space and
provided poor support for social comparison. In the Reading Mirror project, we
applied a colored icicle plot [2], which is able to efficiently visualize hierarchical
data in a linear form. This approach provided space-saving support for both
progress tracking and social comparison, i.e., comparing student own behavior
with the progress of the whole class.
2.1 Self-Monitoring Visualization
Reading Mirror visualizes student own reading progress in the context of a hier-
archical textbook visualization, which follow the following structure: lecture →
book chapter → section → subsection (see Fig. ??). Each unit of reading is
represented by a rectangle. The larger the height of a rectangle, the larger num-
ber of pages the chapter, section, or subsection has. This visual variable allows
students to see at a glance which lectures are more heavy in terms of reading
material. To display current reading progress, we colored rectangles represent-
ing readings units with different shades of blue. The color reflects the fraction
of already read pages in a unit. If a rectangle is white it means that the student
has not read any related page. Otherwise, the darker its blue shade, the more
pages the student has visited in that specific unit.
The reading progress visualization is combined with visualization of student
performance on quizzes. To show quiz performance, next to the progress icicle
plot we added a small bar graph which reflects success rate on answering the quiz
associated with each section (see Fig. 1). Here the red portion of the bar shows
the proportion of incorrect answers while the green portion shows the fraction
of correct answers on that quiz.
2.2 Social Comparison
The design of the social comparison part of the Reading Mirror was motivated
by recent findings in information visualization research, which indicated that
correlation tasks (i.e. detecting if two data distributions were similar or not) are
better supported when presented with mirrored graphs [12]. This work confirmed
earlier findings stating that capability of the human visual system for detecting
visual differences between two regions is more efficient when they are laid out
Reading Mirror: Social Navigation and Social Comparison for Electronic ... 3
Fig. 1. Detail of the individual progress visualization and what visual variables convey
as mirror images of each other, compared to repeated translations of each other
[13].
Considering this, the visualization component that allows learners to compare
their own work with the rest of the class was designed as a mirrored version
of the individual progress icicle plot (see 2 in Fig. 2). Here, the left side of the
visualization shows the aggregation of the class behavior and the right side shows
personal progress.
Students could interact with the visualization by either clicking a section,
which takes them to the first page belonging to the corresponding lecture read-
ing, or by mouseovering the rectangles which shows the values reflecting stu-
dents/class reading progress and quiz answering performance (see 3 in Fig. 2).
3 Classroom Studies
To evaluate the Reading Mirror system, we performed a sequence of classroom
studies in three different courses: graduate courses on Information Retrieval and
Database Management and an undergraduate course Introduction to Object-
Oriented Programming. We used open textbooks and licensed proprietary ma-
terial as readings in the system and prepared a full set of quizzes for each book.
Approximately 200 students have used the system over several semesters.
In this paper we review some data collected during our studies in a graduate-
level Information Retrieval class over Spring and Fall 2018 terms. In this class,
students had to use the platform weekly to review the readings related to the
upcoming lecture. In addition, for each lecture they had to answer a series of
short quizzes associated with the sections they had to read.
The course had 11 topics/lectures supported by the reading system. In the
analysis below we do not consider the first lecture activity logs because learners
4 J. Barria-Pineda et al.
Fig. 2. Overview of Reading Mirror system. (1) Traditional table of content. (2) Mir-
rored progress visualization - (2a) average group progress at left, (2b) individual
progress at right. (3) Tooltip that is showed when a user mouseovers on a lecture sub-
section with information about own student and average class progress/performance
Reading Mirror: Social Navigation and Social Comparison for Electronic ... 5
during that time period could still add/drop the course so usage data is very
sparse. A total of 80 students used the system actively every week in both terms
(39 in Spring 2018 and 41 in Fall 2018). In order to measure the effect that the
social comparison component of the progress visualization had on the students,
we explored two different setups:
1. Spring 2018 : In this semester, we turned on and introduced the social com-
parison features between Lectures 7 and 8. We expected that this design will
help us to compare student behavior (reading progress/quizzes’ performance)
between the first seven and the last three lectures.
2. Fall 2018 : Here, we incorporated the Social Comparison at the beginning
of the course. In this case, we were interested to study the effect of Social
Comparison in a long-term setup. It also allowed us to more reliably assess
the effect of social comparison using the earlier semester as a baseline.
4 Preliminary Results
4.1 Performance Impact
With the aim of quantifying the influence of the social comparison component, we
calculated the average success rate of students before and after it was enabled in
the reading system in Spring 2018, and throughout the whole term in the case of
Fall 2018 (see Fig. 2). We expected that the presence of social comparison will
encourage students to work harder reaching higher success level in their quiz
answers.
In the Spring 2018 semester, we found no difference in success rate on an-
swering quizzes with and without social comparison visualization. During the
first weeks it was 0.63 (SD=0.08), while after its inclusion its value was 0.63
(SD=0.05). We hypothesize that it could happen because the change was intro-
duced too late in the term, so the students did not have enough time to get a
stronger influence from having access to others performance.
Indeed, in the following term (Fall 2018) where social comparison was of-
fered from the beginning, we observed that students success rate was gradually
increasing. While the overall profile of success was similar (likely reflecting vary-
ing difficulty of different chapters), starting from Lecture 3, student performance
in the social comparison condition was always higher than the in the condition
with individual progress tracking only (Fig. 3).
One of the reason that could explain this difference is that the students need
time to get used to social comparison and . We can see in Fig. 3 that after the
first three topics Fall 2018’s students started to differentiate from Spring 2018’s
students. This coincides with the fact that in Spring 2018, with only three weeks
of social comparison, we could not see any expected change in their behavior (i.e.
increase in their previous average performance on quizzes). Notwithstanding the
above, one could hypothesize that the noticeable average different in success rate
between both terms could be explained by having better performer students in
Fall 2018, but we found that there were no significant differences in average
6 J. Barria-Pineda et al.
grades’ performance in both terms (M Spring2018 = 88.0, SD Spring2018 =
4.6 and M F all2018=88.8, SD F all2018=7.3).
Fig. 3. Average students’ performance on quizzes on each topic supported by Reading
Mirror
4.2 Subjective Feedback
After using the system throughout the term, students were asked to fill out a
survey in order to get their opinion about the main features of the online reading
system (see Fig. 4). The survey included questions related to:
– How important was for them accessing to information about their own
progress/performance.
– How important was for them to have access to the average progress/performance
of the rest of the class.
– How well the visualization supported the goals of reflecting own and others
performance.
– How the introduction of the Social Comparison influenced their behavior in
the system (only Spring 2018 case).
As the Fig. 4) shows, student opinion about the key system features was
highly positive. It was interesting to observe that the students were slightly
more positive about the value of tracking own progress than the ability to see
the class progress. They also considered the ability to compare quiz progress
Reading Mirror: Social Navigation and Social Comparison for Electronic ... 7
Fig. 4. Summary of students’ qualitative feedback
slightly higher than reading progress. Also, more than a half of the students felt
that they altered their behavior due to the social comparison features.
Furthermore, we allowed students to give their opinion about how the system
could be improved. Some of the comments they gave were the following: “The
hierarchy bar may not be a good way to visualize the reading sections especially
when the sections have too many pages that can’t be properly visualize”. Indeed,
while the Reading Mirror visualization shows the “big picture” of comparative
behavior, the ability to zoom on a specific chapter or section might be important
for better comparison. We plan to add this ability in future versions.
5 Conclusion and Future Work
In this paper, we introduced the Reading Mirror interface, which enhances stu-
dent textbook reading with progress tracking, social navigation, and social com-
parison features. Early results of two classroom studies demonstrated that the
students’ perception of the system is very positive. However, by comparing stu-
dents performance on answering quizzes in both studies we can hypothesize that
social comparison might take some weeks to have an influence in students.
In our future work, we plan extending the social visualization to include
estimations of students knowledge inferred from their reading behavior and per-
formance on quizzes. On the other hand, we are working on using the knowledge
modeling to recommend most relevant external learning resources. Ultimately,
we would like to combine both visualization and recommendation approaches in
order to explain why an specific external learning content was suggested given
their current level of knowledge.
8 J. Barria-Pineda et al.
References
1. Agrawal, R., Gollapudi, S., Kenthapadi, K., Srivastava, N., Velu, R.: In: Proceed-
ings of the First ACM Symposium on Computing for Development, ACM DEV’10.
https://doi.org/10.1145/1926180.1926204
2. Barlow, T., Neville, P.: A comparison of 2-d visualizations of hierarchies. In: IEEE
Symposium on Information Visualization, InfoVis 2001. pp. 131–138 (2001)
3. Brusilovsky, P., Chavan, G., Farzan, R.: Social adaptive navigation support for
open corpus electronic textbooks. In: De Bra, P., Nejdl, W. (eds.) Third Inter-
national Conference on Adaptive Hypermedia and Adaptive Web-Based Systems
(AH’2004). Lecture Notes in Computer Science, vol. 3137, pp. 24–33. Springer-
Verlag (2004)
4. Brusilovsky, P., Somyürek, S., Guerra, J., Hosseini, R., Zadorozhny, V., Durlach,
P.: Open social student modeling for personalized learning. IEEE Transactions on
Emerging Topics in Computing 4(3), 450–461 (2016)
5. Ericson, B.J., Guzdial, M.J., Morrison, B.B.: Analysis of interactive features de-
signed to enhance learning in an ebook. In: Proceedings of the Eleventh Annual
International Conference on International Computing Education Research,ICER
’15. pp. 169–178. ACM (2015)
6. Farzan, R., Brusilovsky, P.: Social navigation support through annotation-based
group modeling. In: Ardissono, L., Brna, P., Mitrovic, A. (eds.) 10th International
User Modeling Conference. Lecture Notes in Artificial Intelligence, vol. 3538, pp.
463–472. Springer Verlag (2005)
7. Farzan, R., Brusilovsky, P.: Social navigation. In: Brusilovsky, P., He, D. (eds.)
Social Information Access: Systems and Technologies, pp. 142–180. Lecture Notes
in Computer Science, Springer, Cham (2018)
8. Guerra, J., Hosseini, R., Somyürek, S., Brusilovsky, P.: An intelligent interface
for learning content: Combining an open learner model and social comparison
to support self-regulated learning and engagement. In: Proceedings of the 21st
International Conference on Intelligent User Interfaces (IUI ’16). pp. 152–163. ACM
(2016)
9. Guerra, J., Parra, D., Brusilovsky, P.: Encouraging online student reading with
social visualization. In: AIED Workshops (2013)
10. Hsiao, I.H., Bakalov, F., Brusilovsky, P., König-Ries, B.: Progressor: social navi-
gation support through open social student modeling. New Review of Hypermedia
and Multimedia 19(2), 112–131 (2013)
11. Kokkodis, M., Kannan, A., Kenthapadi, K.: Assigning educational videos at appro-
priate locations in textbooks. In: Proceedings of the 7th International Conference
on Educational Data Mining (EDM). pp. 201–204 (2014)
12. Ondov, B.D., Jardine, N., Elmqvist, N., Franconeri, S.L.: Face to face: Evaluating
visual comparison. IEEE Transactions on Visualization and Computer Graphics
25, 861–871 (2018)
13. Treder, M.S.: Behind the looking-glass: A review on human symmetry perception.
Symmetry 2(3), 1510–1543 (2010)
14. Wang, S., Walker, E., Wylie, R.: What matters in concept mapping? maps learners
create or how they create them. In: André, E., Baker, R., Hu, X., Rodrigo, M.M.T.,
du Boulay, B. (eds.) Artificial Intelligence in Education. pp. 406–417. Springer
International Publishing, Cham (2017)
15. Zyto, S., Karger, D., Ackerman, M., Mahajan, S.: Successful classroom deployment
of a social document annotation system. In: Proceedings of the SIGCHI Conference
on Human Factors in Computing Systems. pp. 1883–1892. CHI ’12, ACM (2012)