Visualizing Student Models for Social Learning with Parallel IntrospectiveViews Fedor Bakalov I-Han Hsiao Peter Brusilovsky Birgitta König-Ries Institute for Computer School of Information School of Information Institute for Computer Science, University of Jena Sciences, University of Sciences, University of Science, University of Jena Ernst-Abbe-Platz 2 Pittsburgh Pittsburgh Ernst-Abbe-Platz 2 Jena 07743, Germany 135 N. Bellefield Ave. 135 N. Bellefield Ave. Jena 07743, Germany +49 3641 946435 Pittsburgh PA 15260, USA Pittsburgh PA 15260, USA +49 3641 946430 +1 412 624 9437 +1 412 624 9404 fedor.bakalov@uni- birgitta.koenig- jena.de ihh4@pitt.edu peterb@pitt.edu ries@uni-jena.de report our work on social interface for visualizing open ABSTRACT student models in the context of parameterized This paper reports a novel visual metaphor of parallel views assessment/self-assessment questions. on open student models and a new design of the IntrospectiveViews interface implementing this metaphor. The social interface presented in the paper expands our The interface allows visualizing not only the student’s own earlier work on improving student access to parameterized model, but also displaying parallel views on the models of questions, which support student learning and self- her peers and the cumulate model of the entire class or assessment of their knowledge. Main goal of that work was group. In the paper, we describe the application of parallel helping students to locate the right educational content at IntrospectiveViews for visualizing models representing the right time. In the past, we explored semantic and student progress on the QuizJET parameterized questions. personalized approaches to guide students. Semantic approach means building ontology of topics and using it to Categories and Subject Descriptors arrange the questions into topic folders on an e-learning portal. It provides students an approximate guidance to the H.5.2 [User Interfaces]: Graphical user interfaces (GUI); question within the logical structure. Personalized approach means using student models and adaptive navigation General Terms support to guide students to personally relevant questions. Design; Human Factor This paper explores social guidance approach motivated by collaborative tagging systems. In a collaborative tagging Keywords system a user is frequently guided to the right information Visualization, Open Student Model, Assessment socially, by other users. Exploring tagged bookmarks of a user who is similar to you usually helps finding valuable 1. INTRODUCTION resources. Similarly, we hoped, exploring semantically Engaging students with social learning technologies has structured progress of learners similar to you can help to become an important trend in modern e-learning. One of the select most appropriate learning content. Semantically- biggest challenges is to provide support in the social structured social visualization could be very important on learning context, at the same time allowing students feel in both stages of social guidance: recognizing peers as similar control. One of the popular solutions to address the issue of and grasping their progress to select most useful content. control is so-called open student modeling, an approach that This paper investigates the value of a specific social allows the students to observe and reflect on their progress. visualization approach - parallel visualization of multiple In particular, visual approaches for open student modeling student models using IntrospectiveViews - in finding the were explored to provide students with an easy-to-grasp and right questions. In the next section, we provide a short holistic view on their progress [2-3, 8]. However, most of literature review on the related work. The system and study the open student modeling research focuses on a design are presented in the following section. Finally, we representation of an individual student ignoring the social summarize this work and discuss the future research plan. aspect of learning. In contrast, several social visualization approaches explored in e-learning context [1] focus mainly 2. RELATED WORK on student communication and collaboration rather than on 2.1. Open Student Modeling their progress. Our work attempts to explore the prospects A range of benefits have been reported on opening the of open student modeling and student progress visualization student models to the learners, such as increasing the in the context of modern social e-learning. In this paper, we learner’s awareness of the developing knowledge, Workshop on Visual Interfaces to the Social and Semantic Web (VISSW2011), Co-located with ACM IUI 2011, Feb 13, 2011, Palo Alto, US. Copyright is held by the author/owner(s). difficulties and the learning process, students’ engagement, projects that applied parameterized question generation in motivation, and knowledge reflection [2-4]. Dimitrova et al. e-learning systems for programming domain [12-14]. A [5] explore interactive open learner modeling by engaging range of architectures for exploring adaptive technology on learners to negotiate with the system during the diagnosis dedicated adaptive interfaces has been reported in [15-16]. process. Chen et al. [7] investigated the active open learner The added value of adaptive link annotation for an e- models in order to motivate learners to improve their learning portal has also been studied in [17]. academic performance. Both individual and group open learner models were studied and demonstrated the increase 3. SYSTEM AND STUDY DESIGN of reflection and helpful interactions among teammates. In [22], Bull & Kay described a framework to apply open user 3.1. QuizJET: Parameterized Questions of Java Evaluation Toolkit models in adaptive learning environment and provided QuizJET is a system for authoring and delivery of many in-depth examples. parameterized questions for the Java programming 2.2. Social visualization in E-learning language. It generates parameterized questions for Vassileva and Sun [6] investigated the community assessment and self-assessment of students’ knowledge of a visualization in the online communities. They summarized broad range of Java topics. Each QuizJET question (Figures that social visualization allows peer-recognition and 1.a and 1.b) asks the student to predict the results of provides students the opportunity to build trust in others execution of a specific Java program (i.e., mentally execute and in the group. Bull & Britland [23] used OLMlets to the program and enter the final value of some variable of research the problem of facilitating group collaboration and the text to be printed by the program.) All questions are competition. The results showed that by optionally parameterized, i.e., include a random parameter, which releasing the models to their peers increases the discussion QuizJET instantiates when the question is delivered to a among the students and encourages them to start working student. As a result, the student can attempt to answer the sooner. CourseVis [8] is another example, yet one of the same question multiple times with different values of the few open user model systems that provide graphical parameter, which helps to achieve the mastery level. The visualization for multiple users of groups to teachers and implementation and functionalities of QuizJET were learners. It helps instructors to identify problems early and described in detail in [18]. prevent some of the problems with distance learning. QuizJET questions are semantically organized into topic 2.3 Parameterized Assessment folders and displayed under a tree structure through the This paper is focused on open student models that are built course management portal (Figure 1.c). In addition, to based on the student’s performance on parameterized every question we associate a set of concepts, which questions. Parameterized technology essentially uses semantics is defined in a domain ontology. The domain randomly generated parameters within a small amount of ontology (http://www.sis.pitt.edu/~paws/ont/java.owl) was question templates generated by the content creators and developed by the TALER Lab at the School of Information produce many similar, yet sufficiently different questions. Sciences, University of Pittsburgh. It is formalized in the As demonstrated by a number of projects such as CAPA OWL-Full ontology language and contains more than 300 [9], WebAssign [10], COMBA [11], parameterized classes related to the Java programming language. The questions can be used effectively in a number of domains classes are connected via three relations: subClassOf, allowing to increase the number of assessment items, partOf-hasPart and relatedTo. decrease authoring efforts and reduce cheating. A few (a) (b) (c) Figure 1. The presentation (a) and evaluation results (b) of a QuizJET question; Hierarchical portal interface (c) Figure 2. Parallel IntrospectiveViews. Left pane – visualization of the student’s own progress; right pane – visualization of a peer’s progress. The circular sectors represent the lectures and the annular sectors represent the topics of individual lectures. The shades of the sectors indicate whether the topic has been covered and for the covered ones, denotes the progress the student has made. Color screenshots available at: http://www.minerva-portals.de/research/introspective-views/ perform better in the class. Also, it can allow finding peers 3.2. IntrospectiveViews Interface IntrospectiveViews was first proposed for scrutinizing who can help with difficult topics, hence increase the social semantically-enriched user interest models in [19, 20]. In interaction among the students. the interface user interests can be visualized as a set of In this paper, we demonstrate the application of parallel keywords displayed on a circular surface, which is IntrospectiveViews for visualization of student progress on gradually painted in shades between red and blue according QuizJET questions semantically organized into topics and to the hot-and-cold metaphor. The background color of lectures. Figure 2 shows parallel IntrospectiveViews for a keywords represents the degree of interest, i.e., the items on student in a class on Object-Oriented Programming (OOP). the red surface are the ones that the user is interested in and The visualization consists of two panes: the left pane the items in the blue one are the ones that she is not displays the student’s own model, whereas the right one interested in. In addition, the items can be grouped in displays someone else’s model. By default, the right pane circular sectors by type, i.e., the semantic class they belong displays the average progress of the entire class, but the to (e.g. person, company, country, etc.). Also, following student can switch to the progress of a specific classmate by Shneiderman's visual information seeking mantra [21], selecting the classmate’s name from the combobox menu IntrospectiveViews offers an overview of all items present located at the top of the right pane. Each pane visualizes the in the model, allows for zooming into different parts of it student progress in the form of a pie chart consisting of and filtering according to different criteria, and provides circular sectors representing the class lectures. The lectures details on demand. are displayed in a clockwise order denoting their pre- requisite sequence, i.e., the order they are taught in the 3.3. Integration of QuizJET and IntrospectiveViews class. Lectures may consist of one or several topics, which We have adapted IntrospectiveViews to fit the context of are represented as annular sectors placed within the circular social learning by introducing the parallel views on student sector of the corresponding lecture. For example, in Figure models. The adapted version visualizes not only the 2, Basic Concepts is the first lecture in the Introduction to student’s own model, but also the models of her peers and Object-Oriented Programming class and consists of three the class on average. We believe that the visibility of topics, namely, Variables, Objects, and Classes. The radius activities of others can motivate the student to progress and (width) of annular sectors denotes the amount of readings, quizzes, and exercises assigned to the topic. In a similar way, the span of circular sectors indicates the amount of learning content assigned to the corresponding lecture. Such a representation allows the student to easily estimate the amount of work she has to spend on each individual topic or lecture. The shade of each annular sector denotes whether the topic has been covered and, for the covered ones, indicates the progress the student has made with respect to the topic. The sectors painted grey represent the topics that have not been covered yet, whereas the sectors painted a shade from the color range red to green represent the sectors that have been already covered. For the covered topics, the interface displays the student progress. The progress, in the current implementation, is the ratio of successfully completed Figure 3. Quizzes of the selected topic. quizzes to the total quiz count in the topic. If the ratio equals 0, i.e., no quiz has been successfully completed, the sector is painted red. If it equals 1, i.e., all quizzes have 4. SUMMARY AND FUTURE WORK been completed, the sector appears green. The shades in the In this paper, we have presented a novel visual metaphor of range between red and green denote partial completion of parallel views on open student models in social learning the quizzes. systems. We have described the implementation of this metaphor using the integration of the IntrospectiveViews Similar to the previous design of IntrospectiveViews [19, interface and the QuizJET student modeling approach. We 20], the current implementation supports a number of believe that such a visualization can substantially enhance information visualization tasks postulated by Bed social learning, foster a better cooperation among students, Shneiderman [21]. It allows getting an overview of the improve their performance, and become an attractive tool entire model, but it also allows zooming into a certain part for accessing learning materials. of it in order to get a better view, which is especially important for visualizing models that consist of a large Currently, the interface is being used in the Object-Oriented number of topics. Also, it provides details on demand, e.g., Programming class at the School of Information Sciences at by clicking a sector, the interface will display the contents the University of Pittsburgh. Upon the completion of the of the corresponding topic, in the current implementation, class, we plan to evaluate the interface by asking the the list of quizzes for the topic (Figure 3). For each quiz, the students to fill out evaluation questionnaires. This will help interface provides a visual cue indicating the student us to find out its problems and weaknesses, which we will progress and displays the total number of attempts the address in the next versions. student has made on the quiz and the number of successful In addition to that, we plan to add a number of new attempts. By clicking a quiz label the interface will display features. One of the most important extensions planned is the quiz in a new window. In that way, having found the comprehensive privacy management. Our aim is to uncompleted quizzes, the student can quickly open each of provide the student with flexible mechanisms to control the them and complete the pending task. access of others to her model. Apart from that, we plan to Such visualization can help the student to plan her class enhance the zoom function by displaying different levels of work by providing an overview of her progress in the class details at different zoom levels. Also, we are going to and showing the topics that she has already completed and extend the comparison mode to allow the student to view the ones that she has to work on. In addition to that, the multiple progress charts at one time as well as to sort them ability to view someone else’s progress allows the student either by the overall progress or the progress in a specific to quickly find the peers that can help with a difficult topic topic. or quiz. For example, if the student experiences difficulties in completing some quizzes, using the parallel views, she REFERENCES can find a classmate who has already successfully [1] Vassileva, J. Toward Social Learning Environments. completed those quizzes and ask for help. Finally, the IEEE Transactions on Learning Technologies, 2008, ability to view the average progress of the entire class 1(4), p. 199–214. allows the student to relate her progress to the one of the [2] Bull, S. (2004) Supporting learning with open learner whole class and estimate whether she is ahead or behind of models. In: Proceedings of 4th Hellenic Conference the class on average. on Information and Communication Technologies in Education, Athens, Greece, September 29 - October 3, 2004, pp. 47-61 [3] Mitrovic, A., Martin, B.: Evaluating the Effect of [14] Martin B. & Mitrovic A. (2002) Automatic problem Open Student Models on Self- Assessment. Int. J. of generation in constraint-based tutors. In 6th Artificial Intelligence in Education 17(2), 121–144 International Conference on Intelligent Tutoring (2007) Systems (Its’2002) (eds S.A. Cerri, G. Gouardères & [4] Zapata-Rivera, J.D., Greer, J.E., (2004). Visualizing F. Paraguaçu), pp. 388–398. Berlin: Springer-Verlag, and inspecting Bayesian belief models. In: Biarritz International Journal of Artificial Intelligence in [15] Brusilovsky P. & Sosnovsky S. (2005) Individualized Education IJCAI, 14, pp. 1–37. . IOS Press. exercises for self-assessment of programming [5] Dimitrova, V., Self, J., Brna, P. (2001) Applying knowledge: an evaluation of QuizPACK. ACM Interactive Open Learner Models to Learning Journal on Educational Resources in Computing 5, Technical Terminology. Proc. UM 2001, Springer, pp. Article No. 6. 148-157. [16] Yudelson, M. and Brusilovsky, P. 2005. NavEx: [6] Vassileva J., Sun L. (2007) Using Community Providing Navigation Support for Adaptive Browsing Visualization to Stimulate Participation in Online of Annotated Code Examples. IOS Press. Communities.e-Service Journal, 6 (1), 3-40. [17] Hsiao, I.,Brusilovsky, P., Yudelson, M. and Ortigosa, [7] Chen, Z.H., Chou, C.Y., Deng, Y.C., and Chan, T.W. A. (2010). The Value of Adaptive Link Annotation in (2007). Active Open Learner Models as Animal E-Learning: A Study of a Portal-Based Approach. In: Companions: Motivating Children to Learn through Proceedings of 20th conference on Hypertext and Interaction with My-Pet and Our-Pet, International hypermedia, Toronto, Canada, Page 223-228. Journal of Artificial Intelligence in Education, 17(3): [18] Hsiao, I., Sosnovsky,S. and Brusilvsky, P. (2010) 217-226. Guiding Students to the Right Questions: Adaptive [8] Mazza, R. & Dimitrova, V. (2007). CourseVis: a Navigation Support in an E-learning System for Java graphical student monitoring tool for supporting Programming, Journal of Computer Assisted instructors in web-based distance courses. Learning, Volume 26 Issue 4, Pages 270 - 283. International Journal of Human-Computer Studies, 65, [19] Bakalov, F., König-Ries, B., Nauerz, A., and Welsch, 2, 125–139. M. IntrospectiveViews: an Interface for Scrutinizing [9] Kashy E., Thoennessen M., Tsai Y., Davis N.E. Semantic User Models. In Proc. of the 18th Intl. Conf &Wolfe S.L. (1997) Using networked tools to on User Modeling, Adaptation and Personalization enhance student success rates in large classes. 27th (2010). ASEE/IEEE Frontiers in Education Conference. [20] Bakalov, F., König-Ries, B., Nauerz, A., and Welsch, Pittsburgh. M. Scrutinizing User Interest Models with [10] Titus A.P., Martin L.W. & Beichner R.J. (1998) Web- IntrospectiveViews. In Adjunct Proc. of the 18th Intl. based testing in physics education: methods and Conf on User Modeling, Adaptation and opportunities. Computers in Physics 12, 117–123. Personalization (2010). [11] Sitthisak O., Gilbert L. & Davis H.C. (2008) Deriving [21] Shneiderman, B. The eyes have it: A task by data type e-assessment from a competency model. In Eighth taxonomy for information visualizations. In Proc. of IEEE International Conference on Advanced Learning the 1996 IEEE Symposium on Visual Languages Technolo- gies July 1st–July 5th (eds J. Cordeiro, J. (1996). Filipe & S. Ham- moudi), pp. 327–329. [22] Bull, S. & Kay, J. (2007). Student Models that Invite [12] Brusilovsky, P., Sosnovsky, S. and Shcherbinina, O. the Learner In: The SMILI Open Learner Modelling 2004. QuizGuide: Increasing the Educational Value of Framework, International Journal of Artificial Individualized Self-Assessment Quizzes with Intelligence in Education 17(2), 89-120. Adaptive Navigation Support. in World Conference [23] Bull, S. & Britland, M. (2007). Group Interaction on E-Learning, Washington, DC, USA: AACE. Prompted by a Simple Assessed Open Learner Model [13] Kumar A. (2005) Generation of problems, answers, that can be Optionally Released to Peers, in P. grade and feedback – case study of a fully automated Brusilovsky, M. Grigoriadou & K. Papanikolaou tutor. ACM Journal on Educational Resources in (eds), Proceedings of Workshop on Personalisation in Computing 5, - Article No. 3. E-Learning Environments at Individual and Group Level, User Modeling 2007.