Towards a Learning Dashboard for Community Visualization

Abstract

Learning dashboards (LD) are commonly applied for monitoring and visual analysis of learning activities. The main purpose of LDs is to increase awareness, to support self assessment and re ection and, when used in collaborative learning platforms (CLP), to improve the collaboration among learners. Collaborative learning platforms serve as tools to bring learners together, who share the same interests and ideas and are willing to work and learn together { a process which, ideally, leads to e ective knowledge building. However, there are collaboration and communications factors which a ect the e ectiveness of knowledge creation { human, social and motivational factors, design issues, technical conditions, and others. In this paper we introduce a learning dashboard { the Visualizer { that serves the purpose of (statistically) analyzing and exploring the behaviour of communities and users. Visualizer allows a learner to become aware of other learners with similar characteristics and also to draw comparisons with individuals having similar learning goals. It also helps a teacher become aware of how individuals working in the groups (learning communities) interact with one another and across groups.

Keywords

• Visual analysis Learning dashboards Visualization

-

An increasing number of social learning platforms has been appearing over the past years. The social aspect supports learners to build relations with people who share similar interests, knowledge, and learning goals [ 10 ]. Social learning applications inspire educational designers, researchers and teachers, by bringing groups of people to work and learn together, and form communities through a process known as collaborative learning [ 9 ].

The primary goal of collaborative learning platforms is to support learners in sharing ideas and information, and working together on learning-related tasks. As a result, the participants learning process produces enhanced results [ 1 ]. However, the level of participation between the group members varies. Where some of the participants contribute to discussions and sharing knowledge and resources, others put less e ort into collaboration. One solution to increase the collaboration might be to provide an overview about the productivity of each participant, giving some sort of feedback, using visualizations. For example, when showing the number of messages, answers, and shared resources, the participant can easily monitor her/his collaboration. Also, the learner can follow the tra c surrounding the group she/he is collaborating with. As a result, the user might be motivated to take more responsibilities for the own success as well as for the whole group. The same participant can use the visualizations to articulate thoughts about the amount of collaboration within the group. For example, having analyzed the visualizations which show the distribution of the collaboration of each group member, the user may feel that someone is a free-rider which could lead to discussing this within the group. As a consequence, the discussion could raise this participants awareness of the entire group process [ 8 ]. Furthermore, by considering the overview of the group processes and activities, the teacher or the administrator of the platform can observe if provided strategies for the collaboration are working as expected, assess whether the quality of comments and shared documents are increasing or decreasing over the time, and understand why students are putting less (or more) e ort into collaboration.

Visualizations facilitate discovering, analyzing and understanding communities. They help us to analyze the characteristics of the communities, to understand how learning happens there, and if and how this process is supported. Furthermore, visualizing learners activities supports self-awareness and re ection. It enables learners to discover the learning activities of the peers, to compare own progress with those of the peers respectively. Consequently, this may increase the awareness and lead learners to invest more e ort into the collaboration. Thus, in this paper we introduce a system, called Visualizer, which can be used to visualize and (statistically) analyze the learner's activities. Furthermore, our system gives users control over which aspects of the data has to be considered and also about which statistics have to be de ned. In contrast to the existing systems [ 2 ], Visualizer is context sensitive (data, persons, tasks) and supports personalization. The system is able to automatically de ne a set of appropriate visualizations and further to lter a subset based on target user's visual preferences and interest. Note that, a target user in our context may either be (i) a learner, who considers her personal data or those of the peers, or (ii) a teacher/administrator who considers the data of a speci c user or of the entire group.

The paper is structured as follows: Section 2 presents previous research conducted in the area of learning dashboars. Section 3 brie y introduces the learning dashboard and Section 4 demonstrates how the dashboard can be used to observe the behaviour of di erent communities build within a class. Finally, the Section 5 summarizes and concludes the paper.

Related Work

Most of the collaborative learning- or social platforms provide a high volume on learning materials but only a few o ers possibilities to monitor and analyze the learning process of the collaborators visually. Supplementary, these systems fall short of de ning the whole visualization process automatically. They implicitly de ne which data value is being mapped to which visual component, leaving no scope to impose further con gurations. Furthermore, there is no support for displaying, exploring and analyzing di erent aspects of data points simultaneously. Yet, the greatest aw of these methods is not accounting for user preferences, background, interest and task. In following we investigate the most relevant systems more in detail.

Classroom Salon [ 12 ] is an on-line social collaboration tool that allows instructors to create, manage, and analyze social networks (Google+, Twitter, Facebook, etc.) to improve student learning process. Students use this tool to create, comment on, and modify documents in collaboration with their colleagues. The instructors, however, bene t from it to monitor the social networks and to gauge both student participation and individual e ectiveness. This tool integrates a small size on basic visualizations (bar chart and pie chart) which are tailored for basic statistics. In contrast, Visualizer includes di erent kind of visualizations (thirteen in total) for di erent type of data| temporal, categorical and numerical. Furthermore, it supports more complex statistics, calculating the average, count, maximum and minimum of the users learning activities, shared resource respectively.

Slice 2.0 [ 13 ] is a framework that interconnects tablets of students with slides used by the teacher. The framework allows students to submit notes and questions on teachers slides the teacher can interact with. For instance, the user can monitor and visualize the notes or even display the notes of a particular student for a group discussion on a large display. The framework supports re ection and group discussion but not the evaluation of students or group progress by providing e.g., statistics about each students and whole groups learning activities.

ALAS-KA [ 14 ] is an interactive tool with the aim to support teachers and students by analyzing learning process in online courses. Using prede ned metrics the tool transforms raw log data into meaningful information that can be used by the actors who intervene in the learning process. The metrics cover, the total use of the platform, correct progress on the platform (how good the interactions have been), time distribution of the use of the platform and user's behaviours when solving exercises such as hint avoidance etc. Although the tool is very e ective, it lacks exploring di erent metrics simultaneously. For instance, it is not possible to investigate visually if there is correlation between the time distribution and the number on exercise abandonments. Visaulizer allows such an analysis using the linking and brushing techniques. The idea behind linking and brushing is to connect two or more views of the same data, so that a change to the representation in one view a ects the representation in the other. This means in our case, when selecting a time point on the rst bar chart, the per3

Approach

centage of the exercise abandonments for the same time would be highlighted automatically on the second bar chart.

The learning dashboard { Visualizer { aims to support users (learner/ teacher/administrator) in visualizing and exploring their data. It is basically divided into two major parts: i) a dataset table for representing and managing the uploaded tabular data, and 2) the dashboard where the user visually analyzes and explores the data. After loading the data, the user rst selects the data elds (= data attributes or columns in a tabular dataset) that should be visualized. Then, the system recommends a list of appropriate visualizations that address the characteristics of the selected elds [ 3 ]. Now, the user can select any of the recommended visualizations from the list and use it for exploring and analyzing the data. Note that every time when user selects a visualization, the visualization will be created automatically without requiring the user to manually formulate the mappings of data elds onto visual properties supported by that visualization [ 6 ].

The following sections brie y introduce each part of the system individually. 3.1

Dataset Table

The Dataset Table provides users with basic functionality for processing their data after uploading it. Using the functions integrated into the table, user can aggregate or lter the data, rename the columns or rows, change the data type of the data elds (integer ! number, integer ! string, number ! string, data ! string) or just merge the columns. Once the user decides that the data represented by the table has the appropriate form, the data is sent to the learning dashboard where the visual exploration can take place. 3.2

Learning Dashboard

The learning dashboard supports visual exploration by recommending only the visualizations that best address the characteristics of the data. Once the dashboard obtains user's data, it carries out two important technical steps. First, the data attributes will be analyzed and categorized in two groups: categorical and numerical data (see left side of Figure 1). The data that cannot be aggregated are categorized as categorical- (strings, dates, location), while those that can be aggregated are categorized as numerical data (integers, numbers, oats). Having the data categorized into two groups, the user can now select the elds she would like to present visually. As a response to this selection, the dashboard de nes a list of visualizations that are appropriate for the selected data. The dashboard informs the user about the recommended visualizations by highlighting the icons each representing a particular visualization (see Figure 1 right, VisPicker). When user clicks a highlighted icon, the corresponding visualization will be automatically created and displayed in an individual window. Note, Visualizer supports the creation of multiple visualizations that are then shown side by side (see Figure 1).

The dashboard provides di erent way for adjusting and personalizing the visual interface. For instance, the user can resize or move the windows, rename the visualizations, change the background color and font, and even change the con guration (mapping of data onto visual properties) of a visualization. To recon gure a visualization, a dialog is used where all meaningful mappings for a particular visualization can be selected. Yet, the personalization of the interface can go beyond merely adjusting the properties of the available visualizations. In the case that the available visualizations cannot properly address user's task and goal, technically versed users can integrate their own visualization components into the running dashboard with a minimum of e ort. To make this possible, Visualizer provides a wizard that guides the user in (i) uploading the necessary les and (ii) de ning and describing the components of the new visualization that are needed for a valid mapping between data and the visualization. On the programming side, implementation of a very simple API for accessing the data, and (optionally) for view coordination, must be provided by the user.

Various interactive features empower users to uncover insights from the visualized data. Similar to the Dataset Table, the dashboard provides certain operations for processing the data, such as aggregation, ltering, sorting, and others. Another important feature the dashboard provides is coordinated linking and brushing [ 5 ]. The main idea behind linking and brushing is to combine di erent visualizations to overcome the shortcomings of single techniques. Interactive changes made in one visualization are automatically re ected in the other ones. Visually, the data values selected by the brush retain their original color, while data elements not selected by the brush are shown in gray.

Finally, Visualizer provides user the opportunity to save the full current state of the dashboard. To achieve that, the user just clicks the "Get Bookmark Link" that creates a con guration link. The captured state of the dashboard includes all created visualizations including their positions, size, and the data mapping, all performed data transformations (aggregations, lters etc.), and the current brush. When opening the bookmark link in the browser, another (or the same) user can instantly recreate this previous state of the dashboard. 4

Visualizing communities in formal learning

Visualizing groups or communities allows a learner to become aware of other learners with similar characteristics and also to draw comparisons with individuals having similar learning goals. It also helps a teacher become aware of how individuals in the groups (communities) that she/he de nes interact with one another and across groups.

For instance, we collected and analyzed data of a semester course with 392 rst year students organized in a blended learning format [ 4 ]. For this study we used the following three datasets: { Motivational Beliefs and Self-Regulation Strategies (MBSRS) questionnaire: A printed version of the questionnaire on motivational beliefs and selfregulation [ 11 ] used together with consent and demographics questions. Note that the course script had embedded questions as quick control for students. { Activity logs: User interaction with course organization pages, content pages, the practical exercises pages, and the interaction with the quizzes. Summary quizzes about each topic were available for one week after the face-to-face class. { Performance and control phases: Points obtained in quizzes as control measures and points in the nal exam as performance measure.

Out of 392 students, 170 (140M,40F) completed the MBSRS questionnaire. We used the collected data to analyze self-regulation behaviour in activity streams. We correlated self-regulation behaviour and performance and used activity coding and sequence analysis to de ne behaviour trajectories. After agglomerative hierarchical clustering of these trajectories, a solution with four clusters was chosen. The clusters illustrated patterns of behaviour divided in 4 groups: Cluster 1 inactive: students who did not engage with the course material (N=94), Cluster 2 continuously active: students who engaged with course material before each class (N=90), Cluster 3 procrastinators: students who become active for deadlines (N=62), Cluster 4 probers: students that concentrate on exercises and quizzes repeatedly (N=75).

The dashboard in Figure 2 shows the groups of students that we previously clustered. The top left bar chart illustrates the four groups in relation to the test points and the top right bar chart illustrates the 4 cluster in relation with the days active on the platform. The bottom left radar chart shows the average of planning, monitoring and regulating activities of each group and the bottom right radar visualizes motivational and self-regulating strategies scores from a subjective instrument in contrast with planning, monitoring and regulation activities carried out per session.

Figure 3 represents the dashboard corresponding to the behaviour of each group and the interconnection between bar and radar chart: when selecting a group in the bar chart (clusters by test point or by days active in the platform) the dashboard highlights the behavior of this particular group regarding to the total average on planning, monitoring and regulating activities (left radar chart) and regarding to motivational, self-regulating strategies, plani cation, monitoring and regulation activities per session. Concretely, Figure 3 illustrates the behaviour of cluster 1 (Inactive). Regarding to the test points (left bar chart) cluster 1 present the lowest performance and regarding days active in the platform also the least number of active days (top right bar chart). With regard to SRL (bottom left radar chart and bottom right radar), in average, they evidenced the right amount of planning, monitoring and regulation activities with average time spent on those activities slightly below the highest performing group (Cluster 2). But, they were not active with enough intensity, as the top right bar-chart shows, less days active. Particularly, they reported a lower motivation and a lower SRS. 5

Conclusion

Visualizer o ers useful features to view and compare behaviour of user and user groups in learning platforms. To do so, it requires that the data provides elds identifying communities (and users) for each activity. The data can then be summarized for community and user, so that activities are assigned to the corresponding community even for users that belong to more than one community. This Visual Analytics dashboard is a vehicle for analysis, that can be used iteratively, interleaved with analytics steps to calculate interesting features of the data and illustrate them. Once the analysis reaches a level of maturity, the full con guration of the dashboard interface can be saved, shared, and used to relate the story in the data. In future, we will focus on guiding the user (learner/teacher) in exploring her data by automatically recommending the next interaction (aggregation, ltering, brushing etc.).

Acknowledgment

This work is funded by the European Horizon 2020 research project AFEL (grant nr. 687916) and CONICET. The Know-Center GmbH is funded within the Austrian COMET Program - managed by the Austrian Research Promotion Agency (FFG).

References

1. Gerry Stahl . What we know about cscl and implementing it in higher education . Chapter Building Collaborative Knowing: Elements of a Social Theory of CSCL , pages 53 - 85 . Kluwer Academic Publishers, Norwell, MA, USA, 2004 .

2. Katrien Verbert , Sten Govaerts, Erik Duval, Jose Luis Santos, Frans Van Assche, Gonzalo Parra , and Joris Klerkx . Learning dashboards: an overview and future research opportunities . Personal and Ubiquitous Computing , 18 ( 6 ): 1499 - 1514 , Aug 2014 .

3. Belgin Mutlu , Eduardo Veas, and Christoph Trattner . Vizrec: Recommending personalized visualizations . ACM Transactions on Interactive Intelligent Systems , 6 ( 4 ): 31 : 1 - 31 : 39 , November 2016 .

4. Anal a , Cicchinelli , and Eduardo E. Veas , and Abelardo , Pardo, and Viktoria , Pammer-Schindler , and Angela Fessl , and Carla Barreiros , and Stefanie Lindstaedt . Finding traces of self-regulated learning in activity streams . In Proceedings of the 8th International Conference on Learning Analytics and Knowledge , LAK 2018 , Sydney , NSW , Australia, 2018 .

5. Gerwald Tschinkel , and Eduardo E. Veas , and Belgin Mutlu , and Vedran Sabol . Using Semantics for Interactive Visual Analysis of Linked Open Data . In Proceedings of the ISWC 2014 Posters & Demonstrations Track a track within the 13th International Semantic Web Conference , ISWC 2014, Riva del Garda , Italy, 2014 .

6. Erhard Rahm , and Philip A. Bernstein . A survey of approaches to automatic schema matching . The VLDB Journal , 10 ( 4 ): 334350 , 2001 .

7. Rene F. Kizilcec , and Mar Perez-Sanagust n , and Jorge J. Maldonado . Self-regulated learning strategies predict learner behavior and goal attainment in Massive Open Online Courses . In Journal Computers&Education . Volume 104 , Pages 18 - 33 , 2017 .

8. Sten Govaerts , Katrien Verbert, Joris Klerkx, and Erik Duval . Visualizing Activities for Self-re ection and Awareness . In advances in Web-Based Learning, ICWL , 2010 .

9. Janssen Jeroen , Erkens Gijsbert, Kanselaar Gellof, Jaspers Jos . Visualization of Participation: Does It Contribute to Successful Computer-supported Collaborative Learning . In Journal Computers&Education . Volume 49 Issue 4, Pages 1037 -1065 , 2007 .

10. Jose Luis Santos, Sten Govaerts, Katrien Verbert, Erik Duval . Goal-oriented visualizations of activity tracking: a case study with engineering students . In Proceedings of the 2Nd International Conference on Learning Analytics and Knowledge , LAK , 2012 .

11. Paul R. Pintrich and Elisabeth V. De Groot . Motivational and self-regulated learning components of classroom academic performance . Journal of Educational Psychology ( 1990 ), 3340 .

12. Roberto Martinez Maldonado, Judy Kay, Kalina Yacef, and Beat Schwendimann: An interactive teachers dashboard for monitoring groups in a multi-tabletop learning environment . In: Proceedings of the 11th international conference on intelligent tutoring , 2012 .

13. Wade Fagen , and Sam Kamin: Developing device-independent applications for active and collaborative learning with the SLICE framework . In: Amiel T , Wilson B (eds) Proceedings of world conference on educational multimedia, hypermedia and tele- communications , 2012 .

14. Jose A. Ruiperez-Valiente , Pedro J . Mun ~oz- Merino , Derick Leonya, and Carlos Delgado Kloos: ALAS-KA: A Learning analytics extension for better understanding the learning process in the KHAN Academy platform . In: Computers in Human Behavior , 139 - 148 , 2015