In recent years, several researchers have been developing methodologies, technologies and systems to support the assembly of learning services, tools and resources in personal learning environments (PLEs). The overall goal is to enrich or even replace traditional learning management systems like Moodle and Blackboard with mash-ups of widgets and services that can be combined and configured in a flexible way, according to the specific needs of the user. In this paper, we describe our approach to visualize user interactions with widgets and services within such personal learning environments. These visualizations enable the exploration of learner behavior within PLEs. The major objective is to improve and evolve PLE related research and development according to feedback mechanisms based on empirical observation. In this paper, we present an overview of our method to capture usage behavior and a first prototype of a visualization dashboard that enables the analysis and interpretation of these data as a basis for evaluation.
The development and proliferation of Web 2.0 technologies has
impacted the way users interact with information and with each
other. Web-based communities, wikis, blogs and social networks
have experienced an exponential growth of both users and
content, leading to potentially viral social networking,
collaboration, communication and resource sharing opportunities.
The abundance of these technologies and services creates many
new opportunities in various areas. One of those areas is
Technology Enhanced Learning (TEL) that aims to bring together
new technological developments and learning models to support
learning processes. The ROLE project [
In this paper, we present a schema to generate usage data within
widgets and services in a uniform way. Then, we present our
dashboard that enables the visualization of usage data as a basis to
detect changes in usage patterns. The purpose is to detect
variations in the use of PLEs based on changes in usage patterns
with widgets and services. The dashboard also provides insights
into whether other similar widgets and services are also affected.
The Latour’s actor-network theory (ANT) [
The paper is organized as follows: in the next section, we present a schema for representing usage data in a uniform way. Section 3 presents the objectives of analyzing these data to detect changes in usage patterns and evolutions in PLEs. Implementation details of the visualizations and the back-end infrastructure to store usage data are presented in Section 4. A use case is presented in Section 5. Conclusions and future work are described in Section 6.
This paper focuses on visualizing usage data to enable awareness
of user activities in a PLE and the evolution of widget usage.
PLEs have high evolvable characteristics [
CAM was developed to enable the capturing of usage data from a
variety of applications, such as widgets and services that are
aggregated in PLEs. CAM captures all kinds of user actions and
can capture information about:
the user
the application used
the action type (i.e. read, write, save, print, etc.)
the resource on which the action occurred
additional contextual information that may be available,
such as time, location of the user, operating system or
information related to the session or the IP address
CAM, which will be standardized by the CEN WS-LT working
group on social data [
CAM data can be analyzed to provide an overview about: Where (i.e. in which tools) did the action take place? This enables discovery of popularity, usage bursts and trends of tools. It can also uncover patterns like applications becoming unpopular, rising stars and new applications taking over older applications.
When are the actions generated? This information is for instance useful in controlled environments such as formal learning environments where activities are usually scheduled. It is also quite useful in less controlled or more blended environments, to understand when learners are actually active.
What happens in the environment? The dashboard makes it possible to zoom in on specific action types and resource types, so that we can study in detail what users are doing with resources. • • • • • • • •
We have implemented a tool to visualize CAM data. The
architecture of this tool is composed of three main components
(Figure 1):
•
•
•
•
•
•
CAM storage layer. This layer supports storing and
retrieving usage data. The information is exposed by a
SOAP service developed by Fraunhofer-Institut für
Angewandte Informationstechnik (FIT) [
Pre-analysis layer. The pre-analysis layer pre-calculates statistics to avoid performance issues when users interact with the dashboard. This layer connects to the CAM storage layer and saves aggregated information in its own database, which is exposed by REST services. CAM dashboard. The dashboard is implemented in HTML and JavaScript using the Google Chart Tools JavaScript library [2].
Figure 2 shows a typical screen of the dashboard. It contains three visualizations that provide information about where, when and what is happening in learning environments.
Where and when: An annotated time line visualization (Figure 2, visualization labeled with number 2) shows the total activity and the activity of every application over time. The annotated time line at the bottom enables the user to restrict the period of time that the visualization shows.
When: A vertical bar chart (labeled with number 3) shows the average activity by day of the week.
What: The horizontal bar chart with label 4 shows the activity based on the type of action that users perform. The second horizontal bar chart with label 5 shows activity based on the type of resource involved in the action. They can provide information about what kind of actions and resources are popular. At the top of the dashboard (label 1), there is the option of filtering per application. The modification of this filter affects all visualizations. The charts are also interlinked. Table 1 presents which actions trigger updates of other visualizations.
This use case describes the behavior of a specific widget in a PLE
environment, deployed during a course at RWTH Aachen
University during the period May to July 2010. After this period,
the environment was occasionally used in an informal way. In this
PLE, four widgets were used. The widgets use Open Social [
• ABC Testing widget. This widget was only used during the first two weeks (this information is also displayed in the dashboard). • Cam Widget. This widget tracks the Open Social communication and translates this communication to CAM. Users can deactivate or activate tracking of their data. • Role Web 2.0 Knowledge Map. This widget allows to
search for articles by entering keywords. • XMPP Multiuser Chat. This widget enables chat functionality between different users based on the XMPP technology. In this use case, we will focus on the XMPP Multiuser Chat widget because it is the most active in terms of event communication providing us more information about its particular characteristics. We will now explain how we can derive the conclusions from: 1. Detect changes on usage patterns: When we select theXMPP Multiuser Chat in part 1 one of Figure 2 and we obtain an overview of the overall activity (Figure 3). The annotated time line chart (Figure 3) enables us to see that the activity was concentrated during the period from May to July 2010. After this period, the activity was reduced considerably. In the “events per type of action” chart (Figure 3), we can see that people enter to room chats more than sending messages (if we zoom into the decreasing period, this behavior is emphasized). A possible reason is that the chat room was not used actively for communication. 2. Evaluate whether other similar widgets are affected: In this use case, other similar widgets do not exist. However, we can compare with the general behavior, selecting the tag “Total” (Figure 2 label 1). We can see that the total activity decreased proportionally to the XMPP Multiuser activity. 3. Detect which changes have
been introduced in the environment: There are no remarkable modifications. The widget ABC was not used anymore after two weeks, but it did not affect the overall activity (Figure 4). There are no remarkable modifications such as activity of a new widget or the activity of a widget decreasing before the others. 4. Evaluate how the changes can affect to the first variation in the behavior pattern: If the activity of one widget decreases before the others, it could point to usability issues with a specific widget. However, the activity always decreases proportionally in all the widgets. In summary, this use case illustrates:
From July to August 2010, there is a remarkable variation of the behavior. During this period, the widgets were progressively less used.
As the activity decreased proportionally in each widget, we cannot identify any of them as the influence of the change.
The visualization of usage data enables to detect usage patterns in the use of widgets and their composition in PLEs. Changes in these patterns can be caused by software reasons, but also by other external influences. In our use case, the external influences were the duration of the course.
The dashboard is ongoing work, but we have some preliminary conclusions.
Based on ANT premises, the non-human entities have an important role on the software evolution. The dashboard aims to be useful in the detection, variation and explanation of usage patterns as illustrated in the aforementioned use case, so that PLE development can be grounded in feedback loops from analysis of actual use by the intended target users.
Although a specification like CAM provides some interoperability for usage data, problems appear with the semantic interpretation of the information. For instance, if the definition of an action is not agreed upon, then different actions may be merged if they have the same name. Most of these issues can be solved defining some restrictions and using a vocabulary, and providing some technical guides to use the specification. However, this is a difficult trade-off because if the vocabulary management is too restricted, the specification will not be adopted.
An evaluation of the dashboard is planned that will measure the
usability and usefulness of the dashboard for such purposes. The
current implementation of the dashboard uses real-life data
tracked in a computer science course at the RWTH Aachen
University. The evaluation will focus on usability quality
components [
The research leading to these results has received funding from the European Community Seventh Framework Programme (FP7/2007-2013) under grant agreement no 231396 (ROLE). Katrien Verbert is a Postdoctoral Fellow of the Research Foundation - Flanders (FWO). [2] Google Chart tools
http://code.google.com/apis/charttools/index.html network. Proceedings of 2nd Intl. Workshop on Software Evolvability. IEEE computer society press.