=Paper=
{{Paper
|id=None
|storemode=property
|title=Detecting and Reflecting Learning Activities in Personal Learning Environments
|pdfUrl=https://ceur-ws.org/Vol-931/paper10.pdf
|volume=Vol-931
|dblpUrl=https://dblp.org/rec/conf/ectel/NussbaumerSNKA12
}}
==Detecting and Reflecting Learning Activities in Personal Learning Environments==
https://ceur-ws.org/Vol-931/paper10.pdf
Detecting and Reflecting Learning Activities in
Personal Learning Environments
Alexander Nussbaumer1 , Maren Scheffel2 , Katja Niemann2 , Milos Kravcik3 ,
and Dietrich Albert14
1
Knowledge Management Institute, Graz University of Technology, Austria
{alexander.nussbaumer,dietrich.albert}@tugraz.at
2
Fraunhofer Institute for Applied Information Technology,
{maren.scheffel,katja.niemann}@fit.fraunhofer.de
3
Lehrstuhl Informatik 5, RWTH Aachen University
kravcik@dbis.rwth-aachen.de
4
Department of Psychology, University of Graz, Austria
dietrich.albert@uni-graz.at
Abstract. This paper presents an approach for supporting awareness
and reflection of learners about their cognitive and meta-cognitive learn-
ing activities. In addition to capture and visualise observable data about
the learning behaviour, this approach intends to make the leaner aware of
their non-observable learning activities. A technical approach and partial
implementation is described, how observable data are used to support
reflection and awareness about non-observable learning activities. Basis
for the technical solution is the extraction of key actions from log data
of the interaction of users with resources. Furthermore, a taxonomy of
learning activities derived from self-regulated learning theory is used for
matching its elements with actually performed actions.
Keywords: learning analytics, learning activities, self-regulated learn-
ing personal learning environments, widget, ontology
1 Introduction
In the recent years a trend became very popular to create small applications for
specific purposes with limited functionalities. A second trend became popular in
the technology-enhanced learning area, that systems and technology appeared
that allow to create learning environments by mashing up such small applications
(e.g. iGoogle5 ). The European research project ROLE6 aims to achieve progress
beyond the state of the art in providing personal support of creating user-centric
responsive and open learning environments. Learners should be empowered to
create and use their own personal learning environments (PLE) consisting of
different types of learning resources.
5
http://www.google.com/ig
6
http://www.role-project.eu
125
Detecting and Reflecting Learning Activities in Personal Learning Environments
Strategies have been developed for supporting the creation of such PLEs
which are in fact bundles of widgets. Ideally, such widget bundles should include
widgets that support the performance of several cognitive and meta-cognitive
learning activities, in order to be used for self-regulated learning. Beside widgets
for domain-specific activities, there is also a need for meta-cognitive activities,
such as goal setting, self-evaluation, or help seeking (see [1]). For the support
of the usage of widget bundles, learning analytics approaches have been imple-
mented. The learners’ interactions with widgets and resources are stored and
graphically displayed. In this way support for reflection and awareness about
the own behaviour is provided.
Existing work in the field of learning analytics typically focuses on collecting
and visualising directly observable data of learner behaviour. For example the
approach presented in [2] describes how student data is collected and how this
data is correlated to the achievement in terms of learning progress. Another
example presented in [3] describes how typically activities of students using
Learning Management Systems (LMS) are captured and used for predictions. In
contrast to these approaches, this paper tries to identify way how meta-cogntive
and non-observable cognitive behaviour can be captured and used for feedback
to the learner. Hence, this paper makes an approach to make the learner aware of
the own cognitive and meta-cognitive processes that cannot be directly observed.
This paper presents an approach to support awareness and reflection of the
non-observable cognitive and meta-cognitive learning activities. Section 2 de-
scribes the underlying pedagogical approach (learning ontology and self-regulated
learning) and the technical basis (extraction of key actions from captured usage
data). Section 3 takes into account these underlying concepts and presents a new
approach to support awareness and reflection, which includes a pedagogical and
technical perspective.
2 Related Work and Baseline
2.1 Contextualised Attention Metadata and Visualisation
Previous work has been done in the context collecting log data in a structured
way and visualising these data. Contextualised Attention Metadata (CAM) cap-
tures the interactions of users with resources and tools. Each time a user performs
an activity with a resource (e.g. a document) in the context of a tool, a dataset
structured according to CAM is created and stored. In this way the behaviour
of users can be tracked [4].
A tool that exploits CAM information for making users aware about the own
learning behaviour is CAMera [5]. CAMera provides simple metrics, statistics
and visualizations of the activities of the learner. It also visualizes a social net-
work based on email communication. CAMera is not restricted to PLEs, but can
also use CAM data created by desktop applications. The objective of CAMera
is stimulating self-monitoring of the user.
The Student Activity Meter (SAM) and the CAM Dashboard are two further
applications that demonstrate how CAM data can be used to support reflection
126
Detecting and Reflecting Learning Activities in Personal Learning Environments
of the learner [6, 7]. SAM applies visualization techniques to enable understand-
ing and discovery of patterns from monitoring data. Depending on the level
of detail in the data, different metrics are provided, like basic time spent and
resource use or forum view and post actions. The overall goal of SAM is to as-
sist both teachers and learners with reflection and awareness of what and how
learners are doing. This can be especially useful for self-regulated learning, where
learners are in control of their own learning. The CAM dashboard aims to enable
students to reflect on their own activity and compare it with their peers.
2.2 Key Action Extraction
In [8] an approach is presented how key actions can be extracted from CAM data.
The extraction of key actions is done by analysing CAM data with techniques
used in the research field of computational linguistics. Using methodologies from
text analysis it is aimed to find patterns within the recorded activities. It is
assumed that key actions can semantically represent the session of learners they
are taken from. In order to find repeated string patterns, the collected CAM
data are analysed with the so-called n-gram approach. The following example
illustrates the technique in a simplified way:
A B C A B D B C A B A A C D
The letters represent the actions of users in a session. The merging of n-grams
is possible if the frequency of the new key action is above a set threshold. Let’s
assume the threshold in this example was set to 2. As no monograms are below
that threshold, all of them are used for further calculations. The bigrams AA,
AC, BD, BA, CD and DB only occur once. Hence, they are discarded from further
calculations and can consequently neither be a key action nor part of one. This
example ends with two key actions, the tetragram BCAB which occurs twice
and D. The detailed approach can be found in [8].
2.3 Self-regulated Learning and Learning Ontology
A model for Self-regulated Learning (SRL) in the context of PLEs has been pro-
posed in [9]. This approach is based on a modified version of the cyclic model
for SRL as proposed by Zimmerman [10]. It states that SRL consists of four
cognitive and meta-cognitive phases (or aspects) that should happen during the
self-regulated learning process, which are planning the learning process, search
for resources, actual learning, and reflecting about the learning process. In addi-
tion to these phases and in order to operationalise them, a taxonomy of learning
strategies and learning techniques (in short SRL entities) has been defined and
assigned to the learning phases. Following the ideas presented in [11], learning
strategies and techniques are defined on the cognitive and meta-cognitive level
and are related to the cyclic phases in order to define explicit activities related
to the SRL learning process.
127
Detecting and Reflecting Learning Activities in Personal Learning Environments
Learning strategies and techniques have also been assigned to widgets stat-
ing that these techniques are supported by the respective widgets. The basic
assumption of creating good PLEs is that the assembly of widgets to a wid-
get bundle should follow a pedagogical approach. Assembling widgets to a PLE
then follows some guidelines which underlying constructs should be contained
and how they should be assembled [12]. The general goal is that a bundle con-
sists of widgets for different cognitive and meta-cognitive activities, so that a
learner has available at least one widget for the most important learning activ-
ities. Examples for meta-cognitive learning activities are goal setting, searching
for resources, or time management. Examples for cognitive activities are brain-
storming, mind mapping, or note taking. While this approach helps for creating
suitable bundles for SRL, it does not help learners how to use such bundles. The
approach presented in this paper addresses this gap.
3 Detection and Reflection of Learning Activities
The goal of this paper is not only to monitor and visualise the observable actions,
but also to monitor the cognitive and meta-cognitive activities that are not di-
rectly measurable. To this end the measurable actions are mapped to cognitive
and meta-cognitive learning activities. To be precise, the key actions extracted
from the CAM data analysis (see Section 2.2) are mapped elements of the learn-
ing ontology (see Section 2.3). The mapping is partially done by the learner
herself, but also supported by an algorithm that takes into account the previous
manual matchings.
3.1 Technical Approach
The overall approach from a technical perspective is depicted in Figure 1. The
learning environment where CAM data is captured is a ROLE space with a set of
widgets. Each widget logs CAM data according to the actions of the learning. In
particular, this includes the actions that a learner performs on the widgets or the
documents represented by the widgets. The CAM data are stored in the CAM
service which is basically a database for CAM events that receives these events
over a REST interface. The analysis component accesses these CAM events, in
order to detect key actions. This is done in the same way as described in Section
2.2 and [8], respectively.
The learning ontology consists of cognitive and meta-cognitive learning ac-
tivities describing typical learning activities. It is modelled in RDF format and
stored within a service that exposes this ontology over a REST interface (using
SPARQL queries). This allows for retrieving lists of learning activities from this
service.
The core component of this approach is the matching component where key
activities are mapped to learning activities. It consists of a user interface and
a back-end service. In the user interface the learner can manually assign learn-
ing activities to extracted key actions. Based on previous assignments, learning
128
Detecting and Reflecting Learning Activities in Personal Learning Environments
activities can be recommended for each of the key actions of the user. So the
learner has not to do the whole assignment work, but can chose from a few
possibilities or just approves the recommended assignment. The back-end ser-
vice provides the key actions for each user and also offers the recommendations.
These recommendations are based on previous assignments that are stored in an
assignment database.
Fig. 1. The conceptual approach.
3.2 Pedagogical Approach
The pedagogical perspective of the presented approach focuses on the the re-
flection and awareness aspects of the learning process. In contrast to existing
approaches where learners are made aware of their observable actions, this ap-
proach intends to make learners aware of their non-observable cognitive and
meta-cognitive activities. Based on literature review a taxonomy of learning ac-
tivities has been created that describe typical learning activities. In order to
match observable and non-observable activities, the learner is presented with
the key actions of their own learning behaviour. Then the learner should assign
which cognitive or meta-cognitive activity is represented by the respective key
actions. This assignment task should stimulate the learner to think about the
cognitive and meta-cognitive learning activities. In addition, the learner gets
suggestions for learning activities that are candidates for the observable perfor-
mance. This mixture of active assignment and support through the suggestions
for assignments makes up the pedagogical approach.
129
Detecting and Reflecting Learning Activities in Personal Learning Environments
3.3 Implementation
Several components of this approach have already been implemented in the con-
text of previous work. A widget container where widgets can be added to a
widget bundle has been developed in the ROLE project. The CAM service is
used to collect CAM data from the widgets and makes them accessible for other
components. The key action detection algorithm has already been implemented
and described in [8]. A learning ontology and a service to make it accessible
has been developed in the context of a mashup recommender for supporting the
creation of widget bundles.
New development needed for this approach is the component that matches
observed key activities with learning activities from the ontology. This compo-
nent will consist of a widget as front-end for the user and a Web services as
back-end for the widget. The back-end provides recommendations for assign-
ments of key actions with learning activities to the leaner. The learner actually
commits assignments, which is stored in a database and used for further recom-
mendations. The recommendation algorithm takes into account all committed
assignments.
4 Conclusion and Outlook
This paper presented an approach for supporting awareness and reflection of
learners about their cognitive and meta-cognitive learning activities. In contrast
to typical learning analytics solutions, this approach focuses on non-observable
learning activities that should be made aware and stimulated. Observable track-
ing data are analysed and key actions are extracted. By assigning learning ac-
tivities to these key actions learners should become aware about the cognitive
and meta-cognitive learning activities.
A technical approach is presented that supports this pedagogical approach.
While some components of the technical approach are already available, others
are under development. Next steps include the development of the assignment
and recommendation component. This component integrates the existing compo-
nents and provides the user interface for the learner. Further work also includes
the evaluation of the first prototype regarding its usefulness.
Acknowledgements The work reported has been partially supported by the
ROLE project, as part of the Seventh Framework Programme of the European
Commission, grant agreement no. 231396.
References
1. Dabbagh, N., Kitsantas, A.: Supporting Self-Regulation in Student-Centered Web-
Based Learning Environments. International Journal on e-Learning 3(1) (2004)
40–47
130
Detecting and Reflecting Learning Activities in Personal Learning Environments
2. Romero-Zaldivar, V.A., Pardo, A., Burgos, D., Kloos, C.D.: Monitoring student
progress using virtual appliances: A case study. Computers and Education 58(4)
(2012) 1058–1067
3. Macfadyen, L.P., Dawson, S.: Mining LMS data to develop an early warning system
for educators: A proof of concept. Computers and Education 54(2) (2010) 588 –
599
4. Schmitz, H.C., Kirschenmann, U., Niemann, K., Wolpers, M.: Contextualized
Attention Metadata. In Roda, C., ed.: Human Attention in Digital Environments.
Cambridge University Press (2011) 186–209
5. Schmitz, H.C., Scheffel, M., Friedrich, M., Jahn, M., Niemann, K., Wolpers, M.:
CAMera for PLE. In Cress, U., Dimitrova, V., Specht, M., eds.: Learning in
the Synergy of Multiple Disciplines. Volume 5794 of Lecture Notes in Computer
Science. Springer Berlin / Heidelberg (2009) 507–520
6. Govaerts, S., Verbert, K., Klerkx, J., Duval, E.: Visualizing activities for self-
reflection and awareness. In Luo, X., Spaniol, M., Wang, L., Li, Q., Nejdl, W.,
Zhang, W., eds.: Advances in Web-Based Learning ICWL 2010. Volume 6483 of
Lecture Notes in Computer Science. Springer Berlin / Heidelberg (2010) 91–100
7. Govaerts, S., Verbert, K., Duval, E., Abelardo, P.: The student activity meter
for awareness and self-reflection. In: Proceedings of CHI Conference on Human
Factors in Computing Systems,, ACM (May 2012) Accepted.
8. Scheffel, M., Niemann, K., Pardo, A., Leony, D., Friedrich, M., Schmidt, K.,
Wolpers, M., Kloos, C.: Usage pattern recognition in student activities. In Kloos,
C., Gillet, D., Crespo Garca, R., Wild, F., Wolpers, M., eds.: Towards Ubiquitous
Learning. Volume 6964 of Lecture Notes in Computer Science. Springer Berlin /
Heidelberg (2011) 341–355
9. Fruhmann, K., Nussbaumer, A., Albert, D.: A Psycho-Pedagogical Framework for
Self-Regulated Learning in a Responsive Open Learning Environment. In Ham-
bach, S., Martens, A., Tavangarian, D., Urban, B., eds.: Proceedings of the In-
ternational Conference eLearning Baltics Science (eLBa Science 2010), Fraunhofer
(2010)
10. Zimmerman, B.J.: Becoming a Self-Regulated Learner: An Overview. Theory Into
Practice 41(2) (2002) 64–70
11. Mandl, H., Friedrich, H.: Handbuch Lernstrategien. Hogrefe, Göttingen (2006)
12. Berthold, M., Lachmann, P., Nussbaumer, A., Pachtchenko, S., Kiefel, A., Albert,
D.: Psycho-pedagogical mash-up design for personalising the learning environment.
In Ardissono, L., Kuflik, T., eds.: Advances in User Modeling. Volume 7138 of
Lecture Notes in Computer Science. Springer Berlin / Heidelberg (2012) 161–175
131