This paper presents an approach for supporting awareness and reflection of learners about their cognitive and meta-cognitive learning activities. In addition to capture and visualise observable data about the learning behaviour, this approach intends to make the leaner aware of their non-observable learning activities. A technical approach and partial implementation is described, how observable data are used to support reflection and awareness about non-observable learning activities. Basis for the technical solution is the extraction of key actions from log data of the interaction of users with resources. Furthermore, a taxonomy of learning activities derived from self-regulated learning theory is used for matching its elements with actually performed actions.
In the recent years a trend became very popular to create small applications for specific purposes with limited functionalities. A second trend became popular in the technology-enhanced learning area, that systems and technology appeared that allow to create learning environments by mashing up such small applications (e.g. iGoogle5). The European research project ROLE6 aims to achieve progress beyond the state of the art in providing personal support of creating user-centric responsive and open learning environments. Learners should be empowered to create and use their own personal learning environments (PLE) consisting of different types of learning resources.
Strategies have been developed for supporting the creation of such PLEs
which are in fact bundles of widgets. Ideally, such widget bundles should include
widgets that support the performance of several cognitive and meta-cognitive
learning activities, in order to be used for self-regulated learning. Beside widgets
for domain-specific activities, there is also a need for meta-cognitive activities,
such as goal setting, self-evaluation, or help seeking (see [
Existing work in the field of learning analytics typically focuses on collecting and visualising directly observable data of learner behaviour. For example the approach presented in [2] describes how student data is collected and how this data is correlated to the achievement in terms of learning progress. Another example presented in [3] describes how typically activities of students using Learning Management Systems (LMS) are captured and used for predictions. In contrast to these approaches, this paper tries to identify way how meta-cogntive and non-observable cognitive behaviour can be captured and used for feedback to the learner. Hence, this paper makes an approach to make the learner aware of the own cognitive and meta-cognitive processes that cannot be directly observed.
This paper presents an approach to support awareness and reflection of the non-observable cognitive and meta-cognitive learning activities. Section 2 describes the underlying pedagogical approach (learning ontology and self-regulated learning) and the technical basis (extraction of key actions from captured usage data). Section 3 takes into account these underlying concepts and presents a new approach to support awareness and reflection, which includes a pedagogical and technical perspective. 2 2.1
Previous work has been done in the context collecting log data in a structured way and visualising these data. Contextualised Attention Metadata (CAM) captures the interactions of users with resources and tools. Each time a user performs an activity with a resource (e.g. a document) in the context of a tool, a dataset structured according to CAM is created and stored. In this way the behaviour of users can be tracked [4].
A tool that exploits CAM information for making users aware about the own learning behaviour is CAMera [5]. CAMera provides simple metrics, statistics and visualizations of the activities of the learner. It also visualizes a social network based on email communication. CAMera is not restricted to PLEs, but can also use CAM data created by desktop applications. The objective of CAMera is stimulating self-monitoring of the user.
The Student Activity Meter (SAM) and the CAM Dashboard are two further applications that demonstrate how CAM data can be used to support reflection of the learner [6, 7]. SAM applies visualization techniques to enable understanding and discovery of patterns from monitoring data. Depending on the level of detail in the data, different metrics are provided, like basic time spent and resource use or forum view and post actions. The overall goal of SAM is to assist both teachers and learners with reflection and awareness of what and how learners are doing. This can be especially useful for self-regulated learning, where learners are in control of their own learning. The CAM dashboard aims to enable students to reflect on their own activity and compare it with their peers. 2.2
In [8] an approach is presented how key actions can be extracted from CAM data. The extraction of key actions is done by analysing CAM data with techniques used in the research field of computational linguistics. Using methodologies from text analysis it is aimed to find patterns within the recorded activities. It is assumed that key actions can semantically represent the session of learners they are taken from. In order to find repeated string patterns, the collected CAM data are analysed with the so-called n-gram approach. The following example illustrates the technique in a simplified way:
A B C A B D B C A B A A C D
The letters represent the actions of users in a session. The merging of n-grams is possible if the frequency of the new key action is above a set threshold. Let’s assume the threshold in this example was set to 2. As no monograms are below that threshold, all of them are used for further calculations. The bigrams AA, AC, BD, BA, CD and DB only occur once. Hence, they are discarded from further calculations and can consequently neither be a key action nor part of one. This example ends with two key actions, the tetragram BCAB which occurs twice and D. The detailed approach can be found in [8]. 2.3
A model for Self-regulated Learning (SRL) in the context of PLEs has been proposed in [9]. This approach is based on a modified version of the cyclic model for SRL as proposed by Zimmerman [10]. It states that SRL consists of four cognitive and meta-cognitive phases (or aspects) that should happen during the self-regulated learning process, which are planning the learning process, search for resources, actual learning, and reflecting about the learning process. In addition to these phases and in order to operationalise them, a taxonomy of learning strategies and learning techniques (in short SRL entities) has been defined and assigned to the learning phases. Following the ideas presented in [11], learning strategies and techniques are defined on the cognitive and meta-cognitive level and are related to the cyclic phases in order to define explicit activities related to the SRL learning process.
Learning strategies and techniques have also been assigned to widgets stating that these techniques are supported by the respective widgets. The basic assumption of creating good PLEs is that the assembly of widgets to a widget bundle should follow a pedagogical approach. Assembling widgets to a PLE then follows some guidelines which underlying constructs should be contained and how they should be assembled [12]. The general goal is that a bundle consists of widgets for different cognitive and meta-cognitive activities, so that a learner has available at least one widget for the most important learning activities. Examples for meta-cognitive learning activities are goal setting, searching for resources, or time management. Examples for cognitive activities are brainstorming, mind mapping, or note taking. While this approach helps for creating suitable bundles for SRL, it does not help learners how to use such bundles. The approach presented in this paper addresses this gap. 3
The goal of this paper is not only to monitor and visualise the observable actions, but also to monitor the cognitive and meta-cognitive activities that are not directly measurable. To this end the measurable actions are mapped to cognitive and meta-cognitive learning activities. To be precise, the key actions extracted from the CAM data analysis (see Section 2.2) are mapped elements of the learning ontology (see Section 2.3). The mapping is partially done by the learner herself, but also supported by an algorithm that takes into account the previous manual matchings. 3.1
The overall approach from a technical perspective is depicted in Figure 1. The learning environment where CAM data is captured is a ROLE space with a set of widgets. Each widget logs CAM data according to the actions of the learning. In particular, this includes the actions that a learner performs on the widgets or the documents represented by the widgets. The CAM data are stored in the CAM service which is basically a database for CAM events that receives these events over a REST interface. The analysis component accesses these CAM events, in order to detect key actions. This is done in the same way as described in Section 2.2 and [8], respectively.
The learning ontology consists of cognitive and meta-cognitive learning activities describing typical learning activities. It is modelled in RDF format and stored within a service that exposes this ontology over a REST interface (using SPARQL queries). This allows for retrieving lists of learning activities from this service.
The core component of this approach is the matching component where key activities are mapped to learning activities. It consists of a user interface and a back-end service. In the user interface the learner can manually assign learning activities to extracted key actions. Based on previous assignments, learning activities can be recommended for each of the key actions of the user. So the learner has not to do the whole assignment work, but can chose from a few possibilities or just approves the recommended assignment. The back-end service provides the key actions for each user and also offers the recommendations. These recommendations are based on previous assignments that are stored in an assignment database. The pedagogical perspective of the presented approach focuses on the the reflection and awareness aspects of the learning process. In contrast to existing approaches where learners are made aware of their observable actions, this approach intends to make learners aware of their non-observable cognitive and meta-cognitive activities. Based on literature review a taxonomy of learning activities has been created that describe typical learning activities. In order to match observable and non-observable activities, the learner is presented with the key actions of their own learning behaviour. Then the learner should assign which cognitive or meta-cognitive activity is represented by the respective key actions. This assignment task should stimulate the learner to think about the cognitive and meta-cognitive learning activities. In addition, the learner gets suggestions for learning activities that are candidates for the observable performance. This mixture of active assignment and support through the suggestions for assignments makes up the pedagogical approach. 3.3
Several components of this approach have already been implemented in the context of previous work. A widget container where widgets can be added to a widget bundle has been developed in the ROLE project. The CAM service is used to collect CAM data from the widgets and makes them accessible for other components. The key action detection algorithm has already been implemented and described in [8]. A learning ontology and a service to make it accessible has been developed in the context of a mashup recommender for supporting the creation of widget bundles.
New development needed for this approach is the component that matches observed key activities with learning activities from the ontology. This component will consist of a widget as front-end for the user and a Web services as back-end for the widget. The back-end provides recommendations for assignments of key actions with learning activities to the leaner. The learner actually commits assignments, which is stored in a database and used for further recommendations. The recommendation algorithm takes into account all committed assignments. 4
This paper presented an approach for supporting awareness and reflection of learners about their cognitive and meta-cognitive learning activities. In contrast to typical learning analytics solutions, this approach focuses on non-observable learning activities that should be made aware and stimulated. Observable tracking data are analysed and key actions are extracted. By assigning learning activities to these key actions learners should become aware about the cognitive and meta-cognitive learning activities.
A technical approach is presented that supports this pedagogical approach. While some components of the technical approach are already available, others are under development. Next steps include the development of the assignment and recommendation component. This component integrates the existing components and provides the user interface for the learner. Further work also includes the evaluation of the first prototype regarding its usefulness.
Acknowledgements The work reported has been partially supported by the ROLE project, as part of the Seventh Framework Programme of the European Commission, grant agreement no. 231396. 2. Romero-Zaldivar, V.A., Pardo, A., Burgos, D., Kloos, C.D.: Monitoring student progress using virtual appliances: A case study. Computers and Education 58(4) (2012) 1058–1067 3. Macfadyen, L.P., Dawson, S.: Mining LMS data to develop an early warning system for educators: A proof of concept. Computers and Education 54(2) (2010) 588 – 599 4. Schmitz, H.C., Kirschenmann, U., Niemann, K., Wolpers, M.: Contextualized Attention Metadata. In Roda, C., ed.: Human Attention in Digital Environments.
Cambridge University Press (2011) 186–209 5. Schmitz, H.C., Scheffel, M., Friedrich, M., Jahn, M., Niemann, K., Wolpers, M.: CAMera for PLE. In Cress, U., Dimitrova, V., Specht, M., eds.: Learning in the Synergy of Multiple Disciplines. Volume 5794 of Lecture Notes in Computer Science. Springer Berlin / Heidelberg (2009) 507–520 6. Govaerts, S., Verbert, K., Klerkx, J., Duval, E.: Visualizing activities for selfreflection and awareness. In Luo, X., Spaniol, M., Wang, L., Li, Q., Nejdl, W., Zhang, W., eds.: Advances in Web-Based Learning ICWL 2010. Volume 6483 of Lecture Notes in Computer Science. Springer Berlin / Heidelberg (2010) 91–100 7. Govaerts, S., Verbert, K., Duval, E., Abelardo, P.: The student activity meter for awareness and self-reflection. In: Proceedings of CHI Conference on Human Factors in Computing Systems,, ACM (May 2012) Accepted. 8. Scheffel, M., Niemann, K., Pardo, A., Leony, D., Friedrich, M., Schmidt, K., Wolpers, M., Kloos, C.: Usage pattern recognition in student activities. In Kloos, C., Gillet, D., Crespo Garca, R., Wild, F., Wolpers, M., eds.: Towards Ubiquitous Learning. Volume 6964 of Lecture Notes in Computer Science. Springer Berlin / Heidelberg (2011) 341–355 9. Fruhmann, K., Nussbaumer, A., Albert, D.: A Psycho-Pedagogical Framework for Self-Regulated Learning in a Responsive Open Learning Environment. In Hambach, S., Martens, A., Tavangarian, D., Urban, B., eds.: Proceedings of the International Conference eLearning Baltics Science (eLBa Science 2010), Fraunhofer (2010) 10. Zimmerman, B.J.: Becoming a Self-Regulated Learner: An Overview. Theory Into
Practice 41(2) (2002) 64–70 11. Mandl, H., Friedrich, H.: Handbuch Lernstrategien. Hogrefe, Go¨ttingen (2006) 12. Berthold, M., Lachmann, P., Nussbaumer, A., Pachtchenko, S., Kiefel, A., Albert, D.: Psycho-pedagogical mash-up design for personalising the learning environment. In Ardissono, L., Kuflik, T., eds.: Advances in User Modeling. Volume 7138 of Lecture Notes in Computer Science. Springer Berlin / Heidelberg (2012) 161–175