=Paper=
{{Paper
|id=None
|storemode=property
|title=Supporting Self-Regulated Learning in Personalised Learning Envrionments
|pdfUrl=https://ceur-ws.org/Vol-945/paper10.pdf
|volume=Vol-945
|dblpUrl=https://dblp.org/rec/conf/ltec/KroopBNA12
}}
==Supporting Self-Regulated Learning in Personalised Learning Envrionments==
https://ceur-ws.org/Vol-945/paper10.pdf
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
Supporting Self-Regulated Learning in Personalised Learning Envrionments
Sylvana Kroop Marcel Berthold Alexander Nussbaumer Dietrich Albert
Centre for Social Innovation, Knowledge Management Knowledge Management Knowledge Management
Vienna, Austria. Department Institute Graz University of Institute Graz University of Institute Graz University of
Technology & Knowledge Technology, Graz, Austria Technology, Graz, Austria Technology, Graz, Austria
kroop@zsi.at marcel.berthold@tugraz.at alexander.nussbaumer@tugraz.at dietrich.albert@tugraz.at
Abstract. The advantage of Personal Learning Environments potential to enable and facilitate both informal and formal
(PLEs) is to empower a learner in taking control over his/her learning.
own learning process. The shift from just being controlled by The paper presents three PLE scenarios which have
a teacher towards taking control by oneself in a self- been developed in the ROLE project. In real world
regulated learning (SRL) way can be basically initialised by
testbeds learners are confronted with new ways of
providing learning environments that can be personalised
and individually adapted or created instead of using 'one learning by working with the provided PLE scenarios.
size fits all' learning environments. A lot of research and While the use of any PLE should trigger self-regulated
development on this subject has been done in the EU-Project learning it is especially the third and last PLE scenario
ROLE (role-project.eu). In this context extensive which has been implemented a consequent mechanism to
experiments have been conducted with widget-based PLEs. support SRL.
Scenarios have been created, implemented, tested and This paper investigates the attitudes and reasons for
evaluated in real world settings. The contribution of this acceptance of PLE technology by students and teachers.
paper is the presentation of a) three widget-based PLE
scenarios, b) evaluation results on comparing the value of II. THE CHALLENGES OF SELF-REGULATED LEARNING
the presented PLE scenarios and c) evaluation results on IN PERSONAL LEARNING ENVRIONMENTS
comparing students and teachers point of views against the
presented PLE scenarios including SRL aspects. A. SRL in Technology-enhanced Learning
Environments
Keywords; personal learning environments, self-regulated
learning, open educational ressources, widgets, evaluation In the field of self-regulated learning (SRL) research it
results is often pointed to the important role of learners’ strategic
use of cognitive and metacognitive strategies to regulate
I. INTRODUCTION their learning [2], [3], [4]. Still many learners show
difficulties in applying concrete metacognitive strategies
Responsive Open Learning Environments (ROLE) are such as planning, goal setting, monitoring, evaluating and
based on the idea of Personal Learning Environments as a result perform less successful [5]. For this reason,
(PLEs) by exploiting Cloud Computing Technology much work has been focused on the assessment of
(examples are presented in chapter III). Instead of using students’ SRL strategies to support the learning behaviour
traditional learning environments which provide tools and accordingly. This work is usually bound to highly
content by one single provider and are often owned by controlled learning environment such as intelligent
one specific educational organization ROLE exploits all (tutoring) systems [5], [6], [7]. However, understanding,
existing and developing open educational sources scaffolding or/and facilitating students’ SRL skills is
including all popular Web2.0 resources such as especially important in (responsive) open learning
Wikipedia, YouTube or Flickr. Historically the idea of environments. In such open environments goals are less
PLEs is based on the fact that most learning takes place clear and obvious; therefore students might not necessarily
informally, in different contexts and scenarios, and that be able to predict the outcome of the learning activity or
content is not provided by one single provider. Following the optimal learning path.
this idea ROLE provides a framework essentially Nevertheless, it could be found that PLEs provide
consisting of “enabler spaces” on the one hand and tools, opportunities to enhance SRL skills, especially
content, services on the other hand [1]. Using this metacognitive skills, but learners need additional help and
equipment everyone is invited to individually create guidance [5] during the learning process. In this regard the
his/her PLE. In PLE research it is seen as essential to have concept of freedom and guidance comes into play. The
a learner challenged by offering him/her to create their concept of freedom and guidance is important, because
individually controlled and preferred learning highly motivated learners attain a better learning
environment in order to trigger and motivate more self- performance if they have more control over their learning,
regulated learning. Moreover this approach has the but lower motivated learners attain better learning
47
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
performance if they get more guidance [8]. Issing noted learning phases of a SRL PM to learning strategies,
that this is also applicable to hypermedia learning techniques and activities [11]. In addition, it is shown
environments. how these SRL entities are linked to tool functionalities
In this regard it should be envisioned to develop and therefore bridge psycho-pedagogical information and
services and learning environments that can be adapted to learning tools like widgets in our presented case studies.
the individually degree of guidance and freedom according
III. SCENARIOS
to the learner’s needs and therefore offer the learner an
optimal balanced level of control and responsibility for his This chapter describes three widget-based PLE
or her learning environment [9]. scenarios which were evaluated (see chapter IV to VI).
In the ROLE project the basic equipment for creating
B. A Self-Regulated Learning Process Model PLEs has been developed according to the idea of an easy
Procedure drag and drop system of widgets. Browser-based
In PLEs learners are in the position to create their own prototypes have been developed like sketched in Figure 2.
learning environment and shape it to their personal needs
and learning objectives. In order to provide support in such
an open learning approach an underlying and psycho-
pedagogical sound model which represents the theoretical
backbone of open environment learning has been defined,
the Self-Regulated Learning Process Model (SRL PM).
The SRL PM builds on the cyclic self-regulated learning
model proposed by Zimmerman [10], which describes the
learning process via three learning phases, namely Figure 2: Browser- and Widget-based PLE concept
forethought, learning and self-reflection. In open learning
environments this three learning phase model was On the one hand a repository (widget store) is
extended to reflect the need of selecting web-based necessary to store and administrate useful widgets. On the
learning resources, mostly widgets, to build and mash-up a other hand an enabler space (widget space) is necessary to
PLE. have learners their individually preferred widgets
This extension leads to the four phase SRL PM integrated, used and managed in their personal style.
including the phases of: (1) learner profile information is Starting from this provided prototype essentially
defined or revised, (2) learner finds and selects learning consisting of Widget Store and Widget Space the creation
resources, (3) learner works on selected resources, and (4) of PLEs has been tested in real world use cases and
learner reflects and reacts on learning strategies, scenarios which are described in the following sections.
achievements and usefulness (see Figure 1) [9].
A. Scenario I
In the first scenario learners were provided with the
ROLE Widget Store [12] but they could also make
extended use of widgets by using iGoogle gadgets [13];
(iGoogle gadgets: here the Google term for widgets).
Furthermore, learners had the choice to either use iGoogle
[14] or the ROLE sandbox [15] as an enabler space.
In the following the ROLE widget store is described as
well as an example how ROLE widgets have been
Figure 1: Self-Regulated Learning Process Model (SRL PM) integrated and used in iGoogle.
According to this model, especially meta-cognitive
activities are supported by focusing on the
recommendation of learning activities which can be
performed through the usage of learning resources and
therefore enhance self-regulated learning.
ROLE services such as the Mash-Up Recommender
Widget (see Figure 7) offer guidance and help learners by
presenting recommendations and according explanations,
without limiting the degree of freedom, as the learner can
freely choose between the recommendations made by
ROLE services or other alternatives. This concept is
based on an ontology that builds on a connection of Figure 3: ROLE Widget Store
48
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
The ROLE Widget Store (Figure 3) is a living system Google translator all at the same time for comparisons of
and repository of open educational resources. It hosts and translations [12].
offers all kind of learning widgets. For registered
developers and users it is possible to “add a new widget”
(see icon on the upper navigation in Figure 3) whenever
they have found or created a useful widget with
pedagogical value. Everyone interested in these kinds of
open educational resources can make use of it.
Figure 5: ROLE Translator Widget embedded in the Desktop-
Sidebar
This kind of PLE is created to efficiently work on a
text document. While reading or writing a text in a
foreign language the ROLE translator widget is always
visible and usable in the desktop-sidebar. A click on the
sidebar-widget-icon will open the widget like sketched in
Figure 4: Widget Space iGoogle filled with ROLE widgets Figure 5. The widget will stay in the front while copying a
term from the document in the background to transfer this
Figure 4 [16] shows one example of a browser- and
term to the translator widget. The translation is shown
widget-based PLE. In this example the iGoogle
including the resource of translation (dict.cc, Wikipedia,
environment hosts a PLE. The widgets were added from
Google, etc.). This mean of widget integration should
the ROLE Widget Store.
ensure a very efficient way of learning and working. It
This scenario had already been tested by students in
enables the user to learn new terms by using the widget
2011 at an early stage of development. Results of this
but without losing sight of the text document. Moreover,
evaluation were already presented and discussed at
using several resources of Web2.0 based translations
PLE2011 conference [17], [18].
stimulates the user to have a more critical reflection of the
B. Scenario II offered translations.
The following use case is not an implemented
prototype, but a mock-up which has been created as a
consequence of early stage evaluations [17]. A result of
these early evaluations was the desire of some users to not
be constrained to a browser-based widget-space, but to
use single widgets wherever and whenever they want, e.g.
on a desktop and offline.
The mock-up scenarios presented in Figure 5 and
Figure 6 have been used to discuss and evaluate taking
into account teachers’ and students’ perspectives (see
chapters IV to VI). Both mock-up scenarios are designed
with the idea to be not restricted to use the widgets within Figure 6: Vocabulary Trainer Widget embedded in the Browser-
a browser-based widget space like iGoogle. Moreover Sidebar
instead of using a collection of widgets at the same place
it should also be possible to select and use only one very Figure 6 presents the use of a vocabulary trainer
specific widget. widget which can be opened in the browser sidebar right
Thus, choosing between several means of next to the text a user is working on. While reading the
(personalised) integrating and using the offered widgets text in a foreign language terms might appear a user is not
should be one distinctive added value of all widgets in the familiar with and wants to systematically train them. Then
ROLE Widget Store. the terms can be added to the vocabulary trainer widget.
Figure 5 presents the use of the ROLE translator The widget has been implemented a slightly modified
widget which accesses and displays the results of different Leitner system [19]. Thus, vocabulary can be trained
popular resources such as LEO.org, dict.cc, Wikipedia, efficiently by using this widget. For translations the same
Web services are used as in the mentioned ROLE
49
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
Translator widget. Moreover Flickr is used to suggest the ROLE Widget Store. Instead of the four SRL phases,
pictures for visualising the terms. The widget has four the template can also consist of learning activities on a
functionalities represented by four tabs: “Add”, “List”, finer granularity level, namely learning strategies and
“Train” and “Stats”. A detailed description of this widget learning techniques. Such templates can be created using a
and further widget bundles can be found at the ROLE special authoring tool [21].
Showcase Platform [20]. The MR can be used to provide guidance on different
Important for the presented evaluation is the fact that levels and for different stakeholders (e.g. teachers,
these mock-up scenarios give ideas of some other ways workplace learners, students, beginners, and advanced
how to use the offered widgets from the ROLE Widget students or experts). A high level of guidance is necessary
Store. for instance for beginners and can be prepared by a
complete predefined PLEs based on a specific template by
C. Scenario III
a teacher or tutor. Later the tutor can share this PLE with
Scenario III presents an implemented prototype to her students who can use it or modify. A lower level of
mash-up PLEs which is called “Mash-Up Recommender” guidance can be provided if the teacher just shares the
(MR, see Figure 7). template with the students, so that they have to create
their own PLE. For example, a teacher could select the
SRL entities goal setting, resource searching, note taking,
and reflecting for a template. Teachers or learners using
this template could easily search these SRL entities for
widgets and include them in a PLE. In this way the PLE
consists of widgets for each SRL entity. Learning
strategies are on a higher abstraction level, which results
in an increased number of widgets that can be
recommended. Learning techniques are on a lower
abstraction level, which leads to a smaller number of
related widgets that can be recommended. While in the
first case the learner gets more widgets recommended and
thus less guidance, in the second case the level of
guidance is higher because of the smaller number of
Figure 7: Mash-Up Recommender Widget (MR) recommended widgets. For a detailed description of the
MR and its technical background see [22].
The unique aspect of the MR is the fact that it
services as a gate and a guide to access the large number IV. FOCUS GROUPS AND EVALUATION
of widgets and gadgets available on the web in a The evaluation took place equally in two focus groups:
reasonable self-regulated way. For this purpose the MR • Teachers: The three scenarios were presented, tested
templates are based on learning activities related to the and evaluated in a teacher workshop taking place at
SRL Process Model described in chapter II. the Aha-Conference 2012 in Vienna [23]. In total 8
The main purpose of the MR is to support the self- participants (4 male, 4 female) from Austria and
regulation of learners in mashing up their learning Germany took part. The age ranged from 27 to 55
environments. Therefore, psycho-pedagogical information (Average age: 40.43). Most of them were teachers at
is transferred into applicable recommendation by using the schools or universities. But there were participants
MR widget. The MR widget can be seen as a filtering who also worked as consultant or technical support at
system that provides more or less widgets that can be higher education institutions.
added to the PLE depending on the used template. The
• Students: The three scenarios were evaluated in the
MR contains a predefined template called SRL template.
The SRL template can consist of the four basic SRL same way in a test bed at the University of Vienna
phases “Planning”, “Searching”, “Learning” and within a course called “Didactical Design” (Sylvana
“Reflecting” which are displayed in the upper navigation Kroop). The course was for 25 Master at the Faculty
of the MR (see Figure 7). Each category contains a of Informatics in summer semester 2012. 22 students
number of relevant widgets, e.g. the category “Reflecting” (11 male, 11 female) regularly participated in the
contains widgets such as recording tools, writing tools, prototype evaluation. The age of students ranged from
mind map tools etc. To have the SRL template adequately 23 to 48 (Average age: 28.48). They all studied in the
working according these four SRL phases a ROLE field of computer science. Some of them were teachers
ontology [11] service has been implemented for the who already taught at schools but still enjoyed their
respective functionalities of the SRL entities (learning academic training. Thus, in the discussion some
strategies, techniques and activities). The ontology students evaluated the scenarios from a teacher’s point
predefines associated widgets which will be returned by of view.
50
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
Although quantitative as well as qualitative data were The question regarding a possible ease of the
collected in the evaluation with both focus groups this personal learning process was altogether also rated most
paper only presents the quantitative results due to page positive in scenario 3: The mean value is 4.50. But at the
limits of this paper. same time there is also the highest standard deviation of
Quantitative data were essentially collected by a short 1.41 revealing a wider disagreement among the
questionnaire in the end of testing and discussing the respondents in this question. In contrast to scenario 3 the
three scenarios. To investigate the main research question worst result is displayed for scenario 1 with a mean value
if and why these PLE scenarios will be accepted or of 2.50. Moreover in this case respondents do agree most
rejected by students and teachers two more concrete indicated by the lowest standard deviation of 0.93. In
questions were ask to think about while testing and other words: While the teachers come to the agreement
discussing each of the three scenarios: that scenario 1 will tend to be an additional burden instead
• The first question was on worsening / improvement of of easing the personal learning process scenario 3 is rated
learning outcome; much better by teachers but with a broader variance of
• the second question on the technical including opinions.
cognitive and time-wise burden / ease of personal Altogether the results in both questions show a
learning process. coherent picture for the three evaluated scenarios: While
The answer categories ranged on a six-point-Likert- scenario 1 can be assumed to be potentially rejected by
scale from 1: worsening to 6: improvement resp. 1: teachers scenario 3 tends to be accepted.
burden to 6: ease, which means: the higher the value the
better the acceptance of the respective scenario.
VI. RESULTS OBTAINED BY STUDENTS
Figure 9 presents the results obtained through the
V. RESULTS OBTAINED BY TEACHERS questionnaire which was filled out by 19 students after
Figure 8 presents the results obtained through the finishing their group discussions.
questionnaire teachers filled out after finishing the group
discussions at the end of the workshop.
Figure 9: Results of Students Workshop (N=19)
Figure 8: Results of Teacher Workshop (n=8) The question regarding a possible improvement in
learning was again rated best in scenario 3: The mean
The graphic shows the mean values and the standard value increased from 3.55 in scenario 1 to 4.11 in scenario
deviation (in brackets) for the three scenarios. Each of the 2 up to 4.68 in scenario 3. The standard deviation (sd)
scenarios was rated by eight teachers according to the two shows that the respondents differ most in rating scenario 1
evaluation criteria described in chapter IV. Due to the (sd=1.34) followed by scenario 2 (sd=1.20) and scenario 3
small number of participants no inference statistical (sd=1.11). In other words: Students not only rated
analyses were conducted. scenario 3 best but also agreed in the answers of this
The question regarding a possible improvement in question in scenario 3 most.
learning was answered most positive in scenario 3: Mean The question regarding a possible ease of the
value of improvement of learning increased from 4.14 in personal learning process was also rated best in scenario
scenario 1 to 4.67 in scenario 2 up to 5.14 in scenario 3. 3 with a mean value of 4.29. Students also agreed in the
The standard deviations show that respondents do not answers of scenario 3 most (sd=1.28) while they had the
differ very much in the assessment of the three scenarios broadest variance of opinions in scenario 1 (sd=1.60)
concerning improvement in learning; it ranges from 1.03 which was rated lowest with the mean value of 3.00.
to 1.07. It tends to be consensus in this question.
51
1st International Workshop on Cloud Education Environments (WCLOUD 2012)
Considering a significance test scenario 3 is [6] C. Conati and K. Vanlehn, “Toward computer-based support of
meta-cognitive skills: A computational framework to coach self-
significantly better than scenario 1 in both questions explanation,” International Journal of Artificial Intelligence in
(Improvement: F2,36 =5.48, p=0.008; Ease: F2,36 =4.52, Education, vol. 11, pp. 389–415, 2000.
p=0.018). Due to the small sample this can be randomly [7] R. Azevedo, A. Johnson, A. Chauncey, and C. Burkett, Eds.,
and thus is not further discussed. “Self-regulated learning with MetaTutor: Advancing the science of
Altogether the results in both questions show again a learning with Metacognitive tools,” in New Science of Learning:
Cognition, Computers and Collaboration in Education, Khine,
coherent picture for the three evaluated scenarios: While Myint Swe and Saleh, Issa M., Amsterdam: Springer, 2012, pp.
the results of scenario 1 neither show a clear tendency to 225–247.
be rejected nor to be accepted scenario 3 clearly tends to [8] L. J. Issing and P. Klimsa, “Instruktions-Design für Multimedia,”
be accepted by students in this comparison of PLE in Information und Lernen mit Multimedia und Internet: Lehrbuch
scenarios. für Studium und Praxis, J. Issing and P. Klimsa, 3. vollständ.
überarb. Auflage., Weinheim: Beltz, 2002, pp. 151–178.
VII. CONCLUSION [9] K. Fruhmann, A. Nussbaumer, and D. Albert, “A Psycho-
Pedagogical Framework for Self-Regulated Learning in a
The use of widgets within a widget space such as Responsive Open Learning Environment,” Proceedings of the
iGoogle was evaluated positive in its easy technical International Conference eLearning Baltics Science (eLBa Science
handling but negative in the challenge to efficiently 2010), pp. 125–138, 2010.
support daily learning activities. Thus there is neither [10] B. J. Zimmerman, “Becoming a Self-Regulated Learner: An
Overview,” Theory Into Practice, vol. 41, no. 2, pp. 64–70, 2002.
acceptance nor a clear rejection of scenario 1.
[11] M. Berthold, I. Dahn, A. Kiefel, P. Lachmann, A. Nussbaumer,
Better accepted was the use of single widgets wherever and D. Albert, “ROLE Learning Ontology. An approach to
and whenever learners wants them to use (e.g. in a structure recommendations for self-regulated learning in
desktop-sidebar or browser-sidebar, online and offline) personalized learning environments,” in Learning Innovations and
Quality (LINQ), 2012, pp. 161–175.
sketched in scenario 2
[12] © ROLE Consortium 2009-2012, “ROLE Widget Store:”
Best accepted was the idea to support self-regulated [Online]. Available: http://www.role-widgetstore.eu/.
learning (SRL) by using a four-phases activity model [13] “iGoogle gadgets.” [Online]. Available:
while learners are challenged to select widgets from a http://www.google.com/ig/directory?hl=de&synd=open.
wide variety (scenario 3). The idea to connect different [14] “iGoogle.” [Online]. Available: http://www.google.com/ig.
stages of SRL (Planning, Searching, Learning, Reflecting) [15] “ROLE sandbox.” [Online]. Available: http://role-sandbox.eu/.
with corresponding widgets was seen most needed and [16] S. Kroop, “Theory and Practice of Personal Learning
most useful. Environments (PLEs): A prototype evaluation.,” Prezi store (full
presentation), 2011. [Online]. Available:
ACKNOWLEDGMENT http://prezi.com/hz8nsn9k3pkq/theory-and-practice-of-personal-
learning-environments-plesa-prototype-evaluation/.
The research leading to these results has received
[17] S. Kroop, “Theory and Practice of Personal Learning
funding from the European Community’s Seventh Environments (PLEs): A prototype evaluation,” in Proceedings of
Framework Programme (FP7/2007-2013) under grant PLE2011 Conference, Southampton, UK, 2011.
agreement no 231396 (ROLE project). [18] S. Kroop, Collected presentations of PLE2011 conference, 2011.
[Online]. Available: http://www.slideshare.net/PLE_SOU/theory-
and-practice-of-ples-a-prototype-evaluation.
REFERENCES [19] S. Leitner, So lernt man lernen., 18. Auflage. Freiburg: Verlag
Herder, 2011.
[1] S. Govaerts, K. Verbert, D. Dahrendorf, C. Ullrich, M. Schmidt,
M. Werkle, A. Chatterjee, A. Nussbaumer, D. Renzel, J. Santos, [20] © ROLE Consortium 2009-2012, “Learn a foreign language by
M. Scheffel, E. Duval, M. Friedrich, and E. Law, “Towards reading text,” ROLE Showcase Platform. [Online]. Available:
Responsive Open Learning Environments: the ROLE http://role-showcase.eu/widget-bundle/learn-foreign-language-
Interoperability Framework,” in Proceedings of the 6th European reading-text.
Conference on Technology Enhanced Learning (EC-TEL 2011): [21] A. Nussbaumer, M. Scheffel, K. Niemann, M. Kravcik, and D.
Towards Ubiquitous Learning, Springer Heidelberg. LNCS 6964, Albert, “Detecting and Reflecting Learning Activities in Personal
2011, vol. 6964, pp. 125–138. Learning Environments,” in Proc. of the 2nd Workshop on
[2] M. Boekaerts, “Metacognitive experiences and motivational state Awareness and Reflection in Technology-Enhanced Learning
as aspects of self-awareness: Review and discussion,” European (artel12) at European Conference for Technology-Enhanced
Journal of Psychology of Education, vol. 14, no. 4, pp. 571–584, Learning 2012 (EC-TEL), Saarbrücken, 2012.
1999. [22] A. Nussbaumer, M. Berthold, D. Dahrendorf, H.-C. Schmitz, and
[3] H. Mandl and H. F. Friedrich, Handbuch Lernstrategien, 1st ed. M. Kravcik, “A Mashup Recommender for Creating Personal
Hogrefe-Verlag, 2006. Learning Environments,” in The 11th International Conference on
[4] P. Winne and A. Hadwin, “The weave of motivation and self- Web-based Learning (ICWL2012), Romania, 2012, vol. 7558, pp.
79–88.
regulated learning,” in Motivation and Self-Regulated Learning:
Theory, Research, and Applications, Dale H. Schunk and Barry J. [23] S. Kroop and M. Berthold, “Personalisierte Lernumgebungen.
Zimmerman., New York: Taylor & Francis, 2008. Unterstützung von selbstreguliertem Lernen,” Workshop at [aha:]
[5] M. Bannert, “Effects of Reflection Prompts when learning with Konferenz 2012 - Lernen gestaltet Zukunft. Vienna, 13-Apr-2012.
[Online]. Available: www.ahakonferenz.at/WS-E6a_Kroop.pdf.
Hypermedia,” Journal of Educational Computing Research, vol.
35, no 4, pp. 359–375, 2006.
52