=Paper=
{{Paper
|id=Vol-1850/TEFA2016_paper_2-3
|storemode=property
|title=Facilitating Portfolio-driven Learning in a Personal Learning Environment
|pdfUrl=https://ceur-ws.org/Vol-1850/TEFA2016_paper_2-3.pdf
|volume=Vol-1850
|authors=Alexander Kiy,Franka Grünewald,Matthias Weise,Ulrike Lucke
}}
==Facilitating Portfolio-driven Learning in a Personal Learning Environment==
https://ceur-ws.org/Vol-1850/TEFA2016_paper_2-3.pdf
Facilitating Portfolio-driven Learning in a
Personal Learning Environment
Alexander Kiy, Franka Grünewald, Matthias Weise, and Ulrike Lucke
University of Potsdam,
August-Bebel-Str. 89, 14482 Potsdam, Germany
firstname.lastname@uni-potsdam.de
Abstract. In universities, diverse tools and software systems exist that
each facilitates a di↵erent teaching and learning scenario. A deviating
approach is taken by Personal Learning Environments (PLE) that aim
to provide a common platform. Considering e-portfolios as an integral
part of PLEs, especially portfolio-based learning and assessment have to
be supported. Therefore, the concept of a PLE is developed further by
enabling the products of di↵erent software systems to be integrated in
portfolio pages and finally submitted for feedback and assessment. It is
further elaborated how the PLE approach is used to support the contin-
uous formative assessment within portfolio-based learning scenarios.
1 Introduction
Even if pedagogic concepts of Personal Learning Environments (PLE) and port-
folio work appeared already in the 80s [7], only nowadays corresponding software
systems started existing. Over the last years, portfolio-based learning, teaching
and assessment gained increased popularity in the pedagogic debate as an in-
strument to foster e.g. self-reflection, self-regulation and to provide a way, with
which one is able to demonstrate skills and competence acquisition in a certain
domain.
Although the first e-portfolio systems and PLEs appeared contemporane-
ously in the early 2000s, especially in the ongoing debate the definition and
division of the terms e-portfolio and PLE are considered di↵erently. Either they
are perceived individually [10] or they are combined together, whereby the e-
portfolio term mostly occurs as a part of the PLE [1, 4]. Whereas in most cases
the way to o↵er an e-portfolio system consists in either providing a stand-alone
installation (e.g. Mahara, PebblePad), in integrating the functions in an existing
Learning Management Systems (e.g. Moodle, Blackboard) or in adapting exist-
ing software systems such as blogs or social systems, especially for PLEs quite
diverse approaches are pursued [5]. Becaues the main focus of e-portfolios lies in
collecting, presenting and sharing of materials and resources, it has to be dealt
e.g with cross-system access and export of materials, with ownership, control
and user management as well as with various requirements for formal formative
assessment. However, when considering e-portfolio work as a part of a PLE, most
cross-system issues are supposed to be solved.
2 Alexander Kiy, Franka Grünewald, Matthias Weise, and Ulrike Lucke
But the assessment topic should not only be addressed exclusively from a
technical and teachers perspective, but from the students’ point of view as well.
Thus, the possibility of integrating informal feedback and peer review mecha-
nisms will be tackled in addition to formal assessment mechanisms driven by
end-semester grading of teachers.
An important aspect throughout the learning process is the continuity and
variety of the feedback and assessment opportunities (e.g several types, points
and assessment partners shall thus be supported). Here, the contribution presents
a portfolio workflow, which covers di↵erent e-portfolio scenarios and considers
aspects of formative assessment and feedback. Afterwards some selected results
from the evaluations conducted are presented. The contribution closes with some
insights into the current portfolio systems as a part of the PLE.
2 E-Portfolios and Self-Regulated Learning
E-portfolio systems and PLEs are linked to pedagogical concepts such as self-
regulated (SRL) and self-reflective learning [3, 8]. In this context, e-portfolio
scenarios (e.g. reflection, development, presentation portfolios [2]), can be used
for documentation and reflection on the self-regulated learning process. There-
fore, e-portfolio systems include at least some limited possibilities for formative
assessment and feedback.
Even if e-portfolio systems di↵er in functionality [4], the so called portfolio
workflow is a crucial part of each of them. To facilitate this portfolio workflow,
most systems o↵er possibilities to create pages, populate these pages with content
from various sources, share pages with people or publicly and request feedback
or assessment. In some cases, collaborative aspects are added too.
This portfolio workflow, from page creation up to formative assessment or
feedback, can be linked to Zimmermans cyclical model of self-regulation, in-
cluding the forethought, performance control and self-reflection phase [11]. Each
of these phases can be supported by tools, e.g. by tools for students to foster
goal setting, strategic planning, competence measurement or tools for teachers
in order to simplify the monitoring of student activity [6].
Especially when classes reach a certain count, the formative assessment and
the monitoring of student activity become a bottleneck. If additionally SRL is
seen as a micro-level concept, it reaches a certain new complexity. According to
Saks and Leijen, SRL focuses on the task execution and therefore students’ learn-
ing tasks can be defined by teachers too [9]. As communication and reflection on
learning processes with di↵erent user groups (e.g. students, peers, teachers) play
an increasing role in portfolio scenarios, new possibilities to discuss self-reflection
results and feedback are required [3].
Approaches to foster SRL and formative assessment have to tackle these
organizational problems and have to provide new solutions and tools. As long as
e-portfolio systems lack in providing adequate support for teachers and students,
formative assessment remains a time-consuming task not applicable for the broad
mass of teachers.
Facilitating Portfolio-driven Learning in a Personal Learning Environment 3
2.1 Portfolio Workflow within a Personal Learning Environment
Based on di↵erent teaching and e-portfolio scenarios, performed with e-portfolio
systems such as Elgg, Mahara and Exabis, as well as a requirement analysis
with teaching and e-learning experts, the following generalized portfolio workflow
was derived. The portfolio workflow starts with the creation process. In the
creation process, the user needs to determine the name of the page and can
choose between three templates which are designed to cover the most common
e-portfolio scenarios. In particular, the user can create an e-portfolio page with a
wiki, a blog or an application allowing the creation of web content. The scenario-
based templates contain the following predefined applications:
– page with a wiki: wiki, tag cloud, comments and feedback
– page with a blog: blog, bookmarks, tag cloud, comments and feedback
– page with web content: web content display, comments and feedback
If nothing fits the users’ needs, it is possible to create an empty page enabling
the user to place any of the applications o↵ered by the PLE on the page (e.g. users
profile, personal tasks, acquired competencies, documents and resources, study
plan and many more). Additionally, the user can label the page as a normal
page or as a portfolio page. The term portfolio page indicates a more formal
process where learners submit a page to the teacher until a determined date and
the teacher assesses the page. A normal page is designed to allow uncomplicated
sharing of pages e.g. to other students and can be used for self-reflection or self-
regulated learning which might just require peer-feedback. The page type can
be changed at any time. The newly created page is then listed in the My Pages
application (see fig. 1) within the PLE platform.
Fig. 1. Page Overview providing information about the title, the page type, the people
the page is shared with, the modified date and the submission status.
Additionally to the page creation with the use of the My Pages application,
the PLE allows the user to create individual pages in his PLE too. These pages
4 Alexander Kiy, Franka Grünewald, Matthias Weise, and Ulrike Lucke
are similar to the empty pages and can be placed outside of the portfolio context.
With regard to SRL, these pages are visible in the My Pages application to allow
easy sharing or an integration into the portfolio process at any time. The pages
listed in the My Pages application can be directly accessed and modified. The
user is not restricted to the predefined applications inherited from the previously
listed scenarios. He can remove existing applications or add new ones provided
by the platform to adopt the page to individual needs.
Sharing a page can be achieved in the My Pages application. It is possible to
share a page to particular users or portal wide. Sharing does not lock the page
and can be canceled at any time. A recipient of shared pages can access and
manage the pages in the Pages Shared by Other Users application. In case of an
individual sharing, the corresponding user is notified.
A portal wide sharing of a page is not communicated to the users of the
platform. Shared pages can be viewed but not modified by the recipient. Shared
pages are moved to the profile of the user to indicate that they are publicly
accessible either by a particular user or by everyone. Corresponding to the page
type, the shared page is then listed in the dedicated sections normal pages or
portfolio pages of a users’ profile. If every sharing is withdrawn, the page is moved
back to the users’ private area. Furthermore, feedback and formative assessment
can be requested at any time for pages that have previously been shared. How
this is realized from the technical and workflow perspective and how it can be
used for formative assessment will be discussed in the following section.
2.2 Formative Assessment & Portfolio Workflow
To o↵er a possibility for formal assessment, shared pages can additionally be
submitted. This locks the page and therefore prohibits any modifications by the
owner. Since there are possibly multiple sharings, the owner can submit the page
to multiple recipients as well. This supports SRL by allowing peer-feedback at
multiple points in time and on formal feedback by the teacher at the end of a
previously defined time span as well.
A submitted page is visible in the Pages Shared by Other Users application
and highlighted to the recipient to indicate the need for a review. The owner is
able to cancel the submission at any time. Thus the user has full control over his
data at any time, but is also responsible for submitting the content in time for
assessment. To make modifications of pages after a given deadline transparent,
the recipient is able to see when the user has submitted the page.
To finish the reviewing process, the recipient needs to mark the submission
accordingly. The need to mark the process as finished leaves the decision open
on how the feedback should be given or how the assessment results are commu-
nicated to the owner of the page. For instance, the reviewer is able to comment
directly on the submitted page, write a message inside the platform, write an
e-mail or discuss with the owner personally. When each single review process is
marked as finished, the page will be unlocked and can be modified on the basis
of the given feedback. At a later point in the learning process, a new round of
feedback or assessment can be started. Since the form of feedback is open, it can
Facilitating Portfolio-driven Learning in a Personal Learning Environment 5
Fig. 2. Generalized Portfolio Workflow with feedback and formative assessment options
be given on the learning process, on competencies gained throughout the last
phase of learning, on the learning content and also on the learning results that
are produced in form of an e-portfolio. Continuous feedback and improvement
are thus supported throughout the whole learning phase.
For example, a teacher might ask the students to give peer feedback at three
points in time. The students all continuously write their e-portfolios in the form
of blogs. At those three pre-defined points in time, each student will share his
page with one other student and request feedback from him. The other student
might then write an e-mail or at best attach a comment containing the review
to the blog of the student he should give feedback to. When the feedback was
received, the portfolio page will be unlocked again. At the end of the semester
the lecturer wants to start the final assessment. He then asks all students to
share their portfolio pages with him and request feedback from him. All portfolio
pages will then automatically be locked for further editing when the feedback
was requested. The lecturer will now also be able to look into previous comments
of the peer review. On the basis of the peer reviews and the portfolio page itself
he may now give textual assessment.
This scenario shows possibilities for peer feedback as well as formative as-
sessment. The di↵erent forms of feedback leave the opportunity to the teacher to
give private or public feedback. Page comments can easily be archived together
6 Alexander Kiy, Franka Grünewald, Matthias Weise, and Ulrike Lucke
with the pages, whereas using e-mails as medium for the feedback enables a strict
separation of the learning product and the feedback and assessment results.
3 Testing and Evaluation
First of all, a functional and usablity test was conducted aiming at finding er-
rors in the e-portfolio workflow. Therefore, a test plan containing step by step
instructions about tasks to accomplish and questionnaires were handed out to
10 selected test persons (e-learning experts, domain experts and five teachers).
They were asked to fill them out while running through the portfolio creation,
sharing and feedback/assessment process. One to three tasks were posted in the
categories navigation, creating portfolio pages, releasing portfolio pages, feed-
back/assessment and other functions such as renaming and deleting. Afterwards,
the portfolio processes and interfaces were iterated according to the user feed-
back.
On the basis of the user tests, it turned out that the double presence of
pages (e.g. learning tasks and categories in the portfolio context can be di↵erent
than in the general non-portfolio area) within the portfolio and personal context
are confusing. Furthermore, the wording of actions concerning the portfolio and
feedback workflows, such as ”feedback” (for informal feedback as well as feed-
back involving grading) and ”submit” (for submitting the portfolio for feedback
and assessment) were discussed critically. Additionally, it was desired to have
pages explicitly called ”portfolio pages” for formal learning settings and assess-
ment. Nevertheless, users still wanted the same functionality without the term
”portfolio”. The following changes were made:
– provide a possibility to mark pages as portfolio pages and inverse
– feedback can be given in any way the teacher or student desires
– provide new filter options to quickly access created pages
– di↵erentiation between giving access and requesting feedback/assessment is
possible
– when withdrawing the sharing of a page, it will be moved to the place from
which it had been shared
Subsequently, larger functional and usability tests with the same user group
as the previous, 12 users this time, were executed. Those aimed at testing device
compatibility, conformance to expectations, usability and detecting technical
errors. Therefore, instructions including questionnaire items were handed out to
those selected test persons asking them to return the results to us.
In terms of recognizability, the evaluation indicated that users had problems
with the concept of being able to personalize each page within the system. In
terms of comprehensibility and user expectations, the di↵erent areas of publish-
ing one’s content were difficult to understand. More tool tips to explain what
is happening, the need for software training and help documentation were the
consequences as well as a simplification of wizards and wording.
Currently, pilot testing with three lecturers, each with 30 students, within the
university is conducted. They are using the platform in their university courses
Facilitating Portfolio-driven Learning in a Personal Learning Environment 7
after being given an introduction into the functions o↵ered by the platform. In
parallel to the course, regular feedback sessions with the lecturers are held and
an in-depth study with the students and lecturers in the end of the testing phase
is planned. The feedback sessions included oral feedback so far. The lecturers
all reported that the functionality of the platform is very supporting for their
scenarios. Those scenarios contain information transfer from the lecturer to the
students, collaboration and exchange amongst the students as well as individual
portfolio writing with a formal feedback process in the end. Lecturers report that
the platform currently well supports students in document collection, creation
and sharing and allows the lecturers regular topic-based online exchange with
their students. The formal feedback and grading process is scheduled for the end
of this semester and is thus not finished yet. Nevertheless, the current feedback
is very positive and motivating so that a formal evaluation is planned.
The formal evaluation will be conducted after the feedback and grading pro-
cess is finished. A questionnaire containing items for usage and handling, utility
of the software for the tasks, relation to the topic as well as connection to the
user interest was developed and will be handed out to the students. An interview
guideline to gain structured qualitative feedback from the lecturers was devel-
oped as well. One aim is to find out whether the platform supports the current
learning scenario, for example by simplifying administrative processes, feedback
and grading and supporting collaboration amongst the students. Furthermore,
it shall be identified whether the advantages for the students such as autonomy
and data control overweight in the PLE as opposed to Moodle or Mahara.
4 Conclusion and Future Work
This contribution showed how portfolio-driven learning, teaching and assessment
can be facilitated by extending a PLE. Therefore, the implemented portfolio
workflow including the feedback and assessment process within the PLE was
described. The associated functional and usability evaluation followed by an
overview about the ongoing pilot testing phase summed up the current work.
In the following semester, the extension of the pilot phase to up to 12 lec-
turers after a silent launch of the platform is planned. Also, a second round of
accompanying an international graduate college with the help of the PLE and
continuously evaluating it is scheduled.
So far, only the portfolio workflow of creating, collecting, sharing and as-
sessing pages is thouroughly implemented. Further tasks were already derived
and conceptualized directly from the previous requirement analysis. For example
the need to support the self-directed learning approach will be addressed in the
future. There is a tremendous request to simplify the aggregation of content on
portfolio pages. Therefore, a special editor will be used. In order to support life
long learning, the export and import of portfolios on the basis of standards has
to be supported. Additionally, the presented applications will be extended to
even be usable for collaborative e-portfolio scenarios.
By o↵ering tools that enable on the one hand the learners to manage their
own learning tasks and goals, and that on the other hand provide teachers with
8 Alexander Kiy, Franka Grünewald, Matthias Weise, and Ulrike Lucke
various functionalities for assessment and defining competencies for their courses,
self-directed learning can be supported. Thus, further work will need to be done
on competence frameworks, possibly specialized on certain subjects, as well as
adjacent assessment tools and scenarios that help students and teachers to fol-
low up and grade the competencies gained. This is one option to support self-
monitoring and motivation of students, by allowing them access to the progress
they make in terms of their competencies.
Acknowledgments
Special thanks to Mrs. Maren Schulze for the conception and design of the plat-
form, as well as the valuable help for carrying out the user tests. Furthermore,
we want to thank Mrs. Liane Beuster, Mr. Michael Krause and Mr. Jörg Hafer
for the priceless suggestions.
References
1. Attwell, G.: e-Portfolios - the DNA of the Personal Learning Environment? (2008)
2. Baumgartner, P.: Educational Scenarios with E-portfolios - a Taxonomy of Appli-
cation Patterns. In: Sojka et al. (ed.) SCO 2011. pp. 3–12. muni Press (2011)
3. Gläser-Zikuda, M., Fendler, J., Noack, J., Ziegelbauer, S.: Fostering self-regulated
learning with portfolios in schools and higher education. In: Orbis scholae. pp.
67–78 (2011)
4. Himpsl, K., Baumgartner, P.: Evaluation of E-Portfolio Software. International
Journal of Emerging Technologies in Learning (iJET) 4(1), 16–22 (2009)
5. Kiy, A., Lucke, U.: Technical Approaches for Personal Learning Environments:
Identifying Archetypes from a literature review. In: Proc. of the 16th IEEE Int.
Conf. on Advanced Learning Technologies (ICALT) (2016), will be published
6. Konert, J., Bohr, C., Rensing, C., Bellhäuser, H.: PeerLA - Assistant for individ-
ual learning goals and self-regulation competency improvement in online learning
scenarios. In: Proc. of the 16th International Conference on Advanced Learning
Technologies (ICALT). pp. 52–56 (2016)
7. Lorenzo, G., Ittelson, J.: An Overview of E-Portfolios (July 2005),
https://net.educause.edu/ir/library/pdf/ELI3001.pdf
8. Nussbaumer, A., Kravcik, M., Albert, D.: Supporting Self-Reflection in Personal
Learning Environments Through User Feedback. In: Kravcik et al. (ed.) Proc.
of the 2nd International Workshop on Personalization Approaches in Learning
Environments (PALE). Montreal, Canada (July 2012)
9. Saks, K., Leijen, Ä.: Distinguishing Self-directed and Self-regulated Learning and
Measuring them in the E-learning Context. Procedia - Social and Behavioral Sci-
ences 112, 190 – 198 (2014)
10. Salinas, J., Marin, V., Escandell, C.: A Case of an Institutional PLE: Integrat-
ing VLEs and E-Portfolios for Students. In: Proc. of the PLE Conference 2011.
Southampton (July 2011), http://journal.webscience.org/569/
11. Zimmerman, B.: Becoming a self-regulated learner: An overview. Theory Into Prac-
tice 41(2), 64–70 (2002)