In universities, diverse tools and software systems exist that each facilitates a di↵erent teaching and learning scenario. A deviating approach is taken by Personal Learning Environments (PLE) that aim to provide a common platform. Considering e-portfolios as an integral part of PLEs, especially portfolio-based learning and assessment have to be supported. Therefore, the concept of a PLE is developed further by enabling the products of di↵erent software systems to be integrated in portfolio pages and finally submitted for feedback and assessment. It is further elaborated how the PLE approach is used to support the continuous formative assessment within portfolio-based learning scenarios.
Even if pedagogic concepts of Personal Learning Environments (PLE) and
portfolio work appeared already in the 80s [
Although the first e-portfolio systems and PLEs appeared
contemporaneously in the early 2000s, especially in the ongoing debate the definition and
division of the terms e-portfolio and PLE are considered di↵erently. Either they
are perceived individually [
An important aspect throughout the learning process is the continuity and variety of the feedback and assessment opportunities (e.g several types, points and assessment partners shall thus be supported). Here, the contribution presents a portfolio workflow, which covers di↵erent e-portfolio scenarios and considers aspects of formative assessment and feedback. Afterwards some selected results from the evaluations conducted are presented. The contribution closes with some insights into the current portfolio systems as a part of the PLE. 2
E-portfolio systems and PLEs are linked to pedagogical concepts such as
selfregulated (SRL) and self-reflective learning [
Even if e-portfolio systems di↵er in functionality [
This portfolio workflow, from page creation up to formative assessment or
feedback, can be linked to Zimmermans cyclical model of self-regulation,
including the forethought, performance control and self-reflection phase [
Especially when classes reach a certain count, the formative assessment and
the monitoring of student activity become a bottleneck. If additionally SRL is
seen as a micro-level concept, it reaches a certain new complexity. According to
Saks and Leijen, SRL focuses on the task execution and therefore students’
learning tasks can be defined by teachers too [
Approaches to foster SRL and formative assessment have to tackle these organizational problems and have to provide new solutions and tools. As long as e-portfolio systems lack in providing adequate support for teachers and students, formative assessment remains a time-consuming task not applicable for the broad mass of teachers.
Based on di↵erent teaching and e-portfolio scenarios, performed with e-portfolio systems such as Elgg, Mahara and Exabis, as well as a requirement analysis with teaching and e-learning experts, the following generalized portfolio workflow was derived. The portfolio workflow starts with the creation process. In the creation process, the user needs to determine the name of the page and can choose between three templates which are designed to cover the most common e-portfolio scenarios. In particular, the user can create an e-portfolio page with a wiki, a blog or an application allowing the creation of web content. The scenariobased templates contain the following predefined applications: – page with a wiki: wiki, tag cloud, comments and feedback – page with a blog: blog, bookmarks, tag cloud, comments and feedback – page with web content: web content display, comments and feedback
If nothing fits the users’ needs, it is possible to create an empty page enabling the user to place any of the applications o↵ered by the PLE on the page (e.g. users profile, personal tasks, acquired competencies, documents and resources, study plan and many more). Additionally, the user can label the page as a normal page or as a portfolio page. The term portfolio page indicates a more formal process where learners submit a page to the teacher until a determined date and the teacher assesses the page. A normal page is designed to allow uncomplicated sharing of pages e.g. to other students and can be used for self-reflection or selfregulated learning which might just require peer-feedback. The page type can be changed at any time. The newly created page is then listed in the My Pages application (see fig. 1) within the PLE platform.
Additionally to the page creation with the use of the My Pages application, the PLE allows the user to create individual pages in his PLE too. These pages are similar to the empty pages and can be placed outside of the portfolio context. With regard to SRL, these pages are visible in the My Pages application to allow easy sharing or an integration into the portfolio process at any time. The pages listed in the My Pages application can be directly accessed and modified. The user is not restricted to the predefined applications inherited from the previously listed scenarios. He can remove existing applications or add new ones provided by the platform to adopt the page to individual needs.
Sharing a page can be achieved in the My Pages application. It is possible to share a page to particular users or portal wide. Sharing does not lock the page and can be canceled at any time. A recipient of shared pages can access and manage the pages in the Pages Shared by Other Users application. In case of an individual sharing, the corresponding user is notified.
A portal wide sharing of a page is not communicated to the users of the platform. Shared pages can be viewed but not modified by the recipient. Shared pages are moved to the profile of the user to indicate that they are publicly accessible either by a particular user or by everyone. Corresponding to the page type, the shared page is then listed in the dedicated sections normal pages or portfolio pages of a users’ profile. If every sharing is withdrawn, the page is moved back to the users’ private area. Furthermore, feedback and formative assessment can be requested at any time for pages that have previously been shared. How this is realized from the technical and workflow perspective and how it can be used for formative assessment will be discussed in the following section. 2.2
To o↵er a possibility for formal assessment, shared pages can additionally be submitted. This locks the page and therefore prohibits any modifications by the owner. Since there are possibly multiple sharings, the owner can submit the page to multiple recipients as well. This supports SRL by allowing peer-feedback at multiple points in time and on formal feedback by the teacher at the end of a previously defined time span as well.
A submitted page is visible in the Pages Shared by Other Users application and highlighted to the recipient to indicate the need for a review. The owner is able to cancel the submission at any time. Thus the user has full control over his data at any time, but is also responsible for submitting the content in time for assessment. To make modifications of pages after a given deadline transparent, the recipient is able to see when the user has submitted the page.
To finish the reviewing process, the recipient needs to mark the submission accordingly. The need to mark the process as finished leaves the decision open on how the feedback should be given or how the assessment results are communicated to the owner of the page. For instance, the reviewer is able to comment directly on the submitted page, write a message inside the platform, write an e-mail or discuss with the owner personally. When each single review process is marked as finished, the page will be unlocked and can be modified on the basis of the given feedback. At a later point in the learning process, a new round of feedback or assessment can be started. Since the form of feedback is open, it can be given on the learning process, on competencies gained throughout the last phase of learning, on the learning content and also on the learning results that are produced in form of an e-portfolio. Continuous feedback and improvement are thus supported throughout the whole learning phase.
For example, a teacher might ask the students to give peer feedback at three points in time. The students all continuously write their e-portfolios in the form of blogs. At those three pre-defined points in time, each student will share his page with one other student and request feedback from him. The other student might then write an e-mail or at best attach a comment containing the review to the blog of the student he should give feedback to. When the feedback was received, the portfolio page will be unlocked again. At the end of the semester the lecturer wants to start the final assessment. He then asks all students to share their portfolio pages with him and request feedback from him. All portfolio pages will then automatically be locked for further editing when the feedback was requested. The lecturer will now also be able to look into previous comments of the peer review. On the basis of the peer reviews and the portfolio page itself he may now give textual assessment.
This scenario shows possibilities for peer feedback as well as formative assessment. The di↵erent forms of feedback leave the opportunity to the teacher to give private or public feedback. Page comments can easily be archived together with the pages, whereas using e-mails as medium for the feedback enables a strict separation of the learning product and the feedback and assessment results. 3
First of all, a functional and usablity test was conducted aiming at finding errors in the e-portfolio workflow. Therefore, a test plan containing step by step instructions about tasks to accomplish and questionnaires were handed out to 10 selected test persons (e-learning experts, domain experts and five teachers). They were asked to fill them out while running through the portfolio creation, sharing and feedback/assessment process. One to three tasks were posted in the categories navigation, creating portfolio pages, releasing portfolio pages, feedback/assessment and other functions such as renaming and deleting. Afterwards, the portfolio processes and interfaces were iterated according to the user feedback.
On the basis of the user tests, it turned out that the double presence of pages (e.g. learning tasks and categories in the portfolio context can be di↵erent than in the general non-portfolio area) within the portfolio and personal context are confusing. Furthermore, the wording of actions concerning the portfolio and feedback workflows, such as ”feedback” (for informal feedback as well as feedback involving grading) and ”submit” (for submitting the portfolio for feedback and assessment) were discussed critically. Additionally, it was desired to have pages explicitly called ”portfolio pages” for formal learning settings and assessment. Nevertheless, users still wanted the same functionality without the term ”portfolio”. The following changes were made: – provide a possibility to mark pages as portfolio pages and inverse – feedback can be given in any way the teacher or student desires – provide new filter options to quickly access created pages – di↵erentiation between giving access and requesting feedback/assessment is possible – when withdrawing the sharing of a page, it will be moved to the place from which it had been shared
Subsequently, larger functional and usability tests with the same user group as the previous, 12 users this time, were executed. Those aimed at testing device compatibility, conformance to expectations, usability and detecting technical errors. Therefore, instructions including questionnaire items were handed out to those selected test persons asking them to return the results to us.
In terms of recognizability, the evaluation indicated that users had problems with the concept of being able to personalize each page within the system. In terms of comprehensibility and user expectations, the di↵erent areas of publishing one’s content were dicult to understand. More tool tips to explain what is happening, the need for software training and help documentation were the consequences as well as a simplification of wizards and wording.
Currently, pilot testing with three lecturers, each with 30 students, within the university is conducted. They are using the platform in their university courses after being given an introduction into the functions o↵ered by the platform. In parallel to the course, regular feedback sessions with the lecturers are held and an in-depth study with the students and lecturers in the end of the testing phase is planned. The feedback sessions included oral feedback so far. The lecturers all reported that the functionality of the platform is very supporting for their scenarios. Those scenarios contain information transfer from the lecturer to the students, collaboration and exchange amongst the students as well as individual portfolio writing with a formal feedback process in the end. Lecturers report that the platform currently well supports students in document collection, creation and sharing and allows the lecturers regular topic-based online exchange with their students. The formal feedback and grading process is scheduled for the end of this semester and is thus not finished yet. Nevertheless, the current feedback is very positive and motivating so that a formal evaluation is planned.
The formal evaluation will be conducted after the feedback and grading process is finished. A questionnaire containing items for usage and handling, utility of the software for the tasks, relation to the topic as well as connection to the user interest was developed and will be handed out to the students. An interview guideline to gain structured qualitative feedback from the lecturers was developed as well. One aim is to find out whether the platform supports the current learning scenario, for example by simplifying administrative processes, feedback and grading and supporting collaboration amongst the students. Furthermore, it shall be identified whether the advantages for the students such as autonomy and data control overweight in the PLE as opposed to Moodle or Mahara. 4
This contribution showed how portfolio-driven learning, teaching and assessment can be facilitated by extending a PLE. Therefore, the implemented portfolio workflow including the feedback and assessment process within the PLE was described. The associated functional and usability evaluation followed by an overview about the ongoing pilot testing phase summed up the current work.
In the following semester, the extension of the pilot phase to up to 12 lecturers after a silent launch of the platform is planned. Also, a second round of accompanying an international graduate college with the help of the PLE and continuously evaluating it is scheduled.
So far, only the portfolio workflow of creating, collecting, sharing and assessing pages is thouroughly implemented. Further tasks were already derived and conceptualized directly from the previous requirement analysis. For example the need to support the self-directed learning approach will be addressed in the future. There is a tremendous request to simplify the aggregation of content on portfolio pages. Therefore, a special editor will be used. In order to support life long learning, the export and import of portfolios on the basis of standards has to be supported. Additionally, the presented applications will be extended to even be usable for collaborative e-portfolio scenarios.
By o↵ering tools that enable on the one hand the learners to manage their own learning tasks and goals, and that on the other hand provide teachers with various functionalities for assessment and defining competencies for their courses, self-directed learning can be supported. Thus, further work will need to be done on competence frameworks, possibly specialized on certain subjects, as well as adjacent assessment tools and scenarios that help students and teachers to follow up and grade the competencies gained. This is one option to support selfmonitoring and motivation of students, by allowing them access to the progress they make in terms of their competencies.
Special thanks to Mrs. Maren Schulze for the conception and design of the platform, as well as the valuable help for carrying out the user tests. Furthermore, we want to thank Mrs. Liane Beuster, Mr. Michael Krause and Mr. J¨org Hafer for the priceless suggestions.