Kelluwen is a learning community composed of teachers, students and researchers who are devoted to assessing and promoting ICT-mediated learning strategies, focused on improving students' socio-communicative skills. Kelluwen has a Web platform (http://app.kelluwen.cl) that supports b-learning activities allowing teachers to create, use and share instructional design leveraging Web 2.0 tools. This paper presents the development of an e-portfolio to be included in the Kelluwen platform, which aims at improving the support for awareness and reflection processes of students and teachers during their didactic experiences. Considering the requirements of the learning community, the eportfolio is divided into four sections: Works, Evaluations, Statistics and Work's Gallery. The tool developed is evaluated in a pilot experience and we conclude that it enriches the learning processes by facilitating their comprehensive evaluation.
Kelluwen is a learning community the purpose of which is to improve the
communication skills of elementary and high-school students introducing the use of
collaborative web tools and social networks in learning processes, combining online and
faceto-face (b-learning) didactic activities [
This work seeks to enrich the web platform, considering the peer-revision module with new features that support the application of different evaluative strategies, including views to facilitate the awareness and reflection of both students and their teachers about the learning process. The new tool being developed is an eportfolio, which in addition to supporting a comprehensive evaluation process, provides both teachers and students with a space to manage all the products developed in a didactic experience.
The question that guides this research is the following: Can the proposed eportfolio tool make a contribution to students’ and teachers’ awareness and reflection processes about their didactic experience? 2 2.1
There are several ways of classifying evaluations. The most common classifications
consider aspects such as functionality, timing or who evaluates [
The concept of e-portfolio has several directions. While some articles define
eportfolio as a platform for the organization of student-created artifacts [
Reflection mechanism introduced in the e-portfolio: A common factor is the use of
the e-portfolio to improve learning by reflection. For instance, in [
Register of evaluations as part of the e-portfolio: A common trend is observed in
terms of registering evaluations in the same e-portfolio. In [
Sharing e-portfolio artifacts: E-portfolios encourage the sharing of works and
provide tools to collect critical feedback from other students. In [
Finally we remark that we use the concept of e-portfolio in the sense of an space to organize the processes and outcomes of learning activities during a limited period of time and not in sense of life-long e-portfolio. 2.3
Awareness and Reflection in b-learning Environments
[
Considering the main best practices in the use of e-portfolios found in the literature and the requirements of the community of teachers who have participated in Kelluwen, four modules were developed in the e-portfolio: Works, Evaluations, Statistics and Work Gallery. A stable version of the platform that includes the Portfolio is found in http://app.kelluwen.cl/, accessible through a simple register. Below is a description of the Evaluation and Statistic modules, given their relevance in the awareness and reflection about the learning process of students and teachers. 3.1
In this section of the e-portfolio, the Student View allows access to the different types of evaluations, depending on the activity that is being performed. Fig. 1 shows the module in which the evaluation displayed are team performance and work assessments. A simple diagram represents the evaluations with arrows between the student and her team mates. Each evaluation has a guideline, for instance, the team performance evaluation includes questions like: “She was responsible in fulfilling tasks”, “She helped his groupmates when they needed” or “She contributed to the group learning process”. In the case of work assessments, the questions are more specific to the subject area: “It is included in the slideshow a reflection about the conflicts experienced during the study period” or “A previous organization is observed in carrying out their slideshow”. In the experience assessment, there are more general questions: “The learning experience managed to satisfy a present need in your schooling” or “You think that criticism of the twin classrooms serve you to guide your learning”. Each guideline includes a space to make comments or explain the achievement levels assigned. The role of this open comment is to promote the students’ reflection process. 3.2
This section deals with statistics or analytics of the results of the evaluation processes, considering visualizations that summarize the evaluations that each student or group gets from the different stakeholders involved, using two types of charts: i) histograms that show the frequency of each achievement level considering aggregated criteria of the evaluation guidelines; ii) radial chart that represent the most frequent achievement level for each disaggregated criterion. Additionally, different comparisons are made based on the type of evaluation: when dealing with performance assessments, self and co-evaluations are compared, while in the case of product evaluations, peer evaluations are compared with the evaluation performed by the teacher (see Fig. 2).When evaluating the experience, the evaluation of all students are compared. 4
During 2015, a pilot experience in the use of the Kelluwen Portfolio was conducted in two ninth grade twin classes at the Laico High School (classroom 1) and Martin Luther King High School (classroom 2), both in the city of Valdivia. The DD applied was “Building a Slide Show about the 2nd Half of the Twentieth Century” in the suject of History, Geography and Social Sciences. A total of 60 students, 31 from classroom 1 and 29 from classroom 2, arranged in nine groups per classroom, participated in the experience that took place during October 2015. Both teachers developed all the activities with their students, including all the proposed evaluation instances. All the groups posted works (68 in classroom 1 and 167 in classroom 2). During the experience, three activities of the Design were related to the evaluation, as follows: i) group co-evaluation, where 21 students in classroom 1 (68%) and 23 in classroom 2 (79%) completed the evaluation of their group mates; product co-evaluation where the 18 groups were assigned to reviewers between classes (twin classes) and completed the evaluation; iii) eco-evaluation, where nine students from classroom 1 (29%) and 25 from classroom 2 (86%) completed the evaluation of the didactic experience. 4.1
A survey was applied among the 60 students to capture their perception about
Portfolio’s usability, adapted from the proposal in [
A qualitative analysis of the Portfolio was performed by means of a survey developed by the research team and applied to all the students during the pilot experience. The survey looked into the students’ perception about the implications of the Portfolio on the learning and evaluation processes, and also of the tool’s usefulness. The survey is organized in three parts. The first part focuses on the Works module; the second on the Evaluations module; and the third part contains questions about the Statistics and Gallery modules. The questions are statements that express a positive or negative valuation of the e-portfolio’s functions. There are four levels of responses: strongly disagree (MD), slightly disagree (LD), slightly agree (LA) and strongly agree (MA). The neutral level was discarded to force an expression of positive or negative opinion. The results of the perception survey show that for all the positive statements regarding the usefulness of the Portfolio, most students either strongly or slightly agreed, with percentages above 70% between both options. Regarding the negative statements about the usefulness of the Portfolio, students’ responses were more heterogeneous without a clear trend unlike the case of positive statements. The distribution of this survey’s answers is shown in detail in the Appendix.
Fig. 3 shows the results of the survey questions that more directly address –in the opinion of students– the impacts of the Portfolio on the reflection about their learning process. Fig. 3(a) shows the results of a question that inquiries whether the evaluations performed made them reflect on their own learning process; student responses are somewhat heterogeneous, with an agreement of 50% for students in classroom 1 and 75% in classroom 2. Fig. 3(b) shows the results of a question referred to the valorization of the evaluations’ graphic summaries in promoting reflections about teamwork. In this case, 82.5% of classroom 1 students strongly or slightly agreed, while this was true for 85.7% of classroom 2 students. 4.3
Two discussion focus groups were conducted made up by students from both classrooms in the pilot experience. The purpose of this activity is to understand the meanings attributed by students to their participation in the experience, considering the valorization of the e-portfolio as an environment to support the reflection and motivation of their learning processes. The classroom 1 group was formed by six students. This focus group suggests: i) Broad approval to the e-portfolio: (a) (b) The main attributes mentioned are: ease of use; interaction with students from other schools; all works are available in the same place; and that it can be used both at school and home. ii) Group co-evaluation was a matter of debate as there was a lack of consensus on its proper use by classmates. Classroom 2 group was made up by five students. The following can be summarized from the conversation: i) All of them stated to like the portfolio, that it was something new and fun to use, ii) Most thought that the evaluations were easy to use and some said that they would like to add comments per criterion in addition to the achievement level. One student mentioned that this way, the teachers could also explain their evaluations. 5
The results of the usability study show that students qualified the e-portfolio in the acceptable category, with a usability index of 71.1 in classroom 1 and of 74.6 in classroom 2; both values are within the “good” usability range. This outcome is consistent with the results collected with other evaluation tools, where students suggest several elements to improve the Portfolio’s usability.
The results of the perception survey show that most students positively assess the Portfolio as a tool that supports work posting, comprehensive learning evaluation and the emergence of moments of reflection and awareness in the context of the didactic experience developed.
Discussion groups confirmed the outcomes of the prior surveys and complemented them by specifically arranging the critiques to the tool being proposed. One element worth highlighting is the diversity of perceptions around the co-evaluation process of the team’s performance; while it was extremely well assessed by student from classroom 2 as a reflection driver for the development of teamwork, the students from classroom 1 considered it an uncomfortable and unfair process. On this regard, it should be mentioned that the two classrooms involved in the pilot experience are part of two different school situations which could probably explain such diverging opinions.
Based on the outcomes emerged, we can conclude that our e-portfolio meets the role of contributing to the awareness and reflection of students about their learning process, particularly concerning the development of teamwork skills, as well as in feedback processes of works posted in the platform. Another relevant aspect is the role of teachers in the design of didactic experiences, which in this case were directly related to the development of different evaluation guidelines which made it possible for such guidelines to be extremely well contextualized and therefore, to be perceived as easy to be developed by students. On the other hand, the Portfolio can also support teachers when monitoring the work of students, enabling them to compare their own perceptions of the work performed by students regarding the evaluations they get in the peer review.
Most of the findings or confirmations that the e- portfolio is useful to support the process of awareness and reflection of students are transferable to other learning communities and are related to the focus on a comprehensive assessment process , which not only emphasizes the products if not the underlying processes.
Future work is expected to complement the feedback provided to students and
teachers with analytics of the activity in the portfolio of the participants of the didactic
experience, such as those proposed in [
Results of the Perception Survey on the Portfolio’s Usefulness
Questions T5. It was useful for my group to be able to see the works posted in the Portfolio. 5 Em1y Kexnpoewctiendg ltehaerneivnagl.uation guidelines beforehand was useful to better understand 9 E2. Knowing the work evaluation guidelines before was useful to improve my work. 5 E3. The Portfolio shows clearly which activities are to be submitted to evaluations. 8 E4. Performing evaluations and seeing their answers did not make me reflect about 8 my learning.
E5. I didn’t like performing digital evaluations. 6 E6. Co-evaluations helped our groups to improve their collaborative work. 7 E7. The results of the evaluations helped me learn about which objectives were 10 achieved and which weren’t.
E8. I was unable to see the works of other groups during the evaluation. 6 E9. It was easy to evaluate other groups. 12 E10. Interacting with other groups during evaluation was useful to improve our 7 work. tRe1am.Sweolfr-kevpaelrufaotrimonanacnes.wers versus co-evaluation answers made me reflect about my 6 R2. I was unable to compare how other groups and the teacher evaluated me. 10 R3. I was able to better understand the performance of my work group by looking at 8 the charts in the Portfolio. aRn4s.wIetrhsinokftihteisevaagluoaotdionidse.a for the Portfolio to include charts to be able to see the 10 R5. I think charts make it difficult for me to understand evaluations. 5 R6. The information in the radial chart was useful for me. 9 R7. Being able to see works of other groups in Portfolio was useful for my learning. 7 R8. Being able to see other works in the Gallery was useful to guide our own work. 3 Frequencies TD LD LA TA 16 15 14 1
This research was supported by DID-UACh and FONDECYT under Grant no. 1150545 “Redes Sociales y Medios de Comunicación: Modelo de análisis basado en minería de datos para la comprensión del ecosistema informativo chileno en internet y la educomunicación ciudadana en la red”.