=Paper=
{{Paper
|id=Vol-2250/WS_DK_paper5
|storemode=property
|title=Cooperative Learning with Transparent Learning Outcomes Using a Mobile App
|pdfUrl=https://ceur-ws.org/Vol-2250/WS_DK_paper5.pdf
|volume=Vol-2250
|authors=Julian Dehne,Martin Kapp,Sven Strickroth,Ulrike Lucke
|dblpUrl=https://dblp.org/rec/conf/delfi/DehneKSL18
}}
==Cooperative Learning with Transparent Learning Outcomes Using a Mobile App==
https://ceur-ws.org/Vol-2250/WS_DK_paper5.pdf
Daniel Schiffner (Hrsg.): Proceedings of DeLFI Workshops 2018
co-located with 16th e-Learning Conference of the German Computer Society (DeLFI 2018)
Frankfurt, Germany, September 10, 2018
Cooperative Learning with Transparent Learning
Outcomes using a Mobile App
Julian Dehne1, Martin Kapp1, Sven Strickroth1 und Ulrike Lucke1
Abstract: Competence based learning and e-portfolios are integral parts of modern teaching
repertoires. Implications of these techniques concerning cooperative learning and teaching are
discussed. We developed a mobile application for cooperative learning using micro reflections that
relies on the competence database COMPBASE as a backend for storing learning outcomes and
peer reviewed reflections. COMPBASE2 is a framework for saving and comparing learning goals
and formulated competencies across systems. It includes badges as a formative assessment tool.
The results are linked to the bigger picture of learning outcome oriented CSCL tools.
Keywords: micro-reflections, learning outcomes, database, group awareness
1 Introduction
Learning outcomes have been in the spotlight as a tool to make learning processes more
manageable and easier to monitor. The same argument has been used against them
blaming output-oriented pedagogies for the loss of in-depth reflective thinking.
However, there is a third stand of thinking that argues that learning outcomes should not
be seen as a way to quantify education but as a means of making learning process more
transparent for everyone. The student can use them to communicate his or her interests
to the teacher as well as the teacher explain the sequence of his lectures to the student.
The government can use the learning outcomes to highlight some aspects the political
system deemed necessary. Finally, European universities can use them to simplify
mobility for students. From a technical point of view learning outcomes are valuable
metadata that can be used to improve learning tools. If used with the right motivation
they can be valuable to improve reflection processes and collaborative work.
In this paper we present the concept of 'micro reflections' which we use as a term for a
number of sentences a learner writes about how he or she feels about his/her status or
progress with regards to a learning goal. We use the term 'learning goal' as a short form
for formulated learning outcome that adheres to a taxonomy or is semi-structured. It is
also assumed that learning goals are not simply changes in disposition but can be partly
demonstrated with an activity in a LMS (Learning Management System). Further, it is
assumed that the combination of performing an activity and the comments of teachers or
1
Institut für Informatik/Universität Potsdam, Lehrstuhl für Komplexe Multimediale Anwendungsarchitekturen,
August Bebel Straße 19, 14482 Potsdam, vorname.nachname@uni-potsdam.de
2
https://github.com/juliandehne/competence-database - access 19.07.2018
Julian Dehne, Martin Kapp, Sven Strickroth und Ulrike Lucke
peers hereto suffices as an evidence that the learning goal has been reached. However,
computer generated recommendations about learning goals are not the subject of this
study.
The term 'transparent learning outcomes' is defined as learning outcomes that are visible
to all the roles in the system such as the teacher, the student and the administrative body.
The implemented system is based on an ontology and ensures consistency within the
learning outcomes. It is also used as the backbone of the mobile application
implemented. The mobile application is used to test the concept of micro reflections and
their impact on collaborative learning.
2 Current State of Research
The Oxford Dictionary of Education defines competencies as “‘The ability to perform to
a specified standard“ [Wa09]. In spite of its brevity this definition emphasizes the two
aspects this paper relies on: The possibility to link a competence to a certain action being
performed and the existence of a standardized curricula that can be leveraged as a
framework. Action competence has been described in more detail in [We01, p. 3] and
[Si09, p. 544]. In the context of e-learning the term ’competency’ has several meanings
which result in different modelling and implementation of competency aware systems:
For instance, the industrial approach sees effective management of competencies as a
way to foster human resources development [Hi10]. The main reason to deal with
competencies or knowledge this way lies in the advantage of bridging knowledge gaps
which in turn improves communication processes. Here the economic benefit is most
visible. Competencies are then described according to the need to rate a person’s ability
to do a certain job. In consequence, they are modeled in a numerical fashion (e.g., 1-6, 0-
100%). Another approach focuses on the idea that learning objects such as assignments
or documents contain useful metadata [Lu03]. This knowledge is used to create a model
of the user that has been in touch with the learning objects. Recommender systems are a
logical consequence of this approach. More sophisticated teaching approaches based on
the constructivist paradigm harness the additional information available [No04]. Here
the competencies are modeled in a more complex fashion incorporating the metadata of
the documents, the activities of the user that can be monitored in e-learning systems and
the output generated by the learner.
If the learner is brought into focus there are two use cases. On the one hand the
assessment based on competency [Il13] and on the other hand cooperative learning
[Cr96]. A competence model of the students opens up the possibility of composing study
groups based on similarities and dissimilarities in knowledge and learning styles.
MoodlePeers [Ko16a] aims at group formation using discrete learning outcomes as a
validation step. PeerLA [Ko16b] is a tool that is an editor for self-regulated learning
goals. Both approaches have the idea in common that learning outcomes or goals can be
utilized to foster self-regulation. Like the COMPBASE system they use Moodle plugins
Cooperative Learning with Transparent Learning Outcomes
and could be integrated. Other related work focuses on presenting learning content
depending on the context the learner is in [Mo14]. Different types of contexts such as
natural context (weather, geoposition) or personal context (’User A is related to user X’)
are defined. If integrated, the similarity of learning goals and corresponding groups
could be used to enhance the user context.
3 Collaborative Learning/Teaching and Learning Outcomes
Cooperative learning can be viewed from an e-learning perspective from different
angles: computer supported creation of good learning groups [Ch07], fnding good
patterns for group formations [KD11] or assisted self-regulated group formation. Self-
regulated learning [BC06] includes creating groups based on personal learning
preferences and other factors. Group awareness raises the motivation and the sense of
responsibility and ownership. This way self-regulated learning can be controlled by a
different factor than the individuals’ motivation [Bu11]. Another factor for successful
cooperative learning other than the group composition are the learning goals the group
shares. One possibility is that the teacher has set the learning goals for the class. If this is
the case, these learning goals can be very different from the learning goals of the
individuals in question resulting in unintended learning outcomes. Moreover,
formulating learning outcomes that match the teacher’s expectations can be problematic,
too [HS02]. Learning goals serve to standardize cooperative learning processes. In
informal settings they can be used to mainstream the efforts invested. However, it is not
possible at the moment to compare formulated learning goals automatically. Nature
language processing is needed to segment formulated learning goals and allow reasoners
to compare learning goal aggregates. For this reason, an automatic group composition
algorithm using learning goals does not exist yet to the knowledge of the authors.
Therefore, we focused group awareness and reflection processes instead of group
composition. Computer supported collaborative teaching can be viewed from different
angles: Distance learning that includes more than one teacher is regarded as ICT
supported collaborative learning. Usually, the support is limited to providing a
communication platform with video chat and file exchange. Having teachers reflect on
their colleagues practice, however, is less common. Learning goals offer the possibility
to plan activities more consistently and reflect on them collaboratively. A study at our
institution [DN17] has shown that more than 30 per cent of students regard their study
experience as inconsistent complaining about redundancy or conflicting information
given in different classes. The margin is even higher if more than one faculty is involved
in the study program. The most notable group of e-learning tools that enhance reflection
are e-portfolio systems. Making use of an e-portfolio system as a reflection tool in an
institutional manner conflicts with the learner’s priority to pass exams, which usually
focusses on knowledge (of facts or procedures), rather than in-depth analytical thinking
and reflective depth. Mobile learning can resolve this issue if used in combination with
micro reflections. Small time frames like when sitting in public transport or between
Julian Dehne, Martin Kapp, Sven Strickroth und Ulrike Lucke
lessons can be used effectively by the students to reflect upon their progress, plan their
next steps or comment the progress of their peers. Assessment of micro reflections can
be difficult. It is possible to ask students to merge their micro reflection into a bigger
essay. If grades are required, this may be the only viable option. However, we suggest
that the micro reflections should be used additionally. If this is the case, assessment
using the Open Badges standard3 is suitable. Badge systems can be complementary to
traditional performance assessments such as certificates and diplomas. Open Badges is
the term for an open standard for the creation, allocation, display and storage of digital
badges. The concept comes from the Mozilla Foundation in 2011. This standard
stipulates that every institution issues digital badges. As an implementation for the
badges the Moodle competency framework is used. However, using a more sophisticated
process of baking the competency badges is deemed suitable barring that the Mozilla
framework allows different identity management system to hook into its baking
mechanism. The concepts would then relate in the following way: The competencies are
represented by set of learning outcomes and they count as attained when there is a badge
that links the learning outcomes to user evidences in form of activities. The activities
may contain reflective practices.
Taking all the above shown into consideration the learning outcomes fulfill four
purposes:
1. They provide a framework for the micro reflections making it easier for the
students whose learning style is not adapted to keeping a diary.
2. They enable collaborative teaching by making learning goals transparent
between the contexts of different courses or individuals.
3. They enable peer reflection processes. The peers have a framework which to
judge their fellow students’ progress by.
4. They enable informal formative assessment using the Open Badges standard.
4 Implementierung
The developed mobile app gives an overview of the course goals from the user’s Moodle
courses. The learning goals are separated into views of those reached and not reached.
There is a view for the user’s digital badges and for the creation of his own learning
goals. Inside the learning goal menu the user can self assess the progress and the time he
spent, enter his or her micro-reflections by answering reflective questions, give peer
feedback to activities, and look at the same information from other students. The
interface for the teacher is implemented as a Moodle plugin, where he or she can create
learning goals for the given course context and link Moodle activities to them. It is
possible to link reflective questions the teacher created for the course, too.
3
https://openbadges.org/ - access 19.07.2018
Cooperative Learning with Transparent Learning Outcomes
Figure 1: App navigation structure
The app was implemented for Android and iOS using React Native4. As shown in the
figure above the app consists of two basic UI views depending on the user’s login state:
Login or Tabbar layout. The tab bar contains tabs with the unreached learning goals, the
reached learning goals, the badges and a menu, where the user can log out. Each tab
contains a navigator to make it possible to navigate back and forth. In order to formulate
personal learning goals the user needs to select a verb that describes an observable
behavior. Inside the app all learning goals are displayed in the form: ’I + [verb] + [free
wording]’. A formal grammar has been published previously [DL15]. This makes it
4
https://facebook.github.io/react-native/ - access 19.07.2018
Julian Dehne, Martin Kapp, Sven Strickroth und Ulrike Lucke
easier for the learner to understand and create learning goals and at the same time forces
the use of a consistent format.
Figure 2: Screenshot of the Lernreflex Application. Because of privacy reasons, we did
not take screenshots during the evaluation at the school. The above screenshots show the
implemented app used in the context of inquiry based learning.
These connections between learning goals and other entities are stored in the
COMPBASE. It is connected to Moodle using a plugin and by extending Moodle’s web
services. The app never accesses Moodle directly. All the data it needs comes from the
COMPBASE and is, if not stored already, synchronized from Moodle. The Information
flow and general component structure is shown in the figure 3.
Cooperative Learning with Transparent Learning Outcomes
Figure 3: System architecture overview
When having opened the app users can view their learning goals ordered by course or
learning template, which is a method of grouping learning goals according to a specific
topic. It is possible for them to give feedback to all other students in the course they
participate in and view their self-assessment and the answers they have given to
reflective questions. If a student is given feedback for an activity connected to a learning
goal, the activity is marked as finished and the learning goal is considered as reached. In
an earlier prototype the teacher had to approve every single activity of the learning
outcome to be marked as reached. This slowed down the process significantly, which is
why this new method was chosen.
5 Evaluation
In order to evaluate the app, it was tested with 15-25 students in a preliminary study with
9th and 11th graders during a 90 minutes computer science class in each grade. The
students were introduced to the app and explained their functionalities. It was also
mentioned that everything connected to a learning goal could be seen by the other app
users. The students were then anonymously given credentials to log into the app. They
were instructed to work on tasks in groups of two. The groups would then share their
user names in order to collaborate. This way anonymity was achieved between different
groups of students but collaboration facilitated within single groups. Before starting the
work they were told to answer all the reflective questions in the corresponding learning
goal for the task. There were two learning goals set up for each course. The goals were
set to fit the teacher’s planned class content, so that the test would just be part of a usual
Julian Dehne, Martin Kapp, Sven Strickroth und Ulrike Lucke
lesson. After some time the students were told to assess their progress and the time
needed to reach the learning goal and to give their partner feedback inside the app for the
activity.
Figure 4: Result table. The source is the master thesis of Martin Kapp
The interactions show that the app stimulates awareness and some metacognitive
activities. However, further evaluation is needed to come to conclusions concerning
learning effects. It should be considered using the app at the university level, too.
Cooperative Learning with Transparent Learning Outcomes
6 Discussion and Future Work
The app has been evaluated in a school classroom setting. However, results have been
biased by the students’ preferences. Thus, the real impact of the app for cooperative
learning still needs empirical validation. Using the app at university level is promising as
students are more likely to take part in self-regulated learning activities that can be
supported by micro reflections. Group composition based on perceived learning progress
using this app is possible but not evaluated yet. Different patterns have been suggested
during the tests. These are: group composition based on similar competency patterns,
groups with different time management and groups based on perceived progress.
Another area that needs further experiments are the collaborative teaching aspects.
During our tests we used the app with two teachers (the regular teacher and our staff
member). It remains an open question whether the system scales with the number of
teachers who use the app to communicate learning goals with their intersection of
students.
7 Literaturverzeichnis
[BC06] Bødker, S.; Christiansen, E.: Computer Support for Social Awareness in Flexible
Work.Computer Supported Cooperative Work (CSCW), 15(1):1–28, 2006.
[Bu11] Buder, J.: Group awareness tools for learning: Current and future directions.
Computers in Human Behavior, 27(3):1114–1117, 2011.
[Ch07] Christodoulopoulos, C. E. et al.: A Group Formation Tool in an E-Learning Context.
In: 19th IEEE International Conference on Tools with Artifcial Intelligence, 2007.
IEEE ComputerSoc, Los Alamitos, Calif., S. 117–123, 2007.
[Cr96] Cranton, P.: Types of group learning. New Directions for Adult and Continuing
Education, 1996(71):25–32, 1996.
[DL15] Dehne, J.; Lucke, U.: An Infrastructure for Cross-platform Competence-based
Assessment. In: CHANGEE/WAPLA/HybridEd@ EC-TEL. S. 29–38, 2015.
[DN17] Dehne, J.; Nguyen, T.: A Model of Media supported Inquiry Based Learning (IBL).
2017.
[Hi10] Hirata, K. et al.: Competency profciency ontology. In (et al., T. Hirashima, Hrsg.):
Workshop proceedings of the 18th international conference on computers in education.
Faculty ofEducational Studies, Universiti Putra Malaysia, Serdang, S. 292–299,
December 2010.
[HS02] Hussey, T.; Smith, P.: The Trouble with Learning Outcomes. Active Learning in
Higher Education, 3(3):220–233, 2002.
[Il13] Ilahi, M. et al.: Competence Web-based Assessment for Lifelong Learning. In:
Proceedings of the First International Conference on Technological Ecosystem for
Enhancing Multiculturality. TEEM ’13, ACM, New York, NY, USA, S. 541–547,
2013.
Julian Dehne, Martin Kapp, Sven Strickroth und Ulrike Lucke
[KD11] Karakostas, A.; Demetriadis, S.: Adaptation patterns as a conceptual tool for designing
the adaptive operation of CSCL systems. Educational Technology Research and
Development, 59(3):327–349, 2011.
[Ko16a] Konert, J., Bellhäuser, H., Röpke, R., Gallwas, E., Zucik, A.: MoodlePeers: Factors
relevant in learning group formation for improved learning outcomes, satisfaction and
commitment in E-learning scenarios using GroupAL (accepted for publication).
S. 390–396, 2016.
[Ko16b] Konert, J. et al.: PeerLA: Assistant for individual learning goals and self-regulation
competency improvement in online learning scenarios. In: Proc. of the 16th IEEE
International Conference on Advancing Learning Technologies (ICALT), S. 52–56.
IEEE CPS, Los Alamitos, 2016.
[Lu03] Lucke, U.: The role of metadata for the automated generation of educational
courseware. In (Roy, W., Hrsg.): Proceedings of the 2nd European Conference on e-
Learning (ECEL). Academic Conferences, S. 93–104, October 2003.
[Mo14] Moebert, T.; Jank, H.; Lucke, U.; Kröske, B.: Ein generalisierter Ansatz zur
kontextsensitiven Anpassung in mobilen E-Learning-Umgebungen Conference of the
German Computer Society (DeLFI 2014), Freiburg, Germany, September 15, 2014.
S. 205–212, 2014.
[No04] Noelting, K.: Learner Centrism and Constructivism: New Paradigms for E-Learning?
In (Cantoni, L.; McLoughlin, C., Hrsg.): World conference on Educational
Multimedia, Hypermedia and Telecommunications. Association for the Advancement
of Computing in Education (AACE), Chesepeake, S. 2434–2441, Juli 2004.
[Si09] Sitthisak, O. et al.: Transforming a Competency Model to Parameterised Questions in
Assessment. In (Filipe, J.; Cordeiro, José, Hrsg.): WEBIST 2009 - Proceedings of the
Fifth International Conference on Web Information Systems and Technologies. Jgg. 18
in Lecture Notes in Business Information Processing, Springer, Lissabon, S. 390–403,
October 2009.
[Wa09] Wallace, S.: A dictionary of education. Oxford paperback reference. Oxford University
Press, Oxford und New York, 2009.
[We01] Weinert, F.: Concept of competence: A conceptual clarifcation. In (Rychen, D.S.;
Salganik, L.H., Hrsg.): Defning and selecting key competencies, S. 45–65. Hogrefe
and Huber Publishers, Seattle, 2001.