=Paper=
{{Paper
|id=None
|storemode=property
|title=Tool Support for Reflection in the Workplace in the Context of Reflective Learning Cycles
|pdfUrl=https://ceur-ws.org/Vol-931/paper5.pdf
|volume=Vol-931
|dblpUrl=https://dblp.org/rec/conf/ectel/KrogstieP12
}}
==Tool Support for Reflection in the Workplace in the Context of Reflective Learning Cycles==
https://ceur-ws.org/Vol-931/paper5.pdf
Tool support for reflection in the workplace in the
context of reflective learning cycles
Birgit R. Krogstie1 and Michael Prilla2,
1
Norwegian University of Science and Technology, Sem Sælands vei 7-9, 7491 Trondheim,
Norway
birgitkr@idi.ntnu.no
2
University of Bochum, Institute for Applied Work Science, Information and Technology
Management, Bochum, Germany
michael.prilla@rub.de
Abstract. This paper describes a model for Computer Support Reflective
Learning (CSRL) as a conceptual framework to support the design, application
and evaluation for tools supporting reflection as a learning mechanism at work.
The CSRL model has been derived from theory and inspired by empirical work
done in the MIRROR project. It contains necessary steps of reflection, which
form a reflection cycle and are linked to corresponding tools and additional
support mechanisms such as scaffolds to enable computer supported reflective
learning. It is accompanied by a procedure to use it for the design and analysis
of reflection tools in real cases. The model and the procedure to apply it have
been evaluated in the MIRROR project. This paper reports on results of this
evaluation.
1 Introduction
Developing solutions to improve reflective learning in the workplace is a main objec-
tive in the MIRROR research project, which is an integrated research project funded
under the FP7 of the European Commission. MIRROR seeks to provide tools to em-
power and motivate employees to learn from reflection on tacit work practices and
personal experiences. MIRROR applications offer computer-supported reflective
learning (CSRL) tools for individual, social, creative, game-based as well as organiza-
tional reflection and real-time learning. The project consortium includes five test bed
organizations representing a variety of organizational characteristics and user needs,
and the tools under development in the project cover a wide specter of technologies.
Apart from the MIRROR apps, the project produces conceptual tools to support the
development of CSRL solutions. One of these is a reference framework for the devel-
opment of MIRROR apps. The framework includes a model accounting for the role of
technology in reflective learning processes – the MIRROR CSRL model - and a set of
conceptual tools supporting app development and their use in the test beds.
This paper is addressing the MIRROR CSRL model in a first version and the ac-
companying stepwise procedure for applying the model to a case of reflective learn-
adfa, p. 1, 2011. 57
© Springer-Verlag Berlin Heidelberg 2011
Tool support for reflection in the workplace in the context of reflective learning cycles
ing in a workplace to aid analysis and design. The procedure was developed, evaluat-
ed and delivered as an integral part of version 1 of the model.
In the paper, we use a detailed example to demonstrate the potential of the ap-
proach, seeking to invite discussion in the TEL research community about the CSRL
model and its use. While theoretically grounded, the focus of the paper is deliberately
practical. To underpin our arguments about the qualities of the model we present re-
sults from an evaluation and discuss further work in light of these results.
In what follows, Section 2 gives a theoretical background and Section 3 presents
the CSRL model. Section 4 outlines the procedure for applying the model to a case.
Section 5 presents an example of use of the procedure. Section 6 addresses the evalua-
tion of the model and Section 7 concludes the paper, addressing further work.
2 Background: Computer supported reflective learning
Reflection is critical to workplace learning, enabling employees to make sense of
complex and dynamic situations [1, 2]. Boud et al. [3] (p. 19) defined learning
through reflection as “those intellectual and affective activities in which individuals
engage to explore their experiences in order to lead to new understandings and appre-
ciations.” In line with this definition, in the MIRROR project we consider reflective
learning to be the conscious re-evaluation of experience for the purpose of guiding
future behavior, acknowledging the need to attend to feelings, ideas as well as behav-
ior associated with work experience.
In the workplace, work and reflection on work are intertwined [1, 2], keeping each
other going and taking inputs from each other. Work creates experiences, and some
experiences are reflected upon. Sometimes reflection takes place close to work and at
other times with some distance. Sometimes, reflective learning is based “just” on
memory, sometimes on data as well.
Reflection on work experience leads to an improved understanding of the experi-
ence and allows for deriving implications, conclusions, or lessons learned. In this way
reflection transforms experience into knowledge applicable to the challenges of daily
work. Reflection and learning thus form a cycle (e.g. [4-7]). The outcome of reflec-
tion on work is applied in the work practice.
Apart from being an individual, cognitive process, reflection has a strong social
dimension [8, 9]: It is often accomplished collaboratively by a team or working unit,
which has a joint task to perform and therefore shares work-related experience.
It is possible to encourage reflection by providing appropriate support. In industrial
settings there are reflection “tools” like project debriefings [10-12] demonstrating the
value of reflection in work life. Most reflective learning at work, however, occurs
without support of technology [13].
Technology has a large potential to increase the efficiency and impact of reflective
learning at work [14-19] and can be applied to informal, everyday learning in the
workplace]. The design space of possible solutions is vast and growing with the
emergence of new technologies potentially applicable to work settings.
58
Tool support for reflection in the workplace in the context of reflective learning cycles
There are many examples of successfully modeling experience-based learning as a
cycle [4-7]. On the basis of work in MIRROR [20], the reflective learning cycle on
work includes a reflection session (the time-limited activity of reflecting – short or
long, informal or formal, planned or spontaneous, individual or collaborative, etc.).
Furthermore, achieving transitions from work to reflection and back are essential,
triggers for reflection and useful outcomes of reflection being key issues.
A model outlining tool support for reflective learning in the workplace should out-
line how work and reflection are connected, support the description of reflective
learning processes and scenarios in different real-life settings, e.g. workplaces, and
thereby aid the recognition of differences and commonalities. Also, it should clarify
the different roles technology can play in supporting reflection [20, 21].
3 The MIRROR model of Computer Supported Reflective
Learning (CSRL)
To support analysis and development of computer supported reflective learning, the
use of technology can be linked to steps in a reflective learning cycle. In the
MIRROR CSRL model [21], steps of reflective learning form a cycle and are linked
to categories of tool use. The learning cycle contains four main steps (see Fig. 1).
Fig. 1. The cycle view of the CSRL model
The diagram in Fig. 1 can be instantiated with a case of reflective in the workplace
comprising several cycles of reflective learning (e.g. an ‘expansion outwards’ of the
model). Each learning cycle can be ‘expanded inwards’ to show more detailed steps
in the specific reflection cycle, as well as associated use of tools to support the pro-
cess. This is shown in Fig. 2, in which the rounded rectangles in the middle of the
diagram show a detailing of the process steps in Fig. 1. The columns of square boxes
to the left and to the right are categories of use of reflection tools supporting the steps:
White boxes indicate tool support for capturing data, dotted boxes for providing data,
light gray boxes for scaffolding the process, and the dark gray ones show use of tools
for simulating the work process.
59
Tool support for reflection in the workplace in the context of reflective learning cycles
Fig. 2. The process steps view of the CSRL model with associated categories of tool use
The model in Fig. 2 is the Version 1 of the CSRL model. For a more detailed ex-
planation of the diagram, see [21]. (Please note that the tool categories in Fig. 2,
based on the ongoing conceptual developments in MIRROR, have been slightly re-
fined as compared to the ones in [21]. The differences are not essential with respect
to the issues addressed in the present paper.)
4 A procedure for applying the CSRL model to a case to
support analysis and design
The CSRL model can be used to describe existing processes or practices of reflective
learning in an organization (e.g. before the introduction of new solutions), to describe
intended use of new solutions (e.g. outlining user requirements), and to describe the
actual practices after these solutions have been introduced. The procedure to use the
model for these purposes contains three main steps: outlining a story of reflective
learning, modeling the reflective learning cycles of the story, and detailing each cycle
with steps and associated tool use.
Step 1: Outline the story of reflective learning
First, the case of reflective learning needs o be explained in context of work pro-
cesses in the organization, using the perspective of its actors. Collaborative work with
scenarios helps elicit rich information from users and the organization, helping users
60
Tool support for reflection in the workplace in the context of reflective learning cycles
and developers to reach a common understanding of the case. For example, develop-
ing stories in which people successfully learn by the aid of reflection tools helps fo-
cusing on tasks and goals as well as on learning outcomes and their application. To
cover the full potential of the tool in the case organization, it important that the story
includes the relevant situations of reflection and tool usage as well as the connections
between them (e.g. results of individual reflection feeding into processes of reflective
learning on the level of the team or organization). Supporting artifacts are textual
descriptions or other representations (e.g. storyboards) outlining the scenario of re-
flective learning.
Step 2: Outline the overall reflective learning process by identifying the learn-
ing cycles (how work and reflection are integrated) and how they are connected.
A key to understanding and supporting reflective learning is to consider transitions
between work and reflection. This includes triggers and circumstances that lead to
reflection and the step of bringing insights from reflection back into work, e.g. ensur-
ing that the outcomes of reflection are brought into use in the work process. Artifacts
supporting this are diagrams instantiating the learning cycle view of the CSRL model
as in Fig. 2 and Fig. 5.
Step 3: For each reflective learning cycle, apply the more detailed process
steps and consider what steps are relevant and how tools are used in each cycle.
By considering tool use for separate cycles, different usage in different situations is
described. This might also create new ideas about tool usage or design, if a tool sup-
porting a reflection session currently does not offer scaffolding of a particular step of
reflection, or it does not capture data that are available and could be of potential use.
Also, it could lead to considerations about similarities and differences in tool use
between different reflection sessions from different cycles in the story, with implica-
tions e.g. for the tailoring of user interfaces for the different sessions. This step can be
supported by artifacts such as diagrams instantiating the process steps view of the
CSRL model (e.g. Fig. 6), with tool categories.
The proposed procedure for applying the CSRL model ends at a point where sever-
al artifacts (e.g. an outlined story of reflection and several diagrams) have been devel-
oped. Depending on the step of the development process, we propose that the artifacts
be useful in different ways: As a resource for design on a more detailed and formally
specified level, as a benchmark for evaluation of the modeled solutions, and/or as a
basis for communication among developers and users, e.g. in the next round of devel-
opment (if an iterative approach is used)
5 An example illustrating the instantiating of the CSRL model
with a case
In this section we show how the CSRL model can be used to describe a real case of
reflective learning in the workplace, following the steps outlined above. The work-
place is one of the MIRROR test beds: a hospital. In the story, reflection is supported
by the ‘Talk Reflection’ app, which helps physicians to reflect on difficult conversa-
tions (talks) with patients and relatives.
61
Tool support for reflection in the workplace in the context of reflective learning cycles
Fig. 3 shows a screenshot from the Talk Reflection app. The form contains fields
for doing the ‘objective’ documenting of the talk: This includes a choice of topic, a
description which it is mandatory for the physicians to provide (1), and self-
assessment of own feelings in the situation (2). The form also has a field for personal
reflections (personal note) (3). The physicians can share notes with colleagues and
comment on each other’s notes.
Fig. 3. Screenshot from the Talk Reflection app with the form for documenting a relative talk
Step 1: Outline the story of reflective learning
In this story of reflective learning, the perspective is that of the physician (Fred), who
is a participant in all the reflection sessions mentioned in the story. Additionally, it his
colleagues are also reflecting, which provides important input to Fred’s reflection. To
facilitate the later modeling steps, the story has been divided in three parts.
Part 1) An assistant physician (Fred) is working in the stroke unit. Every time a patient is hos-
pitalized with a stroke the relatives are very concerned about what happened and might hap-
pen. One day Fred has to explain an older man that his wife has suffered a bad stroke and that
she might not recover because it took them to long to get to the hospital. He explains what has
happened in her brain and that because of the stroke she might die. He also explains that they
will need a decision from the husband whether they should take life-extending measures or not.
Suddenly the man gets very angry and shouts at Fred that it is his fault, that he had paid his
health insurance for years, that he demands the best treatment for his wife, and that he thinks
the hospital staff are not willing to do everything they can. Fred is stunned and does not know
how to react. Fortunately a nurse coming into the room is able to calm the old man down and
explain to him that they are doing everything possible to save the life of his wife.
During the day Fred keeps thinking about this episode and finally finds a moment to docu-
ment it in the Talk Reflection app. He first documents the case objectively the way it is required
for the patient’s case file, filling in the description (e.g. explaining that he was stunned and did
not know how to react to the aggression) and using the self assessments e.g. to quantify his
feelings in the situation. He proceeds to add a personal note, reflecting about his experience
and formulating the conclusion that he should perhaps have asked a nurse to participate in the
62
Tool support for reflection in the workplace in the context of reflective learning cycles
conversation in the first place. He shares the documentation and his notes with other assistant
physicians that he trusts, to allow them to comment on it in the Talk Reflection app.
Part 2) Next time he logs into the app, several of his colleagues have commented on his
documentation. Most have written that they have had similar experiences and that they know
how difficult such situations can be. Others describe similar cases with aggressive relatives.
For instance, one colleague had once been hit by the wife of a patient. Based on these com-
ments, Fred recognizes his case as an example of a more general issue and decides to bring it
up again in the bi-weekly reflection meeting for assistant physicians in the stroke unit.
Part 3) In the bi-weekly meeting the five physicians discuss Fred’s case. Fred starts by
briefly explaining about his experience, suggesting that it points to a general issue. His col-
leagues then explain about their experiences with similar cases. The discussion proceeds with
constructive critique of the various approaches. During the discussion the physicians have the
Talk Reflection app in front of them on their individual iPads and can all look at the infor-
mation that has been shared there. Some of the physicians use the app to make quick notes
about cases not yet documented, and comments to cases already documented. Closing the dis-
cussion in the meeting, the physicians have reached the resolution that it would be best to have
relative talks only when there is at least one other person from staff nearby, as with the nurse in
Fred’s case. They decide to make this a change to their work routines. Using the evaluation
function in the Talk Reflection app, one physician writes down this reflection result. To docu-
ment its rationale he makes a link to the relevant cases discussed in the meeting. He then
shares the documented resolution with all participants of the meeting and the other physicians
of the ward.
Fig. 4. A story of reflective learning with the Talk Reflection app
Step 2: Outline the overall reflective learning process by identifying the learn-
ing cycles (how work and reflection are integrated) and how they are connected.
The diagram in Fig. 5 shows the learning cycles described in the story. We note that
there are three cycles in which Fred the physician is involved (drawn with solid ar-
rows in the figures below). The cycles correspond to the three parts of the story in Fig.
4. In the innermost cycle (Part 1), Fred reflects while documenting his experience.
(see the cycle shown highlighted in Fig. 5). In the next cycle (Part 2), Fred reflects on
the comments provided by his colleagues. Finally, in the outer cycle (Part 3), he re-
flects with his colleagues in the physician’s meeting. To complement the picture of
reflective learning in the story, a cycle capturing the reflection session of Fred’s
commenting colleagues has been included and drawn with dashed arrows and boxes.
External factors influencing the process are not shown in the diagram. For instance,
in the diagram in Fig. 5 the step of initiating the inner reflection cycle is called ‘Time
to document’ – which refers not only that Fred actually has the time, but also to the
fact that the organization has routines for documenting conversations with relatives,
requiring that such documenting be done.
Step 3: For each reflective learning cycle, apply the more detailed process
steps and consider what steps are relevant and how tools are used in each cycle.
In what follows, we show how the CSRL model can be used to outline process steps
and tool use in two of the reflective learning cycles in Fig. 5. A more complete analy-
63
Tool support for reflection in the workplace in the context of reflective learning cycles
sis of the case would have included a detailing of the mid cycles, but we left this out
in the paper for reasons of space.
Fig. 5. The learning cycles in the Talk Reflection app story (the inner cycle in this case marked
in boldface to illustrate the procedure of detailing the cycles)
Fig. 6. Instantiating the process steps and categories of tool use for the Talk Reflection case,
inner learning cycle
64
Tool support for reflection in the workplace in the context of reflective learning cycles
We start with the cycle marked with boldface in Fig. 5, e.g. the inner cycle. This
cycle describes individual use of the app for reflecting on single experiences, i.e. rela-
tive talks just documented (in the same app) as part of the work process.
Fig. 6 shows the process steps diagram instantiated with the inner learning cycle.
The steps from the reference model (see Fig. 6) have been reformulated to more accu-
rately describe what happens in the cycle. Furthermore, steps in the process view of
the CSRL model (Fig. 2) that did not seem relevant to the story have been omitted.
The relevant tool categories have been instantiated with brief explanations of how
the Talk Reflection app supports the process. In the perspective of the reflective learn-
ing cycle, the objective documentation of the relative talk, including behavioral and
emotional aspects, can be seen as capturing of data on work experiences. The app
provides scaffolding for this data gathering. Reflection is triggered and framed, as the
physician is encouraged to write a personal note (implicit in the provision of a per-
sonal note field in the documentation template), reflecting on the objectively docu-
mented experience. The documentation further helps the physician in reconstructing
and understanding the meaning of the experience. Reconstruction, articulation of
meaning, and re-evaluation are closely intertwined in this case. The Talk Reflection
app does not provide scaffolding for re-evaluation of the experience, but supports the
capturing and sharing of the reflection outcome.
For the purpose of illustrating the potential of the model to shed light on different
use of tools to support reflective learning in different reflective learning cycles, we
proceed to instantiate the process steps and tool categories with another cycle in the
Talk Reflection story: the outer cycle, e.g. Part 3 of the story. Here, physicians reflect
in their bi-weekly meeting, the outcome being a decision to implement a change in the
work routines. The outer cycle is shown with bold lines in Fig. 7, the process model
instantiating the cycle is shown in Fig. 8.
Fig. 7. The outer learning cycle in the Talk Reflection app story
65
Tool support for reflection in the workplace in the context of reflective learning cycles
Fig. 8. Instantiating the process steps/categories of tool use for the Talk Reflection case, outer
learning cycle
6 Evaluation of the approach
The CSRL model and the approach of applying it to a particular case were evaluated
in a workshop, which had the additional purpose of informing tool development. The
evaluation took place with seven work groups of 24 MIRROR participants. Each
group focused on a different story of reflective learning with a particular app in a test
bed organization. Every group included at least a developer and a representative of
the test bed, group size ranged from 2 to 5. The development of the tools in question
had already started before the evaluation workshop, and thus modeling was mostly
about refining understanding of the cases and re-designing solutions.
The groups were asked to apply the procedure outlined in Section 3, after an intro-
duction in which an example was briefly presented. Step 1 was slightly shortcut to
give more time for steps 2 and 3: A story about reflective learning with the app in the
test bed had been written prior to the exercise, based on knowledge of the case ob-
66
Tool support for reflection in the workplace in the context of reflective learning cycles
tained through previous collaboration with the test bed1. The groups spent approxi-
mately 1,5 hours on developing diagrams with learning cycles (step 2, as in Fig. 2 and
Fig. 5) and their detailed steps (step 3, Fig. 6) – to make this easier, participants were
not restricted to a certain formalism for the cycle diagrams, but could draw them
freely. Then, 30 minutes were used to individually fill in an evaluation form. 22 forms
were handed in. The questions in the form were on opinions about the exercise as well
as strengths and weaknesses of the model and the procedure for applying it to a case.
Besides others, the evaluation form focused on three key questions. Fig. 9 summarizes
the answers to these questions (Q7, Q6 and Q13 in the evaluation form):
• Did the participants perceive the procedure of applying the model to be useful?
• Did the exercise help refine the understanding of the case (descriptive power and
usefulness for analysis)?
• Did the exercise lead to new design ideas (usefulness for design)?
It can be noted from Fig. 9 that most respondents were positive or at least neutral
about the usefulness of the procedure to apply the model (Q7). In another question,
most participants regarded the structuring of the procedure for analysis and design to
be positive, appreciating the detailed steps and categories of tool use. Regarding the
use of a story of reflective learning as a starting point for the instantiation (step 1 in
the procedure), the following comment captures the essence of several answers: “Us-
ing the story is somehow good AND problematic. [on the positive side] it helps to
focus on usage scenarios [and] to link abstract categories and the story [and] to
involve the external people, [on the negative side] it restricts the instantiation to what
you can have in the story”. One group reported having combined two stories to get a
more complete picture of use of their reflection tool.
Concerning the descriptive power of the model with respect to the particular case
17 respondents of Q6 (Fig. 9) answered that the exercise had added detail to the story
of reflective learning. Regarding the usefulness of the model for design, Q13 (Fig. 9)
was only answered negatively by one participant, two answers were left blank. Thus,
19 of the 22 respondents confirmed that the exercise had given them insights or ideas
about the design of the app in question.
Besides these answers, the participants identified strengths and weaknesses of the
model as a tool for describing CSRL cases and solutions, including what could or
could not be described about the particular app by modeling. As a result, a long wish
list for additional capabilities of the model was derived, providing useful input for the
further development of the CSRL model.
The diagrams produced by the groups showed great diversity, and the groups gen-
erally followed the steps of the procedure, but (as explicitly allowed) adapted the way
of drawing the diagrams to their needs whenever there were aspects that they wished
to include but that were difficult to represent with the model. These adaptations pro-
vided ideas for further development of the model. For some of groups, focus of the
exercise was solely on the cycle diagrams, and discussions about the cases seemed to
evolve around these diagrams. These happened mostly for cases in which the com-
1
In two groups the story had not been written in advance and, but could be outlined during the
exercise quickly, as the participants already knew relevant scenarios for usage of the app.
67
Tool support for reflection in the workplace in the context of reflective learning cycles
plexity was high, the processes of reflective learning included several roles and organ-
izational levels, and, in one of the cases, several apps needed to be coupled. This indi-
cates that the cycle diagrams provided a good basis to understand a case of reflection
(see above) and the use of technology within this case.
Fig. 9: Diagrams summarizing answers to three questions from the evaluation form about the
CSRL model and the procedure for applying it
The evaluation must be considered in light of some validity threats. First, it was
conducted within the MIRROR project, with participants that (to a varying degree)
had prior knowledge of the model. It is thus difficult to conclude from the evaluation
about the general use of the model. Also, as mentioned, the stories and the tools mod-
eled were not new, but rather in a process of continued development. These condi-
tions on the other hand allowed the creation of cycle and process step diagrams within
a short timeframe (more limited than the one presumably needed and preferred in a
typical development process), and as apps were mostly used only within one specific
test bed, the evaluation could be conducted with many groups. However, this also
means that comparison of results across participants and groups is difficult. At the
same time, the differences between the cases ensured a wide range of characteristics
and use cases of reflection to be described with the model, enabling and evaluation on
a broad basis of real cases. Ownership and commitment of the participants with re-
gard to the specific tools made the work more ‘real’ and is likely to have lead to in-
creased motivation to actively participate, but having user organization and developer
working together in a group is representative of a the intended development process.
In addition, the time available for the modeling was less than it (probably) would have
been in a real case. This was taken into account when considering the resulting dia-
grams (e.g. their level of detail or coherence). The outcomes of the evaluation, in
terms of quality and quantity, indicate that the evaluation reaped the benefits resulting
from advantages.
68
Tool support for reflection in the workplace in the context of reflective learning cycles
7 Conclusion and further work
The evaluation of the CSRL model and the associated procedure for its application to
a case provided valuable insights about the usefulness of the model and the procedure,
confirming the potential of the model to aid analysis and design. We will end the
paper by discussing some challenges and future steps.
The focus on a story of reflective learning in which there is a user (persona)
seemed to help focus on user needs. Systematic application of the cycle model helped
to make the transitions between work and reflection explicit, including how reflection
is triggered and how reflection outcomes are made applicable and applied. The mod-
eling of tool usage with the process steps diagram supports a systematic walkthrough
of what is supported by the tool and what might be supported by the tool.
Concerning the capturing of all relevant situations and aspects of tool use, it is crit-
ical that the story of reflective learning covers the relevant scenarios. The fact that one
group during our model evaluation decided to combine two stories suggests that it
may be necessary to have several stories covering the relevant app usage and the per-
spectives of different users. For instance, different stories could focus on the needs
and practices of different personas in the organization.
Results on the descriptive power of the model were promising. However, for the
communication between developers (designers) and users, diagrams cannot substitute
application prototypes (even paper prototypes), which let users try user interfaces and
features. To use the unique advantages, MIRROR uses rapid prototyping as a devel-
opment approach. Using the CSRL model for analysis, in turn, has the advantage of
placing the use of the apps into the context of work processes of an organization,
watching how use of an app in different settings form parts of the larger picture of
reflective learning. Using cycle and other diagrams to provide a visually compact
representation grounded in theory of reflective learning and makes it possible to pre-
sent a rather succinct picture of a CSRL process, which expressive enough to support
discussion among developers and potentially useful for communication with users. To
use the advantages of both approaches, future work will also be concerned with com-
bining the approaches of using the model and using prototypes.
There are a few shortcomings to the approach presented here. First, it would be
useful to have a systematic way of representing external factors impacting on the
reflection processes. Second, reflective learning is closely linked to knowledge devel-
opment in an organization (e.g. individual cases developing to general insights; indi-
vidual experience developing to team and organizational knowledge, and so on; see
[22]), and the model so far lacks the means to represent the levels of this process sys-
tematically. The answers to these challenges are likely to lie in a combination of re-
finement and extension of the CSRL model and refinement of the conceptual tools for
its application, e.g. the procedure for model instantiation discussed in this paper. In
the development of the second version of the model, refinement of the model and the
procedure for its application will go hand in hand.
We plan to apply the model to the same cases in a similar evaluation than de-
scribed above after one year of using the apps. While the initial evaluation largely
focused on intended tool use in the test bed organizations, this next evaluation may
69
Tool support for reflection in the workplace in the context of reflective learning cycles
focus on the modeling of actual tool use, as the MIRROR apps in question will have
been used in the test beds at that time. A comparison of the models of intended and
actual tool use may lead to insights about how the tools fill the intended roles. In this
evaluation, the application of the CSRL model will be used both for evaluation pur-
poses and for feeding back into the (re) design of tools.
While use of the CSRL model is important within the MIRROR project to support
shared conceptual understanding [23] and tool development, we also want it to be
used beyond the scope and time of the project. In this respect it is necessary to expose
the model to development of CSRL solutions outside MIRROR: While we continue to
evaluate it within MIRROR, we would like to encourage other researchers and practi-
tioners to consider applying the first version of the CSRL model for purposes of anal-
ysis and design.
8 Acknowledgement
This work is partially funded by the project ‘MIRROR - Reflective learning at work',
funded under the FP7 of the European Commission (project number 257617). The
authors thank Martin Degeling for input on the Talk Reflection case and Viktoria
Pammer for ideas used in this paper.
9 References
1. Lave, J., The practice of learning, in Understanding Practice: Perspectives
on Activity and Context, S. Chaiklin and J. Lave, Editors. 1993, Cambridge
University Press: Cambridge. p. 20.
2. Schön, D., The Reflective Practitioner1983: Basic Books, Inc.
3. Boud, D., R. Keogh, and D. Walker, Reflection: Turning Experience into
Learning1985: RoutledgeFalmer.
4. Cress, U. and J. Kimmerle, A systemic and cognitive view on collaborative
knowledge building in wikis. Computer-Supported Collaborative Learning,
2008. 3: p. 105-122.
5. Kolb, D.A. and R. Fry, Towards an applied theory of experiential learning,
in Theories of Group Processes, C.L. Cooper, Editor 1975, John Wiley:
London. p. 33-58.
6. Korthagen, F. and A. Vasalos, Levels in reflection: Core reflection as a
means to enhance professional growth. Teachers and Teaching: Theory and
Practice, 2005. 11(1): p. 25.
7. Stahl, G., Building collaborative knowing, in What We Know About CSCL
And Implementing It In Higher Education, J.-W. Strijbos, P.A. Kirschner,
and R.L. Martens, Editors. 2002, Kluwer Academic Publishers: Boston. p.
53-85.
8. Høyrup, S., Reflection as a core process in organisational learning. Journal
of Workplace Learning, 2004. 16(8): p. 13.
70
Tool support for reflection in the workplace in the context of reflective learning cycles
9. vanWoerkom, M. and M. Croon, Operationalising critically reflective
behaviour. Personnel Review, 2008. 37(3): p. 15.
10. Dingsøyr, T., Postmortem reviews: purpose and approaches in software
engineering. Information and Software Technology, 2005. 47: p. 293-303.
11. Kerth, N., Project Retrospectives: A Handbook for Team Reviews 2001:
Dorset House Publishing Company.
12. Krogstie, B.R. and M. Divitini. Shared timeline and individual experience:
Supporting retrospective reflection in student software engineering teams. in
CSEE&T 2009. 2009. Hyderabad: IEEE Computer Society.
13. Schindler, M. and M.J. Eppler, Harvesting project knowledge: a review of
project learning methods and success factors. International Journal of
Project Management, 2003. 21: p. 10.
14. Kim, D. and S. Lee, Designing Collaborative Reflection Support Tools in e-
project Based Learning Environment. Journal of Interactive Learning
Research, 2002. 13(4): p. 375-392.
15. Krogstie, B.R. and M. Divitini. Supporting Reflection in Software
Development with Everyday Working Tools. in COOP. 2010. Aix-en-
Provence, France: Springer.
16. Li, I., A. Dey, K., and J. Forlizzi, Understandinng My Data, Myself:
Supporting Self-Reflection with Ubicomp Technologies, in Ubicomp'112011:
Bejing, China.
17. Lin, X., et al., Designing Technology to Support Reflection. Educational
Technology, Research and Development, 1999. 47(3): p. 43-62.
18. Siewiorek, N., et al., Reflection Tools in Modeling Activities, in ICLS2010,
ISLS: Chicago.
19. Xiao, L., et al. Promoting Reflective Thinking in Collaborative Learning
Activities. in Eighth IEEE International Conference on Advanced Learning
Technologies (ICALT). 2008. Santander, Cantrabria, Spain: IEEE.
20. Pammer, V., et al. Reflective Learning at Work - A Position and Discussion
Paper. in ARNets11- Awareness and Reflection in Learning Networks. 2011.
Palermo, Italy.
21. Krogstie, B., et al., Computer support for reflective learning in the
workplace: A model. , in International Conference on Advanced Learning
Technologies (ICALT) 20122012, ACM: Rome.
22. Prilla, M., V. Pammer and S. Balzert The Push and Pull of Reflection in
Workplace Learning: Designing to Support Transitions Between Individual,
Collaborative and Organisational learning, in EC-TEL2012, Springer:
Saarbruecken, Germany.
23. Krogstie, B.R., et al., Collaborative Modelling of Reflection to Inform the
Development and Evaluation of Work-Based Learning Technologies in i-
KNOW2012, ACM ICPS: Graz, Austria.
71