c 2012 for the individual papers by the papers’ authors. Copying permitted for private and academic purposes. Re-publication of material from this volume requires permission by the copyright owners.
Addresses of the editors:
Wolfgang Reinhardt Computer Science Education Group Department of Computer Science University of Paderborn Fu¨rstenallee 11 33102 Paderborn Viktoria Pammer Know-Center Inffeldgasse 21A 8010 Graz Kamakshi Rajagopal CELSTEC Open Universiteit Valkenburgerweg 177 6419 AT Heerlen The Netherlands kamakshi.rajagopal@ou.nl Thomas Daniel Ullmann Knowledge Media Institute The Open University Walton Hall Milton Keynes MK7 6AA Adam Moore Knowledge and Data Engineering Group School of Computer Science and Statistics Trinity College Dublin, D2 Christian Voigt Centre for Social Innovation Linke Wienzeile 246 1150 Wien lucia.pannese@i-maginary.it
range of disciplines (CSCW, psychology, educational sciences, computer science...) the workshop’s theme is encapsulated in the following question: “What do awareness and reflection mean in the context of TEL, and how can technologies support either?”
”ALECR11 (Augmenting the Learning Experience with Collaborative Reflection)”. AR
search on awareness and reflection in TEL and ii) to provide an interactive experience that connects participants’ research, the co-organizing projects’ latest prototypes and models with real end users’ learning experiences and needs regarding reflection technology. We received 12 submissions, of which 6 were accepted as full papers. The workshop was held on September 18, 2012. The workshop was organised in three sessions, where in the first session papers were presented and discussed that dealt with the topic of awareness whereas in the second session papers on reflection were presented and discussed. The final session was an interactive one, in which the participants collaboratively brainstormed about the connections between awareness and reflection. Moreover, the participants played educational games and worked with simulations, which have then been discussed considering their particular impact on awareness and reflection.
Networks” by Reinhardt et al. provides a theoretically and empirically informed exploration of ’awareness’. Grounded in the analysis of 42 interviews, the authors suggest 6 forms of awareness including being aware of others’ activities, disciplinary differences in doing research or the geographical whereabouts of peers. A convincing argument outlines how these forms of awareness impact each other and lead to a layered model of awareness in research networks (LMARN). Although the LMARN is primarily presented as a heuristic device meant to guide the design of new tools supporting the formation of awareness, the paper also contributes to the wider discussion regarding novel forms of measuring the impact of scientific publications in Science 2.0 media.
layer on top of researcher’s often fragmented communication and storage structures, creating recommendations using Big Data analytics and the social graph. PUSHPIN attempts to build a system that recommends related reading based both on what members of the social graph are also interested in but crucially additionally supported by content awareness of the publications within the system.
Complex Sociotechnical Systems” develop a framework to categorise socio-technical systems according to their purpose with respect to shared situational awareness. Sociotechnical systems may support Perception (being aware of surroundings etc.), Prescription (being able to modify existing plans) and Participation (being able to carry out joint actions). These levels of ’maturity’ as they are called in the paper, are being discussed for individual, team and organisational levels. The paper thus provides a way to categorise, analyse, and understand socio-technical systems with respect to shared situational awareness.
tivities in physical spaces”, Hernandez-Leo et al. discuss how the Signal Orchestration
the context of reflective learning cycles” present a model for Computer Supported Reflective Learning (CSRL), created in the MIRROR project. The authors argue for a 3-step approach to the analysis and design of supportive reflective learning in the workplace, which is illustrated with a case of physicians in a hospital setting. They also present the results of the evaluation of the CSRL model.
Santos, Verbert, and Duval’s paper on “Empowering students to reflect on their activity with StepUp!” advances their interests in using Learning Analytics to build dashboards that visualize their traces through learning material in ways that help learners and/or teachers steer the learning process. Studies of two use cases reveal complex issues surrounding implicit and explicit tracking, the influence of complexity on comprehension and goal setting and evaluate time spent as an indicator of depth of study. They conclude that these issues remain complex and recommend further work on both measuring instruments and visualisation, proposing further deployments of visualisations that embed both individual achievement and reflect that within the wider learning community.
study they have carried on 4 days with 37 college students, where students were reminded to reflect about their learning via SMS, and entered their responses into a specific responsesystem. The idea was that students train the “self-as-a-learner” - alongside the EU goals of fostering life-long-learning. The study suggests, that while students are ready to reflect on their learning activities, they are not used to seeing themselves as active learners. Thomas Ullmann’s paper on “Comparing Automatically Detected Reflective Writings in a Large Blog Corpus” presents work done to identify reflective elements in written text by the example of analysing a corpus from blogs. It uses sophisticated methods of text analysis and shows how the results of this detection compares to the same task assigned to humans. The mechanisms presented in this paper are very promising and can be valuable means to detect and support reflection in organization as well as to identify current issue that need to be known on the organizational level.
flective Learning at Work”, Pammer, Prilla and Divitini present preliminary work that investigates several apps in order to extract sharing functions that have impact on selfreflective learning. The three presented apps may assist knowledge workers to improve their work performance by critically reflecting their past activities.
potential to make learners aware of their self-regulated learning. The research challenge is to infer from measurable low-level data the high-level constructs of self-regulated learning. The goal is to obtain a mapping between key actions extracted from Contextualized
and metacognitive learning activities.
Degeling and Prilla report on their experiences implementing articulation support for collaborative reflection. A theoretical introduction to reflection at the workplace sets the scene to the actual cases studies describing their findings. The central piece of their analysis relies on the reflections carried out by physicians in a hospital. The paper demonstrates the potential benefits of sharing experiences, especially in areas where learning is more the product of past work experience than formal education. However, from a design point of view, the paper also highlights the need for contextual design and frequent end-user interactions, as multiple corrective actions were needed to adapt the technology support to the conditions on site.
ness and Reflection in Technology-Enhanced Learning” group on TELeurope.eu: http://teleurope.eu/artel
Supporting FP7 Projects http://stellarnet.eu http://www.mirror-project.eu http://www.imreal-project.eu http://telmap.org/ http://www.tellnet.eun.org http://mature-ip.eu 8
Papers on Awareness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Papers on Reflection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Organization Committee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Program Committee . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Supporting FP7 Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Wolfgang Reinhardt, Christian Mletzko, Peter B. Sloep, and Hendrik Drachsler
Wolfgang Reinhardt, Pranav Kadam, Tobias Varlemann, Junaid Surve, Muneeb I.
Ahmad, and Johannes Magenheim
Shalini Kurapati, Gwendolyn Kolfschoten, Alexander Verbraeck, Hendrik Drachsler, Marcus Specht, and Frances Brazier
Davinia Hernandez-Leo, Mara Balestrini, Raul Nieves, and Josep Blat
Birgit R. Krogstie, and Michael Prilla
Jose Luis Santos, Katrien Verbert, and Erik Duval
Bernardo Tabuenca, Dominique Verpoorten, Stefaan Ternier, Wim Westera, and Marcus Specht 3 3 4 6 7 8 13 31 47 55 57 73 87
Thomas Daniel Ullmann, Fridolin Wild, and Peter Scott 101
Viktoria Pammer, Michael Prilla and Monica Divitini
Alexander Nussbaumer, Maren Scheffel, Katja Niemann, Milos Kravcik, and Dietrich Albert
Martin Degeling, and Michael Prilla 117 125 133 Understanding the meaning of awareness in
Research Networks Wolfgang Reinhardt1, Christian Mletzko1, Peter B. Sloep2, and Hendrik
Drachsler2 1 University of Paderborn
Department of Computer Science
Computer Science Education Group Fuerstenallee 11, 33102 Paderborn, Germany
{wolle,letris}@uni-paderborn.de 2 Open University of the Netherlands Centre for Learning Sciences and Technologies
6401 DL Heerlen, The Netherlands {peter.sloep,hendrik.drachsler}@ou.nl Abstract The term awareness is often used in the context of CSCW research and connotes re-establishing face-to-face situations in so-called groupware applications. No understanding of it yet exists in the context of networked learning and networks of researchers. In this article we present a succinct description of awareness in Research Networks. It is grounded in guided, semi-structured interviews with 42 researchers that have extensive knowledge of cooperation in networked communities and the awareness issues it raises. From the analysis of the interview data we present six forms and five aspects of awareness in Research Networks. Finally, we present a layer model of awareness that describes how researchers’ awareness is typically spread. 1
Introduction
As early as 1959 Peter Drucker identified that society was moving into a
postindustrial age, which was going hand in hand with a shift from manual to
nonmanual work [
While research is often deemed to be solitary work, international cooperation has become the de facto standard. Large funding programs often even require transnational, interdisciplinary project consortia as it is believed they foster innovation, multiple views on a research topic and promote dissemination in the appropriate Research Networks. Such Research Networks may be viewed as a special kind of Learning Networks [23,28], online social networks whose members are researchers that use various learning services in order to reach individual and shared (learning) goals. Sometimes these goals are externally prescribed, at other times they are formed by the intrinsic motivation to know more about a topic. Research Networks are made up of people that interact with each other. Moreover, in them there are many relevant objects (e.g., publications, events, projects, people) that influence learning, knowledge gain and cooperation, and researchers aim to be aware of this.
Despite the massive impact that Social Media have on the way research is conducted and communicated [17,27,31], it is still scientific conferences, fairs, journals and books that are most often used for the dissemination of research results. Research is currently shifting from closed to open, from hidden to visible and from passive consumptive to active, co-determinative (also see [17]). Even though the way of scientific publication has not changed much in the last 300 years, it does currently and will change massively over the course of the next 10 years. Not only the number of high-quality publication outlets has increased enormously, also the common understanding of authority in research has changed considerably.
Scientific results do not need to be published in access-controlled journals
anymore in order to receive notable attention. The number and citations of
peerreviewed publications are still the de-facto currency when it comes to professional
evaluation of researchers’ work. However, this supremacy is beginning to crumble
as an increasing number of researchers as well as society at large are digesting
premature results that researchers share in blog posts, presentations or tweets. Thus,
there are well-known metrics for the impact of classic publications and there
have to be new metrics that factor in impact and buzz in the Science 2.0 reality.
Lately, many researchers are trying to establish alternative metrics that are able
to assess the impact and reach of scientific publications in Science 2.0 media
(see the #altmetrics movement and their manifesto [18]). Moreover, open access
to scientific publications is gaining significant ground and an ever-increasing
number of institutions are urging their employees not to publish research findings
in closed, pay-to-access outlets or give the full copyright to publishers [
Traditionally the concept of awareness is used in the research field of Computer Supported Collaborative Work (CSCW) to re-establish conditions of faceto-face situations in the online realm, with visual cues showing, for example, who is online or working on a document. Research on awareness support in the CSCW context has often been directly related to the direct improvement of cooperative practices and measurable task performance improvements.
This paper presents parts of a larger study that deals with awareness issues in the context of Research Networks. In particular, we report about our findings on how properly to understand the notion of awareness in Research Networks. We hypothesize that the term awareness is more complex and touches on broader contexts than we know from existing CSCW research. The results of our study go beyond the perception of awareness as being a mere enabler and enhancer of collaborative work processes. The results are based on interviews with 42 researchers that took place in October and November 2010.
First, we introduce the three research questions as well as the method of data gathering, data processing and analysis we applied. After that, we present a definition of awareness in the context of Research Networks that integrates the results of our interviews with established awareness research results. This includes the introduction of various forms and aspects of awareness in Research Networks. Synthesizing these results, we propose a layered model of awareness in Research Networks, which incorporates five layers of awareness. Finally, we summarize the results of our study, give an outlook on future research and discuss important side effects of awareness in Research Networks and practical applications of the introduced model. 1.1
Research questions Three research questions were addressed in the research presented here: 1. How do researchers define awareness in the context of Research Networks? 2. What different forms and aspects of awareness in Research Networks are there? 3. What could a model of awareness in Research Networks look like? 1.2
Method We used open, in-depth and semi-structured interviews as our method of data collection. An interview manual provided the basis for open-ended questioning. Each interview was carried out by one of three different interviewers. In three cases the manual was sent to the interviewees via email beforehand. All participants were interviewed in their normal working context. The participants of the study have been asked explicitly for their approval to record the interview. In most cases the interviews were conducted remotely and recorded using the FlashMeeting service [29]. 1.3
Sampling The total population sampled consisted of all researchers that have been authors within the European Conference on Technology Enhanced Learning or were members of Technology Enhanced Learning (TEL) projects funded within the Framework Programme 7 (FP7).
82 researchers from different research disciplines and different countries were asked for voluntary participation in the interview series via email. More than half of the invitees (43 researchers) agreed to be interviewed. Although 43 interviews were conducted, one recording was not suitable for further analysis due to technical problems. 30 interviews were conducted in German, 12 in English. The age of the interviewees was between 27 and 61 years, 32.5 years on average. 35 out of the 43 participants are male (83%), 7 female (17%). The interviews lasted between 28 minutes and 126 minutes, 51 minutes on average. Table 1 gives the job locations of the interviewees.
Most of the participants are involved in the field of TEL and are in possession of a PhD (44%) or Master (53%) as their highest degree. The extent of professional experience ranges from 1 to 30 years. The scope of research fields of the interviewees includes Computer Science Education, Recommender Systems, Knowledge Management, Human Computer Interaction, Semantic Web as well as Model-based Testing, Social Research and Psychology. The coding of the transcribed interview data took place in multiple iterations and was supported by the Atlas.ti [26] qualitative data analysis software. The continuous process of close reading of the transcripts allowed the identification of concepts and labels, which then were coded in Atlas.ti in constant comparison to previous codes. Atlas.ti supported the merging and renaming of codes. Cooccurrence tests built into Atlas.ti helped spotting inconsistencies in the coding and automatically generated visualizations of code relationships were used to identify patterns. In the following we will quote from the interview transcripts. A 3-tupel, denoting the primary document number in the hermeneutic unit of Atlas.ti, the code number within the document and the line numbers for the precise reference, will follow each quotation. Where needed, the authors translated quotes from German to English.
Approaching a definition of awareness in Research
Networks
Awareness is an integral component of CSCW research. Dourish defined it as
“ awareness is an understanding of the activities of others, which provides a
context for your own activity ” [
Awareness in Research Networks, however, concerns itself not solely with
re-establishing face-to-face situations and direct impacts on bettering task
performance. In Research Networks, awareness has a broader meaning and is related
to trend-spotting, alerts to research results in a certain domain, changes in the
structure of a network, personal changes within a project as well as knowledge
about objects that may help carrying out one’s task (research question 1). The
interviewees pointed out that awareness in Research Networks “ is mainly to
know what sort of people in the same field are doing ” (P13, 15, ll. 9-10) or “ is to
know what is important to me and filter out what is not important to me ” (P27,
36, ll. 40-42). Another researcher stresses, “ If I have to search for something,
that means for me, it’s an active action from my part. That’s not what I think
about awareness. Awareness is something that is keeping remind me about
something, without me actually trying actively to search that information ” (P27, 30,
ll. 12-17). Moreover, “ awareness ... can have impact on the individual method of
operation ... as it triggers reflection ” (P16, 58, ll. 306-320). Research shows that
the availability of awareness support improves the effectiveness of how
information is spread in communities [
It is quite difficult to keep up with who is doing what in the field, though many
researchers are making quite an effort to monitor the data that is being spread
on the Web by colleagues. In the past years research has explored collaboration
of scientists by means of co-authorships of publications. In the TEL community,
Henry et al., Wild et al. and Reinhardt et al. undertook such endeavors [
Relevant objects in Research Networks Scholarly communication is often understood to primarily refer to the publication of scientific publications. Building on Thorin [30] and in line with Procter et al. [19], we understand scholarly communication to be broader in scope and incorporate all communicative activities carried out by researchers on a regular basis. In particular we include the joint developing of ideas, conducting research and carrying out experiments, discussing ideas with one’s Research Network as well as information seeking and dissemination of research outputs formally and informally. Thus, researchers are confronted with a wide number of objects that they either need to be or should be aware of: there are projects the researcher is directly affiliated with, interested in or that are somehow related to the researcher. Documents in any form are one core product of labor for researchers: notably publications written by the researcher herself, publications written by other researchers, as well as deliverables of projects, (micro-)blog entries, rules and regulations, best practice reports. People and groups of people are other objects that having awareness of is paramount. Awareness of people is relevant in multiple aspects at the same time and while it may be important to be highly aware in one particular aspect and not so in others, at other times the situation may be reversed. As researchers are often limited to a fixed domain, awareness of latest trends and new research findings in that domain and associated topics helps researchers to stay informed and up-to-date. Researchers often need to show that they are well informed about the state-of-the-art in their research domain and that they know about the key people, events and projects in that domain.
Grounded in the conducted interviews, this article discerns six different forms of awareness that are partly known from CSCW research as well as five different aspects of awareness (research question 2). Whereas forms describe generic areas of awareness, aspects focus on specific awareness characteristics relevant for the awareness of different objects. 2.2
Six different forms of awareness
1. Activity awareness Activity awareness deals with the past, present and
future of an object. For people this could be realized with “ an activity stream
about people that I am connected to” (P30, 82, ll. 438-439), which would hold
the latest information about their work in general, planned event participations,
new collaborations or published content. From a broader perspective, activity
awareness for a research domain is concerned with the “ state-of-the-art in a
particular research area [...] where things are at the moment, who is contributing
to that area, what is the latest thinking in that area ” (P1, 37, ll. 13-16). Activity
streams and awareness dashboards seem to be helpful tools to support awareness
if they could provide historical data, trend detection and forecasting in order to
make claims like “ this author was very nice 10 years ago, but now is not any
more. To know whose ideas are the current ones, it’s difficult ” (P27, 56, ll.
186191).
2. Cultural awareness Cultural awareness refers to a person’s knowledge and
perceptions about foreign cultures, their values, beliefs and perceptions.
Cultural awareness is crucial when interacting with people from other cultures [20].
At the same time, research cultures differ massively between research domains.
Some interviewees explicitly referred to this by calling it “ culturally informed
awareness, e.g. where computer scientists have another focus than educational
scientists ” (P39, 64, ll. 337-339). Differences exist both implicitly and
explicitly in shared knowledge, social aspects of the research community, practices
and conventions, common theories and cognitive processes, and with respect to
theoretical assumptions. Awareness of those differences becomes increasingly
important, as research projects are ever more multi-cultural and multi-disciplinary.
Whereas training for intercultural competence and sensitizing is very common
in economy, academia is slow at offering it.
3. Social awareness Social awareness describes the things people become
conscious of in a social context. This includes information about the attentiveness of
others, gestures and facial expressions that mirror the emotional state of a person
as well as clues about a person’s interest in a topic. Whereas social awareness is
easily realized when workers are co-located, it has to be mediated in distributed
working environments. [
Besides these forms of awareness, the interviews pointed towards the existence of five aspects of awareness that are relevant in the context of Research Networks. 2.3
Five different aspects of awareness The five different aspects of awareness are relevant in any of the above forms of awareness. The importance of a single aspect, however, strongly depends on the object of interest.
A. The technological aspect of awareness The technological aspect of awareness is strongly affiliated with tools and techniques that are relevant for carrying out tasks. On the one hand there is always the question: “ where do I get the information from? Now we’re on a technological level, which is more or less push or pull ” (P24, 28, ll. 32-34). On the other hand different technologies support different forms of awareness. Answering the question “ Which tool was used to create this object? ” may help repeating research results and understanding the methodology used. Moreover, answers to the questions “ Which tools could I use to accomplish this collaborative task? ” and “ How can I reach this person? ” are direct enablers of social interactions and cooperative work. With the increasing Detecting and Reflecting Learning Activities in
Personal Learning Environments Alexander Nussbaumer1, Maren Scheffel2, Katja Niemann2, Milos Kravcik3, and Dietrich Albert14 Abstract. This paper presents an approach for supporting awareness and reflection of learners about their cognitive and meta-cognitive learning activities. In addition to capture and visualise observable data about the learning behaviour, this approach intends to make the leaner aware of their non-observable learning activities. A technical approach and partial implementation is described, how observable data are used to support reflection and awareness about non-observable learning activities. Basis for the technical solution is the extraction of key actions from log data of the interaction of users with resources. Furthermore, a taxonomy of learning activities derived from self-regulated learning theory is used for matching its elements with actually performed actions.
Keywords: learning analytics, learning activities, self-regulated learning personal learning environments, widget, ontology 1
Introduction In the recent years a trend became very popular to create small applications for specific purposes with limited functionalities. A second trend became popular in the technology-enhanced learning area, that systems and technology appeared that allow to create learning environments by mashing up such small applications (e.g. iGoogle5). The European research project ROLE6 aims to achieve progress beyond the state of the art in providing personal support of creating user-centric responsive and open learning environments. Learners should be empowered to create and use their own personal learning environments (PLE) consisting of different types of learning resources. 5 http://www.google.com/ig 6 http://www.role-project.eu
Strategies have been developed for supporting the creation of such PLEs
which are in fact bundles of widgets. Ideally, such widget bundles should include
widgets that support the performance of several cognitive and meta-cognitive
learning activities, in order to be used for self-regulated learning. Beside widgets
for domain-specific activities, there is also a need for meta-cognitive activities,
such as goal setting, self-evaluation, or help seeking (see [
Existing work in the field of learning analytics typically focuses on collecting
and visualising directly observable data of learner behaviour. For example the
approach presented in [
This paper presents an approach to support awareness and reflection of the non-observable cognitive and meta-cognitive learning activities. Section 2 describes the underlying pedagogical approach (learning ontology and self-regulated learning) and the technical basis (extraction of key actions from captured usage data). Section 3 takes into account these underlying concepts and presents a new approach to support awareness and reflection, which includes a pedagogical and technical perspective. 2
Related Work and Baseline
Contextualised Attention Metadata and Visualisation
Previous work has been done in the context collecting log data in a structured
way and visualising these data. Contextualised Attention Metadata (CAM)
captures the interactions of users with resources and tools. Each time a user performs
an activity with a resource (e.g. a document) in the context of a tool, a dataset
structured according to CAM is created and stored. In this way the behaviour
of users can be tracked [
A tool that exploits CAM information for making users aware about the own
learning behaviour is CAMera [
The Student Activity Meter (SAM) and the CAM Dashboard are two further
applications that demonstrate how CAM data can be used to support reflection
of the learner [
Key Action Extraction
In [
A B C A B D B C A B A A C D
The letters represent the actions of users in a session. The merging of n-grams
is possible if the frequency of the new key action is above a set threshold. Let’s
assume the threshold in this example was set to 2. As no monograms are below
that threshold, all of them are used for further calculations. The bigrams AA,
AC, BD, BA, CD and DB only occur once. Hence, they are discarded from further
calculations and can consequently neither be a key action nor part of one. This
example ends with two key actions, the tetragram BCAB which occurs twice
and D. The detailed approach can be found in [
Self-regulated Learning and Learning Ontology
A model for Self-regulated Learning (SRL) in the context of PLEs has been
proposed in [
Learning strategies and techniques have also been assigned to widgets
stating that these techniques are supported by the respective widgets. The basic
assumption of creating good PLEs is that the assembly of widgets to a
widget bundle should follow a pedagogical approach. Assembling widgets to a PLE
then follows some guidelines which underlying constructs should be contained
and how they should be assembled [
Detection and Reflection of Learning Activities
The goal of this paper is not only to monitor and visualise the observable actions,
but also to monitor the cognitive and meta-cognitive activities that are not
directly measurable. To this end the measurable actions are mapped to cognitive
and meta-cognitive learning activities. To be precise, the key actions extracted
from the CAM data analysis (see Section 2.2) are mapped elements of the
learning ontology (see Section 2.3). The mapping is partially done by the learner
herself, but also supported by an algorithm that takes into account the previous
manual matchings.
The overall approach from a technical perspective is depicted in Figure 1. The
learning environment where CAM data is captured is a ROLE space with a set of
widgets. Each widget logs CAM data according to the actions of the learning. In
particular, this includes the actions that a learner performs on the widgets or the
documents represented by the widgets. The CAM data are stored in the CAM
service which is basically a database for CAM events that receives these events
over a REST interface. The analysis component accesses these CAM events, in
order to detect key actions. This is done in the same way as described in Section
2.2 and [
The learning ontology consists of cognitive and meta-cognitive learning activities describing typical learning activities. It is modelled in RDF format and stored within a service that exposes this ontology over a REST interface (using SPARQL queries). This allows for retrieving lists of learning activities from this service.
The core component of this approach is the matching component where key activities are mapped to learning activities. It consists of a user interface and a back-end service. In the user interface the learner can manually assign learning activities to extracted key actions. Based on previous assignments, learning activities can be recommended for each of the key actions of the user. So the learner has not to do the whole assignment work, but can chose from a few possibilities or just approves the recommended assignment. The back-end service provides the key actions for each user and also offers the recommendations. These recommendations are based on previous assignments that are stored in an assignment database. The pedagogical perspective of the presented approach focuses on the the reflection and awareness aspects of the learning process. In contrast to existing approaches where learners are made aware of their observable actions, this approach intends to make learners aware of their non-observable cognitive and meta-cognitive activities. Based on literature review a taxonomy of learning activities has been created that describe typical learning activities. In order to match observable and non-observable activities, the learner is presented with the key actions of their own learning behaviour. Then the learner should assign which cognitive or meta-cognitive activity is represented by the respective key actions. This assignment task should stimulate the learner to think about the cognitive and meta-cognitive learning activities. In addition, the learner gets suggestions for learning activities that are candidates for the observable performance. This mixture of active assignment and support through the suggestions for assignments makes up the pedagogical approach. 3.3
Implementation
Several components of this approach have already been implemented in the
context of previous work. A widget container where widgets can be added to a
widget bundle has been developed in the ROLE project. The CAM service is
used to collect CAM data from the widgets and makes them accessible for other
components. The key action detection algorithm has already been implemented
and described in [
New development needed for this approach is the component that matches observed key activities with learning activities from the ontology. This component will consist of a widget as front-end for the user and a Web services as back-end for the widget. The back-end provides recommendations for assignments of key actions with learning activities to the leaner. The learner actually commits assignments, which is stored in a database and used for further recommendations. The recommendation algorithm takes into account all committed assignments. 4
Conclusion and Outlook This paper presented an approach for supporting awareness and reflection of learners about their cognitive and meta-cognitive learning activities. In contrast to typical learning analytics solutions, this approach focuses on non-observable learning activities that should be made aware and stimulated. Observable tracking data are analysed and key actions are extracted. By assigning learning activities to these key actions learners should become aware about the cognitive and meta-cognitive learning activities.
A technical approach is presented that supports this pedagogical approach. While some components of the technical approach are already available, others are under development. Next steps include the development of the assignment and recommendation component. This component integrates the existing components and provides the user interface for the learner. Further work also includes the evaluation of the first prototype regarding its usefulness.
Acknowledgements The work reported has been partially supported by the ROLE project, as part of the Seventh Framework Programme of the European Commission, grant agreement no. 231396.
Improving Social Practice: Enhancing Learning Experiences with Support for Collaborative Reflection
Martin Degeling1, Michael Prilla1 1 Ruhr-University of Bochum, Institute for Applied Work Science,
Information and Technology Management
Universitätstr. 150, 44801 Bochum, Germany {martin.degeling, michael.prilla}@ruhr-uni-bochum.de Abstract. In this paper we describe collaborative reflection as a core way of informal learning at the workplace. From three case studies we derived reflection on social practice as a good example for learning at the workplace. The way employees talk to third parties like patients or customers was observed to be a major topic in discussions within teams as it triggers the sharing of experiences about cases and fosters building of mutual understanding of common problems. We identified articulation to be a core part for this kind of reflection and derived requirements which were than implemented in a tool to support reflection on this topic focused on a healthcare setting and tested out application to reflect on talks with relatives of patients.
Keywords: collaborative reflection, learning at work, articulation, social skills
1
Besides technology support for the collaborative learning and extension of
knowledge, there are many skills that cannot be taught like e.g. physics but have to be
learned by experiences made during every day work. Although there is an overlap
between formal learning and learning by experience [
This paper reports on a core way of informal learning at work, namely (collaborative) reflection. Reflection is a learning mechanism that transcends the teaching of facts or the combination of different perspectives to create new knowledge. It rather suggests that re-thinking work practice in the face of current knowledge can support and improve future practice. However, although reflection has been recognized as a frequent and essential part of informal learning and there are hardly any insights into processes of collaborative reflection and their support by tools. This paper describes research aiming at closing the resulting gap. This work will be described in the remainder of this paper by the example of supporting the improvement of social practices at work.
The paper is organized as follows. First we describe a model of individual reflection and informal learning to then broaden the view on collaborative reflection and research done in that area so far. In section 4 we then draw on three case studies in different organizations1. Due to the lack of insights into collaborative reflection and in order to create an understanding of processes associated with it, the studies were conducted in an exploratory manner, including interviews with the groups described above and work observations. As an outcome, the studies shed light on collaborative reflection of social practice in particular (section 5) and on process characteristics of collaborative reflection in general. 2
Collaborative Reflection and Informal Learning at the Workplace
Besides situations of formal learning in dedicated sessions where knowledge is
presented by teachers or facilitators learning at work is often rather informal [
Figure 1 Reflection model by [
Following [
Reflection is therefore closely related to problem based learning (cf.[
The vast majority of research on reflection is done on individual reflection and
most models have a strong individual focus [
Learning by collaborative reflection may then occur when an individual links her
knowledge to the experience of others [
Those criteria also match situations in which groups collaborative rethink situations of social practice and interaction with third parties like customers since those situations are re-occuring in general but each episode is different. 3
Related work: Tools for Informal Learning and Reflection Since reflection is based on going back to past experiences tools to support collaborative reflection and informal learning tools have been researched for quite some time to overcome limitations of fading memories and uncertain remembering. Various approaches were tested on their supportiveness.
One way is to use additional hardware and sensors that automatically collects data
which afterwards can be used to support reflection processes. For example a
SenseCam – a wearable camera that makes photos automatically – was used in [
Others require participants to manually collect information e.g. in [
A third group of authors uses data that is generated during regular work tasks. In
[
All tools developed show the usefulness of collaborative reflection to learn about
past experience. Especially they point to the advantages of additional data to foster
collaborative reflection (cf. [
The nature of collaborative reflective learning: An Analysis
Do deepen our knowledge on reflection and especially collaborative reflection we
organized case studies at three different sites from health care and business
professions. For a deeper analysis of modes and types of collaborative reflection and
tool support cf. [
The second case concerns British care homes for people suffering from dementia. Here, care is not organized around emergencies but on daily work routines and sustainable work with residents of the homes to support self-conscious living as long as possible. At a typical care home, five to seven caregivers work with 40 to 50 residents. As the caregivers have no higher education and get just a two-week training one registered nurse per shift is responsible for medical treatments. What differentiates senior caregivers from junior caregivers is the experiences and time spent in the job. This experience is crucial for the job, as the caring for people with dementia is emotionally demanding, as residents may behave unexpectedly and e.g. shout at staff (situations like this are called “challenging behavior” in care homes). Exchanging insights and reflecting on such cases is already recognized as an important learning mechanism: Caregivers organize what was called in one home “reflective meetings”, during which they talk about experiences with residents that were difficult to cope with. In interviews, especially junior caregivers reported that getting feedback and exchanging experiences with more experienced colleagues is a fruitful way to get better in their job. Other occasions of getting together and collaboratively discussing include the shift handovers, in which the nurses and caregivers from overlapping shifts discuss the status of each resident, e.g. whether they showed unusual behavior, and try to find new ways of handling those residents with problems or challenging behavior.
The third case is an IT consulting company in Germany, which focuses on the provision and adaptation of customer relationship management tools for manufacturing companies. In that company our target group are employees from the sales department, who are responsible for customer acquisition and handling the handover from sales to other (development) departments. Learning in the sales department is mostly self-directed and based on experiences from projects and client encounters. They unregularly receive short trainings e.g. about new software features, which are mostly on the web, but according to employees, the main part of learning to improve professional skills is based in practice and self-evaluation as well as evaluation by others. This is also mirrored in regular meetings of the sales department, in which current client activities are described and the participants discuss critical issue in these activities based on their experiences. 4.3
Analysis: Reflection of social practice as an indispensable task Besides differences stemming from the variation in professions, we observed similarities in all cases. While all organizations offer formal training for their employees, we observed hardly any (official) support for informal collaborative learning based on reflection: In all cases, employees used meetings, breaks or short talks on the hallway to discuss cases, residents or customers with colleagues, to ask for their assistance or to offer insights from their experiences to others. This was especially the case for topics that relate to social interactions with those third parties that could be grouped as “service consumers” (patients, residents and clients in the three cases described).
For example, at the hospital we observed that especially for young physicians talking to relatives was a critical task: They often have to explain difficult medical cases to relatives without a background in medicine and these talks often include conveying bad news like brain injuries patients may never recover from. These interactions are only partly covered in formal educations of physicians. Therefore, getting bad feedback from relatives or finding themselves in unpredicted situations often causes physicians to talk about their experiences to others.
At the care home, we found caregivers to often discuss challenging behavior of patients (e.g. behaving aggressively for no apparent reason) very often. Discussions took place in breaks and meetings with other caregivers. In one meeting, a junior caregiver reported a problem with a woman, who asked when she was allowed to leave the care home several times per day. The caregiver had problems telling her that this is not possible and reported how this affected him emotionally. Senior caregivers in the meeting then reported from their own experiences what could have caused this behavior and explained how they had dealt with similar situations before. This helped the young caregiver to understand how to deal with such situations and showed him that these problems are not only relevant for him. In the meeting, the participants then also agreed on ways to handle the requests of the respective elderly woman that were supposed to be used by all caregivers dealing with her and similar cases in the future.
Reflection topics around social interaction with third parties were also present at the consulting company. We observed consultants to often discuss habits and behavior of their contact persons at a customer as well as how they performed in recent presentations at certain customers. They even reported that these situations would happen often and that they discuss issues with colleagues e.g. if they had been together at a customer’s site. They see the experience from colleagues on how they acted as valuable feedback for improving their abilities and welcome constructive criticism.
It can be seen from the examples that collaborative reflection of social practice is an important and common topic across the various professions we investigated. In all cases we observed people to think and talk about the way they interact with customers or patients. They discussed and compared with colleagues, especially more experienced ones, to improve their skills. 4.4
The process of collaborative reflection and the role of articulation
Besides the identification of topics for reflection, we developed a reference cycle for
collaborative reflection, which is shown in Figure 2. The cycle is intended to derive
requirements and support the implementation of computer support for collaborative
reflection (see [
Figure 2 Model of Collaborative Reflection (cf. [
The cycle shown in Figure 2 can be illustrated with an example of reflecting social practice from the cases presented above. In what follows, we chose the reflection of conversations with relatives as explained in case 1 for this. It should be noted that the cycle is not necessary linear, but that steps are interchangeable. For example, individual reflection may happen during documentation, e.g. when a physician thinks about a conversation while documenting it, and there might be multiple loops of collaborative reflection in several groups before outcomes can be documented.
The cycle starts with the activity of documentation and data capturing, which in the case of conversations is important to support the individuals participating in the talk to remember the situations and their emotions during it in order to come back to them. This sets the stage for later reflection and also enables individuals to sustainably share experiences from talks with others (as part of their practice to talk about them) and discuss them together when there is time for it.
Individual documentation of conversations is helpful for individual reflection and enables physicians to reflect on talks some time afterwards, e.g. after they completed their shift on a stressful day. Similar to offline reflection helpers like diaries, a tool needs to support individuals in going back to past experiences on talks, to remember situations in more detail and to articulate insights stemming from reflection of them.
As observed in the hospital, there is a need to share experiences from conversations and make it available for sessions of collaborative reflection. Tools for this need to enable user to share documented talks and to discuss talks that were shared with them. This is helpful especially in work situations where time constraints are otherwise impeding like during the day of physicians. Moreover, in meetings of physicians, the group can come back to shared documentation and results from asynchronous discussion and start a face-to-face reflection session.
For reflection on conversations to lead to improvement, there is a need to support sustaining outcomes. The lack of means for this is a major shortcoming in daily reflection practice, as it hinders the benefits of reflection from becoming visible to others and to be implemented. The cycle shows that documented outcomes may then serve as input for further reflections, e.g. when a physician changes her way of conducted conversations and makes experiences on these changes.
As visible in Figure 2, articulation is a central activity for collaborative reflection. This can be seen in the example: To start the cycle of reflection, physicians need to document (articulate) the content of talks. Then, they need to articulate their thoughts and perceptions on a conversation as part of individual reflection, as they are otherwise not visible to others. Moreover, for collaborative reflection, they have to articulate their perspectives and thoughts on talk documentation shared with them. To close the cycle, there is a need to express insights taken from collaborative reflection in order to make it sustainable and available for implementation. Therefore, articulation support has to be considered a decisive factor in implementing collaborative reflection support. 4.5
Requirements for collaborative reflection support Besides the importance of articulation derived in the previous session, it is obvious that there is a need for human articulation in reflection of social tasks: These tasks cannot be described (only) by formal criteria and social interactions cannot (only) be learned in formal training. Rather than that, they are subject to informal learning processes, which rely on communication and learning from peers – without articulation, learning is only possible from observation and experiences remain with the individual. Therefore, we regard articulation to be of central importance for the reflection of social interactions as described in this paper.
From the above case studies, we can derive corresponding requirements for articulation support in tools for reflective learning. As a prerequisite for these requirements, we assume that articulation needs to transcend verbal communication in order to become available to a larger audience and for reflection participants to refer to details of articulated experiences. However, noting experiences often problematic due to time pressure and other tasks to be done. For future tool development this implies that:
Articulations have to be easy and unobtrusive to make: Users should be able to document experiences 'on the fly', e.g. in a very simple interface that is easy to use or by voice input. Articulation tasks should not cause much additional effort or need a lot of attention. For example, the articulation of emotions during conversations with relatives should be as easy as possible as they are not necessary for work and would thus possibly not be done by medical staff. Articulation tasks have to be integrated into work tasks: Tools for articulation in reflection should be easily accessible throughout work and be closely related to regular work tasks to lower the burdens of additional tools. In the case of documenting conversations, it should therefore be avoided to cause additional work by requiring physicians to document conversations in the patient’s folder and in an additional reflection tool.
Articulation of experiences has to be accepted as valuable task: Since articulation always causes some effort, tools need to show users that outcomes of articulation and collaborative reflection are helpful – not only to the individual that did the articulation task but also to others participating in reflection sessions. For the reflection of conversations, tools need show users that documenting experiences leads to improvements for their conversations sooner or later. People need to be aware of articulated experiences: For documented experiences to become usable in collaborative reflection, digitally sharing them must result in recipients noticing their availability. This opens up the possibilities for collaboration and mutual commenting. Taking the example of the hospital above it would not be sufficient to add a paper to the patients case folder for documentation of talks because this is only accessible in the patients room. Articulations should be contextualized: As there might be many articulations created over time and as reflection participants look for experiences and insights suiting their respective case or problem, there is a need to contextualize articulations, e.g. by referring to specific cases or actors that took part in experiences. In the example of reflecting conversations with relatives, contextualizing could be done by grouping conversations on the same medical disease or with relatives of the same patient.
The requirements above show how articulation as a key mechanism in collaborative reflection support tools can provide support that can be handled and integrated into daily work easily. In what follows, we describe a sample implementation of these requirements.
Implementing articulation support for collaborative reflection Using the example of reflection conversations with relatives in healthcare, below we present a tool built to support articulation and other reflection activities. In addition, we reflect on experiences with implementing the requirements described above.
The Talk Reflection App – Documenting and Reflecting Relative Talks In close partnership with the hospital described as one case we designed and tested a tool that implements the collaborative reflection model described above and fulfills the requirements described in section 4.4. The aim of the tool shown in Fehler! Verweisquelle konnte nicht gefunden werden. and Figure is to support individual and especially collaborative reflection of conversations physicians have with relatives of patients at the stroke unit.
Figure 3 Individual and collaborative reflection spaces: Each documentation can be viewed, shared and discussed. Assessments displayed in spider graphs for a quick overview.
The basic idea is that physicians working on the ward document conversations they had and open them up to discussion with other physicians. It is already mandatory for all physicians to document conversations they had in the patient’s folder by hand and sometimes also separately on a computer to inform physicians in later shift which therapy was agreed on or which measure to take in case of emergencies. To simplify the documentation process the application we developed is designed for mobile devices like smartphones and tablets.
The documentations are shown on the right side of the screenshot. On the left you can see lists of documentations done by the users itself (1a) by others users that shared the documentation (1b) and documented outcomes of collaborative reflection (1c). The sharing of documents and a list of users that have access to the currently visible document is shown at (2). The only additional efforts physicians have to take is to make short self-assessments and answer questions about how they felt during the conversation or what they think how the conversation partner felt during their talk. These self-assessments are visible only for the person documenting and are afterwards visualized (3) to make simple comparisons between documented conversations and support remembrance. Least at (4) you can see the space for comments and notes. Here annotations and comments of other users are displayed that can be used to report on similar experiences or discuss want went well or wrong in the case documented above.
To support the sustainment of outcomes of reflections we developed a page to overview the list of documentations (Figure ). Here users that did individual reflection or participated in a synchronous or asynchronous reflection session can select on or more cases that they reflected on (3) and document explicit outcomes e.g. changes in procedures or good practice. Outcomes are divided into a short descriptive title (2) and a more detailed description of the outcome that highlights the commonalities of the cases selected (1). Afterwards these documented outcomes are shared among users of the app. 5.2
Implementing articulation requirements: Insights from design
We conducted two workshops with physicians of the hospital. They were planned and carried out as part of a formative evaluation to prepare a broad roll out in the hospital ward. The first workshop with three physicians was focused on utility and applicability of the app. I the second workshop another four physicians tested and evaluated a second prototype to test-drive the rollout in the ward.
Referring to the requirements described in section 4.5 we received valuable feedback. In general users agreed that the application is easy to use and they had fun making documentations with the simple, mobile interface. Nevertheless they had several suggestions for usability improvements like a larger input fields for personal comments and ideas for a more intuitive naming of certain categories. They also discussed a lot about problems with auto-correction of medical terms by the mobile OS and issues with syncing the content of the app with the server resulting from the poor WIFI connection. The fact that all these issues came up during the discussion shows the importance of this requirements and the need to improve user interfaces and input methods to make them less obtrusive.
During our workshops we also discussed better ways to integrate the app into daily work. As shown in Figure 3 we already implemented a button to export documentations by e-mail, which allowed them to copy & paste the documentations into the HIS, but due to the connection issues this did not work out very well. Unfortunately a smoother integration with automatic synchronization, which would be most comfortable, is not possible due to constraints of the IT department and high development costs for program interfaces of the proprietary HIS. Therefore participants proposed to give up the benefits of the mobile device and start using the app on the desktop PC as well where they can easily import and export information from on. This decreases possibilities to document cases outside the physician’s office but they also reported that they used this option not as often as thought upfront. We also stated that the articulation of experiences has to be accepted as a valuable task. During the workshop we observed participants heavily referring to what they wrote when explaining the cases again and using the documentations as additional information to more blurry memories. We also received multiple feedbacks that the app and discussions itself resulted in a higher awareness for the topic of conversations with patients and relatives. On user requests we also added a checkbox that says “I want to talk about this later” to raise awareness for certain cases which participants would regard as unusual or more important. There were also ideas for additional organizational support by introducing a bi-weekly meeting in which assistant physicians could talk about documentations they did face to face in addition to sharing them digitally.
The first feature to support contextualization of articulation we integrated was the self-assessment form. These short questions were regarded as helpful for quick assessments and during the workshops we agreed on questions that would better fit the circumstances like “How likely is it that I will think about this at home”. In line with the model they asked for the ability to document cases more detailed e.g. to be able to select from a list of topics like “therapy”, “diagnostic” or “information”. They argued that this would help to find similar cases more easily.
While the workshops were conducted in a formative approach they showed that the application and the underlying process and requirements are applicable to support collaborative reflection of social practice at the healthcare workplace. The participants had numerous ideas and scenarios how the app could be improved to fit better in their workplace settings and already used it in the workshops to document, share and discuss cases of conversations they had and wanted to reflect about.
In this paper we described the importance of collaborative reflection for learning at work. We focused on reflection as a mechanism for informal learning within groups sharing their experiences. Those are especially relevant for learning for topics like social practice that cannot be learned from articulated knowledge but is a result of a continuous process of comparing own behavior to that of others. From two case studies in healthcare and consulting businesses we identified conversations with customers and patients to be a reoccurring topic in collaborative reflection. As an example we took reflection at a hospital about conversations with relatives and developed two prototypes that where tested with groups of physicians on their applicability to support reflective learning about this topic.
The requirements that were elicitated during the case studies proved to be supportive for tools use. We designed the tool to integrate into daily work as articulation is already part of it. That notes are digitally shareable and less dependent on the paper based patients folder was very much appreciated. In addition the fact that the availability of the app raised awareness for the topic itself and fostered discussions not only in workshops but also off the record e.g. in breaks or spontaneous meetings.
Nevertheless there are improvements to make in the ways physicians can use the app as due to technical restrictions and missing wireless connections it was too difficult to use the app since they had to go to a special room to synchronize data. In addition further work has to be done to simplify technical integration between official documentation and the Talk Reflection App to reduce double work as it sometimes took place during the tests. But as the tests brought promising results and positive feedback we will adapt the process and apps to other domains.
References Boud, D. 1985. Reflection: Turning experience into learning. Kogan Page. Daudelin, M.W. 1996. Learning from experience through reflection. Organizational Dynamics. 24, 3 (1996), 36–48.