Prompting users to facilitate support needs in collaborative reflection

Abstract

Reflection and collaborative reflection are common activities at workplace, in which people alone or together with colleagues revisit experiences to learn for future situations. Often support needs arise during reflection and for example people get stuck and don't know how to continue. We propose to fill this gap with prompting as tool-supported scaffolding mechanism. Based on literature and our own previous work on reflections support this paper presents a concept using prompting to support different goals based on known support needs in tool-supported collaborative reflection. We show an initial implementation of a prototype and an approach how to evaluate it. This work contributes to the (AR)TEL community in that it provides a concrete approach for scaffolding in reflection support to be discussed at the workshop and in that it shows how prompting can be used and implemented in tools to evoke reflection.

Keywords

• collaborative reflection
• prompting
• reflection support

-

Reflection is a common and important activity at workplaces [ 1 ]. It includes three stages: returning to past experiences, reassessing them in the context of today’s experience, and then deriving conclusions about how to behave in future [ 2 ]. Schön differentiated reflection-in-action from reflection-on-action [ 1 ], with the former describing reflection about things currently happening and the latter describing reflection about things which happened in the past [ 1 ].

However in daily work often problems arise which make collaborative reflection difficult for people and for example sometimes they don’t know how to proceed and thus get stuck while reflecting [ 3 ]. Therefore facilitation of reflection is needed. This paper presents a concept using prompting to facilitate different aspects of collaborative reflection. The goal is to help users reflect in work place settings and to overcome barriers. After a related work section the concept will be laid out, followed by the implementation of a prototype alongside an approach on how to evaluate the concept. 2

Related work

This section presents work available on reflection (support) at work, including underlying theories, a reflection model and prompting as the concept used to support reflection in our work.

Individual and Collaborative Reflection

Reflection is mostly conceptualized as an individual, cognitive activity [ 1, 2, 4 ], which creates value by understanding own practice and improving it. There have always been voices on the social side of reflection [ 5 ] advocating the common practice of people to reflect together, but only recently research on reflection in groups has gained momentum [ 6, 7 ]. In this work the term collaborative reflection describes reflection in which multiple persons or groups of people are involved. It has been found that such reflection is common practice at many workplaces, and that it has the benefit of producing results which surpass the reflection results of individuals [ 8 ]. It should be noted, however, that collaborative reflection always includes individual reflection phases such as applying group results to one’s own situation [ 9 ]. Also collaborative reflection needs extra support for communication processes [ 10 ] and reflecting together adds complexity to the process of reflection, as multiple perspectives and contributions need to be coordinated and aligned [ 5 ].

Conceptualizing Collaborative Reflection: Models

Supporting (collaborative) reflection needs an understanding and operationalization of reflection processes that enable the development and implementation of support. There are many models available, including the three-stage model by Boud [ 2 ] and the cyclic reflection model by Kolb [ 4 ], which are used by most work available on reflection. However, these models focus on individual reflection and do not include a perspective on (technology) support for reflection, which make them hard to apply for designing support for reflection in practice.

Krogstie, Prilla and Pammer [ 9 ] developed a cyclic model that includes both a perspective on tool support and individual and collaborative reflection aspects (the “Computer Supported Reflective Learning” (CSRL) model shown in Fig. 1). The model contains four different stages, each having a defined input and output and thus describing how the stages feed into each other. In addition it contains triggers for reflection, which represent situations in which the reflection cycle is started. Tools can use this model by connecting to the stages, phases and triggers in order to facilitate the process and to ensure that required information for each phase is available.

Besides the four stages the CSRL model contains several phases, which describe activities that can be helpful in each stage. For example the Initiate Reflection stage contains phases like Set objective and Involve others (no. 2 in Fig. 1), which suggest that it is helpful to think about the reflection session in advance and that tools need to support bringing people together. Furthermore, the model shows that each stage has specific inputs and outputs, which support transitions between the stages. For example 44 without data about the work no reflection session can be initiated (“a” in Fig. 1) and without explicit outcomes change is not possible (“c”). Emphasizing the iterative nature of reflection, each stage can spark a new Initiate reflection phase, for example if a session did not result in an outcome and the group needs another reflection session. Using the model thus can provide support for tool designers to understand the needs of support for reflection processes [ 9 ]. In the research presented here it was used for this purpose and serves as a basis for describing goals for reflection support. The application of reflection support at work faces many difficulties such as time and space available to reflect or continuity in reflection [ 9, 10 ]. Available work on diminishing these difficulties and supporting reflection at work can be differentiated into conceptual work describing means and strategies to support reflection, and work related to tools, which describes approaches to use tools to enable reflection.

On a conceptual level research often emphasizes the need for facilitation in reflection. Daudelin [ 5 ] emphasizes the need for moderation in reflection groups, Zhu [ 11 ] describes questions (for further information or to provoke further discussion) as important means to facilitate reflection, and van Woerkom and Croon [ 12 ] describe group cultures that include actions such as asking for feedback and questioning groupthink as supportive for reflection. Schön [ 1 ] points to the need for more awareness for reflection in daily work by suggesting the “reflective practitioner”, and Vince [ 13 ] emphasizes the need to establish reflective practice individually and in groups.

Concerning tools we found that there are several approaches, which describe different levels of tool support for various degrees of reflectivity of users. Tools often allow users to document their problems and challenges by either manually writing 45 down what happened [ 14 ] or capturing data about daily work [ 15, 16 ], which already helps structuring one’s thoughts. More advances tools such as ECHOES by Isaacs et al. [ 17 ] remind users to regularly return to such data, reflect on it and implement outcomes of their reflection in practice. Sharing documented issues, which allows people to engage with colleagues in collaborative reflection, is mostly done by sharing the tool used to document experiences with others in face-to-face settings. Online discussion forums widely used in other learning contexts [ 11 ] and other tools supporting groups [ 19 ] have been studied in the context of reflection only on a generic level..

In our work, in which we analyzed the behavior of four different groups using a tool facilitating threaded discussions for experience exchange at workplaces to support reflection [ 20, 21 ], we arrived at insights that support and extend the existing work discussed above. In a comparison between more and less effective reflection groups we found facilitation by group members (e.g., asking each other to contribute) and the provision of guidance in the reflection process to be helpful [ 21 ]. From an analysis of the content created in the tool we found that providing experiences rather than advice and describing emotions during experiences positively influences the creation of outcomes from reflection [ 20 ].

What can be taken away from the existing body of work on reflection support is that there is a need to scaffold (that is, guide, facilitate and structure) the process of reflection in order to enable individuals and groups to learn from and for their work.

Prompting as a means to scaffold Reflection

As described above there is a need for scaffolding in reflection processes to guide people in individual and collaborative reflection situations. Especially in real workplace situations additional problems arise while reflecting for example due to time restrictions or because workers are not trained in reflection. Among these problems, workers can get stuck or are not sure how to proceed, needing new impulses to continue [ 3 ]. Scaffolding can also help to overcome these barriers.

There are different means of scaffolding, including scripts [ 22 ] that guide users of tools through a step-by-step procedure of learning, ensuring that necessary steps are taken, and open learning environments providing awareness and notification to learners in order to show them possible helpful learning activities. It has been argued in learning research that neither of these ends provides a good solution for all learning processes, and that there is a need to carefully balance guidance and freedom in learning processes [ 23 ]. One means of scaffolding that creates this balance is prompting, which has been used in learning environments to facilitate tasks and to stimulate reflecting about those tasks [ 24, 25 ]. Prompts needs to be differentiated from awareness mechanisms and notifications, which often don’t have an instructional character [ 26 ].

Prompts are cues designed to stimulate a certain behavior of the recipient [ 27, 28 ], but they don’t enforce that behavior [ 29 ], leaving the choice whether to react or not to the user. They can be created by oneself (self-prompting), by another person or a system (external-prompting) either randomly, periodically or trigger-based. Usually the person or system creating the prompt (prompter) sends a prompt to the recipient (promptee). Prompts may occur in the form of instructions given to the promptee [ 25 ] 46 or questions to be answered by the promptee [ 25 ], which fits the findings on using questions to stimulate reflection discussed above. From a technical point of view prompts are delivered by a prompting mechanism that may provide different prompts for different situations and purposes. Using a simple example, when setting an alarm clock (prompting mechanism) in the evening, one sets a prompt (the alarm itself) for oneself triggered by a matching time stamp, which most likely has the intention to instruct the user to get up in the morning. To help users with specific tasks it is helpful to have context-specific rather than abstract prompts [ 24 ].

As any scaffolding mechanism may also cause harm if not used in a proper way [ 23 ], prompts may also create negative effects. As an example for reminiscence longer prompts lead to lower response rates (but also elicit longer answers) [ 30 ]. They thus have to be designed carefully and the goal designing prompts is to achieve a tradeoff between the intensity of their usage and the effect on the user.

Related work has found that prompts are not always answered, which was dubbed negative in an education environment [ 31 ], in which prompts were part of the learning assignment. However, given the challenges reflection support faces at work, which often hinder people to reflect at all, we think that even if some prompts remain unanswered in certain cases, they are still helpful in creating opportunities for and scaffolding collaborative reflection at work, since we plan to support workers rather than enforcing the way they are using discussions at their workplace.

Although prompting has been used in the context of reflection as described above there is no concept for using prompts as scaffolding for various aspects of reflection in general. This paper presents such a concept, in which prompts help to facilitate reflection and assist if users are stuck while reflecting. 3

Concept to facilitate reflection through prompts

The following section shows our concept of how prompts can help facilitating (collaborative) reflection and how to overcome various barriers. For this we derived different goals from literature and our own work and connected each of the goals to the CSRL model presented above. We also present examples of prompts supporting those goals.

Goals for Reflection Support

Stage I: Plan and Do Work In the initial stage of the CSRL cycle the main goal is to provide data to reflect upon.

G1 Document surprising experiences in daily work: Reflection is often caused by

the discrepancy of reality in contrast how one expected something to happen [ 1 ]. Baumer calls this breakdowns, which are reasons to start reflecting [ 32 ]. Therefore, complementing means to capture data automatically, there is a need to document experiences in order to reflect about them later [ 10, 14 ]. Prompts for this goal are intended to start reflection and to help people develop a reflective view on their work.

Stage II: Initiate reflection Reflection may end before it has started, meaning that discrepancies are not followed up or no insights are created. Initiating it includes setting objectives, involving others and supporting descriptions and initial reflections on the experiences shared.

G2 Set the objective: Boud et al. argue that reflection is not “and end in itself” and that it is focused towards a goal in order to change future behavior [ 2 ]. Therefore setting an objective allows for a more clearly structured reflection session. Prompts may help users to set an individual or group related objective that helps to guide the discussion. This is also part of the input for the next stage in the reflection cycle [ 9 ]. G3 Involve others: Involving others into problem-solving offers of getting feedback from others and finding solutions together [ 8, 10, 12 ]. Thus in line with recommendation research [ 33 ] there is an need to support reflection participants in inviting others such as people someone is frequently getting input from or people the user never worked with before to benefit from new ideas and insights [ 33 ]. Prompts can explicitly ask for such invitations and may include recommendations.

G4 Support individual reflection when describing experiences: Reflection partici

pants should not only describe a problem but also include first explanations and justifications. Fleck and Fitzpatrick [ 15 ] as well as Hatton and Smith [ 34 ] describe this as two different levels with the first one as just descriptive but not being reflective. In order to move away from that level, prompts may guide people into reflection while starting a new discussion thread by asking to think of explanations and interpretations. G5 Elicit problem description: Prompts can also help eliciting the problem description by asking for a more detailed description. Alternative explanations and different viewpoints are necessary to reach a form of dialogic reflection [ 15 ]. Additionally there might be also the need for prompts to limit the length of posts, since very long posts are often not read fully by others in contrast to very short posts which can’t transport all required information [ 35 ]. This goal aims to help setting up the subsequent stage in the reflection cycle through trying to make sure that enough information is present.

Stage III: Conduct reflection session A major aim of the reflection session is to create outcomes that people may use to improve their practices. This can be done in different ways:

G6 Get people to explicitly link to their experiences rather than giving plain ad

vice: As described above we found that sharing experiences leads to an increased likelihood to get reflective outcomes in tool supported collaborative reflection [ 20 ]. Prompts may support this goal by asking people to argue from experience instead of giving just plain non-reflected advice. This also connects to the phase of making related experiences available in this stage of the CSRL model [ 9 ].

G7 Single Loop Learning & G8 Double Loop Learning [ 36 ]: Both goals support learning while reflecting [ 1, 2 ]. This may occur either on the level of learning for particular tasks or problems (single loop learning, see [ 36 ]) or by deriving more general insights in the nature and prerequisites of work (double loop learning). Prompts may guide reflection participants to either of these outcomes by asking for solutions to the particular problem or by focusing on supporting people to state what they can 48 learn in general respectively how the knowledge can be applied to different problems. This is also helpful for other people who are participating in the discussion thread.

Stage IV: Apply outcome

Most work on reflection does not include the implementation of insights. However, in order to provide effective support this stage is equally important as the other stages. G9 Check whether stage has been reached: In order to proceed to the application of outcomes at least one outcome needs to be mentioned in the reflection session. Prompts may aim towards helping the author/starter of a discussion to identify outcomes or to start a new reflection cycle to e.g. pursue another direction in the discussion or discuss some point in detail. This and G10 are similarly supported by literature as G2 with these goals being more focusing on realizing the objective of the session.

G10 Plan application of reflection outcome: This goal supports user in transforming

ideas from the reflection session into a detailed plan of how to implement the change. This way the outcome might get more concrete. This can help people engaging into multiple reflection cycles with describing a problem first, then reporting back insights and they have learned after that (see G11).

G11 Check application of outcomes of a previous reflection cycle: To support the

implementation of change users can be reminded to regularly check their progress in applying outcomes from reflection [ 17 ]. Prompts may regularly ask users to selfassess their progress or ask them to revisit their goals.

Stage-independent (cross-cutting) goals There are some goals that can be applied to multiple or even all stages of the CSRL model, including training on good reflection practice, support for self-efficacy and the provision of an environment in which reflection works.

G12 Train people how reflection works best: Reflection, though seemingly intuitive, is not easy but has to be learned. Explain to reflection participants how reflection processes work in order to enable them to proceed with reflection is common in some disciplines such as nursing, in which reflective practice is well established and studies explain that being reflective has to be trained [ 37 ]. Prompts may enable users to understand the basic concepts of reflection (based on the work of Schön [ 1 ] and Boud [ 2 ]) and they may provide concrete instructions how to proceed in order to show how collaborative reflection works (e.g., based on the CSRL model [ 9 ] discussed above). Though one challenge might be providing a way to deliver adequate personalized feedback regarding learner’s success. Still the means of prompting itself has been used successfully in teaching and education to stimulate users elaborate their answers or to stimulate self-reflection [ 24, 25 ].

G13 Support people to feel safe in the environment: In discussion settings when

people report of their challenges the fear of being judged or criticized can hinder people from posting [ 38 ]. Prompts for this goal include hints that one could post something anonymously thus avoiding direct criticism which seem to work in education settings [ 39 ], or try to remind people to treat each other fairly [ 40 ].

A prompting concept to implement the goals for reflection support

Table 1 shows on overview how the goals above can be implemented by concrete prompts. Each goal is connected to one or more example prompts.

Goal Example prompts

G1: Document surpris- What did surprise you in your daily work lately? Why? ing experiences in Have a look at your calendar of the last week. What was daily work difficult for you? How? G2: Set the objective What is it you want to know from your colleagues? Ask them a specific question! G3: Involve others Who of your colleagues might help you in this? Mention him in your post to notify him or her (@username).

What has happened? Is there already something you learned from it? G4: Support individual reflection already in first post G5: Elicit problem description G6: Explicitly link to experiences rather than giving plain advice G7: Single Loop Learning

G8: Double Loop

learning G9: Check whether stage has been reached G10: Plan application of reflection outcome G11: Check application of outcomes of a previous reflection cycle G12: Train people how reflection works best

Help your colleagues understand your problem: Try to describe what things you already tried to solve your issue! Suggestions are most helpful if they are based on your experience!

What would you personally suggest as a solution? Why? What have you learned regarding this topic so far from this discussion? What is your personal outcome of this discussion so far? What have you learned so far on an abstract level?

Do you have an idea from this discussion how to change your approach to the topic? If not, phrase a new question. How do you want to implement the suggestions of your colleagues?

Did your plan work? What are your experiences with the change? Tell your colleagues about it.

G13: Support people to

feel save in the environment As can be seen from the goals and corresponding prompts, currently the concept is focused on common problems concerning reflection, and it covers the whole cycle of 50 reflection as described in [ 9 ]. While the goals as well as the prompts may be extended, to the knowledge of the authors this is the only concept that shows how prompting may influence reflection in such as broad way. However, there are several possibilities in extending the concept to pursue facilitate other reflection related aspects:

Linking to creativity support: The concept may be complemented by creativity techniques in order to help users getting new ideas about how to approach their work related challenges and how to merge ideas into solutions. One creativity technique to support this might be directed brainstorming [ 41 ], in which a topic is split up into individual parts having a brainstorming each. For the reflection participants this adds the benefit of choosing from a number of ideas, which is easier than creating new ones [ 42 ]. Santanen et al. also used the means of prompting in their study of directed brainstorming [ 41 ], thus linking this to reflection might be promising, but further research is required to study how the concept of directed brainstorming through prompts is also helpful in reflection settings.

Setting goals: Setting personal goals is beneficial for personal reflection since people can use this to plan reflection topics or sessions [ 43 ]. This is also an opportunity to facilitate reflection through the means of prompting. Prompts could help users in reminding them of looking at their goals to reassess whether the goal has already been met, whether the goal is still worth pursuing or whether the goal is now obsolete. Also prompts can assist users in checking whether discussion contributions are helpful to progress towards their goal.

Sustaining reflectivity: Often people who are more experienced in their rob (role) are less likely to have a reflective approach to their work [ 1, 44 ]. Prompts may evoke and sustain their reflexivity by helping these people to take a role similar to a mentor in which they can contribute with their experience on topics initiated by colleagues. Mentorships often have various benefits also for the mentor like learning from the protégé, getting new work related information, and extending the network [ 45 ]. The approach is to deliver prompts which are not asking the experienced worker directly to think about their work, but to try get their feedback for the work of their mentees, which in turn might possible cause them to think about their work. 4

Implementation and Evaluation: Work in Progress

This section reports on the implementation of a prototype incorporating the aforementioned prompting concept, including an approach for its evaluation.

Implementation

To evaluate the concept we are currently implementing a system supporting reflection in online discussion threads as used in typical tools such as community platforms and learning environments. This enables a group of people working together to talk together about the problems they face in an asynchronous way, not requiring them to find a common time and date [ 10 ] and to exchange experiences with colleagues working in the same organization but in different offices. Using prompts might then facili51 tate collaborative reflection and discussion among the participants in the group. This is especially helpful if used in existing platforms in an organization, as it adds reflection to the platform while preserving a well-known and accepted tool for users1.

The prototype currently features threaded discussions as well as social networking features like profiles, personal messages, friendships, mentioning. It also contains the option to share files and a common news section. Features like mentioning facilitate involving others (G3), making adhering to that type of prompts more simple. Having friendships allows people to expand their network to get automatically get notified of discussions their colleagues participated in.

Fig. 2 depicts a discussion thread in the prototype, showing that a thread was already started and that the user viewing the thread has now the possibility to reply. Just below the thread and above the text input area the prompt is shown asking the recipients to think back whether they also experienced something like this and to tell their experiences (G6). This also shows that the prompts are implemented in a way not forcing the recipients to react in any specific way. The prompts will be later on accompanied by a button containing information that the prompt is trying to help them structuring their posts and thus showing a trajectory of how they can make use of it.

Ideas for the Evaluation of the Prompting Approach

The concept described in this paper will be evaluated with groups of 10-100 users as part of the European project EmployID working as counsellors in public employment services. The platform is intended to support the exchange of experiences among practitioners and offering an environment for reflection. While the evaluation will 1 The mechanism for adding the prompts to existing platforms will be described in the ECTEL 2015 poster session, see [ 46 ], begin late in 2015 we already compiled a list of criteria we plan to use to evaluate whether and to what extend the concept is facilitating collaborative reflection:  Do the prompts change the behavior of the participants? If prompts are successful we should be able to see more activity related to reflection (e.g., more experiences shared or questions asked).  Does more reflection occur in the discussion threads? If the prompts facilitate reflection we should be able to see qualitative and quantitative changes in the reflection outcomes (e.g., more outcomes documented and more satisfaction with the outcomes among participants).  Which prompts help facilitating reflection the most? While our concept is based on thorough literature analysis and previous work it is likely that some prompts are more or less helpful than others. Comparing the outcomes of the two questions mentioned above with the prompt being used may therefore create valuable insights on how to prompt for reflection.  Are there any long-term learning effects following the display of prompts? Prompts should not only guide activity but also help people to become more reflective. If this is successful we should be able to see the changes measures in the first two questions even if we reduce the amount of prompts provided to users. This is also related to using prompts as a scaffold which is reduced bit by bit over time when the learner doesn’t need the scaffold anymore.

In order to answer the questions an evaluation setting with a within-groups design is planned, which includes having two groups in a counter-balanced design. This way one group gets prompts for a certain time and no prompts after that, and the other group starts without getting prompts and then switches to getting prompts. This design has the advantage that the size of the group of participants can be smaller since everybody gets prompts at one point of time. A disadvantage might be that having prompts at first influences the behavior in a long-term way, affecting the time period when said group does not receive prompts anymore. However, this disadvantage could be also used to answer the last question regarding long-term effects of prompts shown to users.

To answer the questions we plan to evaluate the concept on two different levels. The first level is whether the prompts are influencing the content of the discussion, that is, whether the users make use of the suggestions, for example whether displaying a prompt asking to evaluate colleagues did leads to more users mentioned in the posts. The second level involves checking whether those interventions also lead to more reflection outcomes in terms of whether the post indicated that the user either learned something or indicated that the user intends to do something differently in future.

Prompts and reactions to it could be evaluated through a content analysis as demonstrated in [ 20 ]. This may give insights whether the content of written discussions may be influenced by the prompt or not. Prilla et al. recently published a coding scheme for analyzing reflection content [ 20 ], which allows for checking whether the content shows indication that the users learned something or intends to do something differently in future. Content analysis is also an evaluation strategy previously employed to analyze prompts [ 24, 47 ]. Methods like pre- and post-tests on the content like in education settings [ 25 ] may not be feasible in workplace settings. Answering 53 the question whether a user actually saw and respectively read the prompt could be either answered statistically [ 25 ] or in pre-tests technology-supported e.g. with eyetracking. However eye-tracking doesn’t seem feasible in a larger workplace setting. Further work has to be done to make sure that this case is considered in the evaluation.

Prompts trying to evoke a more elaborated answer could also be evaluated on a quantitative level. Measuring the amount of words in the corresponding discussion contributions could give a basic insight, whether people adhered to the prompt. Additionally content coding can assess whether contributions contain more descriptive content about problems or approaches the author already tried to solve the problem.

Prompts aiming to guide users can be also evaluated through content analysis by checking whether the text contains elements the prompt asked for. In order to evaluate whether prompts targeting the involvement of colleagues are successful, a social network analysis may be useful to track changes in the personal network of people reflecting together in online threaded discussions. Prompts suggesting that content could be posted anonymously can also be evaluated without content analysis but only with the information how often the prompt was displayed and how often the feature was used respectively how often the feature was used after the user has seen the prompt once. 5

Conclusion and Outlook

We presented a concept based on using prompts to facilitate tool supported collaborative reflection and to support certain reflective behavior. We based the approach on literature analysis and our own previous studies, and we chose this approach because we think it is most suitable to cover a broad bandwidth of support for reflection.

The main contribution of this paper is the presented concept of goals supported by prompting to facilitate collaborative reflection – to the knowledge of the authors there is no other concept linking prompts to reflection support in such an extent. Additionally the paper already shows a first implementation of a prototype outlining how the concept can be used in a setting with online thread discussion forums. Our work is still in progress, and we think it provides fertile grounds for discussion in the ARTEL workshop.

In future work we plan to evaluate the concept with the presented prototype in a workplace setting. We already presented a rough evaluation concept in the paper which shows how we plan to evaluate whether and how different prompts are feasible for collaborative reflection support in a workplace setting.

Acknowledgements

This work is part of the EmployID (http://employid.eu) project on “Scalable & costeffective facilitation of professional identity transformation in public employment services” supported by the EC in FP 7 (project no. 619619). We thank all colleagues and associated partners for their cooperation and our fruitful discussions. 54

References

1. Schön , D.A. : The Reflective Practitioner: How Professionals Think in Action . Basic Books ( 1983 ).

2. Boud , D. , Keogh , R. , Walker , D. : Reflection: Turning Experience Into Learning . Routledge ( 1985 ).

3. Prilla , M. : Collaborative Reflection Support at Work: A Socio-Technical Design Task . ECIS 2014 Proceedings . ( 2014 ).

4. Kolb , D.A. : Experiential learning: Experience as the source of learning and development . Prentice-Hall Englewood Cliffs, New Jersey ( 1984 ).

5. Daudelin , M.W. : Learning from experience through reflection . Organizational Dynamics . 24 , 36 - 48 ( 1996 ).

6. Baumer , E.P. , Khovanskaya , V. , Matthews , M. , Reynolds , L. , Sosik , V.S. , Gay , G.K. : Reviewing Reflection: On the Use of Reflection in Interactive System Design . In: Proceedings of DIS 2014 ( 2014 ).

7. Marcu , G. , Dey , A.K. , Kiesler , S. : Designing for Collaborative Reflection . In: Proceedings of PervasiveHealth'14 ( 2014 ).

8. Hoyrup , S. : Reflection as a core process in organisational learning . Journal of Workplace Learning . 16 , 442 - 454 ( 2004 ).

9. Krogstie , B.R. , Prilla , M. , Pammer , V. : Understanding and Supporting Reflective Learning Processes in the Workplace: The CSRL Model . In: Hernández-Leo , D. , Ley , T. , Klamma , R. , and Harrer , A . (eds.) Scaling up Learning for Sustained Impact . pp. 151 - 164 . Springer Berlin Heidelberg ( 2013 ).

10. Prilla , M. , Pammer , V. , Krogstie , B. : Fostering Collaborative Redesign of Work Practice: Challenges for Tools Supporting Reflection at Work . In: Proceedings of the European Conference on Computer Supported Cooperative Work (ECSCW 2013 ) ( 2013 ).

11. Zhu , E.: Meaning Negotiation, Knowledge Construction, and Mentoring in a Distance Learning Course . ( 1996 ).

12. Van Woerkom , M. , Croon , M. : Operationalising critically reflective work behaviour . Personnel Review . 37 , 317 - 331 ( 2008 ).

13. Vince , R.: Organizing reflection . Management Learning . 33 , 63 - 78 ( 2002 ).

14. Scott , S.G.: Enhancing Reflection Skills Through Learning Portfolios: An Empirical Test . Journal of Management Education . 34 , 430 - 457 ( 2010 ).

15. Fleck , R. , Fitzpatrick , G.: Reflecting on Reflection: Framing a Design Landscape . In: Proceedings of the 22Nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction . pp. 216 - 223 . ACM, New York, NY, USA ( 2010 ).

16. Pammer , V. , Knipfer , K. , Krogstie , B. , Wessel , D. , Prilla , M. , Lindstaedt , S. : Reflective Learning at Work - A Position and Discussion Paper . In: Proceedings of 1st European Workshop on Awareness and Reflection in Learning Networks (ARNets11) at EC-℡ 2011 ( 2011 ).

17. Isaacs , E. , Konrad , A. , Walendowski , A. , Lennig , T. , Hollis , V. , Whittaker , S. : Echoes from the past: how technology mediated reflection improves well-being . In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . pp. 1071 - 1080 . ACM, Paris, France ( 2013 ).

18. Prilla , M.: User and Group Behavior in Computer Support for Collaborative Reflection in Practice: An Explorative Data Analysis . In: Rossitto, C. , Ciolfi , L. , Martin , D. , and Conein , B. (eds.) COOP 2014 - Proceedings of the 11th International Conference on the Design of Cooperative Systems . Springer ( 2014 ).

19. Lee , S. : Design and analysis of reflection-supporting tools in computer-supported collaborative learning . International Journal of Instructional Technology and Distance Learning . 2 , 49 - 56 ( 2005 ).

20. Prilla , M. , Nolte , A. , Blunk , O. , Liedtke , D. , Renner , B. : Analyzing Collaborative Reflection Support: A Content Analysis Approach . In: Proceedings of the European Conference on Computer Supported Cooperative Work (ECSCW 2015 ) ( 2015 ).

21. Prilla , M. : Supporting Collaborative Reflection at Work: A socio-technical Analysis. AIS Transactions of Human-Computer Interaction . ( 2015 ).

22. Kobbe , L. , Weinberger , A. , Dillenbourg , P. , Harrer , A. , Hämäläinen , R. , Häkkinen , P. , Fischer , F. : Specifying computer-supported collaboration scripts . International Journal of Computer-Supported Collaborative Learning . 2 , 211 - 224 ( 2007 ).

23. Dillenbourg , P. , Järvelä , S. , Fischer , F. : The evolution of research on computersupported collaborative learning . In: Technology-enhanced learning . pp. 3 - 19 . Springer ( 2009 ).

24. Davis , E.A. , Linn , M.C. : Scaffolding students' knowledge integration: prompts for reflection in KIE . International Journal of Science Education . 22 , 819 - 837 ( 2000 ).

25. Thillmann , H. , Künsting , J. , Wirth , J. , Leutner , D. : Is it Merely a Question of “What” to Prompt or Also “When” to Prompt?: The Role of Point of Presentation Time of Prompts in Self-Regulated Learning . Zeitschrift für Pädagogische Psychologie . 23 , 105 - 115 ( 2009 ).

26. Palanque , P. , Winckler , M. , Martinie , C. : A Formal Model-Based Approach for Designing Interruptions-Tolerant Advanced User Interfaces . In: Hussmann, H. , Meixner , G. , and Zuehlke , D. (eds.) Model-Driven Development of Advanced User Interfaces . pp. 143 - 169 . Springer Berlin Heidelberg ( 2011 ).

27. Stichter , J.P. , Lewis , T.J. , Whittaker , T.A. , Richter , M. , Johnson , N.W. , Trussell , R.P. : Assessing Teacher Use of Opportunities to Respond and Effective Classroom Management Strategies Comparisons Among High- and Low-Risk Elementary Schools . Journal of Positive Behavior Interventions . 11 , 68 - 81 ( 2009 ).

28. McKenzie-Mohr , D. : Fostering sustainable behavior through community-based social marketing . American Psychologist . 55 , 531 - 537 ( 2000 ).

29. Fivush , R. , Gray , J.T. , Fromhoff , F.A. : Two-year-old talk about the past . Cognitive Development . 2 , 393 - 409 ( 1987 ).

30. Peesapati , S.T. , Schwanda , V. , Schultz , J. , Lepage , M. , Jeong , S. , Cosley , D. : Pensieve: Supporting Everyday Reminiscence . In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . pp. 2027 - 2036 . ACM, New York, NY, USA ( 2010 ).

31. Greene , B.A. , Land , S.M.: A Qualitative Analysis of Scaffolding Use in a Resource-Based Learning Environment Involving the World Wide Web . Journal of Educational Computing Research . 23 , 151 - 179 ( 2000 ).

32. Baumer , E.P. : Reflective Informatics: Conceptual Dimensions for Designing Technologies of Reflection . In: Proceedings of CHI 2015 ( 2015 ).

33. Manouselis , N. , Drachsler , H. , Verbert , K. , Duval , E.: Recommender Systems for Learning . Springer Science & Business Media ( 2012 ).

34. Hatton , N. , Smith , D. : Facilitating Reflection: Issues and Research. ( 1994 ).

35. Wise , A.F. , Zhao , Y. , Hausknecht , S.N. : Learning Analytics for Online Discussions: A Pedagogical Model for Intervention with Embedded and Extracted Analytics . In: Proceedings of the Third International Conference on Learning Analytics and Knowledge . pp. 48 - 56 . ACM, New York, NY, USA ( 2013 ).

36. Argyris , C. , Schön , D.A. : Organizational learning II: Theory, method, and practice . Addison-Wesley ( 1996 ).

37. Ruth-Sahd , L.A. : Reflective practice: a critical analysis of data-based studies and implications for nursing education . J Nurs Educ . 42 , 488 - 497 ( 2003 ).

38. Platzer , H. , Blake , D. , Ashford , D. : Barriers to learning from reflection: a study of the use of groupwork with post-registration nurses . Journal of Advanced Nursing . 31 , 1001 - 1008 ( 2000 ).

39. Tan , N.Y.L. , Chen , W. , Looi , C. - K.: GroupScribbles as a rapid CSCL tool: Learning experiences of pre-service teachers . Presented at the ( 2009 ).

40. Kottkamp , R.B.: Means for Facilitating Reflection . Education and Urban Society . 22 , 182 - 203 ( 1990 ).

41. Santanen , E.L. , Briggs , R.O., de Vreede , G.: The cognitive network model of creativity: a new causal model of creativity and a new brainstorming technique . In: Proceedings of the 33rd Annual Hawaii International Conference on System Sciences , 2000 . p. 10 pp. vol. 1 - ( 2000 ).

42. MacCrimmon , K.R. , Wagner , C. : Stimulating Ideas through Creative Software . Management Science . 40 , 1514 - 1532 ( 1994 ).

43. McAlpine , L. , Weston , C. , Beauchamp , C. , Wiseman , C. , Beauchamp , J.: Building a metacognitive model of reflection . Higher Education . 37 , 105 - 131 ( 1999 ).

44. Mamede , S. , Schmidt , H.G. : Correlates of Reflective Practice in Medicine . Adv Health Sci Educ Theory Pract . 10 , 327 - 337 ( 2005 ).

45. Ragins , B.R. , Scandura , T.A. : Burden or blessing? Expected costs and benefits of being a mentor . Journal of Organizational Behavior . 20 , 493 - 509 ( 1999 ).

46. Prilla , M. , Blunk , O. : Reflective ℡: Augmenting Learning Tools with Reflection Support . In: Proceedings of the Tenth European Conference on Technology Enhanced Learning . Springer ( 2015 ).

47. Rosenshine , B. , Meister , C. , Chapman , S. : Teaching Students to Generate Questions: A Review of the Intervention Studies . REVIEW OF EDUCATIONAL RESEARCH . 66 , 181 - 221 ( 1996 ).