=Paper=
{{Paper
|id=Vol-1736/paper2
|storemode=property
|title=Reflection Analytics in Online Communities: Guiding Users to become active in Collaborative Reflection
|pdfUrl=https://ceur-ws.org/Vol-1736/paper2.pdf
|volume=Vol-1736
|authors=Oliver Blunk,Michael Prilla,Graham Attwell
|dblpUrl=https://dblp.org/rec/conf/ectel/BlunkPA16
}}
==Reflection Analytics in Online Communities: Guiding Users to become active in Collaborative Reflection==
https://ceur-ws.org/Vol-1736/paper2.pdf
Reflection Analytics in Online Communities: Guiding
Users to become active in Collaborative Reflection
Oliver Blunk1, Michael Prilla1, Graham Attwell²
1
Clausthal University of Technology, Germany
{oliver.blunk, michael.prilla}@tu-clausthal.de
²Pontydysgu, Wales
{graham10@mac.com}
Abstract: As reflection helps practitioners to turn experiences into
learning, communities of practices provide an environment to sup-
port reflection. We present a concept showing how reflection ana-
lytics in online communities of practice can help users to improve
their reflection activity, guiding them to become active reflective
participants. A prototype shows how our concept will be evaluated.
Keywords: reflection, reflection analytics, learning analytics,
community of practice
1 Introduction
Reflection is a common activity at workplaces [1]. Our understanding of reflection is
based on Boud, who describes it as a process with three steps: returning to past expe-
riences, reassessing them in order to learn something for future actions [2]. While
most research focuses on individual reflection or reflection in educational settings, we
focus on collaborative reflection by a group of professionals at work, showing how
reflection helps these groups to learn more than they could individually [3].
In earlier work, we have found that small groups of reflective participants (see 2.1)
might suffer from a lack of time or the willingness of other group members to actively
and frequently engage in reflection, and therefore, in line with [4], we propose to
support collaborative reflection in communities of practice [5]. A community of prac-
tice is comprised of members doing similar work, e.g. working in a certain job role,
and who have similar practices [6]. Although communities of practice can be informal
and loosely organized, a community of practice is often supported by Information and
Communication Technologies (ICT) such as online portals with discussion boards
enabling members to exchange practice [6].
From an organizational perspective, enabling workers to reflect together through a
community of practice has multiple benefits [6]: newer employees can benefit from
the expertise of experienced workers, practitioners can discuss and share tacit
knowledge, and spatially distributed organizations can connect employees working in
different geographic locations. We found that integrating reflection support into
27
Reflection Analytics in Online Communities - ARTEL16
community tools provides benefits compared to offering standalone reflection tools,
as the former integrates reflection into existing communication practices [7].
In the ‘work in progress’ approach presented in this paper, we aim at developing
initial “reflection analytics” to guide reflection by participants in communities. We
lean on the field of learning analytics, to capture and present the activity of learners to
support reflection on their personal learning [8]. This approach has been proven effec-
tive for informal learning, and we believe such an analytics driven approach will also
be effective in supporting reflective learners in communities.
This paper combines the concepts of collaborative reflection, communities of prac-
tice and provision of guidance to users in becoming reflective learners. In this paper,
we describe our concepts, their corresponding background and an initial prototype.
2 Related work
2.1 Group dynamics in collaborative reflection
Models of reflection have been developed by Schön [1] and Boud [2] focusing on the
individual. In practice people often discuss their experiences together and thus reflect
together [3]. To engage in this collaborative reflection, participants need to communi-
cate and discuss their experiences, which is at the core of reflection [7]. This is im-
portant for individual workers as well as for organizations [8].
In previous work we have analysed tools supporting groups reflection. We found
that users assume roles based on the core activities of documenting, commenting and
reading about different experiences, and that collaborative reflection depends on the
distribution of these roles in groups. We found four basic roles [9]:
Documenters: Users focussing on documenting experiences.
Commenters: Users who comment mainly on other’s documented experiences.
Readers: Users reading many shared experiences and associated comments, but
rarely becoming active by writing experiences or commenting on them.
Typical (full) reflection participants: Ideally, users participate equally in all
three activities (see above), thus actively supporting the reflection in the group.
In our analysis we found that active reflection groups either contained a core of typi-
cal reflection participants or a sample of enough documenters and commenters to
provide activity in the reflection groups. We concluded that activating readers to start
documenting and commenting as well as motivating commenters to document and
vice versa is likely to increase reflective learning in the respective groups [9].
2.2 Group dynamics in communities of practice
Communities of practice offer opportunities for informal learning through facilitating
discussions by members around practice, exchanging practices and experiences [6].
By being active in such exchange, learners can reflect upon how to integrate shared
practices and experiences into their own daily practice. This is similar to support for
28
Reflection Analytics in Online Communities - ARTEL16
collaborative reflection, and the roles undertaken in communities of practice show
further similarities.
In their classic model of how users interact in communities of practice, Lave and
Wenger differentiate between a periphery comprised of new members or members
with low levels of activity and the core of the community with a low number of high-
ly active members [10]. Karalis [11] adds additional levels, ranging from passive
observers to transactional and peripheral participants as well as those at the core. A
common role often found in the periphery or passive zone of communities is that of a
“Lurker” [12], similar to the readers we described above. In their concept of legiti-
mate peripheral participation, Lave and Wenger emphasize the positive aspect of
lurking (reading) as a way of getting to know the community before becoming active,
and of learning from others’ experiences [13].
Welser et al. [14] and Jones et al. [12] included in their typology “answer people”,
who mainly answer other users posts instead of writing their own, in a similar way to
our description of commenters. Answer people are not connected to many members in
the community, and interact on the periphery of a community. They can be seen as
peripheral participants in the Karalis model. Furthermore, an analysis of the medical
support community WebMD, by Introne, Semaan and Goggins [15], suggests that
active core members spend a lot of time talking to new users. This suggests that active
core members may play our commenter role. Users who are only active occasionally
seemed to play the role of documenters (posting new content in the community).
However, these findings may be specific to the particular type of community investi-
gated, as users of WebMD seek advice around diseases rather than sharing practices.
Research is also concerned as to how people transition from the periphery of the
community towards the core. An interesting model can be found in the Reader-to-
Leader model [16], which states that by contribution (e.g., enough interesting and
valuable content) and with motivation (e.g. recognition by others) users may increase
their activity from being a reader to being a leader supporting others in communities.
2.3 Learning Analytics
Learning Analytics focuses on helping learners to understand their learning pro-
gress and optimising their learning, by a data driven analysis of action undertaken in
learning environments [8]. However, most learning analytics research and practice
has been undertaken in formal school and university contexts. Critically, much work-
place learning is informal with little agreement of proxies for learning. While learning
analytics in educational settings very often follow a particular pedagogical design,
workplace learning is much more driven by demands of work tasks or intrinsic inter-
ests of the learner, by self-directed exploration and social exchange that is tightly
connected to processes and the places of work [17]. Learning interactions at the
workplace are to a large extent informal and practice based and not embedded into a
specific and measurable pedagogical scenario.
Pardo and Siemens [18] point out that “LA is a moral practice and needs to focus
on understanding instead of measuring.” In this understanding “learners are central
agents and collaborators, learner identity and performance are dynamic variables,
29
Reflection Analytics in Online Communities - ARTEL16
learning success and performance is complex and multidimensional, data collection
and processing needs to be done with total transparency.” This poses issues within the
workplace with complex social and work structures, hierarchies and power relations.
Buckingham Shum & Ferguson [8] have added a focus towards the social aspects
of learning including how learners interact with each other. The focus on the social
aspect of learning analytics is more congruent with the informal and social nature of
learning in communities of practice. Data is presented in a way to allow learners to
take action upon it (actionable data). Showing learners analysis of their own behav-
iour can help stimulate reflection [8]. De Laat & Schreurs [19] demonstrate how soci-
al network analysis (SNA) and content analysis can contribute to learning analytics in
community settings.
3 A concept to support reflection analytics
Our concept aims to balance the structure and roles in a community with respect to
becoming an active reflective participant. The goal is to help users to transition from a
reading role at the periphery to a more active role near the core of a community. To
achieve this, we will deliver personal and group reflection (learning) analytics com-
bined with personalized facilitation depending on the analytics, making users aware
of their current reflection activities.
For this kind of scaffolding, we have to know which role a user is playing while re-
flecting in a ICT supported community of practice. For this we build on the metrics
we used in our previous work on roles and groups in collaborative reflection (e.g.
number of comments per time span, [9]) as well as through social network analysis
([19] and [15], who published an algorithm for SNA), which may help us to analyse
interactions in collaborative reflection, and [12], who describe various metrics for
online discussion forums to measure the activity of users. This work enables us to
analyse the activity of users in real time and to compare it to their peers. Using this
analysis, we can support each user type differently:
Guiding typical reflection participants: Participants can be shown new or less
popular threads to help users by providing their experiences as described in [15].
Guiding documenters: Documenters are likely to have experiences that are help-
ful for others, and therefore should be encouraged to comment on other users’
posts to enable reciprocity in the community. When receiving help by others,
they could get encouraged to help others in turn.
Guiding commenters: Users who often help others by commenting on posts can
be encouraged to also create an occasional post themselves to provide experienc-
es others can relate to in order to foster activity as described by [16].
Guiding readers: Users who are reading a lot can be encouraged to start interact-
ing with the community by for example asking questions to others about issues in
their work life (see [20], who describe this as easier than answering; at least in
Question and Answer forums). Readers also need to be made aware of the value
their comments and posts may have for others. It is important new users are sup-
ported in order to ease them into using the platform and discussion area.
30
Reflection Analytics in Online Communities - ARTEL16
4 First Prototype
Our concept of support for these roles includes two steps. Firstly, we provide reflec-
tion analytics to make users aware of the role they currently play and secondly, we
provide actionable prompts in the form of texts or images (related to activity prompts
as mentioned in [21]) to users, proposing steps they can take to develop their role in
the community like helping others or sharing own issues. Prompts have shown to be
helpful in learning contexts [21, 22] to stimulate recipients to think about their ac-
tions, and we have developed a concept for prompts for collaborative reflection [5].
Our concept is currently work in progress and we have developed a prototype to
evaluate it in practice. Fig. 1 taken from the prototype shows the three different
individual roles in reflection as posts (new threads the person started, measuring
documenter activity), comments (threads the user commented on, commenter
activity), and reads (threads which the user looked into, reader activity). Fig 1. shows
that the current user is reading more than average, writing an average number of
comments, but is not writing many new posts. The prompt displayed in Fig. 1
suggests sharing own experiences, since the analytics shows the user is more of an
answer-type person commenting on others threads.
Fig. 1 Reflection analytics prototype
While the prototype is in its early stages, we are planning to extend it to implement
and evaluate our concept. For example, we will develop the choice of prompts to
analyse not only absolute numbers, but also trends in use and to inform users. Analys-
ing the content created by a user may help us to identify whether the user is really
taking part in collaborative reflection within a discussion (see our other work [23]),
which might improve the choice of prompts, and it may allow us to understand user’s
interests. With the latter information, we may utilise recommendation engines to im-
prove the choice of prompts, for example by recommending specific threads instead
of telling new users to simply read something in order to get used to the community.
Also it might be interesting to analyse whether user prefer to see their development
over time in the community or rather this snapshot-based visualisation.
As we are currently finalizing the work on the prototype, we will be able to show
and discuss these features at the ARTEL workshop. Subsequently we will evaluate the
prototype in a real work environment to understand whether and how it influences
user behaviour and whether and how this influences reflection in the community.
31
Reflection Analytics in Online Communities - ARTEL16
5 Conclusion
While our work is in still in progress with no evaluation having been conducted to
date, we are convinced that our idea of reflection analytics contributes to the overall
work being done in the context of (AR)TEL. It builds on a solid basis of our own and
other research and is likely to help users to understand and improve their reflection
activities in what will then be reflective communities of practice.
6 Acknowledgements
This work is part of the EmployID (http://employid.eu) project on “Scalable & cost-
effective facilitation of professional identity transformation in public employment
services” supported by the EC in FP 7 (project no. 619619). We thank all colleagues
and associated partners for their cooperation and our fruitful discussions.
References
1. Schön, D.A.: The Reflective Practitioner: How Professionals Think in Action.
Basic Books (1983).
2. Boud, D., Keogh, R., Walker, D.: Reflection: Turning Experience Into Learning.
Routledge (1985).
3. Hoyrup, S.: Reflection as a core process in organisational learning. Journal of
Workplace Learning. 16, 442–454 (2004).
4. Nyhan, B.: Collective reflection for excellence in work organizations. In: Cres-
sey, P., Boud, D., and Docherty, P. (eds.) Productive Reflection at Work: Learn-
ing for Changing Organizations. p. 133. Routledge (2005).
5. Blunk, O., Prilla, M.: Prompting users to facilitate support needs in collaborative
reflection. In: Kravcik, M., Mikroyannidis, A., Pammer, V., Prilla, M., and
Ullmann, T.D. (eds.) Proceedings of the 5th Workshop on Awareness and Re-
flection in Technology Enhanced Learning. In conjunction with the 10th Europe-
an Conference on Technology Enhanced Learning: Design for Teaching and
Learning in a Networked World (EC-TEL 2015). pp. 43–57. Toledo, Spain
(2015).
6. Wenger, E.: Communities of practice: Learning as a social system. Systems
thinker. 9, 2–3 (1998).
7. Prilla, M., Blunk, O.: Reflective ℡: Augmenting Learning Tools with Reflection
Support. In: Proceedings of the Tenth European Conference on Technology En-
hanced Learning. Springer (2015).
8. Buckingham Shum, S., Ferguson, R.: Social Learning Analytics. Educational
Technology & Society. 15, 3–26 (2012).
9. Prilla, M.: Supporting Collaborative Reflection at Work: A socio-technical Anal-
ysis. AIS Transactions of Human-Computer Interaction. (2015).
32
Reflection Analytics in Online Communities - ARTEL16
10. Lave, J., Wenger, E.: Situated learning: Legitimate periphereal participation.
Cambridge University Press, Cambridge (1991).
11. Karalis, T.: Situated and transformative learning: exploring the potential of criti-
cal reflection to enhance organizational knowledge. Development and Learning
in Organizations: An International Journal. 24, 17–20 (2010).
12. Jones, R., Sharkey, S., Smithson, J., Ford, T., Emmens, T., Hewis, E., Sheaves,
B., Owens, C.: Using Metrics to Describe the Participative Stances of Members
Within Discussion Forums. Journal of Medical Internet Research. 13, e3 (2011).
13. Preece, J., Nonnecke, B., Andrews, D.: The top five reasons for lurking: improv-
ing community experiences for everyone. Computers in Human Behavior. 20,
201–223 (2004).
14. Welser, H.T., Gleave, E., Fisher, D., Smith, M.: Visualizing the signatures of
social roles in online discussion groups. Journal of Social Structure. 8, 564–586
(2007).
15. Introne, J., Semaan, B., Goggins, S.: A Sociotechnical Mechanism for Online
Support Provision. In: Proceedings of the 2016 CHI Conference on Human Fac-
tors in Computing Systems. pp. 3559–3571. ACM, New York, NY, USA (2016).
16. Preece, J., Shneiderman, B.: The Reader-to-Leader Framework: Motivating
Technology-Mediated Social Participation. AIS Transactions on Human-
Computer Interaction. 1, 13–32 (2009).
17. Ley, T., Klamma, R., Lindstaedt, S., Wild, F.: Learning Analytics for Workplace
and Professional Learning. In: Proceedings of the Sixth International Conference
on Learning Analytics & Knowledge. pp. 484–485. ACM, New York, NY, USA
(2016).
18. Pardo, A., Siemens, G.: Ethical and privacy principles for learning analytics. Br J
Educ Technol. 45, 438–450 (2014).
19. Laat, M. de, Schreurs, B.: Visualizing Informal Professional Development Net-
works: Building a Case for Learning Analytics in the Workplace. American Be-
havioral Scientist. 2764213479364 (2013).
20. Yang, J., Wei, X., Ackerman, M.S., Adamic, L.A.: Activity Lifespan: An Analy-
sis of User Survival Patterns in Online Knowledge Sharing Communities. Pre-
sented at the 4th International AAAI Conference on Weblogs and Social Media,
ICWSM 2010 (2010).
21. Davis, E.A.: Prompting Middle School Science Students for Productive Reflec-
tion: Generic and Directed Prompts. Journal of the Learning Sciences. 12, 91–
142 (2003).
22. Thillmann, H., Künsting, J., Wirth, J., Leutner, D.: Is it Merely a Question of
“What” to Prompt or Also “When” to Prompt?: The Role of Point of Presentation
Time of Prompts in Self-Regulated Learning. Zeitschrift für Pädagogische Psy-
chologie. 23, 105–115 (2009).
23. Prilla, M., Nolte, A., Blunk, O., Liedtke, D., Renner, B.: Analyzing Collaborative
Reflection Support: A Content Analysis Approach. In: Proceedings of the Euro-
pean Conference on Computer Supported Cooperative Work (ECSCW 2015)
(2015).
33