=Paper=
{{Paper
|id=Vol-1776/paper7
|storemode=property
|title=Challenges and Resolutions for Engaging Teachers and Students in Participatory Design of Online Science Learning Resources
|pdfUrl=https://ceur-ws.org/Vol-1776/paper7.pdf
|volume=Vol-1776
|authors=Matthias Heintz,Effie Lai-Chong Law
|dblpUrl=https://dblp.org/rec/conf/nordichi/HeintzL16
}}
==Challenges and Resolutions for Engaging Teachers and Students in Participatory Design of Online Science Learning Resources==
https://ceur-ws.org/Vol-1776/paper7.pdf
Challenges and Resolutions for Engaging Teachers and
Students in Participatory Design of Online Science
Learning Resources
Matthias Heintz, Effie Lai-Chong Law
Department of Informatics, University of Leicester, Leicester, United Kingdom
{mmh21,lcl9}@leicester.ac.uk
Abstract. Participatory Design (PD) is a key HCI approach relying on the
involvement of end-users to provide insights into the development of a
product/system. However, it is proved challenging to engage participants in PD
activities, as shown by our practical experiences in a research project that is
aimed to support teaching and learning of science with online labs and associated
resources. Challenges include identifying convincing arguments about benefits
of participation and demonstrating impacts their participation can have. We
propose the use of an online tool such as PDot to support the traceability of PD
feedback and the possibility of audio input. We will also explore the gamification
of participation.
Keywords: Participatory design; Online labs; Teachers; Students;
1 Introduction
“We have everything ready, but we don’t have participants!” Such a predicament is
faced by a number of researchers planning to collect empirical data with human
participants. While “everything” can include methodological approaches, ethical
approval, research protocols, experimental instruments and other logistics
arrangements, “participants” can be people from all walks of life, depending on the
main goal and research questions of the project concerned. For a research project
targeting school students aged 10-18 years old and their teachers with the respective
population sizes being about 4 million and 0.5 million in England 1, one would expect
that the predicament of participant recruitment might be less likely to occur.
Nevertheless, our practical experiences of conducting Participatory Design (PD)
studies with these target groups contradict such an expectation. In fact, it has been rather
challenging for us to get teachers committed to participating in PD activities. The level
of involvement of teachers, who are gatekeepers of access to students, has had a
significant impact on student participation.
1 https://www.gov.uk/government/publications/number-of-schools-teachers-and-students-in-
england
42
Proc. of Fourth International Workshop on Cultures of Participation in the Digital Age - CoPDA 2016
Gothenburg (Sweden), October 23, 2016 (published at http://ceur-ws.org).
Copyright © 2016 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
In this position paper, we describe the challenges we have faced and resolutions we
have attempted to improve the quantity as well as quality of participation in our PD
activities. We also discuss the implication for ameliorating approaches to engage people
in PD.
2 Background
The context of Participatory Design (PD) activities that we have conducted is a
European research project called Go-Lab (Global Online Science Labs for Inquiry
Learning at School)2. Its overarching goal is to support science teaching and learning
by providing pedagogically as well as technologically sound online laboratories (or
online labs) and scaffolds (or apps), which allow their users to perform interactive
experiments over the Internet [1]. Science teachers and school children from upper
primary school up to pre-university colleges are the main beneficiaries of the project’s
outputs. To facilitate the attainment of the project’s goal, a web-based portal called
Golabz2 has been developed to enable teachers to identify and utilize online labs, apps,
online lessons and other resources for addressing learning needs of their students, who
can also access and explore such resources on their own.
As for all computing technologies, it is crucial to ensure high usability, usefulness
and attractiveness of the online labs, apps and other related resources to elicit positive
experiences in teachers and students, thereby sustaining their motivation and interest in
deploying them. Quality assurance can be achieved through the use of appropriate
human-computer interaction (HCI) methodological approaches, including Participatory
Design (PD) in the early phases of the project and User Experience (UX) evaluation in
the later phases. This entails close collaborations between the HCI, software
development, and pedagogy researchers engaged in the project (Fig. 1).
As depicted in Figure 1, the critical source of input is feedback from end-
users/participants (i.e. teachers and students), who in the context of Go-Lab serve as
informants, according to the IBF (Informed, Balanced, Facilitated Design)
Participatory Continuum Model [2,3]. For “Informant Design”, designers create a
prototype and participants are then asked to provide insights into its quality
improvement.
2 Go-Lab: http://www.go-lab-project.eu/; Golabz: http://www.golabz.eu/
43
Proc. of Fourth International Workshop on Cultures of Participation in the Digital Age - CoPDA 2016
Gothenburg (Sweden), October 23, 2016 (published at http://ceur-ws.org).
Copyright © 2016 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
Fig. 1. Interdependencies of the project’s teams.
3 Challenges
In the course of Go-Lab, based on our ongoing reflections on the perceived attitudes
and observed behaviours of the participants in our PD activities, we have adapted
different strategies and approaches to improve the number of participants and the nature
of their participation. In the subsequent descriptions, we summarize the challenges
encountered into two major categories, namely organizational and personal.
3.1 Organizational Challenges: Recruitment and Implementation
For our workflow of PD, participant recruitment was launched when prototypes,
research protocols, data capturing tools and other setups were near completion.
Recruitment for and implementation of our PD studies, despite the meticulous planning
in advance, have been tricky because of several constraints:
Alignment with the development team’s release plan: The release plan was
constrained by the contracted delivery dates of the project’s products. While this
restricted timeframe could help schedule participant recruitment, the maturity and
stability of the prototypes under development, due to various reasons, might not yet
have reached the expected status for the planned PD activities when they had to be
conducted. This might run into the risk of undermining the participants’ perception
of the project’s final products and influence their ability to give adequate feedback,
especially with participants who are not used to provide feedback on prototypes. To
address these issues we emphatically reminded the participants during the PD
workshops that the prototypes presented were still under development.
Recruitment strategies: A variety of recruitment strategies was used to acquire
participants, including sending invitation letters and flyers about the PD studies to
school head-teachers and to researchers’ personal networks (via email), and
publicizing the studies in other project-related activities. The design of the
promotional materials played a critical role in convincing potential participants. To
be most effective the text should be free from jargon and the images should illustrate
the goal and activities of PD well. Of particular importance is to argue for the
44
Proc. of Fourth International Workshop on Cultures of Participation in the Digital Age - CoPDA 2016
Gothenburg (Sweden), October 23, 2016 (published at http://ceur-ws.org).
Copyright © 2016 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
benefits of participation. This is proved challenging because putting emphasis on the
value of participants’ input for improving the project’s products is apparently not
convincing enough for many teachers, given their indirect relationship with the
project and the timescale at which they could expect to actually benefit from the
results.
School timetabling in connection with teacher/student availability: While a number
of our teacher-oriented PD studies have been organized as teacher training
events/workshops in schools as well as non-school settings, the student-oriented PD
studies have taken place only in schools. The reasoning for the latter is built upon
the assumption that field studies can provide more realistic insights into how the
online labs and other resources would be used in situ, thereby increasing the
ecological validity of the findings. However, the drawback is the reliance on school
timetables and technical infrastructure, which often do not match well with the
project timetable and can be difficult to deal with as technical issues cannot be ruled
out as part of the workshop preparation, but need to be dealt with on the spot during
the running workshop.
3.2 Personal Challenges: Needs and Attitudes
Stakeholders’ diverse needs: Different stakeholders involved have different
concerns about the participation in the PD studies. For teachers, their concerns may
be whether the participation enables them to meet their professional development
need or to address certain educational issues. For students, their concerns may be
whether the participation is enjoyable and allows them to gain new learning
experience. For the HCI researchers the main concern might be, whether their
participation in the form of guiding the PD activities influences the quantity and
quality of data. For other project partners, their concern can be whether their
participation in terms of providing the online resources to be evaluated helps affirm
the value of their research endeavour and enhances the reputation of the project.
Nonetheless, some of these concerns may be incompatible. For instance, the HCI
researchers aim to collect teacher and student feedback on a broad range of the online
lab and app prototypes as early as possible in order to get a comprehensive
understanding of the products quality issues. But this may deter teachers from
participating because of the amount of time required.
Communication: In our PD activities, the teachers and students have typically been
asked to provide their feedback by annotating printouts of the online labs and apps
of interest or by inputting their comments (textual or graphical) into a dedicated
online tool called PDot (Participatory Design Online Tool) [4]. During our
workshops we could observe a wide spectrum of the participants’ attitudes and
behaviours. Some of them have been on the ‘cold’ end, necessitating more
encouragement and “pushes” from a researcher to share their comments on the
online labs, apps and online lessons whereas others, once they had become
comfortable with the situation, have engaged enthusiastically in the tasks given.
Interestingly, some of the teacher and student participants tend to communicate their
feedback directly to the researchers present in the PD study opposed to the tools
45
Proc. of Fourth International Workshop on Cultures of Participation in the Digital Age - CoPDA 2016
Gothenburg (Sweden), October 23, 2016 (published at http://ceur-ws.org).
Copyright © 2016 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
provided to give feedback. One plausible explanation for this tendency is that these
participants may want to ensure that their voices have been heard (this basically
aligns with the philosophy of PD) and to seek direct recognition of their
contributions. Another explanation is that some participants simply prefer the
modality of audio input.
4 Implications for Resolutions
Based on the practical experiences gained from designing and conducting the PD
studies in the Go-Lab project, we have come up with some possible resolutions to
address the aforementioned challenges of engaging participants:
Convincing teacher participants with mutual benefits of participation: As
mentioned, a common but rather ineffective approach is to use the argument “your
input is valuable”. Instead of emphasizing on one-way contribution, a “give-and-
take” strategy may work better. In the context of Go-Lab, the teacher participants
are offered the possibility of creating their own online lessons to address specific
needs of their students with the use of the online labs and apps available in the portal.
Our “bargaining token” can be asking the teachers to provide improvement
suggestions on the resources that they would need for the creation of their online
lessons. In return, the HCI researchers provide the support for such a creation
process. Apparently, this mutual benefit model only works if the teacher participants
are keen to create online lessons. Furthermore, integrating such “give-and-take” PD
studies as a part of teachers CPD (continuous professional development) programme
can also be a useful strategy to make them more attractive for teachers to participate.
Re-engaging participants through presenting changes: According to the notion of
problem ownership [5] in the sense of group problem-solving, people are motivated
to contribute to resolving a problem if they feel responsible for it and their effort in
improving it can really make a difference. Hence, it is relevant to demonstrate to the
participants of our PD studies that the feedback they have given on the previous
version of the online learning resources do have real impacts on their redesign. The
traceability of comments can be supported with the use of a software tool like PDot
[4], which allows the participants to retrieve feedback associated with the digital
artefact of interest (Fig. 2 showing PDot). In addition, PDot enables continuous
interaction between participants and developers/researchers by supporting
consecutive evaluations in the same environment. These features can facilitate the
creation of a sense of problem ownership in the participants, who would then be
more willing to continue participating in subsequent PD and other evaluation work.
Convincing student participants with a bigger picture of participation: As
mentioned earlier, students got involved in the PD studies through their teachers.
While the students could be exempted from taking part in the studies, none of them
have opted out. However, their opting-in did not necessarily imply that they were
engaged in providing feedback. In fact, their motivational level varied and
influenced the quality of the feedback. To enhance the students’ motivation, one can
highlight the role of the HCI researchers in creating and ensuring the quality of
46
Proc. of Fourth International Workshop on Cultures of Participation in the Digital Age - CoPDA 2016
Gothenburg (Sweden), October 23, 2016 (published at http://ceur-ws.org).
Copyright © 2016 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
computing technologies. The PD experience the students gained can give them
practical information to consider a possible future career in Informatics.
Augmenting the modality of participation: For traditional PD approaches, feedback
capture is primarily paper-based where textual description and sketching are the
main data types. In recent years, the use of software tools to support PD work has
increased [6], but the input modalities remain largely the same. Nevertheless, given
the observation that some of our PD participants prefer voicing (literally) their
opinions, we propose augmenting the modality to support audio input. With the
increasing sophistication of speech-to-text technologies, this possibility is viable.
Besides altering the modalities one could also aim to make the use of existing
modalities more “fun”. A viable approach to possibly achieve this, which has been
used successfully in other areas than PD, would be gamifying the PD activities [7].
Through the motivational power of game elements, participants could be encouraged
to give more feedback and do so repeatedly, once they have initially been convinced
to give it a try.
Fig. 2. Screenshot of PDot with three main areas: ❶ Functionality to specify feedback, ❷
Instructions for the participants and ❸ Prototype to evaluate with feedback.
5 Concluding Remarks
How can Participatory Design be conducted without participants? This sounds like a
conundrum. Clearly, PD needs participants. The main challenge lies in how HCI
researchers can engage participants in PD activities and make such activities enjoyable
as well as meaningful to them. This challenge is especially acute when different target
groups with different needs are involved. In the above descriptions, we have discussed
some strategies such as the use of an online tool to enable feedback tracking. Other
promising approaches to be explored to improve the experience of PD participation are
audio input and gamification. This will be a research agenda of our future work to make
participants want to take part in PD rather than have to.
47
Proc. of Fourth International Workshop on Cultures of Participation in the Digital Age - CoPDA 2016
Gothenburg (Sweden), October 23, 2016 (published at http://ceur-ws.org).
Copyright © 2016 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
Acknowledgements. This work was partially funded by the European Union in the
context of the Go-Lab project (Grant Agreement no. 317601) under the Information
and Communication Technologies (ICT) theme of the 7th Framework Programme for
R&D (FP7). This document does not represent the opinion of the European Union, and
the European Union is not responsible for any use that might be made of its content.
6 References
1. de Jong, T., Sotiriou, S., Gillet, D.: Innovations in STEM education: the Go-Lab federation
of online labs. Smart Learning Environments , vol.1, 1, pp. 1-16 (2014)
2. Read, J. C., Gregory, P., MacFarlane, S. J., McManus, B., Gray, P., Patel, R.: An
Investigation of Participatory Design with Children – Informant, Balanced and Facilitated
Design, Interaction Design and Children, pp. 53-64 (2002)
3. Scaife, M., Rogers, Y., Aldrich, F., Davies, M.: Designing for or designing with? Informant
design for interactive learning environments. In: Proceedings of the ACM SIGCHI
Conference on Human factors in computing systems, pp. 343-350. ACM, New York, NY,
USA (1997)
4. Heintz, M., Law, E. L.-C., Govaerts, S., Holzer, A.,Gillet, D.: Pdot: participatory design
online tool. In: CHI '14 Extended Abstracts on Human Factors in Computing Systems (CHI
EA '14), pp. 2581-2586. ACM, New York, NY, USA (2014)
5. Isaksen, S. G., Treffinger, D. J.: Celebrating 50 years of reflective practice: Versions of
creative problem solving. Journal of Creative Behavior, vol. 38, pp. 75-101. (2004)
6. Walsh, G., Druin, A., Guha, M. L., Bonsignore, E., Foss, E., Yip, J. C., Golub, E., Glegg,
T., Brown, Q., Brewer, R., Joshi, A., Brown, R.: DisCo: a co-design online tool for
asynchronous distributed child and adult design partners. In: Proceedings of the 11th
International Conference on Interaction Design and Children, pp. 11-19. ACM, New York,
NY, USA (2012)
7. Walz, S. P., Deterding, S.: The gameful world: Approaches, issues, applications. MIT Press.
(2015)
48
Proc. of Fourth International Workshop on Cultures of Participation in the Digital Age - CoPDA 2016
Gothenburg (Sweden), October 23, 2016 (published at http://ceur-ws.org).
Copyright © 2016 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.