From Participation to Learning: Sensemaking in and for Participatory Design to Facilitate End-User Development Marie Utterberg Modén marie.utterberg@ait.gu.se 0 Svea Kiesewetter svea.kiesewetter@ait.gu.se 0 Thomas Hillman thomas.hillman@ait.gu.se 0 University of Gothenburg, Department of Applied IT , Gothenburg , Sweden

This paper explores how learning can be understood in participatory design. Mutual learning is recognized as a foundation for design outcomes that resonate with principles of stakeholder engagement, but few studies specifically examine participants' learning experiences. To address this gap, we propose sensemaking as a lens to analyze learning opportunities in participatory design projects and as a framework for fostering mutual learning. We apply this lens to analyze the initial phase of an ongoing project focused on designing a teacher-oriented dashboard to enhance critical data literacy in education. Although sensemaking was not initially considered, we retrospectively employ it to analyze the project and suggest its potential use to inform how end-user development could be facilitated.

 eol>education dashboard data literacy participatory design end-user development1 1. Introduction

Mutual learning holds a pivotal position in participatory design. It is acknowledged as a fundamental element in the generation of inclusive and equitable design outcomes that resonate with the principles of stakeholders engaged in the advancement of digital technology [ 1 ] and to understand the need for end-users’ control of design [ 13 ]. However, as Eriksen, Hillgren, and Seravalli articulated in 2020, “many publications and accounts of prior PDC proceedings mention mutual learning but often without further empirical or theoretical elaborations of the concept” [2:1]. Explicit evaluations of learning in participatory design projects are rare with a lack of well-documented evaluations that analyze what participants learn during projects [ 3 ]. It can be argued then that the significant role mutual learning is often described as having in participatory design processes is not fully realized [ 4 ] and that a limited understanding of learning in relation to the practical accomplishment of participatory design limits the benefits for end-user development. Responding to the issue in this paper, we suggest sensemaking [ 5 ] as a lens to analyze opportunities for learning when individuals collaboratively negotiate meaning during their engagement in participatory design work. As a case, the participatory design project examined in this study aimed at designing a dashboard to promote critical data literacy in the field of education. The study is guided by the question, how could the process of designing and integrating an educational technology into teaching activities use sense-making to understand the needs among the individuals involved?

2. Learning in participatory design

Mutual learning is a fundamental principle of participatory design [ 1 ]. It holds that participants (such as designers/researchers and intended users) learn from each other's expertise. This principle can be seen as a part of the heritage from early Scandinavian projects that involved and empowered workers striving to realize democratic ideals in their workplaces as a response to the social and political challenges of the time. By using the shared knowledge gained as a foundation for end-user development, the aim is not just to foster collaboration between developers and users (as seen in participatory design), but also to equip them with new tools and innovative organizational methods [ 14 ].

Amongst those studies that do specifically examine mutual learning empirically, some examine the concept as something readily understandable for project participants asking them to provide self-reported accounts of their mutual learning. For example, Klüber et al. [ 3 ] conducted a short-term evaluation during a single workshop. They polled participants finding that they reported a moderate feeling of mutual learning and suggested more mixing of different participant constellations to increase that feeling. However, while such studies relying on selfreported accounts provide some insight into the potential for participatory design to support mutual learning, they offer little guidance for the practical accomplishment of promoting it. By contrast, Bell and Davis [ 6 ] produced narrative summaries based on video-recordings, field notes, reflections and documentation from a series of workshops with young people. They identified learning opportunities experienced by participants in connection to six learning outcomes such as “development of metacognitive awareness of learning”. They argue that these outcomes can be associated with specific design contributions and that the design process benefited both the young people and the design outcome. The identified outcomes offer some insights into practical aspects of participatory design processes that might promote learning. However, they are conceptualized as individual learning outcomes that can be argued to be only loosely connected to collective or mutual forms of learning through a general appeal to the idea of communities of practice. Overall, this points towards the need for greater empirical engagement with mutual learning itself as a process beyond self-reported accounts and analyses of related, but adjacent phenomena. In this paper, we argue for sensemaking as a theory through which such empirical analyses of mutual learning can be productively conducted.

3. Sensemaking

Conceptually, sensemaking can be understood as the process in which people work to understand novel or confusing situations in organizational settings [ 10 ]. In other words, sensemaking is the process of transforming situations into something that can be clearly understood and expressed in language. Enabling and facilitating subsequent actions to resolve uncertainty and ambiguity. This process is important because it is where meanings take shape, influencing how we see ourselves and how we act [ 5 ]. Accordingly, sensemaking has been portrayed as a useful lens to understand learning.

According to Daft and Weick [ 8 ], sensemaking consists of three stages that constitute the overall learning process. First, the initial stage is called scanning, which involves closely monitoring the environment and searching for information. Scanning is typically triggered when people encounter an unexpected or unclear situation that elicits a need for explanation or solution. Specific cues from the environment are actively extracted and interpreted to facilitate the search for valuable information that can enhance understanding of an issue or situation. This ‘noticing and bracketing’ [ 5 ] is a way to create new meaning of existing information. To do so both retrospective and prospective changes are considered, encompassing reflection on past experiences and anticipation of future developments. Second, interpretation takes place when the scanned information is given meaning from ambiguous or conflicting cues. Such meaning is often the result of categories or labels that individuals use to help themselves understand and interpret their experiences [ 5 ]. They are socially defined and adapted to local circumstances. For example, as an issue becomes less equivocal, it becomes more clearly interpreted as a problem or an opportunity and it becomes easier to understand the underlying causes of it. Finally, in the third stage, interpretation leads to a new response or action that involves learning [ 8 ]. In this way, sensemaking involves people participating in activities to understand and construct situations they are trying to comprehend [ 10 ].

From a sensemaking perspective, the organization of a movement from uncertainty to meaning making is embodied in written and spoken communication [ 5 ]. Through language, people communicate, express their thoughts and ideas, interpret information, and share knowledge. Language plays a vital role and underpins the sensemaking process, allowing individuals to interact with others to make new meaning of a situation [ 10 ]. In this way, social interactions facilitate sensemaking not just through individual learning but also by revealing different viewpoints through group interaction. By bringing together a diverse range of perspectives and encouraging dialogue, social interactions enhance the sensemaking process by uncovering insights that may otherwise remain invisible to the group.

4. The case

When we engage in online activities such as browsing a website, logging into a bank account, or interacting on social media, we exchange data with multiple companies that provide a variety of functionality, while remaining largely unknown to the users of a digital service. In response, there is a call for critical data literacy skills among citizens and a request for education to help out with developing young peoples’ understanding of the role of digital data in their lives. In doing so, teachers need support that enables them to guide students in comprehending the ongoing datafication of society [ 11 ]. In addressing this concern, participatory design can serve as a valuable method for designing technology that empowers people to shape future possibilities by embracing ‘the big issues’ [ 1 ]. To gain insights into the large number of companies providing digital solutions and the data infrastructure, it becomes necessary to see what goes on ‘behind the curtain’. In line with this, researchers in the present project have developed a tool called InfraReveal. This packet analysis tool helps reveal the records or metadata attached to data packets when digital services are used. InfraReveal displays the locations and identities with which data is shared during internet use, revealing functionalities that constitute the online services they rely on and their providers. By using the infraReveal tool, teachers and students can shine a light behind the interfaces they use and reveal the digital infrastructures at work. In understanding the various actors and technologies that underlie popular cloud-based services used in schools, teachers and students can reveal the characteristics of the data infrastructure that assembles digital technologies in education.

InfraReveal is a tool aimed at accessing specific aspects of exchanged data, unveiling the origin and destination of data packets, as well as the packet's domain name. By doing so, InfraReveal is capable of visually representing the geographical location and categorization of these data packets (see Appendix A). Each packet’s source and destination address, along with its general media or data type, is recorded. The collected data can provide insights, such as identifying the usage of platforms like YouTube and Gmail, including the times they are accessed. Furthermore, it can illustrate the volume of data exchanged in the form of images or servers located in particular countries. In essence, InfraReveal enables an understanding of data flow, facilitating the observation and analysis of data exchange patterns across countries. The development of InfraReveal is part of the research project: Reconfigurations of educational inequality in a digital world (RED) and has ethical approval from the Swedish Ethical Review Authority.

 5. The Method

The participatory design process can be broadly understood as having three phases: Ethnographic fieldwork, Design workshops, and Implementation. The first phase was to adopt an ethnographic approach and we actively engaged in making sense of the studied environment. This involved gathering rich data through observations, interviews, and the collection of documents in three secondary schools. The ethnographic fieldwork also served as a ‘trust builder’ [ 9 ] between researchers and teachers. Secondly, the design workshops consisted of three types of session: constructing data infrastructures, dashboard design, and tool implementation. The first two types of sessions involved a total of 14 teachers. The first session focused on drawing 'graphical elicitations' as a physical representation of their experiences with data flow, storage, and use. The second session involved working with multiple real-time data representations on the InfraReveal dashboard, exploring ideas for modifying the design to enhance usability and align it with teaching objectives. The third type of session had an implementation character. It involved two teachers from one of the schools collaborating with us researchers to develop a detailed lesson plan centered around the use of the InfraReveal tool. In the third phase, teachers and researchers introduced InfraReveal to students in classrooms through co-teaching.

6. Findings and discussion

First, uncertainty arose among teachers as considered and discussed over data infrastructure and digital systems that were in place at their own schools. Second, uncertainty emerged among teachers regarding aspects of InfraReveal they were planning to implement into their instructional practices. Lastly, we in the research team faced uncertainty while re-designing a real-time dashboard. Interacting with real-time data presented a significant challenge, as it required teachers to anticipate and adapt to unfamiliar situations. This added layer of complexity forced teachers to adapt their instruction on the fly, making it difficult to fully prepare and feel confident during lessons. In the context of sensemaking, teachers initially remained in the scanning phase, focusing on descriptions rather than interpreting or explaining the reasons behind specific patterns or data points visualized on the dashboard. However, as they progressed, they began to move beyond mere descriptions to interpretation and explanation of the data. They described their observations using phrases like ‘it looks like’, or ‘there are’, without immediately ascribing meaning, indicating a lack of inference. We, the researchers, on the other hand, were actively scanning for information regarding teachers’ preferences and needs to inform the dashboard design and ensure that the data presented would generate valuable insights and relevant context for teachers. These insights are particularly relevant to the design of educational technology, where participatory design entails mutual learning among participants. This relates to research highlighting that designing educational technology involves not only getting to know the tool but also figuring out how to integrate it into teaching methods to benefit students' learning [ 12 ]. This overlap of learning forms emphasizes how design can serve as a means of learning, but also implies that design must begin with a clear understanding of what needs to be learned through a tool [ 4 ] In this way, participatory design has potential to empower teachers, ensuring their meaningful involvement in addressing a relevant issue, and amplifying their voices and perspectives [ 7 ]. However, while we firmly believe that involving teachers as stakeholders in the design of educational technology is a valuable approach for participatory design practitioners to enhance meaningful participation and contribute to mutual learning, our experiences with the teachers have shown that this is not trivially easy. Teachers expressed the need to be able to adjust features and visualizations to adapt to their instructional contexts, but also learn how to implement the tool in the classroom. Sensemaking can be a useful lens to investigate these processes in relation to complex topics in educational environments. Our analysis identified that while teachers succeeded in collecting information for technical knowledge, they encountered difficulties in interpreting this knowledge and in identifying and assessing alternative actions. This restricted teachers from substantially contributing to the design of the tool as well as in their use of the designed tool for the intended purpose of supporting student learning about critical data literacy. The application of sensemaking as an analytical perspective revealed that we, the researchers, need to increase our efforts to guide teachers in transitioning from the collection of information to its meaningful interpretation and transformation into actionable knowledge in the participatory design process to facilitate end-user development. This means involvement of learning through design with a strong focus on aligning the design with specific learning goals, and that teachers should have the opportunity to reflect on and adapt the technology within their teaching context.

Acknowledgments

We thank all the teachers and students who have actively participated in the development and use of InfraReveal. This research was possible thanks to the generous support of the Riksbankens Jubileumsfond, which funds the project RED (Reconfigurations of Educational In/Equalities in a Digital World) (GI19-1500). We are grateful to the RED team for our continuous conversations on datafication, schooling, and inequalities.

 A. Appendix [1] Susanne Bødker and Morten Kyng . 2018 . Participatory Design that Matters-Facing the Big Issues . ACM Transactions on Computer-Human Interaction 25 , 1 : 1 - 31 . https://doi.org/10.1145/3152421 [2] Mette Agger Eriksen , Per-Anders Hillgren , and Anna Seravalli . 2020 . Foregrounding Learning in Infrastructuring-to Change Worldviews and Practices in the Public Sector . In Proceedings of the 16th Participatory Design Conference 2020 - Participation(s) Otherwise - Volume 1 , 182 - 192 . https://doi.org/10.1145/3385010.3385013 [3] Sara Klüber , Franzisca Maas, Anna Hohm, and Jörn Hurtienne . 2020 . Participant's View: Short-Term Evaluation of Realizing PD Ideals . In Proceedings of the 16th Participatory Design Conference 2020 - Participation(s) Otherwise - Volume 2 , 138 - 142 . https://doi.org/10.1145/3384772.3385145 [4] Wolmet Barendregt , Tilde M. Bekker , Peter Börjesson , Eva Eriksson, and Olof Torgersson . 2016 . Legitimate Participation in the Classroom Context: Adding Learning Goals to Participatory Design . In Proceedings of the The 15th International Conference on Interaction Design and Children , 167 - 174 . https://doi.org/10.1145/2930674.2930686 [5] Karl E. Weick , Kathleen M. Sutcliffe , and David Obstfeld . 2005 . Organizing and the Process of Sensemaking . Organization Science 16 , 4 : 409 - 421 . https://doi.org/10.1287/orsc.1050.0133 [6] Adam Bell and Katie Davis . 2016 . Learning through Participatory Design: Designing Digital Badges for and with Teens . In Proceedings of the The 15th International Conference on Interaction Design and Children , 218 - 229 . https://doi.org/10.1145/2930674.2930705 [7] Bronwyn Cumbo and Neil Selwyn . 2022 . Using participatory design approaches in educational research . International Journal of Research & Method in Education 45 , 1 : 60 - 72 . https://doi.org/10.1080/1743727X. 2021 .1902981 [8] Richard L. Daft and Karl E. Weick . 1984 . Toward a Model of Organizations as Interpretation Systems . The Academy of Management Review 9 , 2 : 284 . https://doi.org/10.2307/258441 [9] Yngve Dahl and Kshitij Sharma . 2022 . Six Facets of Facilitation: Participatory Design Facilitators' Perspectives on Their Role and Its Realization . In CHI Conference on Human Factors in Computing Systems , 1 - 14 . https://doi.org/10.1145/3491102.3502013 [10] Sally Maitlis and Marlys Christianson . 2014 . Sensemaking in Organizations: Taking Stock and Moving Forward . Academy of Management Annals 8 , 1 : 57 - 125 . https://doi.org/10.5465/19416520. 2014 .873177 [11] Luci Pangrazio and Neil Selwyn . 2021 . Towards a school-based 'critical data education . ' Pedagogy, Culture & Society 29 , 3: 431 - 448 . https://doi.org/10.1080/14681366. 2020 .1747527 [12] Ari Tuhkala . 2021 . A systematic literature review of participatory design studies involving teachers . European Journal of Education 56 , 4 : 641 - 659 . https://doi.org/10.1111/ejed.12471 [13] Gerhard Fischer . 2023 . Adaptive and adaptable systems: Differentiating and integrating AI and EUD in: Spano D. (ed.) Proceedings of the 9th International Symposium on End-User Development , 3 - 18 . Springer. [14] Mørch , Anders I. , and Renate Andersen . 2010 . Mutual development: The software engineering context of end-user development . Journal of Organizational and End User Computing (JOEUC )22 , 2 : 36 - 57 .