Ethnographically Informed Studies as a Methodology for Motivation Aware Design Processes Christine Kunzmann, Andreas Schmidt {kunzmann | aschmidt}@fzi.de FZI Research Center for Information Technologies, Karlsruhe, Germany Abstract. While motivational aspects have been recognized as important factors for IT support for learning, it has been difficult to integrate them into requirements engineering processes. We propose ethnographically informed studies as an effective means that has been successfully applied in two research projects as part of their design processes and discuss the remaining challenges. 1. Introduction Motivational aspects have been recognized as a major factor for successful implementation of systems supporting learning at the workplace, but it has been found difficult to tackle to analyse problems and develop solutions in a systematic way (see, e.g., Kunzmann 2009). Typical techniques for requirements elicitation often fail, probably because motivational aspects are frequently not conscious so that we need other ways to learn about them. One promising approach are ethnographically informed studies. In the following, we present experiences from two European projects where we applied ethnographic methods and derive from this recommendations for integration such methods into design processes. 2. Ethnographically Informed Studies Ethnographic research is characterized by immersion into the social environment the studies take place in. While field observation focuses on what happens, ethnographic research also focuses on the why and how things happen (Barnes et al., 2009). By becoming part of the community, the research is capable of understanding internal aspects of observable behaviour. Traditional ethnographic research in anthropology or sociology is based on long periods of time (Fetterman, 1999). Such long periods of time are not practical for usage of such methods in requirements engineering. Therefore, modified methods like rapid ethnography (Millen, 2000) have been proposed, specifically in the context of CSCW (Harper, 2000). The main modifications are compensations for the shorter time periods by a more focused study and combinations of several researchers’ perspectives in a collaborative effort. 2.2 Experiences in MATURE MATURE1 aims at providing a Learning and Maturing Environment that supports knowledge maturing, where “knowledge maturing” refers to goal-oriented learning on a collective level (Schmidt et al., 2009). As part of the first year’s work of establishing a common understanding and laying the foundation for requirements engineering, an ethnographically informed study has been conducted in seven organizations, out of which two were designated application partners.2 The ethnographer teams at each organization already had an established relationship with the organization, which simplified the trust building. Some of the researchers had prior experience with ethnographically informed studies, while also members of the development teams took part in the study. Goal of the study was to learn about real- world knowledge maturing practices. The study has focused on five topics: stages of maturity, types of knowledge, situations and context of knowledge maturing, motivation or incentives, and the role of structures and semantics. The study was divided into three parts. In the first week, at least two researchers visited each organisation, immersed into the work environment, and shadowed individuals throughout their working day. In the 2nd and 3rd week, self-reporting by the shadowed individuals was used for gaining data about specific and well-defined topics over the two week period. In the fourth week, the same researchers returned to the organisation for further ethnographic fieldwork. At the beginning, they discussed the topics of self-reporting, gained a richer picture of organisational activities and interview selected individuals at the end of the fourth week. This fieldwork provided rich qualitative empirical material which was coded following standard structured text analysis, collectively reflected upon and analysed. The results were amalgamated in structured persona descriptions (Maier & Thalmann, 2010), which contained specific sections on motivational aspects (Kunzmann et al., 2009), among others. During the further course of the project, the personas have been used as a major source for the collaborative use case definition process. The empirical grounded has added much higher degree of richness. But what has been found even more important is that the participation of the majority of project partners in the studies (seven out of twelve), particularly individual members of development teams, has been very useful to build a shared context and helped to make discussions more constructive. It could be observed that individual experiences were expressed in narratives that were reused by others in the course of the discussion. As for the motivational aspects, it has been found that the richness of understanding developed by the individual researcher has depended on prior sensitization so that it was concluded that future studies should include longer 1 http://mature-ip.eu 2 for a comprehensive overview see (Kaschig et al., 2009) preparation phases in which the different backgrounds of researcher are exploited for mutual sensitization for important topics. Although originally intended to accompany the whole development process, it has been found that their usage has been replaced by direct interaction with prospective users of the system as part of the participatory design process in the second year. 2.3 Experiences n MIRROR MIRROR3 aims at supporting reflective learning at the workplace. Following the experiences of MATURE, in the first year exploratory user studies have been conducted as a basis for the requirements engineering process. Unlike MATURE, only one ethnographically informed study was conducted at a German hospital, while other studies have used different methods like interviews or questionnaires. Goal of this study was to explore (a) the role of affective aspects and (b) motivational barriers in the transition between reflection on individual, team, and organizational level. The study design4 included a first period of four days with three ethnographers (out of which one had prior experience with ethnography and knew hospital routines) and five persons (both nurses and physicians) who have been shadowed during two shift types. During the study, the participants also wore an unobtrusive sensor belt that recorded acceleration in three dimensions and heart rate. After a break of five days during which a preliminary analysis of the data with respect to “interesting events” has been conducted, the participants were interviewed to clarify topics (contextualized with examples from the ethnographical notes) such as dealing with emotionally straining situations, coping strategies for stress and emotional strain, dealing with mistakes, barriers to sharing insights, and role of meetings. Both interview transcripts and ethnographic notes were analysed using structured text analysis similar to approach taken in MATURE. The results were exchanged and discussed with other partners. The results of these discussions were summarized in a structured persona description (adapted from the structure used in MATURE). While the personas are yet to be fed into the further requirements engineering process, the study has already revealed interesting aspects that influence the motivation to use reflection support apps. Apart from the difficulties to integrate them into daily practice, we also discovered that participants had already developed coping strategies for emotional strain that might be affected by any attempt to make affective transparent, e.g., with the help of sensor data. Furthermore, it could be confirmed that ethnographic research yields a much richer understanding of everyday practice, which in turn helps in turning discussion processes more constructive. The shortened time period has prevented the ethnographers from observing any developments so that we had to rely on interpretations of the participants in the interviews. We have some indications that this might be problematic and does not realize the full potential of ethnographic methods. 3 http://www.mirror-project.eu 4 For more details see Müller et al. 2011 3. Integration into the Design Process Based on the experiences in MATURE and MIRROR, we consider ethnographic methods to be a very helpful method in design processes, particularly when designing complex systems. Considering state of the art design process models (such as Ravenscroft et al., 2011, see fig. 1), which consist of cycles of defining goals (“prioritize”), learning and problematizing (“learn”), designing solutions and evaluating them, the ethnographic methods are particularly helpful for learning phases in early design cycles where a general understanding of the prospective user and their environment needs to be built and possible motivational barriers need to be identified. As these ethnographic studies typically result in narratives that are shared across the project, they are also an important element of building a shared conceptual model that is grounded in real experiences, thus avoiding too abstract models and highly theoretical discussions. Figure 1: Design-based research for TEL (Ravenscroft et al., 2011) In fig. 2, we illustrate this more detail. We propose to use ethnographically informed studies as the starting point for designing systems and using personas based on empirically grounded structured narratives to summarize their findings for a larger team. This is then the input for an iterative design process in which involving the target users at an early stage is crucial for the success. This direct and frequent contact with real individuals will replace personas in the course of the design process. Where this direct contact is not feasible – for whatever reasons -, the personas will probably play a bigger role also in later phases of system design. Figure 2: Integration of ethnographic studies into the design process 3. Conclusions Ethnographically informed studies are valuable instruments for early phases of design processes and should be used more frequently, complementing or replacing purely interview- or questionnaire-based methods. Particularly the immersion, i.e., becoming part of team and learning their perspective yields a richer picture than a pure observation. From our experiences, we recommend the following:  Ethnographer teams should be composed of experienced ethnographers (for methodological advice) and members of the development team (for transferring the results).  The time period should be carefully decided upon, particularly if developments need to be observed.  Prior trust relationships between the ethnographers and the participants help considerably to get a deeper insight within a shorter time period.  Experiences of the ethnographers in the domain also help to compensate for a shorter time period as they have already a general understanding to build upon.  Design processes should make productive use of narratives that are used to discuss use cases and design decisions as they help to communicate efficiently experiences made during the ethnographic research. These narratives should be preferred over formal models or descriptions. Further research will focus on providing more methodological support for inexperienced ethnographers. Acknowledgments This work has been supported by the European Commission within the 7 th Framework Programme as part of the projects MATURE (http://mature-ip.eu) and MIRROR (http://www.mirror-ip.eu). We wish to thank all partners of both projects for the fruitful collaboration. References [1] Kunzmann, Christine; Schmidt, Andreas; Braun, Volker; Czech, David; Fletschinger, Benjamin; Kohler, Silke; Lüber, Verena (2009): Integrating Motivational Aspects into the Design of Informal Learning Support in Organizations. In: 9th International Conference on Knowledge Management and Knowledge Technologies, September 2-4, 2009, Graz, Austria, pp. 259-267 [2] Barnes, Sally-Anne; Bimrose, Jenny; Brown, Alan; Feldkamp, Daniela; Kaschig, Andreas; Kunzmann, Christine; Maier, Ronald; Nelkner, Tobias; Sandow, Alexander; Thalmann, Stefan (2009): Knowledge Maturing at Workplaces of Knowledge Workers: Results of an Ethnographically Informed Study.In: 9th International Conference on Knowledge Management (I-KNOW '09), Graz, Austria, pp. 51-61 [3] Fetterman, D. M. (1999): Ethnography Step by Step, London, SAGE Publications Ltd [4] Millen, D. R. (2000): Rapid ethnography: time deepening strategies for HCI field research. 3rd conference on Designing interactive systems. New York. [5] Harper, R. H. R. (2000): The Organisation in Ethnography: A Discussion of Ethnographic Fieldwork Programs in CSCW. Comput. Supported Coop. Work, Vol. 9, no. 2, pp. 239-264. [6] Schmidt, Andreas; Hinkelmann, Knut; Ley, Tobias; Lindstaedt, Stefanie; Maier, Ronald; Riss, Uwe: Conceptual Foundations for a Service-oriented Knowledge and Learning Architecture: Supporting Content, Process and Ontology Maturing. In: Sebastian Schaffert and Klaus Tochtermann and Tassilo Pellegrini (eds.): Networked Knowledge - Networked Media: Integrating Knowledge Management, New Media Technologies and Semantic Systems, Springer, 2009, pp. 79-94 [7] Ravenscroft, Andrew; Schmidt, Andreas; Bradley, Claire; Cook, John: Designing complex socio- technical systems for informal learning and knowledge maturing in the ‘Web 2.0 workplace’, Journal of Computer-Assisted Learning, forthcoming [8] Maier, R.; Thalmann, S. (2010): Using Personas for designing knowledge and learning services: results of an ethnographically informed study. In: International Journal of Technology Enhanced Learning 2(1/2), pp. 58 - 74