=Paper=
{{Paper
|id=None
|storemode=property
|title=Ethnographically Informed Studies as a Methodology for Motivation Aware Design Processes
|pdfUrl=https://ceur-ws.org/Vol-957/matel11_submission_3.pdf
|volume=Vol-957
}}
==Ethnographically Informed Studies as a Methodology for Motivation Aware Design Processes==
https://ceur-ws.org/Vol-957/matel11_submission_3.pdf
Ethnographically Informed Studies as a Methodology
for Motivation Aware Design Processes
Christine Kunzmann, Andreas Schmidt
{kunzmann | aschmidt}@fzi.de
FZI Research Center for Information Technologies, Karlsruhe, Germany
Abstract. While motivational aspects have been recognized as important
factors for IT support for learning, it has been difficult to integrate them into
requirements engineering processes. We propose ethnographically informed
studies as an effective means that has been successfully applied in two research
projects as part of their design processes and discuss the remaining challenges.
1. Introduction
Motivational aspects have been recognized as a major factor for successful
implementation of systems supporting learning at the workplace, but it has been found
difficult to tackle to analyse problems and develop solutions in a systematic way (see,
e.g., Kunzmann 2009). Typical techniques for requirements elicitation often fail,
probably because motivational aspects are frequently not conscious so that we need
other ways to learn about them.
One promising approach are ethnographically informed studies. In the following,
we present experiences from two European projects where we applied ethnographic
methods and derive from this recommendations for integration such methods into
design processes.
2. Ethnographically Informed Studies
Ethnographic research is characterized by immersion into the social environment
the studies take place in. While field observation focuses on what happens,
ethnographic research also focuses on the why and how things happen (Barnes et al.,
2009). By becoming part of the community, the research is capable of understanding
internal aspects of observable behaviour.
Traditional ethnographic research in anthropology or sociology is based on long
periods of time (Fetterman, 1999). Such long periods of time are not practical for
usage of such methods in requirements engineering. Therefore, modified methods like
rapid ethnography (Millen, 2000) have been proposed, specifically in the context of
CSCW (Harper, 2000). The main modifications are compensations for the shorter
time periods by a more focused study and combinations of several researchers’
perspectives in a collaborative effort.
2.2 Experiences in MATURE
MATURE1 aims at providing a Learning and Maturing Environment that supports
knowledge maturing, where “knowledge maturing” refers to goal-oriented learning on
a collective level (Schmidt et al., 2009). As part of the first year’s work of
establishing a common understanding and laying the foundation for requirements
engineering, an ethnographically informed study has been conducted in seven
organizations, out of which two were designated application partners.2 The
ethnographer teams at each organization already had an established relationship with
the organization, which simplified the trust building. Some of the researchers had
prior experience with ethnographically informed studies, while also members of the
development teams took part in the study. Goal of the study was to learn about real-
world knowledge maturing practices. The study has focused on five topics: stages of
maturity, types of knowledge, situations and context of knowledge maturing,
motivation or incentives, and the role of structures and semantics.
The study was divided into three parts. In the first week, at least two researchers
visited each organisation, immersed into the work environment, and shadowed
individuals throughout their working day. In the 2nd and 3rd week, self-reporting by
the shadowed individuals was used for gaining data about specific and well-defined
topics over the two week period. In the fourth week, the same researchers returned to
the organisation for further ethnographic fieldwork. At the beginning, they discussed
the topics of self-reporting, gained a richer picture of organisational activities and
interview selected individuals at the end of the fourth week. This fieldwork provided
rich qualitative empirical material which was coded following standard structured text
analysis, collectively reflected upon and analysed.
The results were amalgamated in structured persona descriptions (Maier &
Thalmann, 2010), which contained specific sections on motivational aspects
(Kunzmann et al., 2009), among others.
During the further course of the project, the personas have been used as a major
source for the collaborative use case definition process. The empirical grounded has
added much higher degree of richness. But what has been found even more important
is that the participation of the majority of project partners in the studies (seven out of
twelve), particularly individual members of development teams, has been very useful
to build a shared context and helped to make discussions more constructive. It could
be observed that individual experiences were expressed in narratives that were reused
by others in the course of the discussion.
As for the motivational aspects, it has been found that the richness of
understanding developed by the individual researcher has depended on prior
sensitization so that it was concluded that future studies should include longer
1 http://mature-ip.eu
2 for a comprehensive overview see (Kaschig et al., 2009)
preparation phases in which the different backgrounds of researcher are exploited for
mutual sensitization for important topics.
Although originally intended to accompany the whole development process, it has
been found that their usage has been replaced by direct interaction with prospective
users of the system as part of the participatory design process in the second year.
2.3 Experiences n MIRROR
MIRROR3 aims at supporting reflective learning at the workplace. Following the
experiences of MATURE, in the first year exploratory user studies have been
conducted as a basis for the requirements engineering process. Unlike MATURE,
only one ethnographically informed study was conducted at a German hospital, while
other studies have used different methods like interviews or questionnaires. Goal of
this study was to explore (a) the role of affective aspects and (b) motivational barriers
in the transition between reflection on individual, team, and organizational level. The
study design4 included a first period of four days with three ethnographers (out of
which one had prior experience with ethnography and knew hospital routines) and
five persons (both nurses and physicians) who have been shadowed during two shift
types. During the study, the participants also wore an unobtrusive sensor belt that
recorded acceleration in three dimensions and heart rate. After a break of five days
during which a preliminary analysis of the data with respect to “interesting events”
has been conducted, the participants were interviewed to clarify topics (contextualized
with examples from the ethnographical notes) such as dealing with emotionally
straining situations, coping strategies for stress and emotional strain, dealing with
mistakes, barriers to sharing insights, and role of meetings. Both interview transcripts
and ethnographic notes were analysed using structured text analysis similar to
approach taken in MATURE. The results were exchanged and discussed with other
partners. The results of these discussions were summarized in a structured persona
description (adapted from the structure used in MATURE).
While the personas are yet to be fed into the further requirements engineering
process, the study has already revealed interesting aspects that influence the
motivation to use reflection support apps. Apart from the difficulties to integrate them
into daily practice, we also discovered that participants had already developed coping
strategies for emotional strain that might be affected by any attempt to make affective
transparent, e.g., with the help of sensor data. Furthermore, it could be confirmed that
ethnographic research yields a much richer understanding of everyday practice, which
in turn helps in turning discussion processes more constructive.
The shortened time period has prevented the ethnographers from observing any
developments so that we had to rely on interpretations of the participants in the
interviews. We have some indications that this might be problematic and does not
realize the full potential of ethnographic methods.
3 http://www.mirror-project.eu
4 For more details see Müller et al. 2011
3. Integration into the Design Process
Based on the experiences in MATURE and MIRROR, we consider ethnographic
methods to be a very helpful method in design processes, particularly when designing
complex systems. Considering state of the art design process models (such as
Ravenscroft et al., 2011, see fig. 1), which consist of cycles of defining goals
(“prioritize”), learning and problematizing (“learn”), designing solutions and
evaluating them, the ethnographic methods are particularly helpful for learning phases
in early design cycles where a general understanding of the prospective user and their
environment needs to be built and possible motivational barriers need to be identified.
As these ethnographic studies typically result in narratives that are shared across the
project, they are also an important element of building a shared conceptual model that
is grounded in real experiences, thus avoiding too abstract models and highly
theoretical discussions.
Figure 1: Design-based research for TEL (Ravenscroft et al., 2011)
In fig. 2, we illustrate this more detail. We propose to use ethnographically
informed studies as the starting point for designing systems and using personas based
on empirically grounded structured narratives to summarize their findings for a larger
team. This is then the input for an iterative design process in which involving the
target users at an early stage is crucial for the success. This direct and frequent contact
with real individuals will replace personas in the course of the design process. Where
this direct contact is not feasible – for whatever reasons -, the personas will probably
play a bigger role also in later phases of system design.
Figure 2: Integration of ethnographic studies into the design process
3. Conclusions
Ethnographically informed studies are valuable instruments for early phases of design
processes and should be used more frequently, complementing or replacing purely
interview- or questionnaire-based methods. Particularly the immersion, i.e., becoming
part of team and learning their perspective yields a richer picture than a pure
observation.
From our experiences, we recommend the following:
Ethnographer teams should be composed of experienced ethnographers (for
methodological advice) and members of the development team (for
transferring the results).
The time period should be carefully decided upon, particularly if
developments need to be observed.
Prior trust relationships between the ethnographers and the participants help
considerably to get a deeper insight within a shorter time period.
Experiences of the ethnographers in the domain also help to compensate for
a shorter time period as they have already a general understanding to build
upon.
Design processes should make productive use of narratives that are used to
discuss use cases and design decisions as they help to communicate
efficiently experiences made during the ethnographic research. These
narratives should be preferred over formal models or descriptions.
Further research will focus on providing more methodological support for
inexperienced ethnographers.
Acknowledgments
This work has been supported by the European Commission within the 7 th Framework
Programme as part of the projects MATURE (http://mature-ip.eu) and MIRROR
(http://www.mirror-ip.eu). We wish to thank all partners of both projects for the
fruitful collaboration.
References
[1] Kunzmann, Christine; Schmidt, Andreas; Braun, Volker; Czech, David; Fletschinger, Benjamin;
Kohler, Silke; Lüber, Verena (2009): Integrating Motivational Aspects into the Design of Informal
Learning Support in Organizations. In: 9th International Conference on Knowledge Management and
Knowledge Technologies, September 2-4, 2009, Graz, Austria, pp. 259-267
[2] Barnes, Sally-Anne; Bimrose, Jenny; Brown, Alan; Feldkamp, Daniela; Kaschig, Andreas;
Kunzmann, Christine; Maier, Ronald; Nelkner, Tobias; Sandow, Alexander; Thalmann, Stefan
(2009): Knowledge Maturing at Workplaces of Knowledge Workers: Results of an Ethnographically
Informed Study.In: 9th International Conference on Knowledge Management (I-KNOW '09), Graz,
Austria, pp. 51-61
[3] Fetterman, D. M. (1999): Ethnography Step by Step, London, SAGE Publications Ltd
[4] Millen, D. R. (2000): Rapid ethnography: time deepening strategies for HCI field research. 3rd
conference on Designing interactive systems. New York.
[5] Harper, R. H. R. (2000): The Organisation in Ethnography: A Discussion of Ethnographic Fieldwork
Programs in CSCW. Comput. Supported Coop. Work, Vol. 9, no. 2, pp. 239-264.
[6] Schmidt, Andreas; Hinkelmann, Knut; Ley, Tobias; Lindstaedt, Stefanie; Maier, Ronald; Riss, Uwe:
Conceptual Foundations for a Service-oriented Knowledge and Learning Architecture: Supporting
Content, Process and Ontology Maturing. In: Sebastian Schaffert and Klaus Tochtermann and Tassilo
Pellegrini (eds.): Networked Knowledge - Networked Media: Integrating Knowledge Management,
New Media Technologies and Semantic Systems, Springer, 2009, pp. 79-94
[7] Ravenscroft, Andrew; Schmidt, Andreas; Bradley, Claire; Cook, John: Designing complex socio-
technical systems for informal learning and knowledge maturing in the ‘Web 2.0 workplace’, Journal
of Computer-Assisted Learning, forthcoming
[8] Maier, R.; Thalmann, S. (2010): Using Personas for designing knowledge and learning services:
results of an ethnographically informed study. In: International Journal of Technology Enhanced
Learning 2(1/2), pp. 58 - 74