<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Value Stories: Putting Human Values into Requirements Engineering</article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>Interactive Intelligence Group, Delft University of Technology</institution>
          ,
          <addr-line>Mekelweg 4, 2628 CD Delft</addr-line>
          ,
          <country country="NL">The Netherlands</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>Software systems can give rise to ethical issues, with human values such as privacy, autonomy and responsibility at their heart. Such issues often do not become apparent until software has been put to use, and are left to be dealt with after harm has been done. However, these ethical issues are in uenced by decisions made during design. A range of approaches to technology design aims to consider ethical issues and their underlying values during design, when there is more room to shape a technology. These values-oriented approaches guide designers in identifying and analyzing value issues with technology, but do not focus on deriving requirements from identi ed issues. As a result, the knowledge that can be gained from values-oriented methods does not nd its way into requirements engineering processes. To address these challenges, we present a method to extract values and related elements obtained through existing value elicitation techniques, and to document these elements in a format that is amenable to use in further requirements engineering activities. We present a case study as a proof of concept.</p>
      </abstract>
      <kwd-group>
        <kwd>requirements elicitation</kwd>
        <kwd>values</kwd>
        <kwd>ethical issues</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Software introduces new capabilities or modi es existing ones, and changes the
way people do things. This can have desirable and undesirable consequences
or ethical implications, which center on a system's impact on human values.
Examples of such ethical implications include privacy and accountability issues
with electronic patient record systems, bias in search engines, and user autonomy
issues in decision support systems.</p>
      <p>A system's impact on human values often is not considered until the system
has been put to use and its desirable and/or undesirable consequences have
come to light. To an important extent, this impact is the result of the system's
functions and qualities. These are, in turn, the result of decisions made during the
design process. A system's impact on human values need not be an afterthought;
rather, relevant values should be considered early during design, when a system is
still malleable. The design process should identify key values and ensure that the
functions and qualities of the system support these values as much as possible.</p>
      <p>
        In recognition of the need to address value issues during design, values are
receiving increasing attention in some technology design disciplines. Methods
that deal with values explicitly have emerged and matured perhaps most
prominently within the Value Sensitive Design (VSD) framework [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. These methods
guide designers in identifying, conceptualizing, and analyzing values in light of
a technology, evaluating technology, and pro-actively designing technology to
support values.
      </p>
      <p>
        The majority of work on VSD appears in the eld of Human-Computer
Interaction (HCI). This suggests that the concepts and methods introduced by VSD
and related approaches receive limited attention outside HCI. As these methods
tend to be geared towards HCI, it is also not clear how well they can be
integrated with disciplines outside of HCI, such as Requirements Engineering (RE).
Few methods o er little guidance on expressing identi ed values in design by
specifying system functions and/or qualities to support the values in question,
though the need for such speci cation is recognized (e.g., [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]). In RE, the few
approaches that address values (e.g., [
        <xref ref-type="bibr" rid="ref3 ref4">3, 4</xref>
        ]) have not been tested and tried to the
extent that more mature methods in VSD have, and lack widespread recognition.
Thinking about values is not common practice in RE. As a result, values are not
considered to the extent that they could be in design processes that involve RE.
      </p>
      <p>We argue that this calls for ways to identify and analyze value issues in
RE, and to specify functionality to support relevant values. To address these
issues, we present an approach that helps 1. identify (potentially overlooked)
groups of stakeholders; 2. identify and concretize their values; and 3. specify
user stories that describe features to support these values. Our approach helps
designers and stakeholders explore new ideas and issues beyond users, clients,
and business goals, by considering both direct and indirect stakeholders and by
identifying their values before focusing on system features.</p>
      <p>In Section 2, we will discuss related work in VSD and RE. In section 3 we
will present our approach. We will illustrate this approach in a case study in
Section 4, and discuss the approach and draw some conclusions in Section 5.
2</p>
    </sec>
    <sec id="sec-2">
      <title>Related work</title>
      <p>In this section, we discuss key concepts in Value Sensitive Design, the most
prominent values-oriented framework to have emerged within Human-Computer
Interaction. Furthermore, we discuss approaches in Requirements Engineering
that deal with values to some extent.
2.1</p>
      <sec id="sec-2-1">
        <title>Value Sensitive Design</title>
        <p>
          Value Sensitive Design (VSD) [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ] draws on a number of key concepts. Direct
stakeholders are those individuals or organizations that interact directly with a
system or its output. Indirect stakeholders are all other parties a ected by the
use of the system. The latter are often ignored in the design process [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ]. Values
are another key concept, which can be de ned as \what a person or group of
people considers important in life" [1, p. 70]. VSD distinguishes between three
types of values. Explicitly supported values are those that are required to be
supported by the technology. Designer values refer to the designers or researchers
personal values that implicitly guide design decisions. Stakeholder values are
values that are important to some, though not necessarily all, stakeholders of a
technology [
          <xref ref-type="bibr" rid="ref5">5</xref>
          ]. Di erences among values can give rise to value tensions (such as
privacy versus security) among stakeholder groups, when supporting one value
in a technology challenges another.
        </p>
        <p>
          These concepts are central to VSD's three-part methodology, consisting of
conceptual investigations of key stakeholders and values; empirical investigations
of actual or potential stakeholders and contexts-of-use; and technical
investigations aimed at designing new technology to support selected values or examining
how existing technologies support or hinder certain values [
          <xref ref-type="bibr" rid="ref6">6</xref>
          ]. These
investigations can be applied iteratively and integratively throughout the design process.
Suggestions for using VSD include starting with a value, technology or
context of use; identifying direct and indirect stakeholders; identifying bene ts and
harms for each stakeholder group; mapping bene ts and harms onto
corresponding; conducting conceptual investigations of key values; and identifying potential
value con icts [
          <xref ref-type="bibr" rid="ref1">1</xref>
          ].
        </p>
        <p>
          Several speci c methods have been developed within VSD, some of which are
suited to elicit knowledge about values in situ. In the Value Scenarios technique,
designers or stakeholders write scenarios that consider stakeholders,
pervasiveness, time, systemic e ects and value implications of a technology to support
long-term, systemic thinking in interactive design practice [
          <xref ref-type="bibr" rid="ref7">7</xref>
          ]. The Value Dams
and Flows method conducts stakeholder surveys and, based on these, accounts
for values in design by avoiding problematic features, by identifying and
designing for values that stakeholders do wish to see the system embody, and
systematically addressing design tradeo s that concern values. [
          <xref ref-type="bibr" rid="ref8">8</xref>
          ]. Envisioning Cards
[
          <xref ref-type="bibr" rid="ref9">9</xref>
          ] are a design toolkit that guides the design process through explanations of
key themes and associated concepts, and provides focused design activities to
address issues related to these themes in a systematic way. The toolkit consists
of cards that fall into one of four categories of envisioning criteria: stakeholders,
time, values, and pervasiveness. These methods help consider stakeholders, their
values and a system's (long-term) implications on values, but do not guide the
designer in specifying system functions and qualities to support the values in
question.
2.2
        </p>
      </sec>
      <sec id="sec-2-2">
        <title>Requirements Engineering</title>
        <p>
          between users and the system, and help understand requirements [
          <xref ref-type="bibr" rid="ref10">10</xref>
          ] or even
represent functional requirements [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]. The agile development method Extreme
Programming (XP) uses user stories to describe features that provide business
value to a customer [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]. User stories are comparable to use cases and the
process of writing user stories can be seen as brainstorming in RE, which helps
generate creative solutions to speci c problems [
          <xref ref-type="bibr" rid="ref11">11</xref>
          ]. Though these techniques
often involve stakeholders and their needs, few focus explicitly on both direct
and indirect stakeholders and their values.
        </p>
        <p>
          Notable exceptions with regard to values include [
          <xref ref-type="bibr" rid="ref3 ref4">3, 4</xref>
          ]. Thew and Sutcli e's
method uses a taxonomy of users' values, motivations and emotions, and provides
process guidance to elicit and analyze these issues [
          <xref ref-type="bibr" rid="ref3">3</xref>
          ]. Though the method draws
attention to values, it focuses on their implications for the Requirements
Engineering process rather than specifying functions or qualities to support those
values. Koch and colleagues present a method that focuses on approximating
or identifying users' values based on their preferences for key (work) tasks [
          <xref ref-type="bibr" rid="ref4">4</xref>
          ].
The method helps identify user values, but uses these to adapt existing systems
rather than generate new ideas for functions or qualities. Neither of the methods
takes indirect stakeholders into account.
3
        </p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>Our proposal: from values to requirements</title>
      <p>In this section, we propose a technique to elicit requirements of a system to be
developed that is based on a value analysis of the systems stakeholders. By that,
we aim to connect the concepts used in VSD to those used in RE, and provide
researchers in both elds the opportunity to pro t from each others work. The
technique involves the following ve steps.
1. Analyzing the systems stakeholders
2. Analyzing the stakeholders values
3. Providing concrete situations with the values
4. Determining stakeholder needs
5. Creating user stories</p>
      <p>In the remainder of this section, we will explain each step, and give
suggestions for a practical use of the technique.The rst step is to identify the direct
and indirect stakeholders of the system-to-be, by asking who will interact
directly with the system or its output, and who will be a ected by the system or
its output without interacting directly with it, respectively.</p>
      <p>The second step, a value analysis, involves identifying values that are relevant
to the stakeholders identi ed in the rst step. This step can build on various VSD
techniques used to elicit values. For instance, the Envisioning Cards and Value
Dams and Flows techniques mentioned in Section 2.1 involve the analysis of
stakeholders and their values. The envisioning cards deck contains cards with
assignments and question like \generate as list of as many potentially implicated
values as possible", and \what views and values do stakeholders bring to a
system?". The Values Dams and Flows technique involves identifying potential
harms and bene ts of the system to stakeholders, and then analyzing values
underlying the harms and bene ts.</p>
      <p>The third step of our technique involves coming up with one or more
examples of concrete situation(s) that explain(s) why or how a particular value is
important for the stakeholder who holds the value. This is important because
one value can have di erent meanings in di erent situations for a stakeholder.
For example, the value of privacy for a user of a system can mean that the users
personal information stored in the system should not be shared with anyone, or
that the user should be able to use the system on his own.</p>
      <p>These examples of concrete situations are input for the fourth step,
determining stakeholder needs. For each concrete situation, this step determines how
to support the associated value for the stakeholder in question. For instance, to
support or protect a users privacy with regard to undesired sharing of personal
information, the user should be able to indicate which personal information can
be shared with whom and under what conditions.</p>
      <p>The fth step is to write user stories in order to specify early requirements
to meet the stakeholder needs expressed in the previous step. As discussed in
Section 2.2, user stories can be compared to use cases in RE, which can help
understand or represent functional requirements early in the development process.
User stories commonly take the form: As a [role] I want [something] so that
[bene t]. We propose to create user stories according to the following template: As a
[stakeholder] I want [stakeholder need] so that my [value] is promoted/supported
when [concrete situation]. Note that one stakeholder can have multiple values,
one value can have multiple concrete situations, and one concrete situation can
have multiple stakeholder needs, but that one user story is created for each
stakeholder need (with the associated stakeholder, value and concrete situation). It
clari es and concretizes each elicited value in a speci c situation, and describes
a high-level requirement to support the value. This creates a result that draws
on concepts from VSD. Moreover, the form of the result is familiar in agile
development methods and arguably within RE, making it easier to integrate with
existing practices in those elds.</p>
      <p>One way to use this technique in practice is to organize a workshop with
stakeholders of the envisioned system, e.g. domain experts, potential users, and
developers. We suggest the following outline for such a workshop.
A. Short presentation to introduce the participants to values and VSD, e.g. by
providing examples of values in design (assuming that the participants are
not familiar with VSD yet)
B. Identify direct and indirect stakeholders
C. Per stakeholder (this may be a selection of stakeholders identi ed in step B),
identify one or more values and concrete situations
D. Per concrete situation, identify one or more stakeholder needs</p>
      <p>
        The stakeholders identi ed in part B need not correspond with the
participants, as part of the aim of our approach is to spark new ideas by having
participants consider other perspectives than their own, similar to viewpoint
approaches [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. Stakeholder analysis (step 1), value analysis (step 2) and concrete
situations (step 3) are taken together in one part (part C) because in a workshop
setting it is natural to immediately explain a value to the other participants by
providing a concrete situation. In a large group (e.g., when n &gt; 8), part C and
D can be done in subgroups of 4 to 5 participants. In that case, we suggest to
perform part C and D together with all participants for the rst stakeholder
so that everybody understands what is expected of them. The last step of the
technique, creating user stories, does not require new input of the participants,
and can thus be performed by the workshop organizers afterward, though the
resulting user stories should be examined by stakeholders to validate them.
      </p>
      <p>To conclude, we proposed a technique to elicit requirements while accounting
for values. The technique connects important concepts in VSD (stakeholders and
values) to user stories, which can be re ned into more speci c requirements.. The
proposed technique has two bene ts as compared to other requirement elicitation
techniques. First, paying explicit attention to values of direct and indirect
stakeholder may lead to requirements that would not have been discovered otherwise.
Second, knowing the values behind requirements provides an extra perspective
in case of design trade-o s. This perspective allows developers to also reason and
decide about underlying values instead of mere system features.
4</p>
    </sec>
    <sec id="sec-4">
      <title>Case study</title>
      <p>In this section we describe a case study in which we tested the technique proposed
in the previous section by organizing a value-requirements workshop. First, we
will introduce the project in the context of which we performed the workshop,
then we will describe the workshop results, and based on that, we provide an
evaluation of the proposed technique.
4.1</p>
      <sec id="sec-4-1">
        <title>Context</title>
        <p>
          The case study was performed in the context of the IQmulus project [
          <xref ref-type="bibr" rid="ref12">12</xref>
          ]. This
is a 4-year European project in the area of intelligent information management.
The main objective of the project is to enhance decision making by developing
a system (the IQmulus system) that extracts relevant information from large,
heterogeneous geo-spatial data sets. Such a system is needed because new data
acquisition techniques are providing fast and e cient means for multidimensional
spatial data collection, e.g. stereophotogrammetry, airborne LIDAR surveys, and
SAR satellites. These techniques provide extremely high volumes of raw data,
but in order to be useful, the heterogeneous data sets require harmonization and
integration. The IQmulus project aims to make these data more accessible, and
by that, help decision makers to make better choices, e.g. in case of ooding,
ash oods or industrial accidents.
        </p>
        <p>The project consortium involves partners with technical expertise, e.g. in the
development of algorithms for data integration and information ltering, the
visualization of data, or the development of architectures, and partners with
domain expertise, e.g. collecting and using geo-spatial data for marine spatial
planning or for rapid response and territorial management on land. The latter
group are potential future users of the IQmulus system. The case study was
performed in year 1 of the project. At that point, several other user workshops
and requirement elicitation activities had been organized.
4.2</p>
      </sec>
      <sec id="sec-4-2">
        <title>Value Stories Workshop</title>
        <p>The value-requirement workshop was performed with representatives of several
of the IQmulus consortium partners with domain expertise, i.e., some of the
potential users of the IQmulus system. More speci cally, the workshop involved
9 participants with representatives of the following institutes: 2 of IGN (French
National Institute of Geographical Information and Forestry), 2 of Fomi
(Hungarian Institute of Geodesy, Cartography and Remote Sensing), 2 of Regione
Liguria (Genova, Italy), 1 of Ifremer (French Institute for Exploitation of the
Sea), 1 of UBO (European Institute for Marine Studies), 1 of HR Wallington
(Independent Research and Consultancy in Civil Engineering and
Environmental Hydraulics). The workshop was performed such as described in Section 3,
and led by the authors of this paper. The total duration of the workshop was 4
hours (excluding breaks). None of the participants was familiar with VSD before
participation in the workshop. In the remainder of Section 4.2 the results of the
workshop are provided.</p>
        <p>In the rst step, stakeholder analysis, the following 13 direct and indirect
stakeholders of the IQmulus system were identi ed: consortium partners,
public, European Committee, and users, which were divided into managers, GIS
experts, water authorities, scientists, disaster management people, risk
assessment people, spatial planners, data providers, operators (hardware and
infrastructure), and decision makers. The participants agreed that the public and
European Committee are indirect stakeholders, but that the other stakeholders
can be either direct or indirect stakeholders of the IQmulus system.</p>
        <p>From this list, three stakeholders were selected for further analysis. It was
tried to select three stakeholders that are very distinct, but yet in uenced by
each other. The following selection was proposed by the workshop leaders, and
agreed upon by the participants.</p>
        <p>Decision makers who do not directly interact with the system (indirect
stakeholder)
GIS experts who directly interact with the system in order to provide
information to decision makers (direct stakeholder)
Residents of an area with ood risk, representing public (indirect stakeholder)</p>
        <p>For these three stakeholders 31 values were identi ed, 50 concrete situations
and 94 stakeholder needs. This resulted in 94 user stories. Due to space
limitations, we cannot provide all these results here. Instead, we selected two
representative user stories for each stakeholder. Table 1 shows the user stories with
the associated stakeholders, values, concrete situations, and stakeholder needs,
respectively.</p>
        <p>Several interesting observations can be made from the table. The decision
makers value of personal job security can lead to two con icting stakeholder
needs. On the one hand, personal job security may yield a need for metrics
about the uncertainty of information (user story 1), but on the other hand,
it may yield the need to not receive these metrics (user story 2). In the former
case, the decision maker takes responsibility for the interpretation of (processed)
data, and in the latter case, GIS experts take this responsibility. This tension
is related to the GIS experts value of accountability. The more responsibility is
shifted from the decision maker to GIS experts, the more important their need for
information about data and algorithms becomes to account for the information
they delivered to the decision maker (user story 3).</p>
        <p>The table also shows a relation between the values of residents and those of
the decision maker. For residents of a ood area, having information about the
ood risks can increase residents trust in the information provided by decision
makers (user story 5). This is in tension with user story 2, according to which
the decision maker wants information of the type yes or no and safe or unsafe,
instead of information about risk sizes.</p>
        <p>User stories 1, 2, 3 and 5 are all related to the question of how much
information the IQmulus system should provide about risks, uncertainty, and accuracy
of the data it produces. The user stories show the impact of design choices
regarding that question on the values of all three stakeholders. User story 4 and 6
are not direct requirements of the IQmulus system. However, the design of the
IQmulus system may be impacted by the stakeholder needs in these user
stories. For example, the IQmulus system could have a decision support function
that selects central, but safe locations from where voluntary actions could be
coordinated.
4.3</p>
      </sec>
      <sec id="sec-4-3">
        <title>Observations</title>
        <p>The workshop participants generally indicated that they liked the workshop,
that they found it interesting to learn about VSD, and that they thought that
VSD provided a valuable perspective on the use and development of the
IQmulus system, a perspective they had not encountered before. The participants
particularly mentioned that it helped them to re ect on the main aims of the
project. Some of the workshop participants had been involved in organizing user
workshops to obtain requirements for the IQmulus system themselves. They
remarked that the experience and knowledge obtained in these user workshops
helped them a lot in the value-requirement workshop.</p>
        <p>The workshop leaders thought that the value-requirement workshop was
successful in several respects. First, the participants seemed to understand the main
ideas of VSD and what was expected of them in the workshop. Second, the
participants were able to accomplish all steps in the workshop. Third, the workshop
evoked discussions and information exchanges among the participants, and that
seemed to contribute in developing a common view on the goals of the IQmulus
projects. A potential drawback of the technique is that it requires a considerable
amount of time. In this workshop, 13 stakeholders were identi ed, but in the
total duration of the workshop, 4 hours, only 3 stakeholders were analyzed.
5</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>Discussion and conclusions</title>
      <p>In this paper, we introduced our approach to integrate values-oriented techniques
with a requirements engineering process. Our approach helps requirements
engineers and stakeholders identify values and translate them into a form that is
suitable for use in the further requirements engineering process. In a case study,
we demonstrated that our approach yields value stories user stories, a concept
familiar in requirements engineering, that include values.</p>
      <p>Future work should evaluate this approach in a design process, focusing on
its ease of use and usefulness. In particular, evaluations should aim to assess the
extent to which the approach helps designers without experience in dealing with
values identify and understand value issues, and successfully formulate
requirements to address these issues. Furthermore, future work can further support the
integration of values into further requirements engineering activities by
developing ways to formalize values and their relationships with other entities that
currently gure in requirements speci cation.</p>
      <p>Another promising direction for future work is the development of tool
support. Tools could provide functions to support documentation of identi ed
values, value stories, and related concepts, as well as provide means to visualize the
resulting structure. By using such tools across design projects, designers could
build up a reusable knowledge base or design patterns of values, containing the
range of values encountered, as well as the requirements formulated to support
these values.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Friedman</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kahn</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Borning</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Value sensitive design and information systems</article-title>
          .
          <source>Human-Computer Interaction and Management Information Systems: Foundations. ME Sharpe</source>
          , New York (
          <year>2006</year>
          )
          <volume>348</volume>
          {
          <fpage>372</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Flanagan</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Howe</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nissenbaum</surname>
          </string-name>
          , H. In:
          <article-title>Embodying values in technology: Theory and practice</article-title>
          . Cambridge University Press, Cambridge, UK (
          <year>2008</year>
          )
          <volume>322</volume>
          {
          <fpage>353</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Thew</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , Sutcli e, A.:
          <article-title>Investigating the role of 'soft issues' in the re process</article-title>
          .
          <source>16th IEEE International Requirements Engineering</source>
          ,
          <year>2008</year>
          . RE'
          <volume>08</volume>
          (
          <year>2008</year>
          )
          <volume>63</volume>
          {
          <fpage>66</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Koch</surname>
            ,
            <given-names>S.H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Proynova</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Paech</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Wetter</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>How to approximate users' values while preserving privacy: experiences with using attitudes towards work tasks as proxies for personal value elicitation</article-title>
          .
          <source>Ethics and information technology 15(1)</source>
          (
          <year>2013</year>
          )
          <volume>45</volume>
          {
          <fpage>61</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Borning</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Friedman</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Davis</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lin</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          :
          <article-title>Informing public deliberation: Value sensitive design of indicators for a large-scale urban simulation</article-title>
          .
          <source>In: ECSCW 2005</source>
          , Springer (
          <year>2005</year>
          )
          <volume>449</volume>
          {
          <fpage>468</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Czeskis</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dermendjieva</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Yapit</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Borning</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Friedman</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gill</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kohno</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Parenting from the pocket: value tensions and technical directions for secure and private parent-teen mobile safety</article-title>
          .
          <source>In: Proceedings of the Sixth Symposium on Usable Privacy and Security. SOUPS '10</source>
          , New York, NY, USA, ACM (
          <year>2010</year>
          )
          <volume>15</volume>
          :
          <fpage>1</fpage>
          {
          <fpage>15</fpage>
          :
          <fpage>15</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Nathan</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Klasnja</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Friedman</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          :
          <article-title>Value scenarios: a technique for envisioning systemic e ects of new technologies</article-title>
          . In: CHI'
          <article-title>07 extended abstracts on Human factors in computing systems</article-title>
          ,
          <source>ACM</source>
          (
          <year>2007</year>
          )
          <volume>2585</volume>
          {
          <fpage>2590</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Miller</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Friedman</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jancke</surname>
          </string-name>
          , G.:
          <article-title>Value tensions in design: the value sensitive design, development, and appropriation of a corporation's groupware system</article-title>
          .
          <source>In: GROUP '07 Proceedings of the 2007 international ACM conference on Supporting group work</source>
          ,
          <source>ACM</source>
          (
          <year>2007</year>
          )
          <volume>281</volume>
          {
          <fpage>290</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Friedman</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hendry</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          :
          <article-title>The envisioning cards: a toolkit for catalyzing humanistic and technical imaginations</article-title>
          .
          <source>In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. CHI '12</source>
          , New York, NY, USA, ACM (
          <year>2012</year>
          )
          <volume>1145</volume>
          {
          <fpage>1148</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          10.
          <string-name>
            <surname>Zowghi</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Coulin</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          :
          <article-title>Requirements elicitation: A survey of techniques, approaches, and tools</article-title>
          .
          <source>In: Engineering and managing software requirements</source>
          . Springer (
          <year>2005</year>
          )
          <volume>19</volume>
          {
          <fpage>46</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          11.
          <string-name>
            <surname>Paetsch</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Eberlein</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Maurer</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          :
          <article-title>Requirements engineering and agile software development</article-title>
          .
          <source>In: 2012 IEEE 21st International Workshop</source>
          on Enabling Technologies:
          <article-title>Infrastructure for Collaborative Enterprises</article-title>
          , IEEE Computer Society (
          <year>2003</year>
          )
          <volume>308</volume>
          {
          <fpage>308</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>12. http://www.iqmulus.eu</mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>