<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Equality Requirements for Software Systems: A Survey</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Maryam Ali Al Hinai</string-name>
          <email>maryam.alhinai@hct.edu.om</email>
          <xref ref-type="aff" rid="aff0">0</xref>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Department of Computer Science University of Bristol Bristol</institution>
          ,
          <addr-line>UK ORCiD: 0000-0001-6293-3445</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Information Technology Department Higher College of Technology Al-Khuwair</institution>
          ,
          <country country="OM">Oman</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>-In order to support social sustainability through software systems, we must learn to engineer requirements that foster sustainability conditions within a society. Yet, the notion of social sustainability is very complex, encompassing equality, trust, cultural and religious diversity support, community participation and more. In this paper we present results of a survey-based study on the notion of equality within a range of software users. Do diverse users converge to some common views on equality? Does the diversity of users itself influence these perceptions. And can we formulate requirements statements for these perceptions? We explore these questions through analysis of data from 155 (relatively well educated, English speaking, and technologically literate) survey respondents.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Index Terms—Social sustainability; software; requirements
engineering; equality, survey.</p>
    </sec>
    <sec id="sec-2">
      <title>I. INTRODUCTION</title>
      <p>
        A society is socially sustainable if it has a sound basic
framework that supports its members cooperation at low
transaction costs [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. Shared values, equal rights, and community,
religious and cultural interactions are necessary components
of such a framework [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. What this really means, is that
in a socially sustainable society individuals and groups
interact (e.g., trade, borrow and lend, innovate, learn, govern,
and regulate, etc.) with the maximum personal and social
benefit and minimum expenses, lost opportunities (e.g., due
to discrimination or corruption, etc.) or discomfort, i.e., with
minimum transition costs.
      </p>
      <p>
        As software mediates more and more activities in the
modern societies, many software engineering researchers argue
that it also has a key role to play in supporting social
sustainability. Thus, some researchers [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ], [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] argue that values
drive decision making in software engineering. Consequently,
these values would be embodied within software
implementations and would drive the way that software operates and
structures social interactions around itself. Similarly, Becker
et al. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] note that sustainability values should be explicitly
and intentionally engineered as requirements into software
requirements specifications, and subsequently implemented in
the resultant software systems.
      </p>
      <p>
        Indeed, several pieces of research demonstrate how to
integrate environmental sustainability concerns into software
system, e.g., by reducing resource consumption, and fostering
reuse [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ], or improve technical sustainability of software
Ruzanna Chitchyan
assets by increasing their longevity [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Yet, at present very
little work has been published on how to engineer social
sustainability through software [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Given that, as noted above
[
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], social sustainability encompasses a large set of complex
characteristics (e.g., from cultural to religious and community
interactions, governance and trust) tackling this challenge
would require addressing a single social sustainability
characteristic at time. Thus, in this paper we explore the equality
characteristic of social sustainability.
      </p>
      <p>The key contributions of this paper are two-fold, as it
explores:</p>
      <p>If there is an overall agreement across the wider software
user community on what requirements are relevant for
engineering equality through software, and
If demographic characteristics of individuals are
correlated with their perceptions of the relevance of the
software equality requirements.</p>
      <p>To realise the above two contributions, the paper first
discusses related work to equality and its support through
software in section II. Then presents an overview of how a
number of equality requirements statements (which a software
system should enforce) have been identified from several
previously and independently specified software requirements
documents. Using these requirements statements, the present
research has developed a survey instrument, which has been
used for data collection and analysis that help us deliver the
said contributions. The research methodology used in this
survey design and analysis is detailed in section III. The results
are presented in section IV, with threats to validity discussed
in section V. The paper is concluded with section VI.</p>
    </sec>
    <sec id="sec-3">
      <title>II. EQUALITY REQUIREMENTS</title>
      <p>
        Equality is defined as the right for all members in a
society to enjoy living and getting access to services and
facilities without being discriminated because of their origin,
believes, position, or (dis-)abilities [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. It is an internationally
advocated value that several standards and acts endeavour to
instill. For instance: (i) the ISO 26000 [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] aims at helping
organisations to install such principles as gender equality
and fair treatment; (ii) Social Accountability Standard 8000
[
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] states guidelines on child labour, forced or compulsory
labour, health and safety, freedom of association and collective
bargaining, discrimination and alike; (iii) Equality Act 2010
[
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] aims at reducing socioeconomic inequalities, harmonising
equality laws and tackling discrimination and harassment due
to individuals’ personal characteristics.
      </p>
      <p>
        Equality is often considered from two seemingly
contradictory positions:
one (let’s call it equal equality) taking the viewpoint that
since all members of a society should be equal, they
must all receive the same treatment and equal access to
resources [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. If anyone is not treated equally, it raises
unfairness complaints.
the other (which we’ll call unequal equality)
interpretation [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] suggests that since members of society
differ, equality means providing different treatments to
accommodate the diversities of groups and individuals.
Here, “... failing to provide different treatment is itself
unequal because of the unequal social and economic
position of different groups. . . . in this position, diversity
enhances equality by ensuring that unequals are treated
differently and unequally.”
      </p>
      <p>
        A number of researchers take the equal equality view and
discuss equality in terms of equal resource allocation e.g.,
for equal water [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ] and housing [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ] resource distribution
between members of society. They stand for removing race,
age, religious [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] and gender [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ], [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] discrimination, as
well as inequalities due to geographical location of society
members. The infrastructural access inequalities [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] could be
reduced by enabling equal access to electronic resources [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ],
as well as improving access to community (e.g. child care)
and social (e.g. cultural events) infrastructure [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ].
      </p>
      <p>
        When ICT is concerned, Web Content Accessibility
Guidelines (WCAG) [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] aim to ensure that web content is
accessible, supports equal access and opportunity for people
with disabilities [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], older users, people in rural areas and
developing countries.
      </p>
      <p>
        Those who advocate the unequal equality, argue that equal
distribution does not always entail fairness and it is fair, rather
than equal distribution and access to resources that is essential
for social sustainability [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. Thus, those with greatest need
should be provided with more resources (e.g., babies should
get more milk than adults), those who invest more should get
more back [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] (e.g., if one individual works twice as hard as
the other, she should get twice as much pay), etc.
      </p>
      <p>While each of the above equality-related viewpoints have
valid philosophical grounds, we, as software engineering
researchers, are interested in establishing if there are common
requirements that could be used for engineering equality through
software systems. Thus, we adopted a pragmatic approach
aimed at identifying equality requirements statements relevant
to software systems, as detailed in the following section.</p>
    </sec>
    <sec id="sec-4">
      <title>III. RESEARCH METHODOLOGY</title>
      <p>
        In order to elicit how the software users perceive equality
requirements for software systems, we chose to collect data
through a survey [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]. We followed survey design and analysis
guidelines devised by Kitchenham and Pfleeger [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ] and Van
Selem [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ].
      </p>
      <p>The key objectives of this study are three-fold:
to discern a prioritised ranking of requirements related
to equality, as perceived by the general software user
community;
to investigate the effect of such demographic factors as
users’ expertise, education, gender and religion on their
perception of given equality requirements;
to observe if there is an overall agreement on what
constitutes equality within the general software user
community.</p>
      <sec id="sec-4-1">
        <title>A. Eliciting Equality Requirements Statements</title>
        <p>
          To construct a survey, we first needed to identify equality
requirements statements relevant to software systems. For this
we applied the equality requirements pattern and template
proposed in [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ] on software systems requirements specification
documents1 which were developed previously by independent
requirements engineers from across a variety of application
domains. Most of these documents were selected as they have
been previously used as benchmarks in other requirements and
software engineering-related studies by other researchers. The
documents are sourced from reputable research or software
practitioner organisations, are written in English and well
structured.
        </p>
        <p>
          A number of recurring equality-related requirements
statements were identified. These are general statements that are
applicable to different software system. This was done to
avoid users’ familiarity problem with a specific software
system or domain. It is worth noting that we do not claim
that the statements listed in Table I are the complete and
full requirements for the realisation of equality concern; this
simply is a relevant sub-set obtained through application of the
equality value patterns method [
          <xref ref-type="bibr" rid="ref24">24</xref>
          ] to a set of requirements
documents.
        </p>
        <p>These requirements statements were combined with a few
additional statements, that do not directly relate to equality
(as presented in Table I and discussed in section III-B) and
were used for a survey-based study to observe the perceptions
of the software users on equality and relevance of equality
requirements.</p>
      </sec>
      <sec id="sec-4-2">
        <title>B. Survey Design</title>
        <p>
          A cross-sectional survey instrument was designed for the
data collection purposes [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ]. As we were interested in general
1These documents are: [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], [
          <xref ref-type="bibr" rid="ref16">16</xref>
          ], [
          <xref ref-type="bibr" rid="ref25">25</xref>
          ], [
          <xref ref-type="bibr" rid="ref26">26</xref>
          ], [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ], [
          <xref ref-type="bibr" rid="ref27">27</xref>
          ], [
          <xref ref-type="bibr" rid="ref28">28</xref>
          ], [
          <xref ref-type="bibr" rid="ref29">29</xref>
          ], [
          <xref ref-type="bibr" rid="ref30">30</xref>
          ],
[
          <xref ref-type="bibr" rid="ref31">31</xref>
          ], [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]
software users’ perspectives, with varying demographics
factors (including religion), we chose to utilise an online survey
[
          <xref ref-type="bibr" rid="ref22">22</xref>
          ] format (to fill it the respondents will clearly need to
use software), which was distributed widely through different
communities and lists.
        </p>
        <p>The online survey comprised an information sheet (with a
participation consent form to be confirmed by the
respondents), followed by three self-administered data collection
sections.</p>
        <p>Section 1 of the survey consisted of 21 previously selected
requirements statements (as discussed in section III-A and
summarised in Table I). Here the respondents were asked to
evaluate the importance of each of the given statements for
its’ relevance as a software system’s equality requirement.
The relevance was indicated as irrelevant (NAI) (see Table I,
for statements that do not relate to equality), neither relevant
nor irrelevant (Table I, NINU) to indicate the respondent’s
indecision or lack of knowledge on the relevance of a given
statement, and relevant, where relevance was subcategories
into slightly relevant (SI), relevant (IE), and very relevant
(VIE) options.</p>
        <p>
          Because this is an importance scale, we did not aim to
produce balanced levels. Knowing that a statement is not
important to equality is more relevant than knowing the degree
or depth of unimportance. In the case of unimportance, the
direction is what we are looking for and not the depth. On
the other hand, knowing the degree of importance can help
requirements engineers to prioritise equality requirements and
decide which should be included in the first release and
which to be kept for later releases taking into consideration
resource availability (e.g. time, money, skills, etc.). In this
case, “discrimination . . . between the positive scale positions”
is important [
          <xref ref-type="bibr" rid="ref32">32</xref>
          ].
        </p>
        <p>To identify how well the respondents distinguish the notions
of equality form other requirements relevant to social
sustainability, we mixed into the survey a set of statements that did
not relate to equality. These are statements 1, 4, 5 and 17 in
Table I where:
statement 1 is a security requirement;
statement 4 is a performance requirement;
statement 5 is a robustness requirement;
statement 17 is an availability requirement.</p>
        <p>Section 2 of the survey consisted of two questions aimed
to explicitly elicit the respondents’ notion of equality in terms
of priorities they give to software profitability, usability,
functionality, and equal goal support for various user groups. Here
data was collected in multiple choice options with nominal
scale.</p>
        <p>
          Section 3 of the survey comprised of 6 questions for
collection of demographic information. This section was partly
adopted from the survey by Osho et al. [
          <xref ref-type="bibr" rid="ref33">33</xref>
          ]. Here the questions
aimed to both help characterise the sample of the participating
respondents, and to elicit what effect the demographics have
on the priorities and perceptions of equality. The survey was
concluded with a brief thank you note.
        </p>
        <p>The first version of the survey was piloted with a small
set of participants. The pilot helped to identify and rectify a
number of concerns. Here the wording of the information sheet
was amended, several grammatical mistakes were corrected,
and the demographic information section was moved from the
beginning to the end of the survey. Thereupon, the final version
of the survey was published through a web survey tool2.</p>
      </sec>
      <sec id="sec-4-3">
        <title>C. Population and Respondent Sampling</title>
        <p>
          Since we are interested in the views of the general software
users (i.e., anyone who uses any kind of software), the targeted
population of this study is potentially counted in more than
millions3, with a similarly large numbers of online software
users [
          <xref ref-type="bibr" rid="ref34">34</xref>
          ]. This lead us to choosing sampling methods that
reach a diverse range of respondents and communities. Thus,
the unrestricted sampling [
          <xref ref-type="bibr" rid="ref22">22</xref>
          ] and convenience sampling [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ]
methods were used by publicising the survey questionnaire
through LinkedIn, Instagram, Facebook, Twitter, ResearchGate
and WhatsApp.
        </p>
        <p>
          Additionally, invitation emails were sent to the students and
staff list at the University of Leicester, UK, and, through
academic colleagues, at other counties (such as USA, Brazil,
Germany, and Oman). To encourage varied religious backgrounds
representation, we also explicitly reached out to colleges from
Omani universities through randomly selected staff members
emails available online, asking them to distribute the
participation request locally. As we requested that the colleagues and
respondents forward the participation request to all who they
considered may have been willing to respond to the survey,
we also incorporated elements of snowball sampling (which
is also an unrestricted and convenience sampling method) [
          <xref ref-type="bibr" rid="ref23">23</xref>
          ]
into the data collection.
        </p>
      </sec>
      <sec id="sec-4-4">
        <title>D. Data validation and analysis</title>
        <p>We consider a response valid if all the set questions
have been completed. Clearly, any respondent would have
completed the survey through use of software, and this (i.e.,
software use experience) was the only fundamental qualifying
constraint for participation in this survey. Thus all completed
responses would be valid.</p>
        <p>To analyse the data, we used frequency analysis to describe
the importance of the equality requirements statements as
well as inferential statistical analysis methods to discern the
influence of demographic factors on the equality perceptions.</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>IV. DISCUSSION OF FINDINGS</title>
      <p>Below we first describe the set of our respondents in terms
of their demographic characteristics, then address the above
stated study objectives on equality requirements prioritisation,
perceptions, and influence of the elicited demographic factors
upon these.</p>
      <p>2https://www.onlinesurveys.ac.uk
3http://chrissniderdesign.com/blog/resources/social-media-statistics/</p>
      <p>The respondents’ sample was nearly evenly balanced for
gender, with a slightly higher female participation (by 1.2%),
as illustrated in Table II. The average age of the participants
was 37, with the age characteristic also well distributed in 18
to 64 years old range, but only one over 65 year old participant.
The vast majority of participants were in employment or
education (see Table II). Only 1.9% of the respondents were
novice software users and the majority (45.8%) were of an
advance proficiency (as per Table II).</p>
      <p>The largest group (just over a half) of the respondents report
to be of Muslim background, which could have arisen due
to the previously mentioned more direct invitation of Omani
participants. Christians and Hindus are the next two larger
religious groups (see Figure 1).</p>
      <p>The highest education level of the respondents is rather
biased towards the highly qualified end, with 34.2% holding
PhD and 36.1% Masters degrees (see Table II). There were
no unschooled respondents. Thus, our sample is clearly biased
towards highly educated software users. This, in itself, is not
entirely surprising, as the topic of the survey (software and
equality) as well as method of data collection already presumes
some minimum education and technological literacy levels.</p>
      <sec id="sec-5-1">
        <title>B. Frequency of Equality Statements</title>
        <p>
          The ordering of the responses on the priority of equality
statement for software systems is summarised in Table III and
demonstrated in Figure 2 with a divergent stacked bar [
          <xref ref-type="bibr" rid="ref35">35</xref>
          ].
        </p>
        <p>As Figure 2 shows, statements S10: Multilingual interface,
S14: Software’s usage guidance (e.g., help, tutorials, and
tips), S11: Different information presentation formats (e.g.,
audio, video, text), S20: Different input support formats, and
S3: Support for users across various ages, are the highest
ranked overall relevant stamens (i.e., highest combined slightly
relevant, relevant, and very relevant). All these statements
also have consistently low irrelevance and indecision ranking.
Which clearly indicates that there is an agreement (though
not unanimous) across the respondents that these statements
are closely related to equality.</p>
        <p>Similarly, respondents have consistently marked out the
deliberately introduced unrelated statements (i.e. S1, S4, S5
and S17) as both the least relevant, most irrelevant, and the
most doubted (i.e., neither relevant, nor irrelevant) subset.
However, here too, the respondents were not unanimous in
their ratings, as some respondents have ranked S17 as having
something to do with equality. S17 is a statement on software
availability and may have been perceived as tangentially
relevant to equality by providing users with access at time of their
convenience, without restrictions. Despite these discrepancies,
Background
the overall correct and consistent ranking of the least relevant
statements, indicates that there is a general agreement among
the survey respondents on the statements that do not relate (or
weakly relate) to equality.</p>
        <p>Considering the topic ranking, for the genuinely
equalityrelated statements list (i.e., with exclusion of S1, S4, S5 and
S17) we note that the highest importance is attributed to
statements supporting interaction of the users with the software
(S10, 14, 11, 20), followed by the mixed order of topics
related to user variability support (e.g., age, technical platform
support, etc.). The statements that support stakeholder goals
(S8, 7, 21) come at the bottom part of the list.</p>
        <p>Out of 17 genuine equality-related statements, the least
importantly ranked was S21 (Considering indirect stakeholder
goals) and S15 (availability of softwares shortcuts). This, could
be explained with the fact that the key focus of software is
normally placed on direct software users, with indirect
stakeholders considered thereafter. Clearly, the long-term cumulative
effects of a software system could dramatically affect indirect
stakeholders. For example, the long-term use of Amazon.com
by large number of individuals has gradually undermined
many physical retail shops, and their located environments
and communities (e.g., empty shops in town centres leading to
unattractive social spaces). Yet, normally software users first of
all focus on the direct effect of their immediate interaction with
the software system (e.g., ability to obtain the desired book at
lower price from an online bookstore). This tension between
priorities of direct and indirect stakeholders is, indeed, one of
the characteristics of social sustainability concerns.</p>
        <p>On the other hand, statement S15 might have been rated
least relevant because the extra service of supporting
experienced users was thought of as more related to usability than
equality. It could also be that the extra functionality is viewed
as a privilege given only to expert users. However, having
extra features to for experts does not hinder novice users from
accessing same features in simpler but potentially more time
consuming ways.</p>
      </sec>
      <sec id="sec-5-2">
        <title>C. Equality Goals</title>
        <p>As noted before, the questions in section two were aimed
at understanding what concerns drive the notion of equality
for software systems in the respondents: from profit, to
usability, functionality, and user priorities. While Question2 4
asked to make a choice for the key equality-conducive goals,
Question35 asked which user groups should be supported.</p>
        <p>The results indicate that our respondents are relatively
equally distributed in prioritising support for specific group
goals, provision of same but equal functionality to all users
(without distinction), and simple usability of software as the
key drivers of enabling equality through software.</p>
        <p>For Q2 the largest group of respondents (37.7%) said
that to support equality, the most prioritised goals for each
group should be integrated into software. This suggests that
most respondents consider that equality is supported through
looking at the different stakeholders groups and finding the
best way of allowing them to achieve their goals through the
software system. In Q3 72.4% of this respondent group had
consistently noted that goals of different stakeholder groups
should be implemented to support equality. Thus, this group of
respondents considers equality as equitable support of various
goals of the different user groups for a given software system.</p>
        <p>Another large group of respondents (31.8%) for Q2 said
that to them the best way to enable equality through software
is by providing same functionality at same level to all user
groups. For Q3, the majority (69.4%) of this respondent group
4Statement of Q2: To support equality in a given software application, we
should implement the goals that</p>
        <p>Deliver the highest profit
Provide best usable interface
Provide same level of functionality to all user groups</p>
        <p>Implement the most prioritized goals for each user group.</p>
        <p>5Statement of Question 3: An online shopping software has several groups
of users. Normal users are regular users who use the software to perform
basic shopping through the website. Gold users are those who buy expensive
products that generate high income for the business. Special users are those
who use the application with special request of adjustment to their disabilities
(colour blindness, hearing loss, etc.). Each group of users have different goals
to be implemented by the online shopping application. To support equality
through the online shopping application, which goals will you select to be
implemented in the application.</p>
        <p>The normal users goals The gold users goals
The special users goals None of them
The normal and special users goals
The normal and gold users goals
The special and gold users goals
All of them
also chose that goals of different stakeholder groups should be
supported. Thus, this group considers equality as delivery of
the same service at same level to all user groups for a given
software system.</p>
        <p>The third largest group of respondents (27.9%) for Q2 said
that provision of the best usable interface is the key driver
for equality. In response to Q3, 50.0% of respondents in this
sub-group indicated that goals of all user groups should be
implemented. It could have been expected that those concerned
with the usability of the software would care more about the
users with disabilities (who could face more difficulties with
the software use). However, only 7.1% from this sub-group
indicated the specialist (disadvantaged) user groups as those
who should be especially supported via the software system
to enable equality.</p>
        <p>Furthermore, a small group of respondents (2.6%)
considered profit maximisation as a key direction to enable equality
in Q2, and for Q3 half of this sub-group chose prioritising
requirements of the “gold user” group (i.e., those who buy
expensive packages of service for a given software) as the key
in achieving equality, while the other half chose to support all
user groups.</p>
        <p>In summary, while (as found from Q3 responses) the vast
majority of respondents (64.7%) think that goals of all user
groups should be supported, the perception of the way that
equality should be delivered with software system is varied:
just over one third of survey respondents noted the need to
support goals that each user groups prioritises; another third
focused on equal service level to all - whichever service that
may be and to whomever it is delivered; and the other (slightly
less than) third portion of respondents underlined the usability
of software as key to its support for equality.</p>
      </sec>
      <sec id="sec-5-3">
        <title>D. Impact of Background Factors</title>
        <p>To study what impact background factors may have on
the perception of equality, we consider if and how the
demographic factors affect both equality goal priorities (i.e.,
responses to Q2 and Q3) and ranking of equality-related
statements.</p>
      </sec>
      <sec id="sec-5-4">
        <title>1) Impact on Equality Goal Priority: We start our study of</title>
        <p>the demographic factors impact by formulating a null
hypothesis that the background variables (Q4-9) and equality goals
(Q2 and Q3) variables are independent, with the alternative
hypothesis that they are indeed dependant:</p>
        <p>H0 : Background variables and equality goals variables
are not related.</p>
        <p>Ha : Background variables and equality goals variables
are related.</p>
        <p>
          Given that all variables for respondents background
(questions 4 to 9) and equality goals (questions Q2-3) are
categorical (i.e., nominal and ordinal), chi-square test would
normally be performed [
          <xref ref-type="bibr" rid="ref36">36</xref>
          ] to examine the relationships
between these variables. However, when a cross-tabulation
of these two variables is created, the data contains a high
percentage (more than 20%) of cells with under 5 count.
Consequently, the Fisher’s exact test [
          <xref ref-type="bibr" rid="ref37">37</xref>
          ] is preferred to the
chi-square. Additionally, since the compared variables are at
more than 2 categorical levels (e.g., employment status has
4 options: employed, student, unemployed and retired), the
Fisher-Freeman-Halton Test6 was undertaken.
        </p>
        <p>
          Where the relationships between age factor and Q3’s user
group selection was studied, we utilised the Monte Carlo
method as an alternative to Fisher’s test (to resolve the issues
of high memory intensity posed by the exact tests [
          <xref ref-type="bibr" rid="ref38">38</xref>
          ]).
        </p>
        <p>The test results are detailed in Table IV, from which
we observe that the probability of dependence between the
background variables and goal priorities is negligible for
all demographic characteristics but that of software usage
proficiency. Thus, the above stated H0 hypothesis has to
be accepted for the majority of demographic characteristics
(where p-value is over 0.05 as per Table IV), but rejected for
the software usage proficiency factor.</p>
        <p>Thus, we observe that in our sample of respondents, the
decision on which groups’ goals should be implemented (as
per Q3) in a software system to support equality is related to
respondents proficiency in software usage:</p>
        <p>Half of the novice respondents said that equality would be
achieved by supporting goals of all types of user groups,
but the other half though that support of the “gold users”
goals would lead to better equality.</p>
        <p>The majority of the intermediate, advanced and expert
users considered that goals of all types of groups should
be supported (37.0%, 70.4% and 71.7% respectively).
This could be explained by more experienced users having had
more opportunities to participate in software system use as part
of different groups (e.g., service provider or consumer (e.g.,
seller and buyer), viewer, support team member, developer, or
an administrator, etc.). We expect that such an experience of
broader exposure would, naturally, underline the relevance of
multiple groups of system users (though presently we do not
have sufficient data to substantiated this causality opinion).</p>
        <p>2) Impact on Statement Rating: Inference analysis for the
role of the background factors on the rating of the statements
not-related to equality (i.e., S1, 4, 5, 15, 17) shows that the
rating of the statements 1, 17 and 15 is not affected by the
background factors of the respondents. However, we observe
a relationship between religion of the respondents and their
rating of the statements for S4: Response time and S 5:
Recovery time (at p-value = 0.047 which is under 0.05 for
S4 and p-value = 0.008, which is under 0.01 for S5).</p>
        <p>Many Muslims participants (31.0%) rated statement 4 as
very important to equality. Many Christian (24.1% ) and
Hindu (44.4%) participants found it important to equality,
and a large number of those with other beliefs (38.5%) rated
this statement as slightly important to software equality.
Finally, many of the respondents who did not disclose their
6http://www-01.ibm.com/support/docview.wss?uid=swg21479647
Background factor
religious beliefs (44.4%) rated the statement as neither
important nor unimportant.</p>
        <p>For statement 5, many Christian (34.5%) and Muslim
(36.1%) participants found it very important, and most Hindu
participants (38.9%) rated it as important to equality .
Participants with other religious backgrounds scored a tie
between neither important nor unimportant and slightly
important to software equality with 23.1%. While 33.3%
of those who preferred not to disclose their religion rated
statement 5 as not important.</p>
        <p>
          We observe that both of these statements are related to
time, and previous research has demonstrated that there is a
correlation between time valuation and cultural factors [
          <xref ref-type="bibr" rid="ref39">39</xref>
          ]
[
          <xref ref-type="bibr" rid="ref40">40</xref>
          ]. Thus, it is likely that this relationship is a demonstration
of such a cultural, time-related influence.
        </p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>V. THREATS TO VALIDITY To discuss validity of findings, we consider internal and external validity factors.</title>
      <sec id="sec-6-1">
        <title>A. Internal Validity</title>
        <p>The study design has a large role to play in ensuring that
the results correctly convey the information contained in the
study data. With this respect, internal validity of this study
could be threatened if the statements in Q1 and goals in
Q2 are poorly related to the equality concern. Though this
threat cannot be fully eliminated we have mitigated it as
we have ensured that the equality statements and goals are
representative and are closely related to concerns expressed in
requirements specifications from several independently defined
software system requirements documents.</p>
        <p>Although we discussed in Section III-B the reasons of
having an unbalanced scale, we note that this could pose a
threat to validity if it were to mislead respondents.</p>
        <p>We have also used English language survey, which has been
filled in by participants from other countries, who are likely to
be non-native English speakers. Thus, it is possible that some
respondents may interpreted some of the statements differently
that they were intended. Yet, as English is the most widely
used language for academic research and publications, and the
participants would have had access to transition support (e.g.,
via paper and online dictionaries, and translators), we believe
this was a reasonable choice to make. In addition, we carried
out pilot for the data collection to improve both questions and
statements wording and clarity, as well as structure.</p>
      </sec>
      <sec id="sec-6-2">
        <title>B. External Validity</title>
        <p>
          A threat to external validity can arise if the respondents are
not representative of the population [
          <xref ref-type="bibr" rid="ref41">41</xref>
          ]. This could have been
influenced by the used sampling methods [
          <xref ref-type="bibr" rid="ref41">41</xref>
          ] (see section
III-C). To ensure that our respondents were representative of
different countries, religions, and ages, we posted the survey
request internationally and across various age-groups.
        </p>
        <p>Yet, since we used distribution methods (e.g., LinkedIn,
Research Gate, etc.) accessible to us, it is likely that some
populations with very different views and software use
methods would not have been reached. Indeed, we have reported
that the respondents are rather over-educated compared to
the expected average set of software users, as the request for
participation was widely posted through university lists and
personal requests to university academics.</p>
        <p>Moreover, due to the used distribution and data collection
format, it was not possible to calculate the response rate.
Our respondents are those who volunteered to participant due
to some personal interest; clearly not all who got/read the
participation requested have completed the survey.</p>
      </sec>
      <sec id="sec-6-3">
        <title>Thus, we must note that the generality of results presented in this paper relate to the sub-section of well educated, English speaking, and technologically literate software users.</title>
      </sec>
    </sec>
    <sec id="sec-7">
      <title>VI. DISCUSSION AND CONCLUDING REMARKS</title>
      <p>In this paper we presented a survey-based study on
perceptions of software users on equality. We investigated if the wider
community of software users has a set of “generally agreed
upon” equality goals and equality requirements ratings, that
should be supported via software. We also looked at whether
users’ background characteristics affect they perceptions of
equality.</p>
      <p>The sample of the respondents to our survey is
somewhat skewed, as the respondents are not equally distributed
throughout the world regions, and are mainly well educated,
English speaking, and technologically literate software users.
The findings here are thus related to this particular sample. Our
respondents demonstrated a clear and nearly equal split in their
perception of equality goals: roughly a third of them considers
equality in terms of equal distribution (what we called equal
equality in previous discussion), another third perceives it in
terms of unequal equality whereby each group needs to be
treated differently by supporting their own priorities and needs
to achieve their own goals. Finally, the last third perceives
software equality in terms of more narrowly scoped set of
accessibility requirements. While all these topics form part of
the present equality discourse, this nearly equal split of
priorities was somewhat unexpected. Furthermore, the majority
of respondents also indicated that, in order to be conducive
for equality, a software system should support the whole wide
variety of its user groups.</p>
      <p>For the software engineering professionals this means that
in order to engineer a software system that is perceived to
be conducive to equality characteristic of social sustainability,
that system must:
not only provide accessibility support to all its user
groups,
but also support unequal equality, i.e., to ensure that more
support is provided to those who need it more, more
reward is provided to those who contribute more (i.e.,
groups are positively differentiated with respect to the
goals they want to achieve and support that they need),
yet the equal equality is also observed, whereby
despite the differentiated stakeholder goals, all access to
resources and services are perceived to be equal and fair.</p>
      <p>This, clearly is not an easy task to accomplish, but if either
is not upheld, at least one third of the potential user community
is likely to be disappointed.</p>
      <p>Furthermore, we observe that though there is no unanimous
agreement on which requirement statements are most
important for equality, there is a general convergence of views that
accessibility notions are paramount, closely followed by user
diversity support requirements, and then differentiated goal
support requirements.</p>
      <p>There is also a generic convergence amongst the software
users around the notions that are not relevant to equality.
Though demographic factors (such a religion and user’s
experience) affect these perceptions significantly.</p>
    </sec>
    <sec id="sec-8">
      <title>ACKNOWLEDGEMENTS</title>
      <p>This research is partially funded by the UK EPSRC
Refactoring Energy Systems fellowship (EP/R007373/1).</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>R.</given-names>
            <surname>Goodland</surname>
          </string-name>
          , “
          <article-title>Sustainability: Human, social, economic</article-title>
          and environmental,”
          <source>Encyclopedia of Global Environmental Change</source>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>B.</given-names>
            <surname>Barn</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Barn</surname>
          </string-name>
          , and
          <string-name>
            <given-names>F.</given-names>
            <surname>Raimondi</surname>
          </string-name>
          , “
          <article-title>On the role of value sensitive concerns in software engineering practice</article-title>
          ,”
          <source>in Proceedings of the 37th International Conference on Software Engineering - Volume</source>
          <volume>2</volume>
          , ser.
          <source>ICSE '15</source>
          .
          <string-name>
            <surname>Piscataway</surname>
          </string-name>
          , NJ, USA: IEEE Press,
          <year>2015</year>
          , pp.
          <fpage>497</fpage>
          -
          <lpage>500</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Ferrario</surname>
          </string-name>
          ,
          <string-name>
            <given-names>W.</given-names>
            <surname>Simm</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Forshaw</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gradinar</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. T.</given-names>
            <surname>Smith</surname>
          </string-name>
          ,
          <string-name>
            <surname>and I. Smith</surname>
          </string-name>
          , “
          <article-title>Values-first se: Research principles in practice</article-title>
          ,”
          <source>in Proceedings of the 38th International Conference on Software Engineering Companion</source>
          , ser.
          <source>ICSE '16</source>
          . New York, NY, USA: ACM,
          <year>2016</year>
          , pp.
          <fpage>553</fpage>
          -
          <lpage>562</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>C.</given-names>
            <surname>Becker</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Betz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Chitchyan</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Duboc</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. M.</given-names>
            <surname>Easterbrook</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Penzenstadler</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Seyff</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C. C.</given-names>
            <surname>Venters</surname>
          </string-name>
          , “Requirements: The key to sustainability,
          <source>” IEEE Software</source>
          , vol.
          <volume>33</volume>
          , no.
          <issue>1</issue>
          , pp.
          <fpage>56</fpage>
          -
          <lpage>65</lpage>
          ,
          <year>Jan 2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>M.</given-names>
            <surname>Mahaux</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Heymans</surname>
          </string-name>
          , and G. Saval, “
          <article-title>Discovering sustainability requirements: An experience report</article-title>
          ,” in Requirements Engineering: Foundation for Software Quality: 17th International Working Conference,
          <string-name>
            <surname>REFSQ</surname>
          </string-name>
          <year>2011</year>
          , Essen, Germany, March 28-30,
          <year>2011</year>
          . Proceedings,
          <string-name>
            <given-names>D.</given-names>
            <surname>Berry</surname>
          </string-name>
          and
          <string-name>
            <given-names>X.</given-names>
            <surname>Franch</surname>
          </string-name>
          , Eds. Springer,
          <year>2011</year>
          , pp.
          <fpage>19</fpage>
          -
          <lpage>33</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>J.</given-names>
            <surname>Cabot</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Easterbrook</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Horkoff</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Lessard</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Liaskos</surname>
          </string-name>
          , and
          <string-name>
            <given-names>J.- N.</given-names>
            <surname>Mazo</surname>
          </string-name>
          <article-title>´n, “Integrating sustainability in decision-making processes: A modelling strategy</article-title>
          ,” in Software Engineering-Companion Volume,
          <year>2009</year>
          . ICSE-Companion
          <year>2009</year>
          . 31st International Conference on. IEEE,
          <year>2009</year>
          , pp.
          <fpage>207</fpage>
          -
          <lpage>210</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>C.</given-names>
            <surname>Becker</surname>
          </string-name>
          , “
          <article-title>Sustainability and longevity: Two sides of the same quality?</article-title>
          ”
          <source>in Proceedings of the Third International Workshop on Requirements Engineering for Sustainable Systems, RE4SuSy</source>
          <year>2014</year>
          ,
          <article-title>co-located with 22nd International Conference on Requirements Engineering (RE</article-title>
          <year>2014</year>
          ), Karlskrona, Sweden,
          <year>August 26</year>
          ,
          <year>2014</year>
          .,
          <year>2014</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>T.</given-names>
            <surname>Johann</surname>
          </string-name>
          and
          <string-name>
            <given-names>W.</given-names>
            <surname>Maalej</surname>
          </string-name>
          , “
          <article-title>Position paper: The social dimension of sustainability in requirements engineering</article-title>
          ,”
          <source>in Proceedings of the 2nd International Workshop on Requirements Engineering for Sustainable Systems. Citeseer</source>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9] “Understanding equality,” accessed:
          <fpage>06</fpage>
          -
          <lpage>08</lpage>
          -
          <year>2017</year>
          . [Online]. Available: http://www.equalityhumanrights.com/private-and
          <article-title>-public-sectorguidance/education-providers/secondary-education-resources/usefulinformation/understanding-equality</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          <source>[10] “ Guidance on social responsibility ISO</source>
          <volume>26000</volume>
          ,” Switzerland,
          <year>2010</year>
          , accessed:
          <fpage>04</fpage>
          -
          <lpage>08</lpage>
          -
          <year>2017</year>
          . [Online]. Available: http://www.cnis.gov.cn/wzgg/201405/P020140512224950899020.pdf
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11] “
          <article-title>Social accountability 8000 international standard</article-title>
          ,” New York,
          <year>June 2014</year>
          , accessed:
          <fpage>04</fpage>
          -
          <lpage>08</lpage>
          -
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12] “Equality act
          <year>2010</year>
          c.
          <volume>15</volume>
          ,”
          <year>2010</year>
          , accessed:
          <fpage>04</fpage>
          -
          <lpage>08</lpage>
          -
          <year>2017</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>M. O B</surname>
          </string-name>
          <article-title>´rien, “Equality and fairness: Linking social justice and social work practice</article-title>
          ,
          <source>” Journal of Social Work</source>
          , vol.
          <volume>11</volume>
          , no.
          <issue>2</issue>
          , pp.
          <fpage>143</fpage>
          -
          <lpage>158</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>M. G.</given-names>
            <surname>Bos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. H.</given-names>
            <surname>Murray-Rust</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D. J.</given-names>
            <surname>Merrey</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H. G.</given-names>
            <surname>Johnson</surname>
          </string-name>
          , and W. B.
          <string-name>
            <surname>Snellen</surname>
          </string-name>
          , “
          <article-title>Methodologies for assessing performance of irrigation and drainage management</article-title>
          ,
          <source>” Irrigation and Drainage Systems</source>
          , vol.
          <volume>7</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>231</fpage>
          -
          <lpage>261</lpage>
          ,
          <year>Dec 1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>C.</given-names>
            <surname>Landorf</surname>
          </string-name>
          , “
          <article-title>Evaluating social sustainability in historic urban environments</article-title>
          ,”
          <source>International Journal of Heritage Studies</source>
          , vol.
          <volume>17</volume>
          , no.
          <issue>5</issue>
          , pp.
          <fpage>463</fpage>
          -
          <lpage>477</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>A.</given-names>
            <surname>Chib</surname>
          </string-name>
          and
          <string-name>
            <given-names>A.</given-names>
            <surname>Komathi</surname>
          </string-name>
          , “
          <article-title>Extending the technology-communitymanagement model to disaster recovery: Assessing vulnerability in rural asia,” in Information and Communication Technologies and Development (ICTD</article-title>
          ),
          <source>2009 International Conference on, April</source>
          <year>2009</year>
          , pp.
          <fpage>328</fpage>
          -
          <lpage>336</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>P. R.</given-names>
            <surname>Walsh</surname>
          </string-name>
          , “
          <article-title>Creating a “values” chain for sustainable development in developing nations: where maslow meets porter</article-title>
          ,
          <source>” Environment, Development and Sustainability</source>
          , vol.
          <volume>13</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>789</fpage>
          -
          <lpage>805</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>M.</given-names>
            <surname>Galinium</surname>
          </string-name>
          ,
          <string-name>
            <given-names>I. P.</given-names>
            <surname>Defindal</surname>
          </string-name>
          ,
          <string-name>
            <surname>and I. Melissa</surname>
          </string-name>
          , “
          <article-title>E-learning system introduction: equality in education for teachers in rural area of indonesia</article-title>
          ,” in 2012 Ninth International Conference on Computer Science and Software
          <string-name>
            <surname>Engineering</surname>
          </string-name>
          (JCSSE),
          <source>May</source>
          <year>2012</year>
          , pp.
          <fpage>201</fpage>
          -
          <lpage>206</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <fpage>W3C</fpage>
          , “
          <article-title>Web content accessibility guidelines (wcag) 2</article-title>
          .0,”
          <year>2017</year>
          , working draft, Accessed:
          <fpage>03</fpage>
          -
          <lpage>08</lpage>
          -
          <year>2017</year>
          . [Online]. Available: http://www.w3.org/TR/WCAG21/
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20] --, “Accessibility,”
          <year>2016</year>
          , accessed:
          <fpage>03</fpage>
          -
          <lpage>08</lpage>
          -
          <year>2017</year>
          . [Online]. Available: http://www.w3.org/standards/webdesign/accessibility
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>D. R.</given-names>
            <surname>Forsyth</surname>
          </string-name>
          , “Conflict,” Group Dynamics, pp.
          <fpage>388</fpage>
          -
          <lpage>389</lpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>M. Van Selm</surname>
            and
            <given-names>N. W.</given-names>
          </string-name>
          <string-name>
            <surname>Jankowski</surname>
          </string-name>
          , “Conducting online surveys,
          <source>” Quality and Quantity</source>
          , vol.
          <volume>40</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>435</fpage>
          -
          <lpage>456</lpage>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>B. A.</given-names>
            <surname>Kitchenham</surname>
          </string-name>
          and
          <string-name>
            <given-names>S. L.</given-names>
            <surname>Pfleeger</surname>
          </string-name>
          , “
          <article-title>Personal opinion surveys,” in Guide to Advanced Empirical Software Engineering</article-title>
          . Springer,
          <year>2008</year>
          , pp.
          <fpage>63</fpage>
          -
          <lpage>92</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>M. Al</given-names>
            <surname>Hinai</surname>
          </string-name>
          and
          <string-name>
            <given-names>R.</given-names>
            <surname>Chitchyan</surname>
          </string-name>
          , “
          <article-title>Engineering requirements for social sustainability,” in ICT for Sustainability 2016</article-title>
          . Atlantis Press,
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <given-names>L.</given-names>
            <surname>Zhen</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Cao</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Wei</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O.</given-names>
            <surname>Dilly</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Liu</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Li</surname>
          </string-name>
          ,
          <string-name>
            <given-names>H.</given-names>
            <surname>Koenig</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Tscherning</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Helming</surname>
          </string-name>
          , “
          <article-title>Comparison of sustainability issues in two sensitive areas of china,” Environmental Science and Policy</article-title>
          , vol.
          <volume>12</volume>
          , no.
          <issue>8</issue>
          , pp.
          <fpage>1153</fpage>
          -
          <lpage>1167</lpage>
          ,
          <year>2009</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>R.</given-names>
            <surname>Bijl</surname>
          </string-name>
          , “
          <article-title>Never waste a good crisis: Towards social sustainable development</article-title>
          ,
          <source>” Social Indicators Research</source>
          , vol.
          <volume>102</volume>
          , no.
          <issue>1</issue>
          , pp.
          <fpage>157</fpage>
          -
          <lpage>168</lpage>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>D.</given-names>
            <surname>Sharma</surname>
          </string-name>
          and
          <string-name>
            <given-names>S.</given-names>
            <surname>Shardendu</surname>
          </string-name>
          , “
          <article-title>Assessing farm-level agricultural sustainability over a 60-year period in rural eastern india,” The Environmentalist</article-title>
          , vol.
          <volume>31</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>325</fpage>
          -
          <lpage>337</lpage>
          ,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>K.</given-names>
            <surname>Murphy</surname>
          </string-name>
          , “
          <article-title>The social pillar of sustainable development: a literature review and framework for policy analysis</article-title>
          ,
          <source>” Sustainability: Science, Practice, &amp; Policy</source>
          , vol.
          <volume>8</volume>
          , no.
          <issue>1</issue>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>R.</given-names>
            <surname>Jitpakdee</surname>
          </string-name>
          and
          <string-name>
            <given-names>G. B.</given-names>
            <surname>Thapa</surname>
          </string-name>
          , “
          <article-title>Sustainability analysis of ecotourism on yao noi island</article-title>
          , thailand,”
          <source>Asia Pacific Journal of Tourism Research</source>
          , vol.
          <volume>17</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>301</fpage>
          -
          <lpage>325</lpage>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <given-names>S. A.</given-names>
            <surname>Hosseinijou</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Mansour</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M. A.</given-names>
            <surname>Shirazi</surname>
          </string-name>
          , “
          <article-title>Social life cycle assessment for material selection: a case study of building materials,”</article-title>
          <source>The International Journal of Life Cycle Assessment</source>
          , vol.
          <volume>19</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>620</fpage>
          -
          <lpage>645</lpage>
          ,
          <year>2013</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <given-names>K. R.</given-names>
            <surname>Jones</surname>
          </string-name>
          and
          <string-name>
            <given-names>L.</given-names>
            <surname>Mucha</surname>
          </string-name>
          , “
          <article-title>Sustainability assessment and reporting for nonprofit organizations: Accountability “for the public good”</article-title>
          ,” VOLUNTAS:
          <source>International Journal of Voluntary and Nonprofit Organizations</source>
          , vol.
          <volume>25</volume>
          , no.
          <issue>6</issue>
          , pp.
          <fpage>1465</fpage>
          -
          <lpage>1482</lpage>
          ,
          <year>Dec 2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <surname>I. Brace</surname>
          </string-name>
          ,
          <article-title>Questionnaire design: How to plan, structure and write survey material for effective market research</article-title>
          . Kogan Page Publishers,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <given-names>O.</given-names>
            <surname>Osho</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V. L.</given-names>
            <surname>Yisa</surname>
          </string-name>
          , and
          <string-name>
            <given-names>O. J.</given-names>
            <surname>Jebutu</surname>
          </string-name>
          , “
          <article-title>E-voting in nigeria: A survey of voters' perception of security and other trust factors,” in Cyberspace (CYBER-</article-title>
          <string-name>
            <surname>Abuja</surname>
            <given-names>)</given-names>
          </string-name>
          ,
          <source>2015 International Conference on. IEEE</source>
          ,
          <year>2015</year>
          , pp.
          <fpage>202</fpage>
          -
          <lpage>211</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <given-names>A.</given-names>
            <surname>Lenhart</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Purcell</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Smith</surname>
          </string-name>
          , and
          <string-name>
            <given-names>K.</given-names>
            <surname>Zickuhr</surname>
          </string-name>
          , “
          <article-title>Social media &amp; mobile internet use among teens and young adults</article-title>
          .
          <source>millennials.” Pew internet &amp; American life project</source>
          ,
          <year>2010</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <surname>R. M. Heiberger</surname>
            ,
            <given-names>N. B.</given-names>
          </string-name>
          <string-name>
            <surname>Robbins</surname>
          </string-name>
          et al.,
          <article-title>“Design of diverging stacked bar charts for likert scales</article-title>
          and other applications,
          <source>” Journal of Statistical Software</source>
          , vol.
          <volume>57</volume>
          , no.
          <issue>5</issue>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>32</lpage>
          ,
          <year>2014</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <given-names>R. A.</given-names>
            <surname>Hanneman</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A. J.</given-names>
            <surname>Kposowa</surname>
          </string-name>
          , and
          <string-name>
            <given-names>M. D.</given-names>
            <surname>Riddle</surname>
          </string-name>
          , Basic Statistics for Social Research. John Wiley and Sons,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [37]
          <string-name>
            <given-names>G. H.</given-names>
            <surname>Freeman</surname>
          </string-name>
          and
          <string-name>
            <given-names>J. H.</given-names>
            <surname>Halton</surname>
          </string-name>
          , “
          <article-title>Note on an exact treatment of contingency, goodness of fit and other problems of significance,” Biometrika</article-title>
          , vol.
          <volume>38</volume>
          , no.
          <issue>1</issue>
          /2, pp.
          <fpage>141</fpage>
          -
          <lpage>149</lpage>
          ,
          <year>1951</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          [38]
          <string-name>
            <given-names>C. R.</given-names>
            <surname>Mehta</surname>
          </string-name>
          and
          <string-name>
            <given-names>N. R.</given-names>
            <surname>Patel</surname>
          </string-name>
          , “
          <article-title>Ibm spss exact tests</article-title>
          ,
          <source>” IBM Corporation</source>
          , Cambridge, MA,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          [39]
          <string-name>
            <given-names>K.</given-names>
            <surname>Ishii</surname>
          </string-name>
          ,
          <string-name>
            <given-names>C.</given-names>
            <surname>Eisen</surname>
          </string-name>
          , and
          <string-name>
            <given-names>H.</given-names>
            <surname>Hitokoto</surname>
          </string-name>
          , “
          <article-title>The effects of social status and culture on delay discounting,”</article-title>
          <source>Japanese Psychological Research</source>
          , vol.
          <volume>59</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>230</fpage>
          -
          <lpage>237</lpage>
          ,
          <year>2017</year>
          . [Online]. Available: http://dx.doi.org/10.1111/jpr.12154
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          [40]
          <string-name>
            <given-names>W.</given-names>
            <surname>Du</surname>
          </string-name>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Green</surname>
          </string-name>
          ,
          <string-name>
            <given-names>and J.</given-names>
            <surname>Myerson</surname>
          </string-name>
          , “
          <article-title>Cross-cultural comparisons of discounting delayed and probabilistic rewards,” The Psychological Record</article-title>
          , vol.
          <volume>52</volume>
          , no.
          <issue>4</issue>
          , pp.
          <fpage>479</fpage>
          -
          <lpage>492</lpage>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref41">
        <mixed-citation>
          [41]
          <string-name>
            <given-names>C.</given-names>
            <surname>Wohlin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Runeson</surname>
          </string-name>
          , M. Ho¨ st,
          <string-name>
            <given-names>M. C.</given-names>
            <surname>Ohlsson</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Regnell</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Wessle</surname>
          </string-name>
          <article-title>´n, Experimentation in software engineering</article-title>
          . Springer Science &amp; Business
          <string-name>
            <surname>Media</surname>
          </string-name>
          ,
          <year>2012</year>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>