Social Sustainability Indicators for Software: Initial Review

Abstract

-Software's social sustainability is an important concern that needs an in-depth investigation. The objective of this paper is to understand what social sustainability is, how it is measured today, and how is social sustainability of a software system evaluated today. We present the initial results of a systematic literature review on these questions. Our findings so far highlight a large gap in work on software sustainability assessment. Index Terms-Social sustainability, software, indicators, systematic literature review, assessment.

I. INTRODUCTION

In all developed (and most developing) countries the public is now heavily dependent on software in nearly all walks of life – from email to e-banking and e-voting. Most agree that software applications have changed and largely improved our lives. However, there is a dark side of the story. With all our information available electronically and most activities moving on-line, individuals and nations alike are at social risks that may include (no name a few): • Cyber-crimes such as child bullying and grooming attacks [ 1 ]; • Eroding privacy, and step-by-step move to “total surveillance” societies; • Social ties degradation.

Thus, software engineers must closely engage with the movement on sustainable development (widely inspired in 1987 [ 2 ]) in order to make software products socially sustainable. A significant effort has been expanded into research on numerous topics related to software sustainability (e.g., data center energy efficiency [ 3 ], energy efficient algorithms [ 4 ] and requirements engineering [ 5 ], etc.). Yet, the issue of software effects on social sustainability has barely been studied.

According to [ 6 ], “Social sustainability means maintaining social capital and preserving the societal communities in their solidarity”. Willis, McKenzie and Harris [ 7, 8 ] defined social sustainability as “a positive and long-term condition within communities and a process within communities that can achieve and maintain that condition”. Sustainable software was described as “software whose direct and indirect negative impacts on economy, society, human beings, and the environment resulting from development, deployment, and usage of the software is minimal and/or has a positive effect on sustainable development” [ 9 ]. However, to the best of our knowledge, presently there are no heuristics or metrics to inform and guide software engineers in assessing the effects of a software system on “social capital” or on “positive … condition within communities…”. In order to produce socially sustainable software, the software engineers need a way of assessing, throughout the development process, the effects that the constructed software will have on social sustainability of its intended users. This paper presents the initial results of our ongoing effort towards development of such metrics – a comprehensive review of research related to social sustainability and software.

In order to construct meaningful metrics to measure how software would affect social sustainability, one must first learn what social sustainability really implies over and above some generic definitions. It is also necessary to study any related metrics (which may already exist), and to identify what social sustainability indicators are considered relevant today. We present the preliminary results of our study of these questions obtained via an (on-going) systematic literature review (SLR).

The rest of the paper is structured as follows: section 2 outlines the sources used and specific questions set in the above mentioned SLR, section 3 describes the findings on research questions and section 4 summarizes the work.

II. SYSTEMATIC LITERATURE REVIEW SET UP

The objective of this study is to understand what social sustainability is, how it is measured today, and what has been published with regards to evaluation of software’s social sustainability effects. To investigate these issues, we formulated the following set of research questions: RQ1: What metrics are used for measuring social sustainability? How are they constructed?

This question aims at exploring how social sustainability has been evaluated and what are the specific metrics used for measuring social sustainability. With this question, we aim at exploring the broader literature on the social sustainability issues, regardless of the area of application – whether related or unrelated to software development. This question also aims to review how social sustainability metrics are built and what their bases are.

RQ2: What are social sustainability indicators?

This question aims to study the finer-grained constituents on which the social sustainability metrics are built, and the ways that these constituents are quantified upon. Furthermore, here we will identify what are the common aspects of social sustainability in each area. We will identify common dimensions/constituents used in various domains and how they are customized to adapt to a specific context or domain. RQ3: What is the role of software in social sustainability?

The intention here is to know what is the relationship and use of software applications within social sustainability domain. This question will be used to look at a set of issues, including:

What social sustainability areas of life and activities does software support and how?

What (if any) challenges related to social sustainability could be expected to be addressed via software? RQ4: What are the indicators of software’s social sustainability?

The objective here is to study how software’s social sustainability is assessed. We are interested in knowing the indicators related specifically to software applications. We are also looking at how similar or different are these indicators to indicators in other domains (e.g., agriculture, etc.).

As sources for SLR we used a number of digital libraries, namely ACM, IEEE, Scopus, Springer Link, Web of Science, Applied Social Sciences Index & Abstracts (ASSIA). These libraries where chosen based on their subject coverage of both computer science and social sciences. ACM and IEEE cover computer science and engineering areas. Social sciences and engineering are covered by Scopus and Springer link libraries. Web of science and ASSIA cover social sciences areas to obtain content on (computer-science domain independent) social sustainability.

To select the articles from the digital libraries, we used a combined search string extracted from the above discussed research questions to assure that we get relevant results [ 10 ]. Although the combined search string (which we arrived at after an initial piloting of several search strings) was customized to each digital library, it always covered the topics of "Social Sustainability" AND (metrics OR indicators OR software).

The results of the search and initial screening for this study are shown in Table 1. The excluded sets of papers were either those with no access to abstracts, or not in English, or found to be not relevant to the research questions (i.e., did not address the topic of social sustainability or had no relation to indicators/metrics for social sustainability). Eighty-eight of accepted papers have then been studied (this is an on-going work).

No. of duplicate removed (so

far) 0 ASSIA Web of Science ACM Springer Link Total

No of results returned 1 79 3 832 1116 1 68 2 310 540

No. of duplicate removed (so

far) 0 29 0 7 45 1 9 2 15 88 The following data was extracted from each studied article: • General admin, i.e.,: title, author(s), source, year • Social sustainability indicator • Social sustainability metric • How social sustainability is supported • Type of study (e.g., case study, rigorous analysis, prototype) • Context of study/domain

III. FINDINGS ON RESEARCH QUESTIONS

As we have noted above, this is an on-going work. However, we are now able to review the answers we have so far obtained to the previously set questions. Though these findings will likely evolve to some degree by the time the full SLR is completed, we have observed that the general set of indicators, metrics, and domains has now well stabilized. In other words, review of additional articles does not tend to significantly change/add to the current set of results.

The current findings that address the set research questions are presented below:

A. Construction of metrics used for measuring social

sustainability (RQ1)

1) Assessment frameworks for Social Sustainability

The most commonly used framework for assessment of social sustainability is the life cycle assessment (LCA). This is a “cradle-to-grave” method of evaluating the inputs, outputs and environmental impacts of a product during all phases of its life cycle [ 11 ]. An example of this is: land consumption and environmental emissions in a case of municipal waste management [ 12 ]. The LCA has been adapted to include such social concerns as labour force, communities’ living standards, cultural heritage, freedom, health and safety, equity and poverty prevention [ 12-21 ].

In [ 22 ], a Social Impact Indicator (SII) is applied. SII is based on LCA and is used to calculate social effects such as human resources and stakeholders participation [ 22 ].

In [ 21 ], the LCA is merged with the Economic Input and Output analyses method (EIO) to form economic input–outputbased life cycle assessment (EIO-LCA). The EIO-LCA used to quantify direct and indirect sustainability impacts of U.S construction industries (e.g. indirect work injuries) [ 21 ].

Vulnerability assessment techniques (VATs) were used in [ 23 ] to assess the social impacts resulted from urban redevelopment projects. This was done by identifying the most vulnerable people then assessing the social negative impacts affecting them [ 23 ]. This approach gives insights to policy makers on areas to consider reducing the negative social effect of the project [ 23 ]. Doloi 2012 presented a framework for social performance assessment of infrastructure projects based on Social Network Analysis (SNA) [ 24 ]. The SNA was utilised to identify groups of stakeholders affected by the project (actors), their degree of influence (relationships between actors) and their specific social needs [ 24 ]. Then, the groups’ satisfaction of needs was measured and the project’s social performance was derived [ 24 ].

In [ 25 ], Maslow’s Hierarchy of needs was suggested to be combined with LCA to develop social sustainability measure for organizational decisions. Organizations can use a specified need to derive a social indicator from it. For example, taking into account health as a basic need, an organization considers improved quality of food and health insurance policies as social indicators [ 25 ]. Companies aiming at more social sustainability shall focus on meeting their employees higher order needs (e.g. equity) while others will focus on satisfying the lower order needs such as food [ 25 ].

2) Metrics Construction Process

Based on the reviewed literature so far, we observe that the common way of constructing metrics or methodology to assess social sustainability starts with identification of general or domain specific sustainability assessment guidelines that have been already published. For example, in [ 26 ], the researchers investigated available higher education and campus sustainability assessments frameworks as a starting point for evaluating Malaysian campuses. Guidelines can be local or international. For instance, in [ 27 ], the researchers based their assessment on the International Hydropower Association (IHA) Sustainability Guidelines to evaluate the sustainability of hydropower project in China.

Moreover, established indexes/indicators of assessments (such as Human development index and Wellbeing Index [ 28 ], Vanclay’s definitional list of ‘‘social impacts’’ [ 20 ], Oregon Benchmarks [ 29 ] and European Commission indicators [ 30 ]) could be used as basis to build upon them the assessment 1 variables or to compare the assessment results against them .

Once the general guidelines are chosen and complemented with domain-specific policies, the assessment methodology is then customized to fit a specific domain and case study. In order to do that, academics’ and stakeholders’ contributions are often involved. This is done through interviews, questionnaires or focus groups [ 14, 16, 23, 26, 28, 29, 31-42 ]. Stakeholders’ participation is also a part of evaluating a project’s sustainability [ 27 ]. In study presented in [ 39 ], for instance, the experts who took a part in a customisation phase were selected based on their contribution to the research on future development of dairy farming. In [ 40, 41 ], stakeholders were 1 Although we cannot use the same methods directly, as these are constructed on bases of extensive country-wide surveys of such indicators as life expectancy at birth, mean years of schooling or gross national income per capita. 2

Child labor (i.e., employment of those under 16 years of age) is commonly considered a bad practice in the West. However, we do note that in some involved in selecting or designing indicators for social themes as the available scientific information was limited or nonexistent.

Once the indicators are selected, metrics are constructed with them. For example, in [ 12 ] the social sustainability of Municipal Solid Waste Management system was evaluated by two indicators: damage to human health and income based community well-being [ 12 ]. The damage to human health was calculated by summing the “factors for mortality (measured as years of life lost—YOLL), severe morbidity and morbidity (measured as years lived disabled—YLD)” [ 12 ]. The income based well-being indicator was calculated using the  potential   employment  opportunities  for  ith  level  (labour  hrs/tonne),   the  rate  of  wages  ($/hour)  of  ith  level.  The  value  of  income   generation  from  indirect  activities  ($/tonne)  and  the    cost   of  living  ($/person),  as  shown  in  Fig  1.    

Another clearly emerging threat from the literature review is the current lack of trust towards the sustainability assessment metrics and methodologies. This, we believe, is caused by the relative immaturity of the filed. Some publications propose to tackle this issue by “developing case study banks to translate experiences of using an indicator” [ 43 ]. This work also notes that such banks will help in “… increasing criteria confidence and value usefulness to potential users … through case studies validation checks which can also assist with improving the indicators to meet a satisfactory degree of ‘accuracy’, and ‘credibility’.” This approach has, in fact, been used by a number of other researchers [ 22, 36, 40 ].

B. Social sustainability indicators (RQ2)

Social sustainability indicators should be relevant to the case under investigation. In [ 40 ], it was expressed that sustainability indicators need to satisfy criteria such as causality and sensitivity. Those criteria are to ensure that the indicators are related to the monitored case and they respond to changes in the studied case [ 40 ].

Social sustainability indicators vary depending on the domain. Based on our literature review, so far we have identified over 600 indicators. Looking at the list of indicators and using the keywords and classifications that paper authors had provided, commonly used indicators were identified (regardless of the domain). At the most abstract level, the indicators are divided into two main categories: Community and Culture and Governance. Indicators under the Community category are directly related to individuals and groups within a given society, their health, education, equality, etc. Culture and Governance indicators are concerned with cultural and political issues of a given society. The aggregated categories are demonstrated in Fig. 1 below.

1) Employment indicator comprises several sub-indicators related to employment statistics and job conditions [ 39 ]. The following indicators are examples of what can be used under this category.

• Number of employed women/ “Share of women in leading positions” [ 33, 37, 44 ] • Number of Full time/part time workers [ 45 ] • Utilization of different working time arrangement [ 45 ] • Compensation [ 46 ] • Job opportunities creation [ 37 ] 2) Health indicators set covers the quality of health services provided to the people [ 37 ], health problems reported to authorities [ 15, 46 ], health risks [ 36 ] and health practices [ 46 ] in the community. Health indicators could be used to assess: • Availability and access to drinking water [ 35 ] • Child mortality rate [ 47 ] • “Percentage of workers with health benefits” [ 18 ] • “Contribution to healthy and safe food” [ 36 ] • “Voluntary health measures taken” [ 37 ] 3) Equity category includes indicators that should reveal equality measures to all people regardless of their age, gender, ethnicity and social status. Examples are: • Income/wealth distribution [ 17, 27 ] • Social inclusion [ 40, 48 ] • Diversity of housing infrastructure [ 48 ] • “Provisions for basic needs of disabled, elderly or children with proper access” [ 32 ] • “Fair competition” [ 14 ] 4) Education indicators are related to education facilities provided to the community. This can include: • Number of persons with higher education than secondary school/number of persons between 20–64 years [ 49 ] • Employees educational level/ Literacy levels [ 28, 36, 48 ] • Offered areas of employee training [ 37 ] • Number of student per teacher [ 50 ] • Supporting Educational Institutions [ 46 ] 5) Security indicators are primarily related to crimes as the examples below suggest.

• Personal crime [ 48 ] • Property crime [ 48 ] • Overall crime [ 29, 34 ] • Vandalism [ 34, 51 ] • Juvenile arrests [ 29 ]

6) Services and facilities indicators focus on availability

and access to services and facilities.

Those indicators can be related to schools [ 32, 52 ], health care services [ 32, 34, 52 ], sports facilities [ 32, 52 ], child care and housing [ 48 ]

7) Resilience indicator is related to the communities’ adaptability to changes [ 23, 32, 48, 53 ].

8) Human rights indicators are concerned with, for instance, child labour, forced labour, and discrimination [ 14, 54 ].

9) Social acceptance of technology indicator evaluates the

community’s readiness to implement or use new technology. Knowledge, perception and fear are used as sub-indicators for social acceptance [ 20 ]. Knowledge estimates what is the public level of knowledge about the technology while perception will assess what they think about it (positive – negative). Fear evaluates what issues/ worries the community has about the technology.

10) Social cohesion group of indicators is related to the ties between the community members and their feel of involvement. Some examples are: • Citizens walkability to places in the local area such as shops and community [ 41, 52 ]. • Citizens empowerment by allowing initiations of community activities and voluntary work [ 48, 52 ] or decision making [ 15, 40 ] • Network [ 40, 48, 54, 55 ] and knowledge sharing [ 14, 36, 37, 56, 57 ] • Visible minorities, tolerance, identity [ 40, 48 ] • Accountability and transparent decision making process [ 48 ] 11) Cultural indicator is concerned with preserving the community’s culture. This can include • Respect on cultural heritage and local wisdom [ 14 ] • Respect on customary right of indigenous people [ 14 ] • Local heritage and listed buildings [54] • Protection of cultural heritage [ 27 ] 12) Political indicator considers governmental laws and peoples’ trust in them [ 28 ]. When a given case evaluates social sustainability of an organization, these indicators focus on the organizational policies and employees’ attitude towards them.

While in this section we have summarized the more than 600 social indicators collected form our study into 12 cohesive categories, we must also note that the social sustainability indicators do not, in fact, always adhere to such a simple, flat hierarchy. In truth, they are often interchangeable and overlapping. We attribute this to the previously discussed metrics and methodology adaptation process (see section III. A.1), whereby the metrics and methodology are always customized to suite the domain and the level of granularity relevant to a given case study. For example, employment can be used as an indicator by itself (or a group of indicators, as suggested above) but it can also be used as a sub-indicator to the community’s equity.

The social indicators can also vary based on external and internal view of an organization [ 15, 22, 40, 42, 46 ]. For instance, looking at a farm’s social sustainability internally means relating the social concerns to its employees and workers [ 40, 42 ]. External social sustainability would mean assessing the community affected by the farm or consumes the farm’s products. In [ 40 ], the external indicators were related to animal welfare and health and landscape management.

C. Role of software in social sustainability (RQ3)

The articles related to the role of software in social sustainability suggest that software is often used to:

1) Promote social sustainability.

For instance, in [ 58 ] a prototype of communication software is presented which is to be used as a communication enabler between virtual teams and virtual organization. The software is to support social sustainability by enhancing the social networks.

2) Design for social sustainability.

For instance, in [ 37 ] software is utilized to provide guidance and reminders to researchers and managers while modelling a biotechnological product. The provided knowledge is about social sustainability issues to be taken into account while designing the product. This will help support the process of decision-making.

3) Educate on social sustainability.

For instance, an educational game is used in a study to educate students on sustainability and social responsibility [ 55 ].

4) Assess social sustainability.

For instance, Assefa and Frostel outline a tool for assessing ecological and economic sustainability of energy technologies [ 20 ]. They discuss social indicators to be included in the tool.

We observe that our search on software and “social sustainability” resulted in much fewer articles than expected. This is particularly surprising as we are well aware of a large body of research conducted in the area of human-computer interaction that focuses on various topics of social sustainability (such as stress, usability, loneliness, etc. [ 59 ]). This issue indicates that thought a large effort has been underway for some time in HCI community to address particular human-computer interaction issues, that work has not yet been consolidated under the umbrella of “social sustainability”. As the next step in this research, it is our intention to further study this issue to better address this research question.

D. Indicators of software’s social sustainability (RQ4)

As for any other product, the social sustainability of software can be considered in its production, use, maintenance, and disposal stages. Below are the findings from our literature review on this topic so far:

1) Social sustainability at production process is considered in [ 60 ], where it is suggested to use “country of origin of a material and the manner in which it was produced (for example through child labor2)” as social sustainability indicator.

2) For software use response time and scalability were used to evaluate software prototype that supports social networks and knowledge sharing between virtual teams [ 58 ]. This work also mentions that evaluating the prototype’s performance includes evaluating “the degree of network congestion, the load on servers, the number of 3D objects to manage, and the complexity of the submitted query.” They added that the database will support data availability in different context and data stability. Response time is a relevant indicator for social sustainability in domains where fast access to information is necessary for equality (e.g., financial markets). In more general context, response time and scalability are more related to the sustainability of software itself as inadequate speed and scalability devalue software and complicate evolution.

Another work [ 57 ] provides “… a theoretical basis for a multi-actor system as a simulation tool for social sustainability”. Here software agents and human simulate a social sustainability model [ 57 ]. For this the software agents must be “…equipped with functions of perception, mobility, learning, communication, and coordination…”[ 57 ]. Such functions can be considered indicators for software agents’ social sustainability. The agents were proposed to simulate human individuals and groups’ behaviour related to knowledge generation, knowledge communication and knowledge use [ 57 ]. Knowledge, perception, learning, communication and coordination functions are social sustainability dimensions/indicators Fig 2.).

At present we have not yet identified any work on social sustainability of software maintenance and disposal. 2

Child labor (i.e., employment of those under 16 years of age) is commonly considered a bad practice in the West. However, we do note that in some countries working in programming or tasks like interview transcription for software requirements, etc. could provide a very good future prospect to the children involved.

Similar to the comment in the preceding sub-section, we have observed that there are much fewer social sustainability indicators discussed for software domain, compared to other domains (such as agriculture and supply chain management). This can be attributed to two factors:

i) On the one hand, the software effects on social sustainability are likely to have been studied for individual social sustainability characteristics (such as access to learning or other electronic resources, connectedness, etc.), without aggregating these characteristics under the overall umbrella of social sustainability.

ii) On the other hand, the social effects of software products, once in use, are often indirect, take long term to surface, and are difficult to discern. These effects are the socalled third-order impacts of ICT [ 61 ] which “… are long term indirect effects on the environment that result from ICT usage, like changing life styles that promote faster economic growth and, at worst, outweigh the formerly achieved savings (rebound effects).”

As noted above, review of work that addresses specific characteristics of social sustainability will be the next step in this work.

IV. CONCLUSION

This paper presents the initial results of our work on (ongoing) systematic literature review on social sustainability, its metrics and indicators and its relation to software. So far we have distilled a general social sustainability assessment framework and assembled over 600 social sustainability indicators which are then aggregated into 12 cohesive groups.

A surprise finding of the SLR so far is that, in the 88 reviewed papers - taken from 5 digital libraries - software has virtually no consideration of the concept of social sustainability. Yet, we are aware of significant work (most particularly in HCI community) that has addressed a number of social sustainability features (such as usability, loneliness, etc.). This SLR has not been able to identify such relevant work since that work has not related to the concept of social sustainability explicitly. Thus, our future work will investigate such specific social sustainability dimensions in relation to software development. Upon completion of the literature review, we will work on construction of social sustainability assessment metrics and guidelines for software development. [54] Weingaertner,

Sustainability:

C. and Learning Å. from

Moberg, Exploring

Perspectives on

Social Urban

References

[1] Rybnicek , M. , R. Poisel , and S. Tjoa. Facebook Watchdog : A Research Agenda for Detecting Online Grooming and Bullying Activities . In Systems, Man, and Cybernetics (SMC) , 2013 IEEE International Conference on. 2013 .

[2] Bruntland , G. , Our common future: The world commission on environment and development , 1987 , Oxford: Oxford University Press.

[3] Garg , S.K. , et al., Environment-conscious scheduling of HPC applications on distributed Cloud-oriented data centers . Journal of Parallel and Distributed Computing , 2011 . 71 ( 6 ): p. 732 - 749 .

[4] Héliot , F. , M.A. Imran , and R. Tafazolli . Near-optimal energyefficient joint resource allocation for multi-hop MIMO-AF systems . 2013 .

[5] Roher , K. and D. Richardson . Sustainability requirement patterns . In Requirements Patterns (RePa) , 2013 IEEE Third International Workshop on . 2013 .

[6] Penzenstadler , B. and H. Femmer , A generic model for sustainability with process- and product-specific instances , in Proceedings of the 2013 workshop on Green in/by software engineering2013 , ACM: Fukuoka, Japan. p. 3 - 8 .

[7] Willis , P. , S. McKenzie, and R. Harris , Introduction: Challenges in Adult and Vocational Education for Social Sustainability , in Rethinking Work and Learning, P. Willis, S. McKenzie, and R. Harris, Editors. 2009 , Springer Netherlands. p. 1 - 9 .

[8] McKenzie , S. , Adult and Vocational Education for Social Sustainability: A New Concept for TVET for Sustainable Development , in Work, Learning and Sustainable Development , J. Fien , R. Maclean , and M.-G. Park, Editors. 2009 , Springer Netherlands. p. 177 - 186 .

[9] Dick , M. , S. Naumann , and N. Kuhn , A Model and Selected Instances of Green and Sustainable Software , in What Kind of Information Society? Governance, Virtuality, Surveillance, Sustainability, Resilience, J. Berleur , M. Hercheui , and L. Hilty, Editors. 2010 , Springer Berlin Heidelberg. p. 248 - 259 .

[10] Software Engineering Group, S.o.C.S.a.m. , Keele University, et al., Guidelines for performing Systematic Literature Reviews in Software Engineering . 2007 .

[11] Handbook of Life Cycle Assessment : Operation Guide to ISO Standards. 2002 , Secaucus, NJ, USA: Kluwer Academic Publishers.

[12] Menikpura , S.N.M. , et al., Evaluation of the Effect of Recycling on Sustainability of Municipal Solid Waste Management in Thailand . Waste and Biomass Valorization , 2012 . 4 ( 2 ): p. 237 - 257 .

[13] Ekvall , T. , Nations in social LCA . The International Journal of Life Cycle Assessment , 2011 . 16 ( 1 ): p. 1 - 2 .

[14] Manik , Y. , J. Leahy , and A. Halog , Social life cycle assessment of palm oil biodiesel: a case study in Jambi Province of Indonesia . The International Journal of Life Cycle Assessment , 2013 . 18 ( 7 ): p. 1386 - 1392 .

[15] Labuschagne , C. , A.C. Brent , and S.J. Claasen , Environmental and social impact considerations for sustainable project life cycle management in the process industry . Corporate Social Responsibility and Environmental Management , 2005 . 12 ( 1 ): p. 38 - 54 .

[16] Stamford , L. and A. Azapagic , Sustainability indicators for the assessment of nuclear power . Energy , 2011 . 36 ( 10 ): p. 6037 - 6057 .

[17] Corbière-Nicollier , T. , I. Blanc , and S. Erkman , Towards a global criteria based framework for the sustainability assessment of bioethanol supply chains . Ecological Indicators , 2011 . 11 ( 5 ): p. 1447 - 1458 .

[18] Heller , M.C. and G.A. Keoleian , Assessing the sustainability of the US food system: a life cycle perspective . Agricultural Systems , 2003 . 76 ( 3 ): p. 1007 - 1041 .

[19] Menna , C. , et al., Assessment of ecological sustainability of a building subjected to potential seismic events during its lifetime . The International Journal of Life Cycle Assessment , 2012 . 18 ( 2 ): p. 504 - 515 .

[20] Assefa , G. and B. Frostell , Social sustainability and social acceptance in technology assessment: A case study of energy technologies . Technology in Society , 2007 . 29 ( 1 ): p. 63 - 78 .

[21] Kucukvar , M. and O. Tatari , Towards a triple bottom-line sustainability assessment of the U.S. construction industry . The International Journal of Life Cycle Assessment , 2013 . 18 ( 5 ): p. 958 - 972 .

[22] Brent , A . and C. Labuschagne , Social Indicators for Sustainable Project and Technology Life Cycle Management in the Process Industry (13 pp + 4) . The International Journal of Life Cycle Assessment , 2006 . 11 ( 1 ): p. 3 - 15 .

[23] Pearsall , H. , From brown to green? Assessing social vulnerability to environmental gentrification in New York City. Environment and Planning C: Government and Policy, 2010 . 28 ( 5 ): p. 872 - 886 .

[24] Doloi , H. , Assessing stakeholders' influence on social performance of infrastructure projects . Facilities , 2012 . 30 ( 11 ): p. 531 - 550 .

[25] Hutchins , M.J. , J.S. Gierke , and J.W. Sutherland . Decision making for social sustainability: A life-cycle assessment approach . In Technology and Society , 2009 . ISTAS '09. IEEE International Symposium on. 2009 .

[26] Saadatian , O. , K.B. Sopian , and E. Salleh , Adaptation of sustainability community indicators for Malaysian campuses as small cities . Sustainable Cities and Society , 2013 . 6: p. 40 - 50 .

[27] Liu , J. , et al., Sustainability in hydropower development-A case study . Renewable & Sustainable Energy Reviews , 2013 . 19 : p. 230 - 237 .

[28] Magee , L. , A. Scerri , and P. James , Measuring Social Sustainability: A Community-Centred Approach . Applied Research in Quality of Life, 2012 . 7 ( 3 ): p. 239 - 261 .

[29] Schlossberg , M. and A. Zimmerman , Developing Statewide Indices of Environmental, Economic, and Social Sustainability: a look at Oregon and the Oregon Benchmarks . Local Environment , 2003 . 8 ( 6 ): p. 641 - 660 .

[30] Manos , B. , T. Bournaris, and P. Chatzinikolaou , Impact assessment of CAP policies on social sustainability in rural areas: an application in Northern Greece . Operational Research , 2010 . 11 ( 1 ): p. 77 - 92 .

[31] Chengcheng , F. , J.D. Carrell , and Z. Hong-Chao . An investigation of indicators for measuring sustainable manufacturing . in Sustainable Systems and Technology (ISSST) , 2010 IEEE International Symposium on . 2010 .

[32] Chan , E. and G.L. Lee , Critical factors for improving social sustainability of urban renewal projects . Social Indicators Research , 2008 . 85 ( 2 ): p. 243 - 256 .

[33] Manos , B. , et al., Effects of CAP policy on farm household behaviour and social sustainability . Land Use Policy , 2013 . 31 : p. 166 - 181 .

[34] Bramley , G. and S. Power , Urban form and social sustainability: The role of density and housing type . Environment and Planning B: Planning and Design , 2009 . 36 ( 1 ): p. 30 - 48 .

[35] Ochieng , B.O. , K.F.D. Hughey , and H. Bigsby , Rainforest Alliance Certification of Kenyan tea farms: a contribution to sustainability or tokenism? Journal of Cleaner Production , 2013 . 39 : p. 285 - 293 .

[36] Vasileiadis , V.P. , et al., Sustainability of European maize-based cropping systems: Economic, environmental and social assessment of current and proposed innovative IPM-based systems . European Journal of Agronomy , 2013 . 48 : p. 1 - 11 .

[37] von Geibler , J., et al., Accounting for the social dimension of sustainability: experiences from the biotechnology industry . Business Strategy and the Environment , 2006 . 15 ( 5 ): p. 334 - 346 .

[38] Valdes-Vasquez , R. and L.E. Klotz , Social Sustainability Considerations during Planning and Design: Framework of Processes for Construction Projects . Journal of Construction Engineering & Management , 2013 . 139 ( 1 ): p. 80 - 89 .

[39] Calker , K.V. , et al., Identifying and ranking attributes that determine sustainability in Dutch dairy farming . Agriculture and Human Values , 2005 . 22 ( 1 ): p. 53 - 63 .

[40] Meul , M. , et al., MOTIFS: a monitoring tool for integrated farm sustainability . Agronomy for Sustainable Development , 2008 . 28 ( 2 ): p. 321 - 332 .

[41] Marletto , G. and F. Mameli , A participative procedure to select indicators of policies for sustainable urban mobility. Outcomes of a national test . European Transport Research Review , 2012 . 4 ( 2 ): p. 79 - 89 .

[42] Lebacq , T. , P. Baret , and D. Stilmant , Sustainability indicators for livestock farming. A review . Agronomy for Sustainable Development , 2013 . 33 ( 2 ): p. 311 - 327 .

[43] Hasna , A.M. Embedding sustainability in capstone engineering design projects . in Education Engineering (EDUCON) , 2010 IEEE. 2010 .

[44] Alam , M. , Y. Furukawa , and K. Harada , Agroforestry as a sustainable landuse option in degraded tropical forests: a study from Bangladesh . Environment, Development and Sustainability , 2009 . 12 ( 2 ): p. 147 - 158 .

[45] Blake-Beard , S. , et al., Social sustainability, flexible work arrangements, and diverse women . Gender in Management: An International Journal , 2010 . 25 ( 5 ): p. 408 - 425 .

[46] Sarkis , J. , M.M. Helms , and A.A. Hervani , Reverse logistics and social sustainability . Corporate Social Responsibility and Environmental Management , 2010 . 17 ( 6 ): p. 337 - 354 .

[47] McMichael , A.J. and J.W. Powles , Human numbers, environment, sustainability, and health . British Medical Journal , 1999 . 319 ( 7215 ): p. 977 - 980 .

[48] Landorf , C. , Evaluating social sustainability in historic urban environments . International Journal of Heritage Studies , 2011 . 17 ( 5 ): p. 463 - 477 .

[49] Andersson , K. , et al., Connecting Municipal and Regional Level Planning: Analysis and Visualization of Sustainability Indicators in Bergslagen, Sweden . European Planning Studies , 2013 . 21 ( 8 ): p. 1210 - 1234 .

[50] Ceyhan , V. , Assessing the agricultural sustainability of conventional farming systems in samsun province of Turkey . African Journal of Agricultural Research , 2010 . 5 ( 13 ): p. 1572 - 1583 .

[51] Halme , M. , C. Jasch , and M. Scharp , Sustainable home services? Toward household services that enhance ecological, social and economic sustainability . Ecological Economics , 2004 . 51 ( 1-2 ): p. 125 - 138 .

[52] Rogers , S. , K. Gardner , and C. Carlson , Social Capital and Walkability as Social Aspects of Sustainability . Sustainability, 2013 . 5 ( 8 ): p. 3473 - 3483 .

[53] Lee , G.K.L. and E.H.W. Chan , Evaluation of the urban renewal projects in social dimensions . Property Management , 2010 . 28 ( 4 ): p. 257 - 269 .

Development and Companies and Development , 2011 : p. n/a-n/a.

[55] Gennett , Z.A. , J.A. Isaacs , and T.P. Seager . Developing a social capital metric for use in an educational computer game . in Sustainable Systems and Technology (ISSST) , 2010 IEEE International Symposium on . 2010 .

[56] Oliveira , F.d.C. , Á . Calle Collado, and L.F. Carvalho Leite , Autonomy and sustainability: An integrated analysis of the development of new approaches to agrosystem management in family-based farming in Carnaubais Territory , Piauí, Brazil. Agricultural Systems , 2013 . 115 ( 0 ): p. 1 - 9 .

[57] Faber , N.R. and R.J. Jorna . The use of multi-actor systems for studying social sustainability: Theoretical backgrounds and pseudo-specifications . in Computer Supported Cooperative Work in Design (CSCWD) , 2011 15th International Conference on. 2011 .

[58] Fiumara , G. , et al., Knowledge Representation in Virtual Teams: A Perspective Approach for Synthetic Worlds, in Collaborative Networks for a Sustainable World , L. Camarinha-Matos , X. Boucher , and H. Afsarmanesh, Editors. 2010 , Springer Berlin Heidelberg. p. 619 - 625 .

[59] CHI 2014 : Toronto, ON, Canada. The DBLP Computer Science Bibliography 2014 [cited 2014 22nd May]; Available from: http://dblp.uni-trier.de/db/conf/chi/chi2014.html.

[60] Widok , A.H. , et al. Achieving sustainability through a combination of LCA and DES integrated in a simulation software for production processes . in Simulation Conference (WSC) , Proceedings of the 2012 Winter . 2012 .

[61] Naumann , S. , et al., The GREENSOFT Model: A reference model for green and sustainable software and its engineering . Sustainable Computing: Informatics and Systems , 2011 . 1 ( 4 ): p. 294 - 304 .