=Paper=
{{Paper
|id=Vol-2030/HAICTA_2017_paper76
|storemode=property
|title=Developing an Assessment Tool to Evaluate the Sustainability of Sheep and Goat Farming Systems in Europe
|pdfUrl=https://ceur-ws.org/Vol-2030/HAICTA_2017_paper76.pdf
|volume=Vol-2030
|authors=Konstantinos Zaralis,Laurence Smith,Alejandro Belanche,Emmanuel Morin,Samantha Mullender,Ignacio Martín-García,David Yañez-Ruiz
|dblpUrl=https://dblp.org/rec/conf/haicta/ZaralisSBMMMY17
}}
==Developing an Assessment Tool to Evaluate the Sustainability of Sheep and Goat Farming Systems in Europe==
https://ceur-ws.org/Vol-2030/HAICTA_2017_paper76.pdf
Developing an assessment tool to evaluate the
sustainability of sheep and goat farming systems in
Europe
Konstantinos Zaralis1*, Laurence Smith1*, Alejandro Belanche2, Emmanuel Morin3,
Samantha Mullender1, Ignacio Martín-García2 and David Yañez-Ruiz2
1
The Organic Research Centre, Elm Farm, Hamstead Marshall, Newbury, Berkshire RG20
0HR, UK
2
The Spanish National Research Council, Cmno. del Jueves s/n E-18100 Granada - Spain
3
L'Institut de l'Elevage, Campus INRA - Chemin de Borde Rouge - BP 42118, France
e-mail: Konstantinos Zaralis, kostas.z@organicresearchcentre.com
Abstract. Assessing the sustainability of small ruminant systems is crucial to
ensure their long-term viability, to identify potential areas of improvement, to
uncover trade-offs between different aspects of performance and to,
potentially, demonstrate benefits of particular management strategies.
However, there is scarce literature and guidance regarding optimal tool choice
and effective practical use of such tools with regards to the evaluation of the
sustainability of small ruminant systems. The overall aim of this work was to
select the most appropriate indicators following the coverage of the
sustainability criteria defined by the SAFA framework and to select or create -
if appropriate, a rapid, but effective, assessment tool. Ease of tool use and it’s
easiness of adaptation to include new indicators were key criteria. This paper
describes the selection of the most appropriate indicators and their
incorporation into an adapted version of the Public Goods Tool (PG Tool).
Keywords: Sheep, Goats, sustainability assessments, PG Tool, indicators,
ruminants.
1 Introduction
Sustainability assessment can be defined as an evaluation exercise that directs
decision-making to ensure ongoing feasibility of the production system (Bond et al.,
2012; Hugé et al., 2013; Pope, 2006). Indicator-based sustainability assessment tools
and frameworks can either guide or conduct sustainability assessments (Gasparatos
and Scolobig, 2012; Ness et al., 2007) and vary widely in their scope (e.g.
geographical and sector), target group (e.g. farmers or policy makers), selection of
indicators, aggregation and weighting method and time required (Binder et al., 2010;
Marchand et al., 2014; Schader et al., 2014). Scientific evidence on farm
*
These authors equally contributed to this work.
633
sustainability assessments, as well as the sustainability assessment tools available to
support decision-making, are ever-expanding; however, these assessment tools can
vary enormously in their scope and approach. (Binder et al., 2010; Bockstaller et al.,
2009; Gasparatos and Scolobig, 2012; Marchand et al., 2014; Ness et al., 2007;
Schindler et al., 2015).
Assessing the sustainability of small ruminant systems is crucial as it will help to
ensure their long-term viability, to identify potential areas of improvement and
efficiencies, to uncover trade-offs between different aspects of performance and to,
potentially, demonstrate benefits of particular types of management strategies and the
sector as a whole. Such assessments require physical visits to farms and collection of
data of different nature. However, questions arise on how to navigate between
existing tools, what their key characteristics are and how can one select the most
appropriate tool for this specific purpose. There is scarce literature and guidance
regarding optimal tool choice and the effective and practical use of such tools and
methodologies when it comes to small ruminant systems (De Ridder et al., 2007;
Bernués et al., 2011; Gasparatos and Scolobig, 2012). In addition, the typology of the
sheep and goat sector in Europe consists of a great diversity of farm types in terms of
production aims, farm size, breeds kept, and levels of intensification, including also
organic, Protected Designation of Origin (PDO), Protected Geographical Indication
(PGI), pluriactive farms or breed specific farms (Theodoridis et al 2016). These farm
types are specific to a wide range of environmental and socioeconomic conditions
with local, regional, national or intra-national importance and can vary in
sustainability and their needs for innovation. Therefore, the diversity of the existing
farm types and their production aims across Europe imposes a great challenge for
selecting appropriate indicators and tools to assess the sustainability of the sheep and
goat production systems.
The aim of this work was therefore to identify a) the most suitable indicators and
tools for assessing the sustainability of sheep and goat farms and b) to incorporate the
identified indicators to the most appropriate sustainability tool selected for the
assessments. To do this, we first conducted a thorough review of existing
sustainability assessment tools and frameworks and evaluated the suitability of the
identified tools with the involvement of a number of stakeholders across Europe.
Concurrent to this, specific indicators of sustainability of particular relevance to
sheep and goat systems, focusing on the often under-represented social, economic
and animal health and welfare aspects (e.g. Schader et al., 2014), were identified. The
information collected were analysed in order to select and establish a final tool to be
used for the assessments. The indicators already contained in it and the ease of
adding additional indicators was also taken into account. The methodology followed
is described in detail in the following sections.
2 Review of sustainability tools and indicators
Different terms are used in the literature to describe sustainability assessments,
such as methods, methodological approaches, frameworks, and tools (Marchand et
al., 2014; Schader et al., 2014; Schindler et al., 2015). In this review, we focused on
634
those sustainability assessments that have been developed into tools aimed at
conducting ex-post assessments of the sustainability performance of farms using
indicators. These are called indicator-based sustainability assessment tools.
A range of scientific papers on sustainability assessments relevant to agricultural
systems were identified in the literature (e.g. Alrøe et al, 2016; Alrøe and Noe, 2016;
Olde et al., 2016; Lewis et al., 2010; Marchand et al., 2014; Padel et al., 2015;
Schader et al., 2014; Schader et al., 2016; Smith and Little, 2013). The review of
these and other studies resulted in a long but not exhaustive, list of 103 sustainability
tools; these tools were in turn categorized based on the following criteria: i. the
quantification of sustainability used (functional units; e.g. currency, carbon footprint,
standardised units etc.), ii. farm, product or sector level (spatial scale), iii. whether
the tool was designed for a specific country or region or is more widely applicable
(transferability), iv. Whether it is sector specific (i.e. specific to dairy/crops/etc. or
covers a range of farm types), v. time taken to complete the assessment and vi.
software or platform used. Following this exercise, 21 tools (out of the 103 identified
in the literature) were subsequently selected and prioritised based on their coverage
in view of the FAO’s SAFA (Sustainability Assessment of Food and Agriculture
systems) framework guidelines. A similar approach was conducted by Olde et al.
(2016) for four of the tools included in our analysis (RISE, SAFA Tool1, PG Tool
and IDEA). Their results are shown below, in Table 1.
Table 1. General characteristics of the tools that complied with six selection criteria (Olde et
al. 2016)
Tool Full name Target group Reference Origin
Response Inducing Switzerland (Bern
Häni et al.
RISE Sustainability Farmers University of
(2003)
Evaluation Applied Sciences
Sustainability Food and agricultural
Assessment of Food enterprises, Multiple countries
SAFA FAO (2012)
and Agriculture organizations, and institutes
Systems governments
Gerrard et al. UK (The Organic
PG Tool Public Goods Tool Farmers, policy-makers
(2012) Research Centre)
Indicateurs de
Durabilité des Farmers, policy-makers, Zahm et al. France (multiple
IDEA
Exploitations education (2008) institutes)
Agricoles
635
The FAO’s SAFA guidelines define sustainability in in four domains (i.e.
environmental integrity; economic resilience; social wellbeing and good governance)
and each domain is further subdivided into themes and then sub-themes when
appropriate. Indicator quality is assessed in SAFA based on whether it is outcome-
based (e.g. actual measures of performances such as soil nitrogen surplus), related to
individual farm practices or simply a farm target. Being a well-founded and widely
accepted approach in the sector, SAFA provided a sensible framework to use as basis
for the conduction of our own review of indicators. Therefore, any indicators
additional to those suggested in the SAFA framework were identified and where
these indicators did not fit within an existing theme or sub-theme, new classes were
proposed.
2.1 Stakeholders survey on identifying appropriate indicators and
sustainability tool
Following the literature review of the tools and indicators a list of the potential
indicators and sustainability tools was presented by means of an on-line survey to a
number of sheep and/or goat stakeholders representing farmers in Greece, UK,
France, Italy, Spain, Finland and Turkey. These partners were asked to rank the
indicators in view of their experience and knowledge, on the basis of the most
“appropriate” and farmer/industry “commonly used” indicators. Participants were
also asked to rank and give details of any sustainability assessment tools that they
had used in the past or heard details of. The survey was developed using the
Qualtrics survey platform (Qualtrics, 2016) and was web-based. A total of 35
responses were received from 69 potential participants (researches and stakeholders)
while the 35 responses covered 95 % of the participating stakeholders. The results of
the survey were analysed in view of the industry type i.e. sheep vs. goat organisations
and were compared. In addition, preferences of research vs. industry partners were
also evaluated. Interestingly, there was a high level of agreement between different
stakeholder groups. The results of the survey were used to help compile a final list of
sustainability indicators and identify a suitable assessment tool for the purposes of
this work.
3 Selection of sustainability tool and indicators
In addition to the literature review and stakeholder survey, the development of the
assessment approach was based also on workshop discussions that aimed to i.
identify the most appropriate indicators in all dimensions (i.e. social, economic,
environmental, governance), ii. select the best sustainability assessment tool in view
of the European typology (see report by Theodoridis et al., 2016) of the sheep and
goat farms and iii. ensure that all the key indicators identified in step (i) were
included in the tool selected in step (ii) and if not, an additional list of indicators
would be built to be incorporated in the selected tool.
636
3.1 Selection of the indicators
The decision on the most appropriate indicators (with starting point on survey
results) was discussed with project partners through workshops and on-line
discussions; these meetings helped also to identify additional indicators that were
particularly relevant in more specialist types of production systems. The SAFA’s
general criteria to the categorisation of indicators was applies and selected indicators
fall within following three categories.
Animal health, welfare and livestock management indicators. The final selection
of the specific animal welfare indicators considered the particularities of sheep and
goat farming systems and their production purposes in view of the existing farm-type
variation (e.g. intensive, extensive, PDO, organic, transhumance, etc.). in addition to
the literature review and stakeholders’ feedback, the selection built upon
recommendations from the AWIN project (AWIN, 2015), DairyCare COST Action
and SOLID project experience, as well as latest EFSA recommendations (2014). The
addition of new indicators was based on a multi-dimensional concept (mental and
physical health in harmony with environment and the ability to cope with likely
changes in the environment due to climate change) following the principles of good
feeding, good housing, good health and appropriate behaviour, and considered both
extensive and intensive systems. In the selected indicators, resource-based and
management-based indicators have been combined with questions associated with
direct observations of animals.
Socioeconomic indicators. As previously, the identification of socioeconomic
indicators considered the specificities of sheep and goat farming systems across
Europe (Theodoridis et al., 2016). For the economic indicators, preference was given
to the “gross margin” approach (goat or sheep income, less variable costs), which is
already used by several farmers and requires less time to evaluate the economic
efficiency of livestock enterprises than the “costs of production” approach. The
selected indicators covered a range of socioeconomic aspects such as product quality,
food safety, product certification, integration in the local economy, vulnerability and
long-term profitability, investments farm succession in addition to indicators related
to the on-farm working conditions, health and safety and interaction of the farm with
the wider social environment.
Environmental indicators. These indicators were mostly selected based on the
SAFA themes of atmosphere, water, landscape, soil quality, biodiversity, and
materials and energy. Individual indicators associated with each area were drawn
from a range of sources including Government guidelines and codes of best practice
(e.g. Defra, 2006, Environment Agency, 2004, Natural England, 2007), industry-
facing guidelines (e.g. ADAS and The Organic Research Centre, 2002) and the
SAFA framework itself (FAO, 2012). New indicators were selected in relation to the
use of supplementary feed, on-farm energy efficiency and biodiversity management.
637
3.2 Selection and adaptation of the sustainability tool
The decision on the most appropriate tool was based in principle on its compliance
with the FAO SAFA guidelines, the time taken for the completion of the assessment,
. the scientific rigour of the farm assessment process and the need to adapt the
assessment framework through the inclusion of new indicators identified within the
literature review, stakeholder survey and workshops. It was also apparent such a tool
should be capable of assessing the sustainability of sheep and goat farms across a
range of farm types and will serve the basis for developing a toolbox of assessment
tools and indicators that can be used in future assessments. Additional selection
criteria included the easiness to adapt the tool to include new indicators identified
and the ease of tool use i.e. the expertise and time taken to carry out the assessment.
Following the characterisation of tools according to above criteria it was concluded
that concluded that the Public Goods Tool (PG Tool) provides the best option for a
rapid yet comprehensive framework for assessing the sustainability of the sheep and
goat sector. Amongst other advantages the readily of the PG Tool to be adapted, its
modular structure to follow FAO’s SAFA framework and that fact the tool has been
used successfully in multi-national sustainability assessments within a range of
European projects made it clearly the best choice in view of the aims of the project.
The development of the PG Tool for the specific project involved a) the inclusion
of the indicators identified and b) a comprehensive approach and analysis with
regards to the formulation of the questions to asked, what will be the potential
options for responses, and how each response will be scored. Some questions
required five different answers, given scores 1 to 5, while others required only three
and the scores assigned were 1, 3 and 5. Scores were attributed based on
recommendations for best practice included within guideline documents (e.g. CALU,
and ADAS, 2007) with the higher score representing the best sustainable practice
while some indicators required more than one question in order to be addressed.
Once indicators incorporated in the tool, they were analysed to finalise scoring
criteria. A significant proportion of the new indicators added to the PG tool related to
animal health and welfare, farm livestock management, social sustainability and
governance (the latter area was previously missing from the PG tool). Animal
welfare and social sustainability also tend to be under-represented in sustainability
assessments in general and were therefore given a greater emphasis.
Following incorporation of the new indicators, the PG-Tool underwent systematic
tests in UK, Spain, Greece, France and Italy as part of the WP1, assessing in sheep or
goat farms in these counties, in order to identify possible flaws or errors. Feedback
from farmers was recorded on the applicability and usefulness of the tool and the
time it took to assessment. Results from these assessments suggest the tool provides a
useful framework and identifies areas of poor/good performance. Following farmers’
feedback, considerable attention was paid on the time that the assessment is taking to
be completed. The sustainability assessment using the final version of the tool takes
about 2 to 3 h to be completed.
638
4 Conclusions
This work showed that, despite the growing interest in livestock sustainability
assessments, no specific tools that cover all sustainability domains and types of
production systems were developed with focus on the small ruminant sector.
Therefore, we put forward an assessment approach for selection of appropriate
indicators and sustainability tools that lead in the creation of a rapid, but effective,
assessment tool. We therefore consolidated information and data collected through
the industry partner survey, workshop discussions and literature review and the most
appropriate indicators in all dimensions (i.e. social, economic, environmental,
governance) were identified in addition to the best tool for assessing sustainability of
sheep and goat farms ensuring adaptability to a range of farm types. We concluded
that the Public Goods Tool (PG Tool) was the most appropriate framework for
adaptation as it was the first to fulfil all the key selection criteria (i.e. ease of tool use;
the coverage of a range of sustainability criteria as defined within the SAFA
framework and; the possibility and ease of adapting the tool to include new
indicators). The results of the on-line survey highlighted the importance of keeping
the process of the sustainability assessments relatively short (i.e. between two and
three hours) in order to attack farmer’s interest and keep them engaged through the
process. Thus, during the adaptation of the tool considerable attention was payed to
the time that the assessment is going to take on farm.
Acknowledgments. This work was undertaken as part of the iSAGE Project
(http://www.isage.eu/), with financial support from the European Union’s Horizon
2020 research and innovation programme under grant agreement No 679302. We
also thank professor Raffaele Zanoli for kindly providing access to the Qualtrics
survey platform (Qualtrics, 2016), which was used for the development of the on-live
survey used in this work.
References
1. ADAS and Organic Research Centre (2002), Managing Manure on Organic
Farms. London: Defra.
2. Alrøe, H. and Noe, E., 2016. Sustainability assessment and complementarity.
Ecology and Society 21
3. Alrøe, H., Moller, H., Læssøe, J., Noe, E., 2016. Opportunities and challenges for
multicriteria assessment of food system sustainability. Ecology and Society 21.
4. AWIN, 2015. AWIN welfare assessment protocol for goats. DOI:
10.13130/AWIN_GOATS_2015.
5. Bernués, A., Ruiz, R., Olaizola, A., Villalba, D., Casasús, I., 2011. Sustainability
of pasture-based livestock farming systems in the European Mediterranean
context: Synergies and trade-offs. Livestock Science 139, 44-57.
639
6. Binder, C.R., Feola, G., Steinberger, J.K. (2010) Considering the normative,
systemic and procedural dimensions in indicator-based sustainability assessments
in agriculture. Environmental Impact Assessment Review. 30, 71–81.
7. Bockstaller, C., Guichard, L., Keichinger, O., Girardin, P., Galan, M.-B.,
Gaillard, G., 2009. Comparison of methods to assess the sustainability of
agricultural systems: a review. Sustainable Agriculture. Springer, pp. 769-784.
8. Bond, A.; Morrison-Saunders, A.; Pope, J. (2012) Sustainability assessment:
The state of the art. Impact Assess. Proj. Apprais. 30 , 53–62.
9. CALU and ADAS (2007) Managing Energy and Carbon: The farmer's guide to
energy audits. ADAS Wales.
10. De Ridder, W., Turnpenny, J., Nilsson, M., Von Raggamby, A., 2007. A
framework for tool selection and use in integrated assessment for sustainable
development. Journal of environmental assessment policy and management 9,
423-441.
11. Defra (2014) Application of the public goods tool on conventional farms. Final
report to Defra.
12. Dumanski, J., Eugene Terry, Derek Byerlee, Christian Pieri. (1998) Performance
Indicators for Sustainable Agriculture (Discussion Note) Policies of the World
Bank.
13. EFSA recommendations on animal welfare (2014):
https://www.efsa.europa.eu/en/topics/topic/animalwelfare
14. FAO (2012). Sustainability Assessment of Food and Agriculture systems
(SAFA). FAO, Rome.
15. Gasparatos, A., Scolobig, A., 2012. Choosing the most appropriate sustainability
assessment tool. Ecological Economics 80, 1-7.
16. Gerrard, C.L., Smith, L.G., Pearce, B., Padel, S., Hitchings, R., Measures, M.
(2012) The OCIS PG tool: A tool for assessing the Public Goods provision of a
farm, Sustainable Agriculture Reviews. pp 1-22.
17. Häni, F., Braga, F., Stämpfli, A., Keller, T., Fischer, M., and Porsche, H. (2003).
RISE, a tool for holistic sustainability assessment at the farm level. International
Food and Agribusiness Management Review 6,78–90.
18. Hugé, J., Waas, T., Dahdouh-Guebas, F., Koedam, N., Block, T., 2013. A
discourse-analytical perspective on sustainability assessment: interpreting
sustainable development in practice. Sustainability science 8, 187-198.
19. Lewis K.A. et al. (2010) The climate change mitigation potential of an EU farm:
towards a farm-based integrated assessment. Deliverable 5.1, Final Report,
ENV.B.1/ETU/2009/0052.
20. Marchand, F., L. Debruyne, L. Triste, C. Gerrard, S. Padel, and L. Lauwers
(2014) Key characteristics for tool choice in indicator-based sustainability
assessment at farm level. Ecology and Society 19, 46.
21. Natural England (2007). The Heather and Grass Burning Code. London: Defra.
22. Ness, B., Urbel-Piirsalu, E., Anderberg, S., Olsson, L., 2007. Categorising tools
for sustainability assessment. Ecological economics 60, 498-508.
640
23. Olde, E.M., Oudshoorn, F.W., Sørensen, C.A., Bokkers, E.A., De Boer, I.J.
(2016). Assessing sustainability at farm-level: Lessons learned from a
comparison of tools in practice. Ecological Indicators. 66, 391-404.
24. Padel, S., Gerrard, C., Smith, L., Schader, C., Baumgart, L., Stolze, M. and
Pearce, B., (2015) Further Development of Methodologies for Sustainability
Assessment and Monitoring in Organic/Ecological Agriculture. ORC reports, no.
2015/1. Organic Research Centre, Newbury.
25. Pope, J., Grace, W., 2006. Sustainability assessment in context: Issues of process,
policy and governance. Journal of Environmental Assessment Policy and
Management 8, 373-398
26. Qualtrics (2016), Version 3. Copyright © 2017 Qualtrics. Qualtrics and all other
Qualtrics product or service names are registered trademarks or trademarks of
Qualtrics, Provo, UT, USA. http://www.qualtrics.com.
27. Schader, C., Baumgart, L., Landert, J., Muller, A., Ssebunya, B., Blockeel, J.,
Weisshaidinger, R., Petrasek, R., Mészáros, D., Padel, S., 2016. Using the
sustainability monitoring and assessment routine (smart) for the systematic
analysis of trade-offs and synergies between sustainability dimensions and
themes at farm level. Sustainability 8, 274.
28. Schader, C., Grenz, J., Meier, M.S., Stolze, M. (2014) Scope and precision of
sustainability assessment approaches to food systems. Ecology and Society 19.
29. Schindler, J., Graef, F., König, H.J., 2015. Methods to assess farming
sustainability in developing countries. A review. Agronomy for sustainable
development 35, 1043-1057
30. Smith, L and Litte T (2013) Environmental footprinting for farm businesses
Organic Research Centre, Elm Farm Tony Little.
31. Theodoridis, A. Ragkos, K. Zaralis, S. Mullender and G. Arsenos, (2016) iSAGE
Deliverable No: 1.1 Report on new farm typologies for sheep and goat systems
within the EU, Greece: Aristotle University of Thessaloniki.
32. Zahm, F., P. Viaux, L. Vilain, P. Girardin, and C. Mouchet. 2008. Assessing farm
sustainability with the IDEA method from the concept of agriculture
sustainability to case studies on farms. Sustainable Development 16:271–281.
641