Citizen science and socio-technical perspective: reflection on a possible integration

Abstract

Public participation in scientific research activities is also known as citizen science. It includes a set of methodological approaches for engagement and recruitment of non-expert volunteers coordinated by expert scientists, definition of tasks and scope aimed at organization of a community to achieve research purposes. One of the main goals of citizen science is to find scientific-based solutions to societal needs. Citizen science community acts as a system composed by social and technical components. Public participation is, in fact, supported using information technologies and digital technologies, such as smartphone and personal mobile devices. Participants and experts also organize and coordinate their activities through social media platforms. However, integration of citizen science approaches in a socio-technical perspective is a research topic not yet explored. Scope of this short position paper is to pose some reflection on possible integration starting from a brief review of main characteristics of citizen science.

Keywords

• 1 Citizen science
• public engagement
• socio-technical perspective

1. Introduction and motivation

Citizen science is a set of methodological approaches aimed at increasing public participation in scientific research activities [ 1,2,3 ]. Application of citizen science is transdisciplinary, allowing multiple disciplines to join efforts with experts and volunteers to solve societal challenges [ 4,5,6 ]. Unlike other types of participative approaches, based on crowdsourcing models, citizen science enlarges public participation to all steps of a research project, from data collection to interpretation of results and dissemination of outcomes [ 3 ]. In this perspective, expert researchers engage amateur volunteers in the definition of common and shared research design at the scope to find scientific-based solutions to societal needs and environmental challenges [ 3 ]. In a citizen science approaches, external environment indicates contextual background in which an organized groups of researchers and volunteers act. Contextual background is understood as the set of environmental, geographic, sociodemographics, cultural and jurisdictional factors. Societal challenges can be linked to different issues that threaten contemporary societies: climate change, water resources management, sustainable development, urban mobility, and urban planning, just to report most common application examples. Citizen science can represent a method to support and coordinate public call-to-action aimed at comanagement and co-assessment of public resources – such as water, land, cultural heritage – combining scientific purposes and societal challenges to address decision and policy making processes through bottom-up approaches [ 2,3 ].

Literature usually distinguishes ideal types of citizen science: (1) contributory, (2) collaborated, and (3) co-created [ 1,3,4 ]. Contributory ideal type refers to projects designed by expert scientists. The role of volunteers is limited to data collection [ 1,3,4 ]. In a collaborated ideal type, expert scientists generally define research goals, while volunteers can contribute to refine research design and activities such as data collection and analysis, and dissemination of outcomes [ 1,3,4 ]. In a co-created ideal type, expert scientists and volunteers collaborate, co-designing and co-conceptualizing all stage of the scientific process [ 1,3,4 ].

Citizen science constitutes an innovative research approach aimed at the integration of human and societal perspectives into scientific tasks [ 3 ] by (1) investigating into bidirectional relationships between environmental and socio-economic and cultural systems; (2) modelling new organizational processes supported by flexible interaction between humans and computers [ 7,8 ].

Starting from these conceptual assumptions, this short position paper intends to pose a reflection on possible integration and interpretation of citizen science in a socio-technical theoretical perspective.

2. Citizen science: a combination of social and technical components

Citizen science approaches to scientific research lay on three fundamental pillars:

1. Internal collaboration among participants. 2. Interaction between groups of participants and external environment.

3. Use of Information Communication Technologies (ICT) and digital web-based technologies to support and organize research tasks, internal organization of participants, communication activities and interaction with external environment.

The first two pillar points refer to the social component of citizen science, while the latter to technical component. 2.1.

Social components

The participation of volunteers in a citizen science project is a collective process aimed at identifying scientific-based solutions to collective issues. It implies the construction of structurally organized groups of volunteers, the adoption of co-design research methodologies among all the participants and the definition of common and collegial goals.

A citizen science community is a complex organization generally composed by groups of expert scientists and volunteers, as depicted in table 1. Expert scientists are people who have academic or scientific background [ 2 ]. They usually coordinate and supervise research activities as research design, goals definitions, data collection settings and protocols [ 2 ]. Volunteers constitutes a heterogeneous group with no academic qualification and profound differences in terms of professional expertise, education, personal interests, and socio-cultural background [ 9 ]. Volunteers, therefore, interact and collaborate within a context of social norms, legal frameworks, cultural values, and environmental factors that influence the definition of common research goals, the implementation of methodologies suitable for achieving the purposes and for adoption of suitable tools [ 2 ]. Volunteers can be categorized as (1) decision makers, (2) stakeholders, (3) citizens and general public. Decision makers and stakeholders constitute subgroups of expert amateurs, professionals and local authorities that give their contribution in participative science by expertise and advanced education. Citizens and general public are generally recruited and engaged within organized groups such as civic groups, neighborhood organizations, target communities. In this perspective, volunteers’ engagement acquires a meaning around the sense of community and belonging. Local communities’ participation in citizen science activities enhances the role of citizens into research and decisional processes [ 10 ] and individual motivations to participation. These latter usually refer to: (1) personal interest [ 1,11 ]; (2) scientific knowledge for the better understanding of their environment or to gain political leverage [ 2 ]; (3) improving relationships between people, institutional actors and stakeholders [ 2,7 ] aimed at social learning and co-management of common resources and goods [ 12 ]; (4) promoting joint action and civic participation on environmental topics with socio-economic and cultural implication or that involves socio-organizational aspects [ 3,8 ].

Expertise, education, socio-cultural background, and motivations are key aspects for the definition of roles within a citizen science community and defining the level of participation in scientific tasks. The participatory process, in fact, is articulated in different steps: envisioning and goal settings, model formulation, data collection and cross-checking, model application and evaluation of outcomes [ 7 ].

Decision makers, stakeholders

Citizens, public

Institutions/communities

Universities, academic institutions, research centers

Local authorities, professionals, expert amateurs

Civic groups,

neighborhood organizations, target communities, online communities 2.2.

Technical components

The implementation of web-based technologies into citizen science approaches gives new perspectives for engagement enlarging participation from physical to virtual spaces. In particular, social media systems allow rapid sharing of information at low cost connecting and mobilizing people within online communities. Involving online communities into the citizen science approach implies a new form of participatory organization around (1) remote work teams, (2) collaborative task management, (3) management of the dynamic division of labor, and (4) communication with large audience [ 13 ]. Introduction of web-based technologies and social media tools in citizen science investigation is reshaping socio-spatial networks of participation, projecting the engagement of volunteers from community-based approaches towards virtual contexts [ 14 ]. The adoption of computer-based models – e.g., software platforms, suitable smartphone applications – is functional to: (1) Support recruitment and engagement activities [ 15,16 ] enlarging participation by means of non-intrusive tools, commonly used in daily life of citizens [ 17,18 ]. (2) Support data collection activities and crowdsourcing by means of distributed network of human sensors [ 19 ] that operate at different scale of geographic scale and time. (3) Support communication efforts directed to general public or specific target groups. (4) Analyze social networks patterns to identify potential stakeholders to involve, defining roles and tasks within the organization [ 14,20,21 ]. (5) Investigate internal interactions between members of an online community [ 22 ] and external interactions between users and their context of interaction [ 23,24 ]. (6) Evaluate individual and collective efforts by visualization of results.

(7) Enhance a constant information flow for directional purposes [ 3 ].

These points synthetize the technological framework of citizen science platforms as recruitment pools between projects and volunteers. Platforms such as Zooniverse and Spotteron, for example, offer features and tools for supporting the participation of volunteers in scientific research and knowledge co-production processes through data collection functions from geo-localized observations, social community extensions – such as newsfeeds, forum, comments, liking and user following, data visualization and summary and data visualization dashboard and rating of observations for the data validation and information provided by users. These functionalities are aimed at building operative communities of volunteers that share opinions and experiences in relation to topics of scientific interest, highlighting their impacts on daily experiences.

3. Benefits and limits of citizen science

3.1. Benefits

The benefits of citizen science are measured in terms of raising knowledge through co-production and constant dialogue between citizens and experts around environmental and societal challenges. Benefits and opportunities are bidirectional for experts and volunteers. Experts can broaden availability of data at low cost. Volunteers can benefit within the organization in terms of professionalization, acquisition of skills and awareness, through the possibility of continuous training, dialogue and open discussion with experts and stakeholders [ 3 ]. Citizen science represents a common space for socializing; it creates a sense of belonging that contributes to participants’ performance and outcomes [ 11 ] by improving efficiency, effectiveness and the scope of research processes [ 8 ]. Citizen science creates a space for social inclusivity, dialogue between stakeholders and allows people to express themselves by giving a voice to non-experts [ 8 ]. Social inclusivity—derived from citizen science activities— contributes to mitigate conflicts around environmental and resources management [ 25 ]. It also contributes to improve the economic situation of participants by giving them knowledge and tools for managing local issues [ 25 ].

Citizen science, firstly, encourages dialogue and the exchange of information between citizens and experts, in order to increase awareness on public interest issues and in decision-making processes [ 26,27 ]. Dialogue and information exchange are also functional to collaborative modelling of tools and processes [ 7,28 ], definition of methodologies, protocols and research methods that reflect shared points of view between experts and engaged volunteers [ 29 ]. 3.2.

Limits

The limits related to citizen science can be divided into:

1. Socio-cognitive 2. Technological

As previously reported, citizen science aims to integrate humanistic and socio-cultural aspects into the processes and definition of research protocols to arrive at a co-production of knowledge. Human perceptions and socio-cultural background may affect data collection activities and the fulfillment of tasks assigned to the volunteers [ 7,30 ]. Levels of education or training and cognitive biases can threaten the validity and reliability of volunteers’ observations [ 10,31 ]. Subjective perceptions and biases not only influence interpretation and dissemination of results, but also may have potential negative consequences in economic, human and environmental concerns [ 31 ].

Still on the social implications, citizen science approaches can be influenced by social marginality deriving from controversies, insufficient funding, and barriers to participation related to social marginalization or political and jurisdictional barriers [ 2,8,32,33 ]. Marginalization usually takes origin from social, political and jurisdictional issues that limit the participation to specific groups and exclude others [ 2,10,34 ].

These aspects can be further aggravated by the digital divide. The latter sharpens the differences between social groups often leading to the marginalization of disadvantaged groups. Participation in research projects according to citizen science approaches thus risks being elitist. Such differentiations may be reflected geographically, where network coverage is greater in urban than in rural areas.

From a technical point of view, these issues lead to discontinuity in data collection, exchange and processing of information useful for implementing analytical models and for achieving valid results. Citizen science needs the provision of adequate infrastructures to support communication, (online) training, storage of data collected, to offer analysis and standardize program evaluation [ 4 ]. Appropriate technology helps citizen science projects. Internet and smartphones are fundamental tools to facilitate the participation, but they are not a warranty of data quality and training is needed for the correct use of these technologies in citizen science tasks [34]. The success of participatory approaches, as crowdsourcing and citizen science, does not rely only on technological advances, but also on the capacity to engage people and foster cooperation and coordination between participants and stakeholders around common community concerns [ 17,18 ].

4. A socio-technical perspective for Citizen science

The integration of citizen science into a socio-technical system represents a yet unexplored research topic. Given the previous context, possible research scenarios include analysis of the interactions between people, technologies and the external environment [35]. In particular, research on the topic of citizen science from a socio-technical perspective can be useful to deepen the motivations and human behaviors of the participants within a community of researchers and volunteers, the methods of internal organization and alignment of aims and processes, and finally the contribution of digital technologies to support research activities [36]. The integration of citizen science into socio-technical perspectives constitutes a long and challenging research path in absence of solid reference studies. Here, it is only possible to trace some research input, which must be followed not only by theoretical reflections but above all empirical case studies. Studies on open-source communities and software projects can offer initial insights on the collaboration between volunteers and personal motivations within small organizations with targeted and specific goals [37,38], according to inductive analysis methods in search of general and replicable categorizations from single case studies [37,38].

A socio-technical perspective, in short, can be functional to understand the changes at the organizational level in the modalities of participatory science and the changes both in behavioral and cultural terms (social learning, professionalization of volunteers, greater awareness of social and environmental issues) and in technological terms (use of digital tools in scientific research, new models for collecting and analyzing data from voluntary observations, development of management platforms, digital technologies to direct operational activities).

Socio-technical perspectives can also help to study and understand citizen science communities as work system that relies on human, informational and technical infrastructure [36]. Participants (experts and non-experts) collect data, process and exchange information for knowledge co-production, finding new forms of interaction and cooperation with the support of digital platforms as recruitment and communication tools. As a work system, a citizen science community interacts within a specific environment given by geographical, ecosystem and socio-cultural conditions that affect behaviors and choices with the support of information and digital technologies.

Categories proposed by Alter (2020) such as unity of purpose and compatibility between the participants, alignment of roles and tasks to be performed, sharing of responsibilities, exchange and continuous access of information and interoperability with technologies [36] can represent the pillars for the proper systemic functioning of an organized citizen science community. In this regard, definition of a research design suitable to solve societal and environmental challenges, identification of target groups active in a specific context, social pattern analysis and definition of shared standard protocols of research among experts and volunteers could represent a roadmap towards unity of purpose, alignment of processes and tools modeling at the base of the socio-technical functioning of a citizen science organization [ 3 ].

A community active in citizen science can also be interpreted as a univocal and congruent system of values, in which research objectives are aligned with personal and collective needs [35,39]. The purpose of citizen science can be addressed towards the joint optimization of its social and technical components to ensure the efficiency and validity of research work in connection with social goals and collective satisfaction [35]. On the one hand, the research goals must correspond with collective needs to find scientific solutions to collective problems. On the other hand, the participation of volunteer citizens must be favored by easy access to information and by the assignment of tasks that are easy and intuitive to carry out using non-intrusive and everyday technologies such as smartphones and personal mobile devices [35].

To measure the functioning of a socio-technical system within a citizen science community, it would be useful to carry out environmental and socio-cultural context analysis to identify strengths and weaknesses, causes of potential conflict, legal and cultural obstacles to participation [39]. Context analysis is also useful to identify behavioral and socio-cultural patterns to evaluate the conditions of collaboration and cooperation between volunteers and between them and team of coordinating experts [39]. Finally, at the technological level, the context analysis would help to understand any changes and benefits; understand if technologies help in problem-solving problems or create issues; whether they help overcome cognitive biases that threaten the validity and reliability of citizen science approach to scientific research [39].

5. References

[32] V.J. MacPhail, S.R. Colla, Power of the people: A review of Citizen Science programs for conservation. Biol. Conserv., 249 (2020), 108739. https://doi.org/10.1016/j.biocon.2020.108739 [33] J. Minet, Y. Curnel, A. Gobin, J.P. Goffart, F. Mélard, B. Tychon, J. Wellens, P. Defourny, Crowdsourcing for agricultural applications: A review of uses and opportunities for a farmsourcing approach. Comput. Electron. Agric., 142 (2017), 126–138. https://doi.org/10.1016/j.compag.2017.08.026 [34] M. Andrachuk, M. Marschke, C. Hings, D. Armitage, Smartphone technologies supporting community-based environmental monitoring and implementation: A systematic scoping review.

Biol. Conserv., 237 (2019), 430–442. https://doi.org/10.1016/j.biocon.2019.07.026 [35] S. Sarker, S. Chatterjee, X. Xiao, A. Elbanna, The sociotechnical axis of cohesion for the IS discipline: its historical legacy, MIS Quart., 43 (2019), DOI: 10.25300/MISQ/2019/13747 [36] S. Alter, Dimensions of Integration in Sociotechnical Systems, in: Proceedings of the 6th

International Workshop on Socio-Technical Perspective in IS Development (STPIS 2020), 2020. [37] W. Scacchi (2005). Socio-technical interaction networks in free/open source software development processes. In Software process modeling (pp. 1-27). Springer, Boston, MA. [38] A. Nolte, I. Jahnke, I. A. Chounta, T. Herrmann. Supporting Collaboration in Small Volunteer Groups with Socio-Technical Heuristics. In Proceedings of 16th European Conference on Computer-Supported Cooperative Work-Exploratory Papers. European Society for Socially Embedded Technologies (EUSSET), 2018. [39] S.H. Appelbaum, S. H. (n.d.). Socio-technical systems theory: an intervention strategy for organizational development, Manag. Dec., 35 (1997) 452–463, doi: 10.1108/00251749710173823

References

[1] R. Bonney , C.B. Cooper , J. Dickinson , S. Kelling , T. Phillips , K.V. Rosenberg , J. Shirk , Citizen science: A developing tool for expanding science knowledge and scientific literacy . BioScience 59 ( 2009 ), 977 - 984 . http://dx.doi.org/10.1525/bio. 2009 . 59 .11.9

[2] W. Buytaert , Z. Zulkafli , S. Grainger , L. Acosta , T.C. Alemie , J. Bastiaensen , B. De Bièvre, J. Bhusal , J. Clark , A. Dewulf , et al., Citizen science in hydrology and water resources: Opportunities for knowledge generation, ecosystem service management, and sustainable development . Front. Earth Sci. , 2 ( 2014 ), doi:10.3389/feart. 2014 . 00026 .

[3] A. Spasiano , S. Grimaldi , A.M. Braccini , F. Nardi , Towards a Transdisciplinary Theoretical Framework of Citizen Science: Results from a Meta-Review Analysis . Sustain . 13 ( 2021 ), 7904 . https://doi.org/10.3390/su13147904.

[4] G. Newman , J. Graham , A. Crall , M. Laituri , The art and science of multi-scale Citizen science support . Ecol. Inform . 6 ( 2011 ), 217 - 227 . https://doi.org/10.1016/j.ecoinf. 2011 . 03 .002

[5] C.N. Knapp , R.S. Reid , M.E. Fernández-Giménez , J.A. Klein , K.A. Galvin , Placing transdisciplinarity in context: A review of approaches to connect scholars, society and action . Sustain . 11 ( 2019 ), 4899 . https://doi.org/10.3390/su11184899

[6] F. Nardi , C. Cudennec , T. Abrate , C. Allouch , A. Annis , T. Herman Assumpção , A.H. Aubert , D. Berod , A.M. Braccini , W. Buytaert , et al., Citizen s AND HYdrology (CANDHY): Conceptualizing a transdisciplinary framework for citizen science addressing hydrological challenges . Hydrol. Sci. J . 2020 , doi:10.1080/02626667. 2020 . 1849707 .

[7] A. Voinov , N. Kolagani , M.K. McCall , P.D. Glynn , M.E. Kragt , F.O. Ostermann , S.A. Pierce , P. Ramu , Modelling with stakeholders-Next generation . Environ . Model. Softw., 77 ( 2016 ), 196 - 220 . https://doi.org/10.1016/j.envsoft. 2015 . 11 .016

[8] J. van de Gevel , J. van Etten, S. Deterding , Citizen science breathes new life into participatory agricultural research. A review . Agron. Sustain. Dev . 40 ( 2020 ), doi:10.1007/s13593-020-00636- 1.

[9] J.L. Dickinson , J. Shirk , D. Bonter , R. Bonney , R.L. Crain , J. Martin , T. Phillips , K. Purcell , The current state of Citizen science as a tool for ecological research and public engagement . Front. Ecol. Environ . 10 ( 2012 ), 291 - 297 . https://doi.org/10. 1890 /110236

[10] A. Commodore , S. Wilson , O. Muhammad , E. Svendsen , J. Pearce , Community-based participatory research for the study of air pollution: A review of motivations, approaches, and outcomes . Environ. Monit. Assess . 189 ( 2017 ), 378 . https://doi.org/10.1007/s10661-017-6063-7

[11] R. Crain , C. Cooper , J.L. Dickinson , Citizen Science: A tool for integrating studies of human and natural systems . Annu. Rev. Environ. Resour . 39 ( 2014 ), 641 - 665 . 10 .1146/annurev-environ030713- 154609

[12] E. Villaseñor , L. Porter-Bolland , F. Escobar , M.R. Guariguata , P. Moreno-Casasola , Characteristics of participatory monitoring projects and their relationship to decision-making in biological resource management: A review . Biodivers. Conserv . 25 ( 2016 ), 2001 - 2019 . https://doi.org/10.1007/s10531-016-1184-9

[13] T. Ignat , P. Ayris , I.L.I. Juan , S. Reilly , B. Dorch , T. Kaarsted , A.K. Overgaard , Merry work: Libraries and Citizen science . Insights: UKSG J . 31 , ( 2018 ), doi.org/10.1629/uksg.431.

[14] M.S. Reed , A. Graves , N. Dandy , H. Posthumus , K. Hubacek , J. Morris , C. Prell , C.H. Quinn , C. H. , L.C. Stringer , Who's in and why? A typology of stakeholder analysis methods for natural resource management . Journal of Environmental Management , 90 ( 2009 ), 1933 - 1949 . https://doi.org/10.1016/j.jenvman. 2009 . 01 .001

[15] N. Kankanamge , T. Yigitcanlar , A. Goonetilleke , M. Kamruzzaman , Can volunteer crowdsourcing reduce disaster risk? A systematic review of the literature . International Journal of Disaster Risk Reduction , 35 ( 2019 ), 101097 . https://doi.org/10.1016/j.ijdrr. 2019 .101097

[16] M. Modaresnezhad , L. Iyer , P. Palvia , V. Taras , Information Technology ( IT) enabled crowdsourcing: A conceptual framework . Information Processing and Management , 57 ( 2020 ). https://doi.org/10.1016/j.ipm. 2019 .102135

[17] T.H. Assumpção , I. Popescu , A. Jonoski , D.P. Solomatine , Citizen observations contributing to flood modelling: Opportunities and challenges . Hydrology and Earth System Sciences , 22 ( 2018 ), 1473 - 1489 . https://doi.org/10.5194/hess-22- 1473 -2018

[18] J.P. de Albuquerque , B. Herfort , A. Brenning , A. Zipf , A geographic approach for combining social media and authoritative data towards identifying useful information for disaster management . International Journal of Geographical Information Science , 29 ( 2015 ), 667 - 689 . https://doi.org/10.1080/13658816. 2014 .996567

[19] M.F. Goodchild , Citizen s as sensors: The world of volunteered geography . GeoJ . 69 ( 2007 ), 211 - 221 . https://doi.org/10.1007/s10708-007-9111-y

[20] C. Ruzol , D. Banzon-Cabanilla , R. Ancog , E. Peralta, Understanding water pollution management: Evidence and insights from incorporating cultural theory in social network analysis . Global Environmental Change , 45 ( 2017 ), 183 - 193 . https://doi.org/10.1016/j.gloenvcha. 2017 . 06 .009

[21] S. Uddin , Social network analysis in project management - A case study of analysing stakeholder networks . Journal of Modern Project Management , 5 ( 2017 ), 106 - 113 . https://doi.org/10.19255/JMPM01310

[22] W. Williamson , K. Ruming , Using Social Network Analysis to Visualize the Social-Media Networks of Community Groups: Two Case Studies from Sydney . Journal of Urban Technology , 23 ( 2016 ), 69 - 89 . https://doi.org/10.1080/10630732. 2016 .1197490

[23] Y. Huang , Q. Wu , Y. Hou , Examining Twitter Mentions Between Police Agencies and Public Users through the Lens of Stakeholder Theory . In Proceedings of the 18th Annual International Conference on Digital Government Research (dg.o '17) . Association for Computing Machinery , New York, NY, USA, 2017 , 30 - 38 . https://doi.org/10.1145/3085228.3085316

[24] D. Bojovic , C. Giupponi, Understanding the dissemination and adoption of innovations through social network analysis: geospatial solutions for disaster management in Nepal and Kenya . Journal of Environmental Planning and Management , 63 ( 2020 ), 818 - 841 . https://doi.org/10.1080/09640568. 2019 .1614435

[25] D. Frigerio , P. Pipek , S. Kimmig , S. Winter , J. Melzheimer , L. Diblíková , B. Wachter , A. Richter , Citizen science and wildlife biology: Synergies and challenges . Ethology 124 ( 2018 ), 365 - 377 . https://doi.org/10.1111/eth.12746

[26] B. Hollow , P.E.J. Roetman , M. Walter , C.B. Daniels , Citizen science for policy development: The case of koala management in South Australia . Environmental Science and Policy , 47 ( 2015 ), 126 - 136 . https://doi.org/10.1016/j.envsci. 2014 . 10 .007

[27] S. Brouwer , P. van der Wielen, M. Schriks, M. Claassen , J. Frijns , Public participation in science: The future and value of citizen science in the drinking water research . Water (Switzerland) , 10 ( 2018 ), 1 - 15 . https://doi.org/10.3390/w10030284

[28] L. Basco-Carrera , A. Warren , E. van Beek , A. Jonoski , A. Giardino , Collaborative modelling or participatory modelling? A framework for water resources management . Environmental Modelling and Software , 91 ( 2017 ), 95 - 110 . https://doi.org/10.1016/j.envsoft. 2017 . 01 .014

[29] J. Le Coz , A. Patalano , D. Collins, N.F. Guillén , C.M. García , G.M. Smart , J. Bind , A. Chiaverini , R. Le Boursicaud , G. Dramais , I. Braud , Crowdsourced data for flood hydrology: Feedback from recent citizen science projects in Argentina, France and New Zealand . Journal of Hydrology , 541 ( 2016 ), 766 - 777 . https://doi.org/10.1016/j.jhydrol. 2016 . 07 .036

[30] F.R. Adler , A.M. Green , Ç.H. Şekercioğlu , Citizen Science in ecology: A place for humans in nature . Ann. New York Acad. Sci., 1469 ( 2020 ), 52 - 64 . https://doi.org/10.1111/nyas.14340

[31] C. Conrad , K. Hilchey , A review of Citizen Science and community-based environmental monitoring: Issues and opportunities . Environ. Monit. Assess. , 176 ( 2011 ), 273 - 291 . https://doi.org/10.1007/s10661-010-1582-5