<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Real-Time Crowdsourcing of Health Data in a Low-Income Country: A Case Study of Human Data Supply on Malaria First-Line Treatment Policy Tracking in Nigeria</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Olubayo Adekanmbi</string-name>
          <email>olubayo@datasciencenigeria.ai</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Wuraola Fisayo Oyewusi</string-name>
          <email>wuraola@datasciencnigeria.ai</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ezekiel Adebayo Ogundepo</string-name>
          <email>ezekiel@datasciencenigeria.ai</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Data Science Nigeria</institution>
          ,
          <addr-line>Lagos</addr-line>
          <country country="NG">Nigeria</country>
        </aff>
      </contrib-group>
      <abstract>
        <p />
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>Malaria is one of the leading causes of high morbidity and mortality in Nigeria
despite various policy interventions to frontally address the menace. While a
national malaria policy agenda exists on the use of Artemisinin-based antimalarial
as the first-line drug of choice for the treatment, there have been challenges in the
implementation monitoring across various drug distribution layers, particularly the
informal channels that dominate over eighty percent of the antimalarial drug
distribution value chain. The lack of sustained policy monitoring through a structured
and systematic surveillance system can encourage irrational drug usage, trigger
antimalarial drug resistance, and worsen the disease burden in an economy where
over ninety percent of the population live below the poverty line. We explored the
use of real-time data collection through ordinary local residents, who leverages
low-cost smartphones with an on-device app to run quick mystery shopping at drug
outlets to check recommended malaria treatment drugs in four (4) states across
the country. The instant survey data is collected via guided mystery shopping,
which requires the volunteer participants to answer three basic questions after a 5
10 minutes in-store observation. Each submission is verified with the drug store
picture and auto-generated location co-ordinates. The antimalarial policy
compliance level is immediately determined and can be anonymously aggregated into a
national map for onward sharing with pharmaceutical trade groups, government
agencies and non-profits for immediate intervention via requisite stakeholder
education. This crowd-sourcing effort provides an affordable option that can be scaled
up to support healthcare surveillance and effective policy compliance tracking in
developing nations, where there is a paucity of data as a result of high illiteracy
and infrastructural inadequacy.</p>
    </sec>
    <sec id="sec-2">
      <title>Background</title>
      <p>
        Malaria is responsible for three out of every five outpatient visits in Nigeria [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]. An average Nigerian
adult experiences at least one episode yearly, while an average under-five year-old child records
up to 3 5 incidences annually [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. On a broader measure, malaria infection represents 30%
of hospitalisation cases in Nigeria as at 2016 while also constituting major cause of unintended
pregnancy loss and maternal deaths annually. A recent government statistics indicate that malaria is
responsible for as much as 11% of the total maternal mortality cases in the country [
        <xref ref-type="bibr" rid="ref3 ref4">3,4</xref>
        ]. This places
Nigeria as one of countries with most worrisome burdens of malaria in the world, with as much as
97% of Nigeria’s population being at the risk of malaria.
      </p>
      <p>Nigeria updated its national first-line malaria treatment policy in 2005, which advocated for full
the use of Artemisinin based Combination Therapies (ACTs) because of the widespread resistance
to chloroquine. Two years after the policy kick-off, the nation’s drug regulatory body stopped
the registration of Artemisinin mono-therapies to safeguard the efficacy of ACTs. This has also
been complemented with large-scale national awareness campaign and multi-sectoral stakeholder
engagement under a holistic “Roll Back Malaria” program. The intervention is in alignment with the
global initiative by the World Health Organization (WHO), which recommends the use of Artemisinin
Combination Therapy (ACT), for uncomplicated malaria confirmed with either microscopy or rapid
diagnostic test (RDT).</p>
      <p>
        However, while there are evidence of significant funding by the Federal Government of Nigeria
on mass education aimed to drive the policy message to the prescribers as well as the suppliers,
the reality on ground indicates poor compliance.Independent researchers confirmed a significantly
lower-than-average level of adherence to the national antimalarial drug policy, especially in the rural
and semi-urban areas, where more than seventy percentage of the population lived [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
The most evident explanation for this is the absence of a robust, inclusive and transparent surveillance
system to monitor the implementation of the policy. This is further heightened by the lack of structures
for drug distribution, regulation and control, especially in the informal sector where most drugs are
sold in the country. The nature of access to centres for the treatment for malaria is another nightmare.
The healthcare delivery channels are a diverse range of outlets, dominated by unregistered service
channels like private hospitals, pharmacies, patent medicine dealers and roadside drug hawkers.
These unregistered and informal channels represent as much as eighty percent in most states of the
country [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>
        Available studies suggest that fifteen years after the introduction of ACT policy as first line treatment,
inappropriate prescription of antimalarial drugs persist, at both public and private health facilities
across the country [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ]. A study by [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] indicates that low compliance among local drug vendors in
a Northern State (Jigawa) in respect of stocking and selling of first-line antimalarial. The research
also indicates that more than 80% of the outlets visited were ignorant of and lacked training about
new treatment guidelines. In the same vein, [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] revealed that out of a total of 110 informal drug
retail shops engaged in an interview, as much as 43:1% were unaware of the government policy
on malaria recommended first-line treatment.[
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] indicated as much as 88% of non-compliance rate
in recommended antimalarial sold through private drug stores, with further insights on how huge
variation in pricing makes the cheaper brands of non-ACT the preferred option by the customers who
self-medicate. The current situation portends a huge risk if not curtailed. This presents a huge national
emergency, hence our proposal to explore easy, fast and affordable means for real-time surveillance
to assess the levels of policy compliance across the country in order to drive for accountable and
targeted policy enforcement.
2
      </p>
    </sec>
    <sec id="sec-3">
      <title>Method</title>
      <p>
        Crowdsourcing in healthcare delivery has been identified as an effective way for real-time insight
gathering [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. Its ability to offer complementary insights for public policy making for government
and institutions have also been noted [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. Varied use cases have been explored in advanced
economies[
        <xref ref-type="bibr" rid="ref12 ref14">12,14</xref>
        ], with examples as simple as its use for price transparency tracking [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ].
The proprietary data collection application,DataCrowd,is a simple Android-based survey tool designed
to facilitate real-time data collection in the form of text, picture, video and sound. It has capacity for
offline data capture and geo-fenced data collection capture that validates the physical presence of the
volunteer data collector at the specified drug sales outlet.The volunteer data collectors belongs to a
curated data collector community managed by the organization the authors work with.
2.2
      </p>
      <sec id="sec-3-1">
        <title>Sampling</title>
        <p>In Nigeria, informal drug sales outlets are part of a major pipeline where the populace seek health
services. Both formal and informal drug sales outlets in four (4) states were used for the study based
on the availability and nearness of volunteer data collectors. The approach was a basic convenient
sampling, informed by the exploratory or pilot nature of the study to validate the appropriateness
of the use of ordinary citizen in data collection on antimalarial drug policy tracking based on the
on-the-spot antimalarial drug recommendation to an average walk-in customer. While we recognise
the limitation of this sampling methodology and obvious lack of representation of population, it
serves out intent to confirm the feasibility of the use of ordinary local residents as affordable data
collectors.
2.3</p>
      </sec>
      <sec id="sec-3-2">
        <title>Survey Instrument Design on Crowdsourcing App</title>
        <p>Basic questionnaire that asks questions about the antimalarial drugs recommended in a mystery
shopping exercise were filled into the app. The questions asked are Name of Premise, Type of drug
sales outlets( Pharmacy or Patent and Medicine Vendor),Address of Premise,Name and picture of
antimalarial drug recommended More than 93% of the survey responses were instantly received at
the back-end tracking server within the 10 minutes of submission, accompanied with the coordinate’s
data of the location and the outlet pictures.
2.4</p>
        <p>Quality Assurance
112 data collectors collected data from 217 retail outlets.All the the volunteer data collectors were
trained via easy-to-comprehend, step-by-step, how-to-do-it videos1 were exchanged via Telegram
platform. These were followed with proofs of comprehension.The community of data collector
consists of a mix of citizens, the criteria for selection is based on proficiency in use of mobile
devices and ability to follow instruction. The participants in this survey were paid an equivalent of 1
dollar each for data subscription. A project situation room was created to track the quality of data
submitted based on geo-tagged location coordinates and outlet picture matching using Google map
extrapolation.
Policy non-compliance level is defined by the proportion of respondents whose prescription patterns
do not conform to the appropriate antimalarial drugs stipulated in the national antimalarial treatment
guidelines with respect to the first drug of choice for complicated and uncomplicated malaria
treatment.</p>
        <p>In determining this percentage value, we have calculated the ratio of the sum of all the non-ACT
recommendations made at the visited outlets measured against all the first-line anti-malaria
recommendations made (ACT, non-ACT, other drugs like antibiotics recommended for malaria treatment)
made by the prescribers during the period of study in percentage terms. The observed pattern of
non-compliance ratio showed significant geographical peculiarities and channel variance (rural vs
urban, formal vs informal).The purpose of split along geographical peculiarities is because there are
more informal drug outlets serving the health needs of people in states with predominantly rural
areas.The percentage compliance with the antimalarial first-line policy was significantly lower in
informal outlets in rural states.</p>
        <p>Our research, conducted within a 24-hour timeline in 217 retail outlets across 4 states in Nigeria
confirm the feasibility and viability of crowdsourcing for instant, on-the-spot health policy
implementation using ordinary residents with basic app-based solution.</p>
        <p>This affordable approach can be scaled up with appropriate governance and stakeholder management
framework to support health policy surveillance in developing country; as its simplicity and ease of
deployment leapfrog existing infrastructural challenges and skill gap.</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>Broader Impact</title>
      <p>This works presents a fast, cheap and alternative method to track implementation of health policies
especially in low resource health systems where the budgetary allocation may not make room for
such checks.One of the advantage of adopting this method of policy implementation tracking is that
better and faster decisions about this type of data can be made without assumptions policy failure
when it is simply a non implementation issue.</p>
      <p>This project leveraged community relationships to access data from drug sales outlets. Drug sales
outlets is only a subset of a larger healthcare ecosystem where policies are being implemented
1Link to sample virtual training content for crowdsourcing community https://www.youtube.com/
playlist?list=PL0mGkrTWmp4s1hs5mHpDUfz8y55UFCzSt</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Ladi-Akinyemi</surname>
            ,
            <given-names>T. W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Amoran</surname>
            ,
            <given-names>O. E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ogunyemi</surname>
            ,
            <given-names>A. O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kanma-Okafor</surname>
            ,
            <given-names>O. J.</given-names>
          </string-name>
          ,&amp;
          <string-name>
            <surname>Onajole</surname>
            ,
            <given-names>A. T.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Knowledge and implementation of the National Malaria Control Programme among health-care workers in primary health-care centers in Ogun State, Nigeria</article-title>
          .
          <source>Journal of Clinical Sciences</source>
          ,
          <volume>15</volume>
          (
          <issue>1</issue>
          ),
          <fpage>48</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Scott</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hussain</surname>
            ,
            <given-names>S. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Martin-Hughes</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fowkes</surname>
            ,
            <given-names>F. J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kerr</surname>
            ,
            <given-names>C. C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Pearson</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          , ...&amp; Wilson,
          <string-name>
            <surname>D. P.</surname>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Maximizing the impact of malaria funding through allocative efficiency: using the right interventions in the right locations</article-title>
          .
          <source>Malaria journal</source>
          ,
          <volume>16</volume>
          (
          <issue>1</issue>
          ),
          <fpage>1</fpage>
          -
          <lpage>14</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Obiebi</surname>
            ,
            <given-names>I. P.</given-names>
          </string-name>
          (
          <year>2019</year>
          ).
          <article-title>Adherence to antimalarial drug policy among doctors in Delta State, Nigeria: implications for malaria control</article-title>
          .
          <source>Ghana Medical Journal</source>
          ,
          <volume>53</volume>
          (
          <issue>2</issue>
          ),
          <fpage>109</fpage>
          -
          <lpage>116</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Etuk</surname>
            ,
            <given-names>E. U.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Egua</surname>
            ,
            <given-names>M. A.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Muhammad</surname>
            ,
            <given-names>A. A.</given-names>
          </string-name>
          (
          <year>2008</year>
          ).
          <article-title>Prescription pattern of antimalarial drugs in children below 5 years in a tertiary health institution in Nigeria</article-title>
          .
          <source>Annals of African Medicine</source>
          ,
          <volume>7</volume>
          (
          <issue>1</issue>
          ),
          <fpage>24</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Mangham-Jefferies</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hanson</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mbacham</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Onwujekwe</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Wiseman</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Mind the gap: knowledge and practice of providers treating uncomplicated malaria at public and mission health facilities, pharmacies and drug stores in Cameroon and Nigeria</article-title>
          .
          <source>Health Policy and Planning</source>
          ,
          <volume>30</volume>
          (
          <issue>9</issue>
          ),
          <fpage>1129</fpage>
          -
          <lpage>1141</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Uzochukwu</surname>
            ,
            <given-names>B. S. C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ossai</surname>
            ,
            <given-names>E. N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Okeke</surname>
            ,
            <given-names>C. C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ndu</surname>
            ,
            <given-names>A. C.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Onwujekwe</surname>
            ,
            <given-names>O. E.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Malaria knowledge and treatment practices in Enugu state, Nigeria: A qualitative study</article-title>
          .
          <source>International Journal of Health Policy and Management</source>
          ,
          <volume>7</volume>
          (
          <issue>9</issue>
          ),
          <fpage>859</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Berendes</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Adeyemi</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oladele</surname>
            ,
            <given-names>E. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oresanya</surname>
            ,
            <given-names>O. B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Okoh</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Valadez</surname>
            ,
            <given-names>J. J.</given-names>
          </string-name>
          (
          <year>2012</year>
          ).
          <article-title>Are patent medicine vendors effective agents in malaria control? Using lot quality assurance sampling to assess quality of practice in Jigawa, Nigeria</article-title>
          .
          <source>PLoS One</source>
          ,
          <volume>7</volume>
          (
          <issue>9</issue>
          ),
          <year>e44775</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Oladepo</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brieger</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Adeoye</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lawal</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Peters</surname>
            ,
            <given-names>D. H.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>Awareness of anti-malarial policy and malaria treatment practices of patent medicine vendors in three Nigerian states</article-title>
          .
          <source>African journal of medicine and medical sciences</source>
          ,
          <volume>40</volume>
          (
          <issue>4</issue>
          ),
          <fpage>345</fpage>
          -
          <lpage>352</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Oladepo</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Brieger</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Adeoye</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lawal</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Peters</surname>
            ,
            <given-names>D. H.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>Awareness of anti-malarial policy and malaria treatment practices of patent medicine vendors in three Nigerian states</article-title>
          .
          <source>African journal of medicine and medical sciences</source>
          ,
          <volume>40</volume>
          (
          <issue>4</issue>
          ),
          <fpage>345</fpage>
          -
          <lpage>352</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Créquit</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mansouri</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Benchoufi</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vivot</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Ravaud</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Mapping of crowdsourcing in health: systematic review</article-title>
          .
          <source>Journal of medical Internet research</source>
          ,
          <volume>20</volume>
          (
          <issue>5</issue>
          ),
          <year>e187</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Meisel</surname>
            ,
            <given-names>Z. F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>VonHoltz</surname>
            ,
            <given-names>L. A. H.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Merchant</surname>
            ,
            <given-names>R. M.</given-names>
          </string-name>
          (
          <year>2016</year>
          , March).
          <article-title>Crowdsourcing healthcare costs: Opportunities and challenges for patient centered price transparency</article-title>
          .
          <source>In Healthcare</source>
          (Vol.
          <volume>4</volume>
          , No.
          <issue>1</issue>
          , pp.
          <fpage>3</fpage>
          -
          <lpage>5</lpage>
          ). Elsevier.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Brabham</surname>
            ,
            <given-names>D. C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ribisl</surname>
            ,
            <given-names>K. M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kirchner</surname>
            ,
            <given-names>T. R.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Bernhardt</surname>
            ,
            <given-names>J. M.</given-names>
          </string-name>
          (
          <year>2014</year>
          ).
          <article-title>Crowdsourcing applications for public health</article-title>
          .
          <source>American journal of preventive medicine</source>
          ,
          <volume>46</volume>
          (
          <issue>2</issue>
          ),
          <fpage>179</fpage>
          -
          <lpage>187</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Meisel</surname>
            ,
            <given-names>Z. F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>VonHoltz</surname>
            ,
            <given-names>L. A. H.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Merchant</surname>
            ,
            <given-names>R. M.</given-names>
          </string-name>
          (
          <year>2016</year>
          , March).
          <article-title>Crowdsourcing healthcare costs: Opportunities and challenges for patient centered price transparency</article-title>
          .
          <source>In Healthcare</source>
          (Vol.
          <volume>4</volume>
          , No.
          <issue>1</issue>
          , pp.
          <fpage>3</fpage>
          -
          <lpage>5</lpage>
          ). Elsevier.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Wazny</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Applications of crowdsourcing in health: an overview</article-title>
          .
          <source>Journal of global health, 8</source>
          <volume>(1</volume>
          ).
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>