<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>OpenWHO traffic analysis: Can we predict non-profit course reach by dissemination channel?</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Yu Zhao</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Richelle George</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Heini Utunen</string-name>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Gaya Gamhewage</string-name>
        </contrib>
      </contrib-group>
      <pub-date>
        <year>2019</year>
      </pub-date>
      <fpage>261</fpage>
      <lpage>266</lpage>
      <abstract>
        <p>OpenWHO.org is the World Health Organization's first open online platform, introducing massive online learning into health emergency response. Since its launch in June 2017, the platform has surpassed 100 000 course enrolments, bringing know-how and science to frontline health workers. While most existing research on this topic focuses on traditional and commercial MOOC courses, few findings investigate the operational model of non-profit platforms. In order to understand the pattern of distribution that facilitates improved platform reach, the authors of this paper conducted a traffic analysis based on a selection of OpenWHO's published courses, investigating similar traffic patterns among some of the site's most recent courses, a selection of courses published in both English and French, and some courses with homogenous themes. Preliminary findings show that: 1) A combined usage of search engines and social media distribution channels appears to attract more users 2) Courses with the most enrolments generate user traffic from a wide range of channels 3) A significant number of enrolments come from visitors using mobile devices 4) Users of French courses are rarely reached through social media, but rather through a combination of search engines and the US National Library of Medicine. Further research could be conducted by integrating user professional background and age factors.</p>
      </abstract>
      <kwd-group>
        <kwd>Health Emergencies</kwd>
        <kwd>Epidemics</kwd>
        <kwd>Health Emergency Training</kwd>
        <kwd>MOOC</kwd>
        <kwd>Digital Health</kwd>
        <kwd>Non-Profit</kwd>
        <kwd>Online Learning</kwd>
        <kwd>Platform Learning</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1 World Health Organization</title>
      <p>The aim of this paper is to share findings on the learning reach of OpenWHO.org, the
World Health Organization (WHO)’s first open online course platform tailored for
frontline responders during health emergencies. The article begins with an introduction
to the platform, background and the current distribution channels used for course
dissemination, followed by a summary of existing literature related to the subject; it
then summarizes the methodology for data collection before moving on to preliminary
findings taken from 18 ongoing courses. Finally, limitations of the analysis and
proposed next steps are considered.
2
2.1</p>
      <sec id="sec-1-1">
        <title>Background</title>
        <sec id="sec-1-1-1">
          <title>OpenWHO: WHO’s first open online learning platform</title>
          <p>
            People travel, and so do pathogens. While globalization has fostered thriving
economies, heightened global travel and trade, and spurred societal transformations, it
has also facilitated the spread of infectious diseases caused by high-risk pathogens
across national borders [
            <xref ref-type="bibr" rid="ref1">1</xref>
            ] (Fukuda, K, 2015).
          </p>
          <p>
            Meanwhile, burgeoning digital health technologies are helping to boost health equity
by improving the health of those in low- and middle-income countries, a transformation
which is critical to the achievement of the Sustainable Development Goals [
            <xref ref-type="bibr" rid="ref2">2</xref>
            ] (Sinha
&amp; Schryer-Roy, 2018) and the ideals of the WHO’s constitution and General
Programme for Work 131. Particularly in health education, massive open online courses
(MOOCs) have grown exponentially in popularity in recent years [
            <xref ref-type="bibr" rid="ref3">3</xref>
            ]
(Liyanagunawardena &amp; Williams, 2014) and are a fast-growing sector of the digital
health boom. They can be accessed by anyone, anywhere and at any time, as long as
users can access the internet, are computer or mobile literate [
            <xref ref-type="bibr" rid="ref3">3</xref>
            ] (Liyanagunawardena
&amp; Williams, 2014) and are able to access courses in their spoken languages.
          </p>
          <p>
            OpenWHO.org was created to respond to these challenges and opportunities, aiming
to provide life-saving technical information and training to first responders during
disease outbreaks anywhere in the world [
            <xref ref-type="bibr" rid="ref4">4</xref>
            ] (Zhao et al, 2018). Unlike platforms
hosting traditional MOOCs, OpenWHO features self-paced courses without start and
end dates [
            <xref ref-type="bibr" rid="ref5">5</xref>
            ] (Rohloff et al, 2018), and a flexible MOOC learning approach enabling
professionals to access knowledge anytime. Some of the courses offer a free certificate
to users who reach certain thresholds2.
2.2
          </p>
          <p>How courses are currently disseminated on OpenWHO
─ Currently, course announcements are sent out, in both English and French, to all
registered users via email when a course is published.
─ Short introductory paragraphs are sometimes included in the WHO newsletter and
newsletters of various projects3. A monthly OpenWHO newsletter was also
disseminated to all course producers between June and December 2018.
─ Each course is assigned a unique URL and equipped with “Social media” share
buttons.
─ Expert networks of clinicians, students, social scientists and WHO Collaboration
Centre are targeted; communication is sometimes done using tools such as
WhatsApp.
─ The WHO Website regularly brings in visitors from disease and outbreak-related
pages, as do websites of UN agencies, health institutes and thematic websites
1 www.who.int/about/what-we-do/thirteenth-general-programme-of-work-2019-202
2 For example, users are required to score no less than 80% of the points in all the assessments
of a course to receive a Record of Achievement.
3 A primary example is the Pandemic Influenza Preparedness (PIP) Framework
addressing health subjects. Association sites also contribute routine traffic to the
platform; among them, the US National Library of Medicine 4tops the list.
─ Google search and Bing are two primary search engines
2.3</p>
        </sec>
        <sec id="sec-1-1-2">
          <title>Existing research</title>
          <p>
            Research into the relationship between different distribution channels as a means of
outreach and the success of a MOOC suggests that the nature of a MOOC’s distribution
is a key determinant of the MOOC’s success. [
            <xref ref-type="bibr" rid="ref6">6</xref>
            ] (Shah, Oztok &amp; Wagner, 2015). [
            <xref ref-type="bibr" rid="ref6">6</xref>
            ]
Shah, Oztok &amp; Wagner (2015) claim that the right distribution channel is particularly
important for the long-term sustainability of a MOOC.
          </p>
          <p>
            Selecting the right distribution channel for outreach is also important to ensure a
MOOC’s accessibility to all, including utilising mobile-friendly channels. In recent
years, the use of mobile phones has increased dramatically in low- and middle-income
countries, with 7 billion mobile phone subscriptions worldwide [
            <xref ref-type="bibr" rid="ref7">7</xref>
            ] (Bergström, Fottrell
&amp; Hopkins, 2015) and more than 80% of households across Africa able to access a
mobile phone [
            <xref ref-type="bibr" rid="ref8">8</xref>
            ] (Arie, 2015).
          </p>
          <p>
            Moreover, research from Haan, Lugtig &amp; Toepoel [
            <xref ref-type="bibr" rid="ref9">9</xref>
            ] found that if people already
use a smartphone for specific tasks, their intention to use a smartphone for other tasks,
such as completing surveys, is more likely. As such, they argue that investing in
increasing the ease of mobile usage (such as by incorporating mobile-friendly design)
could significantly increase the number of mobile users [
            <xref ref-type="bibr" rid="ref9">9</xref>
            ].
2.4
          </p>
        </sec>
        <sec id="sec-1-1-3">
          <title>Can variation in dissemination channels predict course reach?</title>
          <p>While most existing research on this topic focuses on traditional MOOC courses on
commercial platforms, few studies investigate the operational model of non-profit
platforms. As such, how variation in the distribution channels used to disseminate free
learning sources may influence course reach remains unclear.</p>
          <p>The authors of this paper conducted a traffic analysis based on a selection of
OpenWHO’s published courses, investigating similar traffic patterns among the site’s
most recent courses, a selection of courses published in both English and French, as
well as among some courses with homogenous themes.
3</p>
        </sec>
      </sec>
      <sec id="sec-1-2">
        <title>Methodology</title>
        <p>The traffic analysis was implemented following four steps:
─ Enrolment and traffic data were collected from eighteen active courses.</p>
      </sec>
    </sec>
    <sec id="sec-2">
      <title>4 disasterlit.nlm.nih.gov</title>
      <p>─ Traffic sources were examined and grouped into ten categories5. Mobile traffic from
search engines and social media was separated from their desktop counterparts
─ The number of users from each traffic source was summed up under each of the
above categories and divided by the total course traffic.
─ Traffic compositions were finally rendered as comparison tables and charts.
4
4.1</p>
      <sec id="sec-2-1">
        <title>Findings</title>
        <sec id="sec-2-1-1">
          <title>Most effective channels: search engines and social media</title>
          <p>5 Email, network website, search engine, search engine mobile, social media, social media
mobile, WHO internal network, WHO website, US National Library of Medicine (NLM) and
Helpdesk tickets.
6 As of 28 February 2019, the total number of AMR course enrolments is 20 167.
7 Pandemic Influenza Vaccines: National Deployment and Vaccination Plans
8 It is worth noting that both the topics of Antimicrobial Stewardship and Pandemic Influenza
have a social media presence which predates OpenWHO, a fact which may predispose these
courses to be more commonly accessed through social media.
It is worth noting that NLM counted as the primary source of traffic in seven courses,
four of which focus on disease-related knowledge. For example, the Pandemic
Influenza Severity Assessment (PISA)_EN hosts 55% of its visitors from the NLM and
27% from search engines.
4.2</p>
        </sec>
        <sec id="sec-2-1-2">
          <title>The more diverse the distribution channels, the greater the enrolments</title>
        </sec>
        <sec id="sec-2-1-3">
          <title>Mobile distribution channels generate significant user traffic</title>
        </sec>
        <sec id="sec-2-1-4">
          <title>Preferred distribution channel differs significantly by language</title>
          <p>further research could be done to investigate why French-speaking users so frequently
access OpenWHO courses using an English website.
5</p>
        </sec>
      </sec>
      <sec id="sec-2-2">
        <title>Limitations and further research</title>
        <p>Two factors impacted findings: 1) The traffic data collected does not reveal information
at the individual level, making it difficult to create links between a user’s distribution
preference and his/her professional background, age range and his/her answer to the
compulsory sign-up questions on OpenWHO9. 2) The start date of each course varies;
this may bring bias into distribution due to a lack of unified marketing interventions for
OpenWHO.</p>
        <p>Further in-depth investigation could focus on comparing these results with
OpenWHO compulsory questionnaire data to discover more precise distribution
preferences among course users.</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          1.
          <string-name>
            <surname>Fukuda</surname>
            ,
            <given-names>K:</given-names>
          </string-name>
          <article-title>Global health security and global epidemic threats in the 21st century</article-title>
          .
          <article-title>Weekly epidemiological record</article-title>
          .
          <source>No. 20</source>
          ,
          <year>2015</year>
          ,
          <volume>90</volume>
          ,
          <fpage>217</fpage>
          -
          <lpage>252</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          2.
          <string-name>
            <surname>Sinha</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Schryer-Roy</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <article-title>M: Digital health, gender and health equity: invisible imperatives</article-title>
          .
          <source>Journal of Public Health</source>
          ,
          <volume>40</volume>
          (
          <issue>suppl</issue>
          _2),
          <fpage>ii1</fpage>
          -
          <lpage>ii5</lpage>
          (
          <year>2018</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          3.
          <string-name>
            <surname>Liyanagunawardena</surname>
            ,
            <given-names>T. R.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Williams</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <article-title>A: Massive open online courses on health and medicine</article-title>
          .
          <source>Journal of medical Internet research</source>
          ,
          <volume>16</volume>
          (
          <issue>8</issue>
          ) (
          <year>2014</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          4.
          <string-name>
            <surname>Zhao</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Utunen</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Stucke</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gamhewage</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Reinfelde</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Farzi</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Attias</surname>
            ,
            <given-names>M:</given-names>
          </string-name>
          <article-title>OpenWHO: integrate massive online learning into health emergency response</article-title>
          .
          <source>Poster presented at the Geneva Health Forum</source>
          <year>2018</year>
          , Geneva, Switzerland (
          <year>2018</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          5.
          <string-name>
            <surname>Rohloff</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Utunen</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Renz</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zhao</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gamhewage</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Meinel</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,:
          <article-title>OpenWHO: Integrating Online Knowledge Transfer into Health Emergency Response (</article-title>
          <year>2018</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          6.
          <string-name>
            <surname>Shah</surname>
            ,
            <given-names>M. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Oztok</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Wagner</surname>
            ,
            <given-names>C:</given-names>
          </string-name>
          <article-title>Drivers of MOOC Sustainability: A Comparison of the US and India</article-title>
          .
          <source>International Journal of the Computer, the Internet and Management</source>
          ,
          <volume>23</volume>
          ,
          <fpage>1</fpage>
          -
          <lpage>1</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          7.
          <string-name>
            <surname>Bergström</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fottrell</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Hopkins</surname>
            ,
            <given-names>H:</given-names>
          </string-name>
          <article-title>mHealth: can mobile technology improve health in low-and middle-income countries</article-title>
          .
          <source>UCL public policy briefing</source>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          8.
          <string-name>
            <surname>Arie</surname>
            ,
            <given-names>S:</given-names>
          </string-name>
          <article-title>Can mobile phones transform healthcare in low and middle income countries?</article-title>
          <source>BMJ</source>
          ,
          <volume>350</volume>
          , h1975 (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          9.
          <string-name>
            <surname>Haan</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lugtig</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Toepoel</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          :
          <article-title>Can we predict device use? An investigation into mobile device use in surveys (</article-title>
          <year>2019</year>
          ).
          <article-title>9 Survey questions include “Where did you hear about OpenWHO?”, with option “friends, organization, other, print media, social media, web search</article-title>
          , WHO website”.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>