Proceedings of EMOOCs 2019: Work in Progress Papers of the Research, Experience and Business Tracks OpenWHO traffic analysis: Can we predict non-profit course reach by dissemination channel? Yu Zhao1, Richelle George, Heini Utunen1, Gaya Gamhewage1 1 World Health Organization Abstract. OpenWHO.org is the World Health Organization’s first open online platform, introducing massive online learning into health emergency response. Since its launch in June 2017, the platform has surpassed 100 000 course enrolments, bringing know-how and science to frontline health workers. While most existing research on this topic focuses on traditional and commercial MOOC courses, few findings investigate the operational model of non-profit platforms. In order to understand the pattern of distribution that facilitates improved platform reach, the authors of this paper conducted a traffic analysis based on a selection of OpenWHO’s published courses, investigating similar traffic patterns among some of the site’s most recent courses, a selection of courses published in both English and French, and some courses with homogenous themes. Preliminary findings show that: 1) A combined usage of search engines and social media distribution channels appears to attract more users 2) Courses with the most enrolments generate user traffic from a wide range of channels 3) A significant number of enrolments come from visitors using mobile devices 4) Users of French courses are rarely reached through social media, but rather through a combination of search engines and the US National Library of Medicine. Further research could be conducted by integrating user professional background and age factors. Keywords: Health Emergencies, Epidemics, Health Emergency Training, MOOC, Digital Health, Non-Profit, Online Learning, Platform Learning 1 Roadmap The aim of this paper is to share findings on the learning reach of OpenWHO.org, the World Health Organization (WHO)’s first open online course platform tailored for frontline responders during health emergencies. The article begins with an introduction to the platform, background and the current distribution channels used for course dissemination, followed by a summary of existing literature related to the subject; it then summarizes the methodology for data collection before moving on to preliminary findings taken from 18 ongoing courses. Finally, limitations of the analysis and proposed next steps are considered. 261 Proceedings of EMOOCs 2019: Work in Progress Papers of the Research, Experience and Business Tracks 2 Background 2.1 OpenWHO: WHO’s first open online learning platform People travel, and so do pathogens. While globalization has fostered thriving economies, heightened global travel and trade, and spurred societal transformations, it has also facilitated the spread of infectious diseases caused by high-risk pathogens across national borders [1] (Fukuda, K, 2015). Meanwhile, burgeoning digital health technologies are helping to boost health equity by improving the health of those in low- and middle-income countries, a transformation which is critical to the achievement of the Sustainable Development Goals [2] (Sinha & Schryer-Roy, 2018) and the ideals of the WHO’s constitution and General Programme for Work 13 1. Particularly in health education, massive open online courses (MOOCs) have grown exponentially in popularity in recent years [3] (Liyanagunawardena & Williams, 2014) and are a fast-growing sector of the digital health boom. They can be accessed by anyone, anywhere and at any time, as long as users can access the internet, are computer or mobile literate [3] (Liyanagunawardena & Williams, 2014) and are able to access courses in their spoken languages. OpenWHO.org was created to respond to these challenges and opportunities, aiming to provide life-saving technical information and training to first responders during disease outbreaks anywhere in the world [4] (Zhao et al, 2018). Unlike platforms hosting traditional MOOCs, OpenWHO features self-paced courses without start and end dates [5] (Rohloff et al, 2018), and a flexible MOOC learning approach enabling professionals to access knowledge anytime. Some of the courses offer a free certificate to users who reach certain thresholds 2. 2.2 How courses are currently disseminated on OpenWHO ─ Currently, course announcements are sent out, in both English and French, to all registered users via email when a course is published. ─ Short introductory paragraphs are sometimes included in the WHO newsletter and newsletters of various projects 3. A monthly OpenWHO newsletter was also disseminated to all course producers between June and December 2018. ─ Each course is assigned a unique URL and equipped with “Social media” share buttons. ─ Expert networks of clinicians, students, social scientists and WHO Collaboration Centre are targeted; communication is sometimes done using tools such as WhatsApp. ─ The WHO Website regularly brings in visitors from disease and outbreak-related pages, as do websites of UN agencies, health institutes and thematic websites 1 www.who.int/about/what-we-do/thirteenth-general-programme-of-work-2019-202 2 For example, users are required to score no less than 80% of the points in all the assessments of a course to receive a Record of Achievement. 3 A primary example is the Pandemic Influenza Preparedness (PIP) Framework 262 Proceedings of EMOOCs 2019: Work in Progress Papers of the Research, Experience and Business Tracks addressing health subjects. Association sites also contribute routine traffic to the platform; among them, the US National Library of Medicine 4tops the list. ─ Google search and Bing are two primary search engines 2.3 Existing research Research into the relationship between different distribution channels as a means of outreach and the success of a MOOC suggests that the nature of a MOOC’s distribution is a key determinant of the MOOC’s success. [6] (Shah, Oztok & Wagner, 2015). [6] Shah, Oztok & Wagner (2015) claim that the right distribution channel is particularly important for the long-term sustainability of a MOOC. Selecting the right distribution channel for outreach is also important to ensure a MOOC’s accessibility to all, including utilising mobile-friendly channels. In recent years, the use of mobile phones has increased dramatically in low- and middle-income countries, with 7 billion mobile phone subscriptions worldwide [7] (Bergström, Fottrell & Hopkins, 2015) and more than 80% of households across Africa able to access a mobile phone [8] (Arie, 2015). Moreover, research from Haan, Lugtig & Toepoel [9] found that if people already use a smartphone for specific tasks, their intention to use a smartphone for other tasks, such as completing surveys, is more likely. As such, they argue that investing in increasing the ease of mobile usage (such as by incorporating mobile-friendly design) could significantly increase the number of mobile users [9]. 2.4 Can variation in dissemination channels predict course reach? While most existing research on this topic focuses on traditional MOOC courses on commercial platforms, few studies investigate the operational model of non-profit platforms. As such, how variation in the distribution channels used to disseminate free learning sources may influence course reach remains unclear. The authors of this paper conducted a traffic analysis based on a selection of OpenWHO’s published courses, investigating similar traffic patterns among the site’s most recent courses, a selection of courses published in both English and French, as well as among some courses with homogenous themes. 3 Methodology The traffic analysis was implemented following four steps: ─ Enrolment and traffic data were collected from eighteen active courses. 4 disasterlit.nlm.nih.gov 263 Proceedings of EMOOCs 2019: Work in Progress Papers of the Research, Experience and Business Tracks ─ Traffic sources were examined and grouped into ten categories 5. Mobile traffic from search engines and social media was separated from their desktop counterparts ─ The number of users from each traffic source was summed up under each of the above categories and divided by the total course traffic. ─ Traffic compositions were finally rendered as comparison tables and charts. 4 Findings 4.1 Most effective channels: search engines and social media Figure 1 displays the clear pattern that a combined usage of search engines and social media channels attracts a significantly higher number of users than combinations of other channels. Ten out of the eighteen courses studied acquired more than 50% of users from a combination of these channels. The most enrolled course Antimicrobial Stewardship 6 acquired more than half of its traffic from social media. Search engines also prove their efficiency in four courses, respectively: Objectif 3 de développement durable FR (69%), Simulation Exercise Management (68%), Pandemic Influenza Vaccines EN 7(66%) and Operational Readiness (Tier 1) EN (54%) 8. 5 Email, network website, search engine, search engine mobile, social media, social media mobile, WHO internal network, WHO website, US National Library of Medicine (NLM) and Helpdesk tickets. 6 As of 28 February 2019, the total number of AMR course enrolments is 20 167. 7 Pandemic Influenza Vaccines: National Deployment and Vaccination Plans 8 It is worth noting that both the topics of Antimicrobial Stewardship and Pandemic Influenza have a social media presence which predates OpenWHO, a fact which may predispose these courses to be more commonly accessed through social media. 264 Proceedings of EMOOCs 2019: Work in Progress Papers of the Research, Experience and Business Tracks Fig. 1. OpenWHO course traffic by distribution channel (as of 28 February 2019) It is worth noting that NLM counted as the primary source of traffic in seven courses, four of which focus on disease-related knowledge. For example, the Pandemic Influenza Severity Assessment (PISA)_EN hosts 55% of its visitors from the NLM and 27% from search engines. 4.2 The more diverse the distribution channels, the greater the enrolments Figure 1 illustrates that the courses with the highest total enrolments generate user traffic from a wider range of channels, in turn driving more than 10% of total traffic to the OpenWHO platform. Utilising multiple channels mitigates the burden each channel carries, meaning that even if one channel becomes unavailable during an emergency, the course is less likely to attract significantly fewer users. 4.3 Mobile distribution channels generate significant user traffic Table 1 demonstrates the average traffic composition for each distribution channel by language. It illustrates that two in three people directed to OpenWHO through search engines and social media are using mobile devices. The findings reaffirm the mobile-friendly approach highlighted in Section 3 of this paper. Table 1. Average traffic composition by language and category Table 2. Average traffic composition by language and category Distribution channels EN FR Email 4% 5% Network Website 4% 5% Search Engine 13% 23% Search Engine mobile 22% 13% Social Media 14% 2% Social Media mobile 17% 10% WHO staff (WIMS) 5% 4% WHO website 7% 2% US National Library of Medicine 13% 37% Helpdesk tickets 0% 1% 4.4 Preferred distribution channel differs significantly by language Table 1 also indicates a radical difference in preferred channels between English and French course learners: while the proportion of users from email, network websites and search engine channels remain approximately equal between the two languages, English-speaking users are much more likely to access courses through social media. In contrast, only 12% of French-speaking users access courses through social media. Instead, almost 40% of them navigated to courses using the NLM. In this respect, 265 Proceedings of EMOOCs 2019: Work in Progress Papers of the Research, Experience and Business Tracks further research could be done to investigate why French-speaking users so frequently access OpenWHO courses using an English website. 5 Limitations and further research Two factors impacted findings: 1) The traffic data collected does not reveal information at the individual level, making it difficult to create links between a user’s distribution preference and his/her professional background, age range and his/her answer to the compulsory sign-up questions on OpenWHO9. 2) The start date of each course varies; this may bring bias into distribution due to a lack of unified marketing interventions for OpenWHO. Further in-depth investigation could focus on comparing these results with OpenWHO compulsory questionnaire data to discover more precise distribution preferences among course users. References 1. Fukuda, K: Global health security and global epidemic threats in the 21st century. Weekly epidemiological record. No. 20, 2015, 90, 217–252 (2015). 2. Sinha, C., & Schryer-Roy, A. M: Digital health, gender and health equity: invisible imperatives. Journal of Public Health, 40(suppl_2), ii1-ii5 (2018). 3. Liyanagunawardena, T. R., & Williams, S. A: Massive open online courses on health and medicine. Journal of medical Internet research, 16(8) (2014). 4. Zhao, Y., Utunen, H. Stucke, O., Gamhewage, G., Reinfelde, M., Farzi, M., Attias, M: OpenWHO: integrate massive online learning into health emergency response. Poster presented at the Geneva Health Forum 2018, Geneva, Switzerland (2018). 5. Rohloff, T., Utunen, H., Renz, J., Zhao, Y., Gamhewage, G. and Meinel, C.,: OpenWHO: Integrating Online Knowledge Transfer into Health Emergency Response (2018). 6. Shah, M. A., Oztok, M., & Wagner, C: Drivers of MOOC Sustainability: A Comparison of the US and India. International Journal of the Computer, the Internet and Management, 23, 1-1 (2015). 7. Bergström, A., Fottrell, E., & Hopkins, H: mHealth: can mobile technology improve health in low-and middle-income countries. UCL public policy briefing (2015). 8. Arie, S: Can mobile phones transform healthcare in low and middle income countries? BMJ, 350, h1975 (2015). 9. Haan, M., Lugtig, B., Toepoel, V.: Can we predict device use? An investigation into mobile device use in surveys (2019). 9 Survey questions include “Where did you hear about OpenWHO?”, with option “friends, organization, other, print media, social media, web search, WHO website”. 266