Natural disasters are harmful events resulting from natural processes of the Earth. Examples of natural disasters include tsunamis, volcanic eruptions, earthquakes, oods, droughts, and other geologic processes. If they a ect populated areas, natural disasters can cause economic damage, injuries, or even losses of lives. It is thus desirable that natural disasters be detected as early as possible and potentially a ected persons be noti ed via emergency alerts. By their pure nature, natural disasters are global phenomena that people refer to by di erent names, for example, the 2014 typhoon Rammasun1 is known as typhoon Glenda in the Philippines. In this paper, we present our ongoing early-stage research on a realtime Wikipedia-based monitoring system for the detection of natural disasters around the globe. The long-term objective is to make data about natural disasters detected by this system available through public alerts following the Common Alerting Protocol (CAP).
1.1 Katastrophenhilfe (BBK,3 \Federal O ce of Civil Protection and Disaster Assistance") in Germany work to ensure the safety of the population on a national level, combining and providing relevant tasks and information in a single place. The United Nations O ce for the Coordination of Humanitarian A airs (OCHA)4 is a United Nations (UN) body formed to strengthen the UN's response to complex emergencies and natural disasters. The Global Disaster Alert and Coordination System (GDACS)5 is \a cooperation framework between the United Nations, the European Commission, and disaster managers worldwide to improve alerts, information exchange, and coordination in the rst phase after major sudden-onset disasters." Global companies like Facebook,6 Airbnb,7 or Google8 have dedicated crisis response teams that work on making critical emergency information accessible in times of disaster. As can be seen from the (incomprehensive) list above, natural disaster detection and response is a problem tackled on national, international, and global levels; both from the public and private sectors. To facilitate collaboration, a common protocol is essential. 1.2
The Common Alerting Protocol
The Common Alerting Protocol (CAP) [
Contributions, Hypotheses, and Research Questions In this paper, we present rst results of our ongoing early-stage research on a realtime comprehensive Wikipedia-based monitoring system for the detection of natural disasters around the globe. We are steered by the following hypotheses. H1 Content about natural disasters gets added to Wikipedia in a timely fashion. H2 Natural disasters being geographically constrained, textual and multimedia content about them gets added to local, i.e., non-English Wikipedias as well. H3 Link structure dynamics of Wikipedia provide for a meaningful way to detect future natural disasters, i.e., disasters unknown at system creation time. 3 BBK: http://www.bbk.bund.de/ 4 OCHA: http://www.unocha.org/ 5 GDACS: http://www.gdacs.org/ 6 Facebook Disaster Relief: https://www.facebook.com/DisasterRelief 7 Airbnb Disaster Response: https://www.airbnb.com/disaster-response 8 Google Crisis Response: https://www.google.org/crisisresponse/ These hypotheses lead us to the following research questions.
Q1 How timely and accurate for the purpose of natural disaster detection is content from Wikipedia compared to authoritative sources mentioned above? Q2 Does the disambiguated nature of Wikipedia surpass keyword-based natural disaster detection approaches, e.g., via online social networks or search logs? 2
Digitally crowdsourced data for disaster detection and response has gained momentum in recent years, as the Internet has proven resilient in times of crises, compared to other infrastructure. Ryan Falor, Crisis Response Product Manager <?xml version="1.0" encoding="utf-8"?> <alert xmlns="urn:oasis:names:tc:emergency:cap:1.2"> <identifier>GDACS_FL_4159_1</identifier> <sender>info@gdacs.org</sender> <sent>2014-07-14T23:59:59-00:00</sent> <status>Actual</status> <msgType>Alert</msgType> <scope>Public</scope> <incidents>4159</incidents> <info> <category>Geo</category><event>Flood</event> <urgency>Past</urgency><severity>Moderate</severity> <certainty>Unknown</certainty> <senderName>Global Disaster Alert and Coordination System</senderName> <headline /><description /> <web>http://www.gdacs.org/reports.aspx?eventype=FL&amp;eventid=4159</web> <parameter><valueName>eventid</valueName><value>4159</value></parameter> <parameter><valueName>currentepisodeid</valueName><value>1</value></parameter> <parameter><valueName>glide</valueName><value /></parameter> <parameter><valueName>version</valueName><value>1</value></parameter> <parameter><valueName>fromdate</valueName>
<value>Wed, 21 May 2014 22:00:00 GMT</value></parameter> <parameter><valueName>todate</valueName>
<value>Mon, 14 Jul 2014 21:59:59 GMT</value></parameter> <parameter><valueName>eventtype</valueName><value>FL</value></parameter> <parameter><valueName>alertlevel</valueName><value>Green</value></parameter> <parameter><valueName>alerttype</valueName><value>automatic</value></parameter> <parameter><valueName>link</valueName>
<value>http://www.gdacs.org/report.aspx?eventtype=FL&amp;eventid=4159</value> </parameter> <parameter><valueName>country</valueName><value>Brazil</value></parameter> <parameter><valueName>eventname</valueName><value /></parameter> <parameter><valueName>severity</valueName><value>Magnitude 7.44</value></parameter> <parameter><valueName>population</valueName><value>0 killed and 0 displaced</value> </parameter> <parameter><valueName>vulnerability</valueName><value /></parameter> <parameter><valueName>sourceid</valueName><value>DFO</value></parameter> <parameter><valueName>iso3</valueName><value /></parameter> <parameter><valueName>hazardcomponents</valueName> <value>FL,dead=0,displaced=0,main_cause=Heavy Rain,severity=2,sqkm=256564.57 </value></parameter> <parameter><valueName>datemodified</valueName>
<value>Mon, 01 Jan 0001 00:00:00 GMT</value></parameter>
<area><areaDesc>Polygon</areaDesc><polygon>,,100</polygon></area>
</info>
</alert>
Listing 1. Common Alerting Protocol feed via the Global Disaster Alert and
Coordination System (http://www.gdacs.org/xml/gdacs_cap.xml, 2014-07-16)
at Google in 2011, remarks in [
Wikipedia is an international online encyclopedia currently available in 287 languages.13 (i) Articles in one language are interlinked with versions of the same article in other languages, e.g., the article \Natural disaster" on the English Wikipedia (http://en.wikipedia.org/wiki/Natural_disaster) links to 74 versions of this article in other languages.14 (ii) Each article can have redirects, i.e., alternative URLs that point to the article. For the English \Natural disaster" article, there are eight redirects,15 e.g., \Natural Hazard" (synonym), \Examples of natural disaster" (re nement), or \Natural disasters" (plural). (iii) For each article, the list of back links that link to the current article is available, i.e., inbound links other than redirects. The article \Natural disaster" has more than 500 articles that link to it.16 Likewise, the list of out9 Twitter: https://twitter.com/ 10 Wikimapia: http://wikimapia.org/ 11 Flickr: https://www.flickr.com/ 12 Wikimedia Commons: https://commons.wikimedia.org/ 13 All Wikipedias: http://meta.wikimedia.org/wiki/List_of_Wikipedias 14 Article language links: http://en.wikipedia.org/w/api.php?action= query&prop=langlinks&lllimit=max&titles=Natural_disaster 15 Article redirects: http://en.wikipedia.org/w/api.php?action=query& list=backlinks&blfilterredir=redirects&bllimit=max&bltitle= Natural_disaster 16 Article inbound links: http://en.wikipedia.org/w/api.php?action= query&list=backlinks&bllimit=max&blnamespace=0&bltitle= Natural_disaster bound links, i.e., other articles that the current article links to, is available.17 By combining an article's in- and outbound links, we determine the set of mutual links, i.e., the set of articles that the current article links to (outbound links) and at the same time receives links from (inbound links). 3.2 Identi cation of Wikipedia Articles for Monitoring Starting with the well-curated English seed article \Natural disaster", we programmatically follow each of the therein contained links of type \Main article:", which leads to an exhaustive list of English articles of concrete types of natural disasters, e.g., \Tsunami" (http://en.wikipedia.org/wiki/ Tsunami), \Flood" (http://en.wikipedia.org/wiki/Flood), \Earthquake" (http://en.wikipedia.org/wiki/Earthquake), etc. In total, we obtain links to 20 English articles about di erent types of natural disasters.18 For each of these English natural disasters articles, we obtain all versions of each article in di erent languages [step (i) above], and of the resulting list of international articles in turn all their redirect URLs [step (ii) above]. The intermediate result is a complete list of all (currently 1,270) articles in all Wikipedia languages and all their redirects that have any type of natural disaster as their subject. We call this list the \natural disasters list" and make it publicly available in di erent formats (.txt, .tsv, and .json), where the JSON version is the most exible and recommended one.19 Finally, we obtain for each of the 1,270 articles in the \natural disasters list" all their back links, i.e., their inbound links [step (iii) above], which serves to detect instances of natural disasters unknown at system creation time. For example, the article \Typhoon Rammasun (2014)" (http://en.wikipedia.org/wiki/Typhoon_Rammasun_ (2014))|which, as a concrete instance of a natural disaster of type tropical cyclone, is not contained in our \natural disasters list"|links back to \Tropical cyclone" (http://en.wikipedia.org/wiki/Tropical_cyclone), so we can identify \Typhoon Rammasun (2014)" as related to tropical cyclones (but not necessarily identify as a tropical cyclone), even if at the system's creation time the typhoon did not exist yet. Analog to the inbound links, we obtain all outbound links of all articles in the \natural disasters list", e.g., \Tropical cyclone" has an outbound link to \2014 Paci c typhoon season" (http:// en.wikipedia.org/wiki/2014_Pacific_typhoon_season), which also 17 Article outbound links: http://en.wikipedia.org/w/api.php?action= query&prop=links&plnamespace=0&format=json&pllimit=max&titles= Natural_disaster 18 \Avalanche", \Blizzard", \Cyclone", \Drought", \Earthquake", \Epidemic", \Extratropical cyclone", \Flood", \Gamma-ray burst", \Hail", \Heat wave", \Impact event", \Limnic eruption", \Meteorological disaster", \Solar are", \Tornado", \Tropical cyclone", \Tsunami", \Volcanic eruption", \Wild re" 19 \Natural disasters list": https://github.com/tomayac/postdoc/ blob/master/papers/comprehensive-wikipedia-monitoring-forglobal-and-realtime-natural-disaster-detection/data/naturaldisasters-list.json happens to be an inbound link of \Tropical cyclone", so we have detected a mutual, circular link structure. Figure 1 shows the example in its entirety, starting from the seed level, to the disaster type level, to the in-/outbound link level. The end result is a large list called the \monitoring list" of all articles in all Wikipedia languages that are somehow|via a redirect, inbound, or outbound link (or resulting mutual link)|related to any of the articles in the \natural disasters list". We make a snapshot of this dynamic \monitoring list" available for reference,20 but note that it will be out-of-date soon and should be regenerated on a regular basis. The current version holds 141,001 di erent articles.
Legend: seed level disaster type level in-/outbound link level English German de:Pazifische Taifunsaison 2014 (inbound link) de:Tropischer
Wirbelsturm (language link) en:Natural (seed article)
disaster (redirect) en:Tropical en:Tropical storm cyclone
(mutual link)
en:2014 Pacific
typhoon season
en:Typhoon
Rammasun (2014)
(inbound link)
en:Disaster
preparedness
(outbound link)
In the past, we have worked on a Server-Sent Events (SSE) API [
In order to detect spikes, we apply exponential smoothing to the last n edit intervals (we require n 5) that occurred in the past 24 hours with a smoothing factor = 0:5. The therefore required edit events are retrieved programmatically via the Wikipedia API.21 As a spike occurs when an edit interval gets \short enough" compared to historic editing activity, we report a spike whenever the latest edit interval is shorter than half a standard deviation 0:5 .
A subset of all Wikipedia articles are geo-referenced,22 so when we detect a spiking article, we try to obtain geo coordinates for the article itself (e.g., \Pazi sche Taifunsaison 2014") or any of its language links that|as a consequence of the assumption in H2|may provide more local details (e.g., \2014 Paci c typhoon season" in English or \2014年太平洋季" in Chinese). We then calculate the center point of all obtained latitude/longitude pairs.
In a nal step, once a given con dence threshold has been reached and upon human inspection, we plan to send out a noti cation according to the Common Alerting Protocol following the format that (for GDACS) can be seen in Listing 1.
We have created a publicly available prototypal demo application deployed23
at http://disaster-monitor.herokuapp.com/ that internally connects
to the SSE API from [
We recall our core research questions that were Q1 How timely and accurate
for the purpose of natural disaster detection is content from Wikipedia compared
to authoritative sources mentioned above? and Q2 Does the disambiguated
nature of Wikipedia surpass keyword-based natural disaster detection approaches,
21 Wikipedia last revisions: http://en.wikipedia.org/w/api.php?action=
query&prop=revisions&rvlimit=6&rvprop=timestamp|user&titles=
Typhoon_Rammasun_(2014)
22 Article geo coordinates: http://en.wikipedia.org/w/api.php?action=
query&prop=coordinates&format=json&colimit=max&coprop=dim|
country|region|globe&coprimary=all&titles=September_11_attacks
23 Source code: https://github.com/tomayac/postdoc/tree/master/
demos/disaster-monitor
e.g., via online social networks or search logs? Regarding Q1, only a manual
comparison covering several months worth of natural disaster data of the
relevant authoritative data sources mentioned in Subsection 1.1 with the output
of our system can help respond to the question. Regarding Q2, we propose an
evaluation strategy for the OSN site Twitter, loosely inspired by the approach
of Sakaki et al. in [
showCandidateArticle(data.article, data.language, disasterObj); } }; }; // fired whenever an article is on the monitoring list var showCandidateArticle = function(article, language, roles) { getGeoData(article, language, function(err, geoData) { getRevisionsData(article, language, function(err, revisionsData) { if (revisionsData.spiking) {
// spiking article } if (geoData.averageCoordinates.lat) {
// geo-referenced article, create map } // trigger alert if article is spiking }); }); getMonitoringList(seedArticle, function(err, data) { // get the initial monitoring list if (err) {
return console.log(’Error initializing the app.’); } monitoringList = data; console.log(’Monitoring ’ + Object.keys(monitoringList).length +
’ candidate Wikipedia articles.’); // start monitoring process once we have a monitoring list var wikiSource = new EventSource(wikipediaEdits); wikiSource.addEventListener(’message’, function(e) {
return parseWikipediaEdit(JSON.parse(e.data)); }); // auto-refresh monitoring list every hour setInterval(function() { getMonitoringList(seedArticle, function(err, data) { if (err) {
return console.log(’Error refreshing monitoring list.’); available user data through its streaming APIs,24 which would be considerably harder, if not impossible, with other OSNs or search logs due to privacy concerns and API limitations. Based on the articles in the \monitoring list", we put forward using article titles as search terms, but without disambiguation hints in parentheses, e.g., instead of the complete article title \Typhoon Rammasun (2014)", we suggest using \Typhoon Rammasun" alone. We advise monitoring the sample stream25 for the appearance of any of the search terms, as the ltered stream26 is too limited regarding the number of supported search terms. In order to avoid ambiguity issues with the international multi-language tweet stream, we recommend matching search terms only if the Twitter-detected tweet language equals the search term's language, e.g., English, as in \Typhoon Rammasun". 5
In this paper, we have presented rst steps of our ongoing research on the creation of a Wikipedia-based natural disaster monitoring system, in particular, we have nished its underlying code sca olding. While the system itself already works, a good chunk of work still lies ahead with the ne-tuning of its parameters. A rst examples are the exponential smoothing parameters of the revision intervals, responsible for determining whether an article is spiking, and thus a potential new natural disaster, or not. A second example is the role that natural disasters play with articles: they can be inbound, outbound, or mutual links, 24 Twitter streaming APIs: https://dev.twitter.com/docs/streamingapis/streams/public 25 Twitter sample stream: https://dev.twitter.com/docs/api/1.1/get/ statuses/sample 26 Twitter ltered stream: https://dev.twitter.com/docs/api/1.1/post/ statuses/filter and their importance for actual occurrences of disasters will vary. Future work will mainly focus on nding answers to our research questions Q1 and Q2 and the veri cation of the hypotheses H1{H3. We will focus on the evaluation of the system's usefulness, accuracy, and timeliness in comparison to other keyword-based approaches. An interesting aspect of our work is that the monitoring system is not limited to natural disasters. Using an analog approach, we can monitor for human-made disasters (called \Anthropogenic hazard" on Wikipedia) like terrorism, war, power outages, air disasters, etc. We have created an exemplary \monitoring list" and made it available.27 Concluding, we are excited about this research and look forward to putting the nal system into operational practice.