Introduction

In the normal course of events, emergencies like natural disasters, military escalation, epidemic outbreaks etc. are almost immediately followed by some response, such as containment and mitigation efforts, counterattack, quarantine, etc., often within minutes or hours, and much of the work on emergency response is concerned with exploiting this short-term dynamics. Yet for preparedness, a key aspect of the work is the identification, ahead of time, of potentially implicated locations (LOC), organizations (ORG), and persons (PER). Here we describe how static repositories of traditional news reports, operating on a much slower (typically, daily) news cycle can be exploited to yield disaster-or accident-implicated named entities. Most of our results are on English data, but the bootstrap method proposed here works for any language, and to show this we evaluate our method on Hungarian as well.

Section 2 describes the current state of the Basic Emergency Vocabulary (BEV) that can be used as the basis of the classifiers we use to select emergencyrelated material in English and other languages. Section 3 outlines the main method of collecting emergency-implicated NERs, Section 4 describes an ad hoc evaluation, and Section 5 offers some conclusions.

Basic emergency vocabulary

The idea that there is a basic vocabulary composed of a few hundred or at most a few thousand elements goes back to the Renaissance -for a more detailed history see [1], for a contemporary list see [6]. The emergency vocabulary serves a dual purpose: first, these words are the English bindings for deep semantic (conceptual) representations that can be used as an interlingual pivot or as a direct hook into knowledge-based (inferential) systems; and second, these words act as a reasonably high-precision high-recall filter on documents that are deemed relevant for emergencies: newspaper/newswire articles, situation reports, etc. In fact, rough translations of these words into a target language T can serve as a filter for emergency-specific text in T, a capability we evaluate on Hungarian in Section 4.

In terms of applications, the basic concept list promises a strategy of gradually extending the vocabulary from the simple to the more complex and conversely, reducing the complex to the more simple. Thus, to define asphyxiant3 as 'chemical causing suffocation', we need to define suffocation, but not chemical as this item is already listed in the basic set. Since suffocate is defined as 'to lose one's life because of lack of air', by substitution we will obtain for asphyxiant the definition 'chemical causing loss of life because of lack of air' where all the key items chemical, lose, life, because, lack, air are part of the basic set. Proper nouns like Jesus are discussed further in [5], but we note here that they constitute a very small proportion (less than 6%) of the basic vocabulary. None of these basic entities, whose list is restricted to names of continents, countries, major cities, founders of religions, etc., are particularly implicated in disasters or accidents, so our method (for a theoretical justification see [7]) involves no seeding for the actual entity categories we wish to learn -our seed lists, minimal as they are, contain only common nouns, verbs, adjectives and adverbs.

At 1,200 items, the basic list was small enough to permit manual selection of a seed emergency list, about 1/10th of the basic list, by the following principles. First, we included from the basic list every word that is, in and of itself, suggestive of emergency, such as danger, harm, or pain. Second, we selected all concepts that are likely causes of emergency, such as accident, attack, volcano, or war. Third, we selected all concepts that are concomitant with emergencies, such as damage, Dr, or treatment. Fourth, because of semantic decomposition, we added those concepts that signal emergencies only in the negative, such as breathe or safe (can't breathe, not safe, unsafe). Fifth, and final, we added those words that will, on our judgment, appear commonly in situation reports, news articles, or even tweets related to emergencies such as calm, effort, equipment, or situation. The full list of these manually selected entries is given in Appendix A.

It is evident that many emergency words are not basic: examples include jettisoning and typhoon. To obtain a good sample, we analyzed the the glossary [2] using the same principles as above. This yielded another 267 words like hazmat or thermonuclear. These were taken, for the most part, from the definitions in the glossary, not the headwords, especially as the latter are often highly specific to the organization of US emergency response procedures, while our goal is to build a language-independent set of concepts, not something specific to American English. The Basic Emergency Vocabulary (see Appendix B) contains 350 emergency-related concepts -none of these are proper names.

Of the 260 words found in the Glossary, only 41 appear on the basic list, and of these, only half (22) were found on the first manual pass over the basic list. In hindsight it is clear that the remaining 19, area authority care develop event exercise field general heat measure officer plan protection range search skin smoke team waste, should also have been selected based on the above principles, especially the last (fifth) one.

The lesson from this is clear: the list has to be built from emergency materials, rather than by human expertise. But there is something of a chicken and egg problem here: to have a good list, we need to have a good corpus of emergency materials, to have a good corpus, we need to build a good classifier, and to build a good classifier, we need a good list. In the next Section we describe a method of jointly bootstrapping the list and the emergency corpus.

Finding the NERs involved

Using the BEV as positive evidence [7], it is possible to select a small, emergencyrelated subset of a given corpus C of articles (we used the New Reuters collection of 806,791 news stories) by a simple, semi-automatic iterative process. First, the articles were indexed by a search engine, and the BEV was used as the initial search query. Of the documents returned by the engine, only the most relevant N were retained. The threshold was selected in such a way that in a window of documents around it, about half should be emergency-related. A linear search from the top would have obviously been infeasible, but with a binary search among D documents with a window size W , N can be found by looking at only W log 2 (D) documents -in our case we only had to look at 80 documents of the entire corpus to select a core set E of about 2,000 emergency-related articles. This is a noisy sample, only about 80% of the documents in it are actually emergency related, and we estimate recall also to be only about 80%, so there may still be about 400 further emergency-related articles in the corpus. An Fmeasure of .8 will not be impressive if our goal was detecting emergency-related articles in a live stream, but here it does not unduly affect the logic of our enterprise: since a random document in the corpus C will be emergency related with probability p = 0.0025, but in the subset E with probability p = 0.8, words in the subsample are far more likely to be emergency-prone. To quantify this, we computed log text frequency ratio ∆ = log(T F (w, E)/T F (w, C)) for each word, and looked at those 1,700 words where this exceed the expected zero log ratio by at least two natural orders of magnitude. Of these, the ratio is greater than 3 for about a quarter (472 items), and greater than 4 for about one in 12 (135).

Since we have only 2k relatively short documents to consider, we ran the NER system from Stanford CoreNLP on these, and collected the results for all 1,700 words. Typically, words are classified unambiguously (label entropy is below 0. Two-thirds of the words in the list are emergency-related common nouns (e.g. levee, floodwater, mudslide). This number is so significant that we could in fact dispense with the BEV, and bootstrap the classifier starting with only two words: emergency and urgent.

Looking at the documents that contain at least one of these two words we can obtain an emergency-related corpus of documents E'. While the top of the list of words that are significantly more frequent in E' than in the background are not quite as good as the actual BEV listed in Appendix B (e.g. it has outright false positives like nirmala), it is good enough for further iteration. The emergency sets obtained from the BEV and from this skeletal list are practically identical, a matter we shall investigate more formally in Section 4 for Hungarian, and so are the lists of NERs.

To summarize what we have so far, we propose to identify emergency-implicated NERs by searching for those NERs that occur in an emergency-related subcorpus considerably more frequently than in the corpus as a whole. Certainly, among the hundreds of thousands of locations in NewReuters, the method puts at the top Hohenwutzen, Slibice, and Oderbruch, still very much exposed to floods of the river Oder, and Popocatepetl, a volcano that has been implicated in half a dozen new eruptions since the corpus was collected. Among persons, the top choices are 'Matthias Platzeck, environment minister in the German state of Brandenburg' and 'government crisis committee spokesman Krzysztof Pomes'.

Evaluation

It is clear that the precision of the system is reasonably high, even at the bottom of the range we get locations like Key Biscayne and good classifier words like around-the-clock. To measure recall is much harder, and it would take manual analysis of larger samples to obtain significant figures. Therefore, we decided to validate the basic idea of iteratively bootstrapping the keyword-and the document-set on a different language, Hungarian. We use the MagyarHirlap collection of some 44,000 newspaper articles, and start with only three words, katasztrófa 'catastrophy', vészhelyzet 'emergency situation', and áldozat 'victim, sacrifice'. (Hungarian doesn't have a word that could be used both as a noun and an adjective to denote emergency.) word 3.68 16 Based on these words, we found a small document set (170 documents) from which we repeated the process. The resulting wordlist required manual editing, primarily to take care of tokenization artifacts, but the top 35 words already show the same tendency, with several emergency-implicated locations (Goma, Rwanda, Chernobyl) and excellent keywords for a second pass such as vulkánkitörés 'volcanic eruption', evakuál 'evacuate', or segélyszállítmány 'relief supplies'. There are also entries such as 33-as '#33' which require local knowledge to understand (there was flooding along route 33 in Hungary at the time) and morphology is a much more serious issue: we see e.g. the locative adjectival form ruandai 'of or pertaining to Rwanda' along with the country name.

∆

Although the TF values are really too small for this, we performed another iteration, obtaining a slightly longer document list, and a much longer wordlist, containing many excellent keywords that could not be obtained by translating the BEV to Hungarian, supporting the observation we already made in regards to English, that manual word selection has low recall. In fact, the wordlist we obtained by a dictionary-based translation of BEV had too many elements (over 2,200) and was dominated by false positives (valid Hungarian translations that corresponded to some sense of English keywords that were not emergencyrelated).

Conclusions

Faced with the problem of building a two-way classifier selecting a small class of emergency reports from a much larger set of other (non-emergency) texts, it is tempting to put the emphasis on non-textual features such as the snowballing of reports from the same area. Here we considered 'emergency' to be a topic on a par with 'sports' or 'computer science' or any other topic in a well-established topic hierarchy, and assumed that reports coming in later will often have reference not just to the event, but to the response as well.

This assumption is clearly borne out by the vocabularies, not so much by the BEV (which was built by knowledge engineering, with the response assumption already built in), as by the lists built iteratively based on very small seeds (in English, two words, in Hungarian, three words). The first iteration already yields words like English evacuee or Hungarian segélyszervezet 'aid organization' that only makes sense in the context of some organized response.

In the near future we plan to use the method for a systematic selection of a far larger emergency corpus (since we build linear classifiers, the time to do it is linear in the size of the corpus), with the expectation that not more than a quarter of a percent of the material in a static news corpus will be selected. Once the corpus is at hand, we can use standard NER techniques to designate persons, organizations, and locations as emergency-implicated. We plan to investigate whether in the context of iterated keyword-weight bootstrap the simple recall-based ranking of selecting and weighing keywords advocated in [7] is outperformed by the slightly more complex Bi-Normal Separation method advocated in [3].

The key benefit of our proposal is that it only requires a collection of documents, typically easily obtained by web crawl even in less well resourced languages -everything else can be bootstrapped from minimal seeds of 2-3 words. In better resourced languages, the iterative keyword selection method used here can be compared to one based on word vectors [4] or we can hybridize the two -we leave this for further research. combustible compromised concentrated concern condition consequence containment contaminate cooling coordinate corrective could counterterrorism crime crisis critical crush damage danger dangerous dead debris decay declaration decontaminate defective defense degrade demolition department designated destroy destruction deteriorate develop device diarrhea die dig disaster discharge disease disperse dose dosimeter downgrade drill drug earthquake effort embargo emergency emission end energy enough environment equipment error escape escort evacuate event exceed exclusion exercise explode explosion explosive expose exposure extreme facility fail failure fall fallout fatality fatigue fault field fight filter fire firefighter fission fissionable flammable flashpoint flesh food force frighten fuel fuse gas general grain grenade half-life harm hazard hazmat headquarters health heat hemorrhage herbicide hospital hot hotline hurricane hurt ice ignition ill illness impact inadequate inadvertent incident infect inflammation ingest inhale injure installation ionization issue jettison launch leak lethal level liason lifethreatening lightning limit loss lost malevolent malfunction management mass meal measure medical microorganism mine missile mitigate mobilize monitoring mortality nausea necessary notify nuclear offensive officer offsite operation organization pain parameter people perimeter pesticde plan plume plutonium poison police pollute pose powerful preparedness prevent problem procedure protect public quarantine quick rad radiation radio radioactive radiology rain range react reactor recovery reentry release rem report request resolution respiration responder response restoration risk rocket rod rule sabotage safe safeguard safety scenario sea search secure security serious severe shelter shield shock shoot sick sickness sink site situation skin smoke snow social soldier spark special speed spill stabilization staging stolen stop strike strong suffocation supply surprise symptom tank target team temperature tent terrorism thermal thermonuclear thick thin threat tornado toxic toxin travel treatment tritium trouble typhoon unconscious uncontrolled unexpected unintended unintentional unstable uranium urgent vehicle victim violation violent vital volatile volcano vomiting vulnerability vulnerable war warhead waste weakness weapon weather wind worry wound zone