This paper describes our approach on \Information Extraction from Microblogs Posted during Disasters" as an attempt in the shared task of the Microblog Track at Forum for Information Retrieval Evaluation (FIRE) 2016 [2]. Our method uses vector space word embeddings to extract information from microblogs (tweets) related to disaster scenarios, and can be replicated across various domains. The system, which shows encouraging performance, was evaluated on the Twitter dataset provided by the FIRE 2016 shared task.
Social media plays a very important role in the
dissemination of real-time information such as disaster outbreaks.
E cient processing of information from social media
websites such as Twitter can help us to pursue proper disaster
mitigation strategies. Extracting relevant information from
tweets proves to be a challenging task, owing to their short
and noisy nature. Information extraction from social media
text is a well researched problem [
A set of tweets, T = ft1; t2; t3 : : : tng and a set of topics, Q = fq1; q2; q3 : : : qmg are given. Each topic contains a title, a brief description and a detailed narrative on what type of tweets are considered relevant to the topic. The tweets given in the task were posted during the Nepal earthquake1 in April 2015. Each topic contains a broad information need during a disaster, such as { availability or requirement of general or medical resources by the population in the disaster a ected area, availability or requirement of resources in a geographical region, reports of relief being carried out by an organization and reports of damage to infrastructure. The main objective of this task is to extract all tweets, ti 2 T that are relevant to each topic, qj 2 Q with high precision and high recall, and rank them in their order of relevance. 3.
This section describes the dataset and resources provided to the shared task participants. A text le containing 50,068 tweet identi ers that were posted during the Nepal earthquake in April 2015, was provided by the organizers. A Python script was provided that downloaded the tweets using the Twitter API2 into a JSON encoded tweet le which was processed during the task. A text le of topic descriptions in TREC3 format was provided, that contained information necessary for the extraction of relevant tweets. The topic le consisted of 7 topics: FMT1, FMT2, FMT3, FMT4, FMT5, FMT6 and FMT7. Each topic consisted of the following 4 sections: <num> : Topic number. <title> : Title of the topic. <desc> : Description of the topic. <narr> : A detailed narrative which describes what types of tweets would be considered relevant to the topic. 4. 4.1
We parsed the JSON encoded tweets and retrieved the following attributes { tweet identi er, tweet, geolocation. From the tweets, we removed the Twitter handles starting with @, URLs and all punctuation marks except the instances of a single \ . " (period) and a single \ , " (comma), using regular expressions. We removed the ASCII characters from the tweets and converted the remaining tweet to lower case characters.
We also preprocessed the <narr> sections of the topic le. We removed the punctuation marks, stop words and converted the text to lower case characters. The preprocessed <narr> sections for each topic was used for building the word bags. *Indicates equal contribution. 1http://en.wikipedia.org/wiki/April 2015 Nepal earthquake 2https://dev.twitter.com/overview/api/tweets 3http://trec.nist.gov 4.2
To build the topic-speci c word bags, the preprocessed <narr> section was manually checked to retain the relevant words for each topic. The topic words were expanded using the synonyms obtained from NLTK WordNet4. The past, past participle and present continuous forms of verbs were obtained using the NodeBox5 library for Python. Vowels, except the initial character, were removed to create unnormalized version of the words which are generally used in Twitter owing to the 140 character limit. The resultant set of words were used to create the word bags for each topic. 4.3
For the topics FMT5 and FMT6, location and organization information was required to be detected from the tweet. To extract the location information, we used the geo-location attribute from the tweets and the Stanford NER tagger6 to extract location names from the tweet text. Similarly, we used the Stanford NER tagger to detect organizations in the tweet text.
We split the tweet le into 10 les containing 5,000 tweets each. The Stanford NER tagger was used in parallel on the 10 splitted les to identify the location and organization, if any. This reduced the computation time by 85%. 4.4
We used the pre-trained 200 dimensional GloVe [
The tweet vectors were created by taking the normalized summation of the vectors of the words in the tweets, which were present in the vocabulary of the pre-trained GloVe model. In cases where the word was not a part of the model vocabulary, it was assigned to the null vector.
# ti =
1
Nv (ti) j=1 # # and, uij = 0 ;
Nv(ti) #
X uij if uij 2= vocabulary where, # ti = Tweet vector of ith tweet, ti.
Nv (ti) = Number of words in ti present in vocabulary. # uij = Vector of ith word in jth tweet.
Similarly, the word bag vectors were created by taking the normalized summation of the vectors of the words in the word bags, which were present in the vocabulary of the pre-trained GloVe model. Out of vocabulary words were assigned to the null vector.
# qi =
1
Nv (qi) j=1 # # and, wij = 0 ;
Nv(qi) #
X wij if wij 2= vocabulary 4http://www.nltk.org/howto/wordnet.html 5https://www.nodebox.net/code/index.php/Linguistics 6http://nlp.stanford.edu/software/CRF-NER.shtml 7http://nlp.stanford.edu/projects/glove/ # qi = Topic vector of ith word bag, qi.
Nv (qi) = Number of words in qi present in vocabulary. # wij = Vector of ith word in jth word bag.
# # The tweet vector, ti and the word bag vector, qi are used to calculate the similarity.
The Word2Vec [
We used cosine similarity measure to calculate the cosine similarity, S between the tweet vector and the topic vector. # # S = cosine-sim(ti; qj )
# # = #ti q#j
jjtijj jjqj jj A high v#alue of S denotes higher similarity between the tweet # vector, ti and the topic vector, qj and vice versa.
For topics such as FMT5 and FMT6, where entity information such as location (LOC) or organization (ORG) was required, the consolidated score, S0 was calculated as follows: where,
S0 = I =
S + I
2 8 <1, if LOC or ORG is present. :0,
otherwise.
The consolidated value, S0 shifts the cosine similarity towards 1 if the location or organization information is present (high relevance) and towards 0, otherwise (low relevance). 5.
JU NLP 1 0.4357 JU NLP 2 0.3714 JU NLP 3 0.3714 0.3420 0.0869 0.3004 0.0647 0.3004 0.0647 0.1125 0.0881 0.0881
The secondary performance obtained in Run 2, 3 is a result of the averaging which approximated the actual cosine similarity value between the tweet and topic vectors. Runs 2 and 3, which are identical in nature, used cosine distance as their similarity metric.
8https://radimrehurek.com/gensim/models/word2vec.html 9https://github.com/manasRK/glove-gensim
In this paper, we presented a brief overview of our system to address the information extraction from microblog data. We have observed that, building word bags which contained all the topic words relevant to the topic showed better results than splitting the word bags. Therefore, Run 1 exhibited better results than the rest. Considering hashtags as a feature should also improve the performance of the system.
As a future work, we work like to explore more sophisticated techniques to build the vectors of the tweets, given the vectors of its constituent words, by considering the sequence of the words into account. We also plan to incorporate more topic speci c features to improve the performance of our system. 7.