In the last few years, social networks like Twitter have been a very useful resource for tracking the events that happened before, during and after an earthquake. Several studies of this topic have applied different techniques like Clustering or Temporal models for extracting these events from Twitter. In this paper, however, we propose a new approach for extracting not only the events that happened in the earthquake but also some of its most prominent features like intensity, epicenter and affected places. We performed a lexical syntactical analysis of Spanish and English tweets in order to find the events that happened, in addition to a semantical analysis using statistical metrics and models like Pointwise Mutual Information(PMI) and Latent Dirichlet Allocation(LDA) for extracting the features of the earthquake. Our results show that, by considering the semantics and syntactics of the tweets, we can extract important events and features of an earthquake, which can be used for online detection and tracking of similar disasters. 1Fortune Magazine: http://fortune.com/2017/02/09/twitterq4-2016/
extracting events and features from disasters like
earthquakes
Several studies have been done about this
topic using different kinds of algorithms and
approaches. Some of the most prominent algorithms
found in the literature are probabilistic models
like LSA and Spatial-Temporal models
In that sense, this paper propose an approach that use a LDA topic model with NLP techniques like Named Entity Recognition and Pointwise Mutual information for not only extracting the events related to an earthquake but also extracting its prominent features like epicenter, intensity and affected places.
Twitter is one of the most popular social networks
in the world. As of February 2017, it has 319
million active users1. Because of its privacy policy,
Twitter is vastly used for reporting information
of events
There are some concepts that have to be
considered in order to perform a proper Event
Detection. Weiler (2016) found out that at least 60%
of the related work considered Ground Truth as
an evaluation metric. In Event Detection, Ground
Truth is a metric for evaluating the quality of the
extracted events by making sure that these events
really happened
Bontcheva (2014) classifies Event Detection
approaches into 3 types: Model Based, Clustering
Based and Based on signal’s processing. On the
other side, authors like
From the state of the art’s analysis, we conclude that the current tools and best approaches for Event Detection used labeled datasets, so there is a great opportunity for studying the semantic and syntactics of the text without having a labeled dataset beforehand. Also, we decided to evaluate our results using a manual Ground Truth, which is widely used for these works. 3
Our approach consist of 4 main steps. These steps are described in Figure 1 and are explained in
greater detail in Section 4:
First we identify the earthquake and get a corpus for a fixed ranged of dates. Next we generate a LDA model for determining clusters of terms for a fixed number of topics. Then we used the Stanford Named Entity Recognizer for identifying the entities and places of the earthquakes. With those entities and places, we performed a Lexical and Syntactical analysis to identify the events based of the related tweet for the identified entity.
Finally, we identified some features of the earthquake using Pointwise Mutual Information(PMI) between each pair of words in the clusters of the LDA model. Then, we select the best ”n” relationships, that are the pairs that have greater PMI. With all this information, we plotted the events in a map of the country using the date and hour of the resulting events from our analysis. 4
We developed 5 experiments with corpus from different earthquakes around the world; these can be seen on Table 1.
In the next subsections, the results for the earthquake that happened on Chile on September 16, 2015 are shown. Then, there are tables with the results of entities and places for the other 4 earthquakes, so that the nature of these results can be explained. 4.1
For the extraction phase, we used the open source application GetOldTweets3 for extracting tweets from more than 2 weeks. Different corpus of tweets were generated, one for each earthquake analyzed. Corpus of Chile’s and Ecuador’s earthquakes are in Spanish. Meanwhile, corpus of Japan’s and New Zealand’s earthquakes are in English. The information of the Chile 8.4 earthquake corpus can be seen on Table 2.
In the same manner, the corpus generated for each earthquake had a different quantity of tweets which are shown on Table 3. For supressing the noise of the corpus, we applied regex and stopwords filtering for getting rid of the links, hashtags and stopwords.
Once the cleaning was finished, we used LDA in order to obtain the topics that were represented by different subsets of words as an initial way of identifying some words or terms that could be relevant on earthquakes domain of knowledge.
We experimented with different values for the amount of clusters generated as well as the words per cluster. We ended up choosing to generate 10 clusters with 20 words each and keep the 14 more relevant words of each cluster since after this threshold almost every word started being unrelated to the earthquakes.
3GetOldTweetsrepository: https://github.com/JeffersonHenrique/GetOldTweets-python
On Table 4 we can see some words related to this specific earthquake, such as ”chile” and ”santiago”, which relate to the place where the earthquake took place, as well as 8 and 4, which in this case indicate the degree of the earthquake: 8.4.
We also obtained some other words we might expect to find on any earthquake such as ”terremoto”, ”magnitud”, ”temblor” and ”sismo”. 4.3
Afterwards, we proceeded to use the Stanford Named Entity Recognizer with previously trained models for Spanish and English, focusing on obtaining places relevant to each earthquake corpus.
On Table 5 the places with higher frequency are presented, aside from Chile which had the highest with 9303. We performed a Lexical and Syntactical analysis as a way to retrieve relevant entities from the earthquakes. In those entities we expected to find some locations in similar manner to Section 4.3, but also some groups, entities or words which were heavily involved in those events, such as affected people or assistance actions.
For this we used the UDPipe4 parser in order to obtain the grammatical categories of each word on a tweet, as well as the syntactic tree.
The grammatical categories which were worked upon were the ’SUBJ’ tag for subjects.
Following this procedure, each tweet was assigned to the subjects found inside it; this was used to obtain tweets grouped by subject and verb. 4.4.1
If we just used all the words tagged as ’SUBJ’ the result would be a large amount of entities. In order to reduce this amount and keep the most relevant entities, a pruning process was performed, where the entities that had between 10% and 100% of the maximum subject frequency where the ones that remained.
After the entities have been pruned, a clustering process occurs, in which the remaining entities that surpass a minimum threshold of similarity were clustered together and treated as the same.
The entities shown on Table 6 where clustered via 2 metrics: 1. Levenshtein distance: minimum replacements/insertions/deletions needed to turn one word into another. 2. Average Entity to entity distance in a tweet The minimum Levenshtein distance in order to cluster two entities together was 20% of the maximum length between the two entities and the minimum average entity to entity distance was 1.5. Both of these values were obtained via experimentation and results reviewing.
On Table 7 a tweet for each entity is shown. Each entity has more tweets similar to these, being the entity ”chile” the one that has the most.
The tweets per entity were classified as an event
inside the earthquakes since they served the
purpose of explaining what happened to each entity
in the corpus.
Pointwise mutual information(PMI) is an
information measure for identifying how much related are
two variables. In information retrieval, it’s vastly
used for identifying words that are very related in
function of their co-occurrences in a document,
that is, how many times those words appear
together in the same document
P (x, y) ⇤ N
log2 P (x) ⇤ P (y)
where P(x,y) is the number of co-occurrences
between the words ’X’ and ’Y’, P(x) and P(y) are the
frequencies of the words X and Y respectively and
N is the number of documents
There has been a wide range of applications of PMI in Natural Language Processing. Rana (2016) found that is one of the most common metric for relationship extraction in Ontology Learning, because it considers the semantic relationship between entities, which is what we wanted for extracting the most prominent features of the earthquake.
We compute PMI between the topics and terms identified in the LDA Model. Based on that, we identified the pair of words with a greater PMI and mapped this pair with a feature of the earthquake. The results are shown on Table 8. We observed that we extracted interesting properties of the earthquake like its intensity and epicenter. In the case of the epicenter, we identified Illapel as the epicenter of the earthquake, which is true based of the news sources. 5
The identified places possibly related to the earthquakes previously mentioned (except the earthquake of Chile from 2015) are shown on Tables 9 ,10 , 11 and 12.
Besides, as it was mentioned on Table 5, all the places shown have the higher frequency without taking in consideration the respective country of each earthquake (Chile, Ecuador, New Zealand and Japan respectively) .
We observed many different places related to an earthquake, not only the country where it happened. This is possible due to different factors such as other countries that might be affected as well shown in Ecuador’s possible places, which includes one of its neighbors Colombia. Also the date of the earthquake can affect the results, such as for Japan’s earthquake, which includes Ecuador since both earthquakes happened around the same week.
Below, entities and tweets associated with each earthquake are shown on Tables 13, 14, 15, 16, in order to see their relevance.
We consider each of the tweets related to an entity as en event. For testing the validity of every event, we compute the manual Ground Truth, which can be seen as ”How many of these events really happened?”. The state of art suggest that having many news sources helps to improve the quality of the results, so we considered some of the most important news sources like CNN, BBC, New York Times and The Associated Press. Once the source news were established, we check every event with their articles that covers a particular earthquake, so if the event was verified by each of the source news, we considered him as valid. The results from this evaluation are presented on Table 17. From the summaries of Weiler (2016) and Winarko (2013), we notice that the values of Ground Truth for many event detection works varies between 60% to 90%, so we got very good results comparing with the state of the art. This results, however, are not very accurate because of different factors such as the size of the corpus, the range of dates and so on. That’s why, there is room for improvement in our evaluation measure. 6.1
Our motivation was to propose a new approach for earthquake detection and tracking systems, so we had to make sure that our approach can be adapted to those systems. That is the reason why we implemented a visualization module that can show the events and entities detected by our approach in real time. 6.2
We used the information obtained from the tweets associated with each earthquake, in order to show in a world map different factors of the earthquakes such as the epicenter, intensity or how was the frequency of tweets at different times. For this purpose we use the software Mathematica5, which is an incredible tool with helpful features, together with the collected information.
To begin with, we show in Figure 2 a regular picture of Chile with some cities that might be affected by the earthquake of 2015.
Then with the collected information we can reflect in that picture how much information were related with each city during different moments in 5Mathematica: https://www.wolfram.com/mathematica/ the range of dates of the related earthquake. To show this we use different colors and sizes for the cities in Figure 3.
The circles for each city are related with the frequency of tweets per day, a small circle means that the city was not mentioned regularly that day and a big circle means that the city is mentioned in a lot of tweets. Besides, the color scale start from green to red, with yellow being in the middle, so the colors can be a variation of those 3 colors, which depends of the quantity of tweets.
According to this, it seems that Valparaiso is not involved in the earthquake, meanwhile Coquimbo, which has some mentions, is really close to the earthquake. In the same way, Illapel is mentioned very frequently over time, which suggest that is the epicenter, and finally Santiago is mentioned a lot some days after the earthquake, because is very close to Illapel and most of the assistance and supplies came from there. 7
From the experiments, we have shown that, by using NLP techniques and tools like Named Entity Recognition(NER) with a LDA model, it is possible to identify events and features from disasters like earthquakes that can be as good as the ones identified by the state of the art. In that sense, this approach has the following advantages: 1. This approach considers Spanish language, which is poorly found in the literature. 2. By considering the semantics and syntactics of the tweets, better features and events were identified. 3. This approach can compete with the state of the art and may get better results if more data for each earthquake is provided. 4. Our approach can be adapted for an Online Earthquake Detection System, because LDA has an online version which is very used in this type of systems.
For future research, we consider to perform an adaptation of this approach for detecting events in real time. The advantage of this work is that LDA is very good for real time detection, like shown by Aiello (2014) in his summary of Event Detection works. Another improvement is on the visualization module, because if we use Google Maps API for automatic detection of the places, this module will be fully automatized. Finally, using a notmanual Ground Truth like the one proposed by Weiler (2016) may provide a better metric for testing our results.