We introduce our methodology for collecting tweets and identifying event-related actionable information using key terms and in ections. The vast amount of user-generated content makes it challenging to detect relevant information. Therefore, we aim to facilitate extracting morphological, syntactic, and semantic features of a key term semiautomatically. The results of our study show that handling the in ections of key terms separately has its advantages: in the disaster scenario that we are investigating we are successful in discovering relevant and irrelevant information e ectively.
Twitter provides a platform that contains a rich source of user generated content. Any user of this platform can post tweets, and language use tends to be rich and creative. Consequently, increasingly large quantities of novel content is awaiting proper analysis on Twitter. Especially, in a disaster context, the need of detecting precise and complete information is in urgent need of a solution.
In this paper, we present our analysis method and apply it on a tweet collection concerning oods. The method consists of collecting data with a single key term (a stem, and its in ections), pre-processing, extracting extensive features for use in machine learning, and determining coherent clusters in tweet subsets. Human input is used when labeling the clusters.
A central concept in our method is the information thread, i.e, a group of related tweets. Relatedness is determined by the expert who uses the method. For example, the word ` ood' has multiple senses, including `to cover or ll with water' but also `to come in great quantities'4. A water manager will probably want to focus on only the water-related sense. At the same time, he will want to discriminate between di erent contextualizations of this sense: past, current, up-coming events, e ects, measures taken, etc. By incrementally clustering and labeling the tweets, the collection is analyzed into di erent information threads.
Here, we report on the e ect that the use of key term in ections has on knowledge discovery in a tweet collection. In the following sections, we summarize the related research, the tool we developed to support our methodology, the data we collected, and the results obtained. 2
Identifying di erent uses of a word is a word sense induction task [
Detecting relevant information in mass emergencies has considerable
complexity and urgency [
We implemented an open source tool, Relevancer6. Relevancer supports the analysis of the tweet collections with the following steps: Preprocessing Retweets that are identi ed by the respective JSON eld of an API response and the tweets where `RT @' occurs at the beginning of the tweet text are eliminated. Moreover, user names and URLs are converted to `usrusrusr' and `urlurlurl' respectively. After that, we exclude exact duplicate tweets.
Feature extraction Any token that occurs in a tweet text is used as a feature. Tokens are detected based on a split on a single punctuation mark or an arbitrary number of space characters. Features can be words, hashtags, letters, numbers, letter and number combinations, emoticons, or 2, 3, or 4 length punctuation mark combinations.
Near-duplicate detection Tweets that contain the same pentagram (here: 5 consecutive words larger than 2 letters), or have a cosine similarity higher
5 The article in the following URL provides an excellent example of the ambiguity caused by lexical meaning and syntax: http://speld.nl/2016/05/22/ man-rijdt-met-180-kmu-over-a2-van-harkemase-boys/ 6 https://bitbucket.org/hurrial/relevancer than 0.85, based on the features extracted in the previous step, are considered as a group of near duplicates. We keep only one tweet from each near-duplicate group of tweets.
Information thread detection Information threads related to the key term and its in ections are detected using an unsupervised clustering method, viz. K-Means. Clustering and cluster selection steps are repeated in iterations until the requested number of coherent clusters is reached. The tweets in the detected clusters are not included in the following clustering iteration. Annotation An expert assigns an information thread label to each cluster based on its coherency and relevancy.
Remaining and new tweets can be clustered or analyzed with the knowledge discovered in previous iterations of the clustering and annotation process. 4
We collected tweets from the public Twitter API7 with the key term ` ood' and its in ections ` oods', ` ooded', and ` ooding' between December 17 and 31 2015. We applied the analysis steps supported by Relevancer to each subset.
Detailed statistics of the collected tweets are represented in Table 1. The number of collected tweets are provided in the column #All. The columns #unique and unique% contain the counts after eliminating the retweets and duplicate tweets and the percentage of unique tweets in each subset of the collection. The unique tweet ratio for the terms ` ooded' and ` ood' is highest and lowest respectively.
Each subset of unique tweets was clustered in order to identify information threads. The annotation was performed with the labels `relevant', `irrelevant, and `incoherent', which are the most general information threads that can be handled with this approach. We generated 50 clusters for each subset. The number of tweets that were placed in a cluster is presented in the column clustered.
A cluster is relevant, if it is about a relevant information thread, e.g. an actionable insight, an observation, witness reaction, event information available from citizens or authorities that can help people avoid harm. Otherwise, the
label is irrelevant. Clusters that are not clearly about any information thread are labeled as incoherent. Respective columns in Table 1 provide information about the number of tweets in each thread. Having many incoherent clusters points towards the ambiguity of a term.
The detailed cluster analysis revealed the characteristics of the information threads for the key term and its in ections. The key term ` ood' (stem form) is mostly used by the authorities and automatic bots that provide updates about disasters in the form of a ` ood alert' or ` ood warning'. Moreover, mentioning the name of the authorities, e.g, ood advisory, enables tweets to fall in the same cluster. The irrelevant tweets using this term are about ads or product names.
Each in ection of the term ` ood' has a di erent set of uses with a considerable number of overlapping uses. The form ` ooded' is mostly used for expressing observations and opinions toward a disaster event and news article related tweets. General comments and expressions of empathy toward the victims of disasters are found in tweets that contain the form ` oods'. Finally, the form ` ooding' mostly occurs in tweets that are about the consequences of a disaster.
Another aspect that emerged from the cluster analysis is that common and speci c multi-word expressions containing the key term or one of its in ections, e.g. ` ooding back', ` ooding timeline', form at least a cluster around them. Tweets that contain such expressions can be transferred from the remaining tweets set to the respective cluster. For example, we identi ed 806 and 592 tweets that contain ` ooding back' and ` ooding timeline' respectively.
Finally, named entities, such as the name of a storm, river, bridge, road, web platform, person, institution, or place, and emoticons enable the clustering algorithm to detect coherent clusters of tweets containing such named entities. 5
A novel tweet collection and analysis methodology was presented in order to discover detailed and precise information about disaster events on a microblogging platform and in a challenging domain. Our results shows that determining and handling separate uses of a key term and its in ections reveal di erent angles of the knowledge we can discover in tweet collections. The majority of the tweets that were not placed in any cluster can be handled with the knowledge we discovered in the annotation phase, which provides insights about the semantic and syntactic space each subset covers.
This methodology can be applied to any term in any language. The labeled tweets can be used to create supervised machine learning models in order to handle the remaining tweets in the collection as well as new tweets. Acknowledgments. This research was supported by the Dutch national research programme COMMIT. We gratefully acknowledge the contribution by Floodtags.