In this paper, we devise an information retrieval system which can lter and rank tweets according to relevance to the query. We devise methods to understand relationships among entities and action verbs from a small set of manually annotated tweets. We further use these relationships to lter tweets and rank them accordingly. Our results (as published by FIRE Microblog Track) show that we have high precision score in detection of topmost 20 tweets.
FIRE 2016 Microblog track [
We manually annotate a random sample of 1000 tweets into speci ed categories of information, a tweet can also belong to multiple groups. For example, a tweet of destruction of a bridge also might convey information about the requirement of basic amenities. Hence, di erent categories of tweets had di erent text attributes that pertain to a speci c information related to that query. We aimed at identifying those text attributes, i.e, combination of di erent words for any particular query. We further created networks of each query's text attributes' combinations. The edges represent the interrelationships among these text attributes which aid in segregation of tweets according to di erent queries. We will describe the methodology in details in later sections.
Tweets are informal, so a vocabulary gap exists even among tweets of same strata. So, we did not depend only on text analysis of named entities, like food packets but rather combined them with the set of important verbs that identi es a correct relationship among those. We weighed the di erent identi ed keywords of each category according to their relevance to the query. We, thereby, could identify tweets due to their presence of relevant keywords for a query. The published results from FIRE suggest we could accurately identify tweets of high relevance of di erent categories with good precision and recall.
We have divided the paper into following sections. We discuss about data collection and pre-processing in the next section, followed by our procedure of identi cation of tweets in section 3 and nally results and discussion in section 4 and conclusion in section 5. 2.
FIRE Microblog Track provided a set of about 50,000 tweet-ids which we used to the access the tweets. We ltered only the relevant tweet information from these tweets, that consist of tweet text, tweet id, etc. We further ltered some tweets from the whole set of tweets. For example, during disasters, there are a number of tweets that express grief, urge people to pray or help. These messages are general messages, hence we made a bag of words that express only urge, request, pray etc. and removed those tweets that contain words from this bag. 3.
In this section, we discuss our procedure. We do not use any external source of information. We use NLTK toolkit1 to perform text based analysis on tweets. We rely on tweet text attributes to lter tweets of relevance. In order to understand the text attributes of tweets, we select a random sample of 1000 tweets from the whole set of 50,000 tweets. We manually group tweets according to di erent queries by FIRE, a tweet can nevertheless belong to di erent categories.
We perform a set of operations on tweet text for every group (as speci ed before). Firstly, we remove the stopwords from these tweets. Stopwords hardly represent any special characteristic of an entity. After removal of the stop words, we use POS Tagger to select only the nouns and verbs from the tweets. We then rank the entities of all tweets according to frequency. We select a subset of these entities according to the ranks, we also include the entities speci ed in the query itself by FIRE. This step gives us a list of the important entities for a speci c query.
Often, an entity to entity matching fails to resolve tweets of di erent genre, i.e., a tweet containing information of medical aids can either highlight availability or requirement of the same. So, we identify the di erent set of possible actions of any entities, to understand the underlying relationships. We further rank the bigrams to identify the set of working verbs to highlight speci c actions. Thus, this set of related working verbs and entities signify tweets of a particular category.
However, there remains a vocabulary gap among di erent tweets of even same category due to their informal structure. Tweets of both requirement and availability of medical resources may contain entities, like blood and working verbs like donate but are completely di erent in meaning. Hence, segregation only on the basis of keywords fails to di erentiate these relationships. We analyze the context of those keywords relationships, which re ects the actual meaning, as in the absence of question tags (like, where, how, what, etc), or request tags (please, etc) in availability based tweets. The segregation of tweets into di erent categories thus requires identi cation of proper entities, actions, and context to understand it's relevance.
We further have ranked an entity and the action verbs according to their importance, which we will explain later. We formulate separate bipartite graphs for each query, that represents the relationships among the entities, actions and context. While a set of nodes represent entities' names, another set of nodes represent the names of verbs (i.e., actions). These relationships were formulated from the manually annotated tweets. We give a brief overview of the speci c words and their relationships for each query in the next section.
We select the di erent types of action verbs from our manually annotated tweets into similar groups. The main action verbs represent donations, transport, relief inf ormation, build. We represent the relationships between these di erent set of action verbs with di erent set of entities in the graphs 1 and 2 and the set of keywords of each group in table 1. Thus a new tweet is selected if it contains the existing relationship, as mentioned by the arrow, i.e., it must contain at least an entity and verb from the nodes the arrow connects.
Words Representing the Nodes o to nepal survivor, victim,
a ect food, water, cloth, blanket, biscuit, power, plane, bus, material, beef,
equipment volunteer, helicopter, item, tool, app
team shelter, tent, house,
home relief, rescue
donate transfer, sell, distribut, suppl, send, sent, deliver, dispatch, o er, land, deploy, transport,
prepar relief, rescue, working, aid, support, engage, rush build need, want, require
In this section, we intend to lter all tweets that mention the requirement or need of some resource, like human resources or infrastructure like tents, water lter, power supply, etc. We studied our manually annotated tweets, and found the main action verbs that denote requirement of resources are, need related or relief related. We highlight the di erent relationships among these various entities in gure 3 and include details of the di erent terms in table 1. Thus, we later select those tweets from the total list of tweets if it contains the relationship represented by the arrow, i.e., it contains at least an entity and action verb from the list of keywords that the arrow connects. 3.2
In this section, we identify messages that mention the availability of some medical resources like blood, blood bank, medicine, etc. Firstly, we distinguish di erent action verbs from the manually annotated tweets that contain information related to this query, the verbs are namely donation, transport, rescue etc. There are some action verbs that are ambiguous in meaning, example need re ects both the need and the availability of resources. On further analysis of need mentioned tweets, we found need is used in availability of resources tweets only in conditional statements (example, if is a conditional clause).
We represent the actions and their corresponding entities in the next two graphs, namely graph 4 and graph 2, the arrows represent the relationships among the both. The table 2 represents the set of keywords for each entity or action. Thus, we lter all those tweets from the whole set of fty thousand tweets which contain the relationships, i.e., at least a keyword from both the nodes of an arrow. There are also some stringent relationships, that comprise of more than just an entity and action name, as illustrated in graph 4.
In this section, we identify messages that mention the requirement of some medical resources like blood, blood bank, medicine, etc. We represent the actions and their corresponding entities in graph 5, the arrows represent the relationships among the both. The table 2 represents the set of keywords for each entity or action. Thus, we lter all those tweets from the whole set of fty thousand tweets which contain the relationships, i.e., at least a keyword from both the nodes of an arrow. 3.4
In this section, we identify messages that mention the damage or restoration of any communication or structural infrastructures. However, the general statements about any structure is not relevant. We lter the possible set of infrastructure names from our manual annotated tweets and the di erent set of actions related to them. After detection of the relationships among the action verb and entity name from the manually annotated tweets, we select only those tweets that contain We visualize the di erent relationships among di erent set of entities in Figure 6, and highlight the set of keywords in table 3.
Words Representing
the Nodes hotel, debris, building, temple, rubble, tower, road,
bridge, house, railway, dam, tent, heritage, monument, power grid, engineer, equipment, electricity reduce, atten, destroy, devastat,
avalanche, damage, restore, capture, collapse,
build, builds devastat, terrif,
heartbreak footage, image, picture
Words Representing the Nodes
relief, rescue, aid build, transfer, sell, distribut, send, sent, deliver, supply, donat, need
deploy, dispatch, lad, transport, y prepare, o er, launch, allow, provide, make, support, engag, rush, help, working, in action volunteer, food, biscuit, shelter, tent, house, home, cloth, blanket power, equipment, material, item, team, helicopter, bus, plane, call, helpline,contact
The above graphs represent di erent entities, and their set of actions for a particular query. For a given query, we match the relationships among the new tweet with the prescribed relationships. Thus, a tweet is selected if it contains the speci ed relationships of entities of that query. We further rank those tweets according to it's relevance to the query in the next section. 4.1
In this section, we rank the selected tweets by their relevance to query. In order to rank the tweets, we score the di erent keyword relationships of a query. The keywords are segregated into two di erent sections, entities and action verbs. We give importance to words that signify better temporal relevance than others, i.e., there is a major di erence between tweets like food items sent to a ected areas by Indian government , India dispatched 500 packets of rice to Nepal and India will dispatch food packets by saturday . We give a brief description of our scoring mechanism. 1. T emporal Importance : An action verb is given more importance if it highlights immediate action rather than future. This is illustrated by Action2 and Action3,
2. Relevance : Some action verbs, represent greater relevance in times of calamity, as expressed in Action1. Similarly, there are some entities (as in Entity1), which are the basis needs of human livelihood, like food and shelter which are more important than information related to other entity (as in Entity2).
The di erent scores of the keywords are given in table 4. Thus, a tweet's score is the summation of it's keywords' scores. We hereby, could rank the tweets by their relevance score accordingly. Result 0.770 0.4344 0.2186 0.2208
In this section, we highlight our results, FIRE Microblog Track matched our selected tweets with a manual annotator's results. We brie y give an explanation of the metrics and our results are depicted in Table 5. The metrics are. 1. Precision at rank 20, i.e., considering up to the top 20 tweets for each topic.
In this paper, we devise a mechanism to extract the contextual, content relationships of entities. We are able to lter tweets of high relevance for di erent queries by matching these relationships. We require a small number of manual annotated tweets to attain our results. 7.