This paper describes the (usupervised) knowledge-based approach to lter relevant tweets for a given entity that is followed by the UNED-READERS system at RepLab 2013. The approach relies on a new way of contextualizing entity names from relative large and broad collections of texts using probabilistic signature models (i.e., discrete probability distributions of words lexically related to the knowledge or topic underlying set of entities in background text collections). The contextualization is intended to recover relevant information about the entity (speci cally, lexically related words) from background knowledge. Results obtained in the ltering task are presented.
In the last few years, Twitter has become an increasingly popular platform for users to communicate with each other by sharing their latest updates in the form of short messages called tweets. 1 Because tweets are compact and fast, Twitter has become widely used to spread and share many kinds of information, such as breaking news, personal updates, spontaneous ideas, or opinions about brands, services, celebrities, etc. [6].
This phenomenon has attracted the attention of information analysts, who see in tweet datasets a valuable input source to carry out a plethora of studies. The large and ever increasing volume of available tweets obliges to make use of automatic methods and tools to interpret and analyze them. Nevertheless, tweets are particularly challenging for being interpreted and analized automatically, since they are shorts piece of texts (i.e., 140 characters) usually posted using a nonstandard language with similarities to SMS style [3, 4].
Currently, a major problem concerning the analysis of tweets is that of deciding whether a given tweet containing the name of an entity is actually refering or not to the entity. This is because entity names are often ambiguous. For instance, a tweet containing the word \Ford" might refer to the motor company, but also might be about the Ford Models (the modeling agency), Tom Ford (the lm director), etc. [5]. 1 http://www.twitter.com
In this paper, we address the above problem by proposing a methodology to contextualize entity names with probabilistic models from relative large and broad text collections, so that a model of these entities can be applied to decide whether a tweet contains or not a reference to a given entity. By \contextualize" we refer to the process of recovering relevant information about the entity (speci cally, lexically related words) from background knowledge. 2
Given a collection of text documents C = fd1; : : : ; dN g with vocabulary V = fw1; : : : ; wM g, we propose to contextualize a set of entity names q = fwi1 ; : : : ; win g in the same domain (e.g., automotive, education, music/artists) according to C (8j 2 f1; : : : ; ng, we assume that wij 2 V) by means of a probabilistic signature model; that is, a discrete probability distribution of words lexically related to the knowledge or topic underlying the set of entities in the collection.
To de ne such a probabilistic signature model for the text element q
(hereafter referred to as the signature model for q), we rely on a posterior probability
distribution of words fp(wjq)gw2V that is de ned from the following equation:
p(wjq) = p(qjw) p(w)
(
The aim is to base the signature model on the likelihood with which words in the vocabulary V are mapped to (or entail) words in q.
Nevertheless, as the posterior fp(wjq)gw2V can actually assign high probability values to either meaningless words (such as prepositions, conjunctions, common verbs, etc.) or words with a relative high prior or frequency in C (for example, ambiguous words with di erent interpretations in the collection), we propose to learn the signature model that contextualizes q from a re ned version of the posterior distribution of words fp(wjq)gw2V , namely, fpq(w)gw2V , that is obtained by minimizing the following cross entropy value:
Hq =
X p(wjq) log (1 w2V ) pq(w) + p(w) where is a mixture weight that accounts for the proportion of \background noise" in fp(wjq)gw2V and fp(w)gw2V is a background probability distribution of words (e.g., the distribution of words in C).
Let z represents the topic underlying q in the context of C. Then, we de ne
the signature model of q as the discrete distribution:
sq(w) / pq(w)
(
To estimate the re ned that minimize Hs;q, we rely on an Expectation Maximization procedure that starting from initial values for fpq(w)gw2V , namely fp(q0)(w)gw, it iteratively approximates the values of fps(w)gw for each w 2 V until convergence by means of the following updates in the kth iteration: where Zw is: p(qk)(w) = P p(wjq)Zw w0 p(w0jq)Zw0 Zw = (1
) p(qk 1)(w)
(
From the above formulation of signature models, we devise an (unsupervised) knowledge-based system for ltering tweets as it was asked by the Replab 2013 ltering task [1].
The RepLab dataset comprises about 105; 000 tweets in the test set, each one containing the canonical name of at least one entity in a selected set of 61 entities from four domains: automotive, banking, universities and music/artists. The source languages of the tweets in this dataset include English and Spanish.
The idea underlying our system is to consider {as positive evidence{ the overlapping between the vocabularies of the tweets and the vocabulary of the topic signature models obtained for the four di erent sets of entity names that comprise each domain of entities in Replab 2013 (i.e., automotive, banking, universities and music/artists).
The signature models were obtained by contextualizing each set of entity names according to webtext (a collection of about 1; 700; 000 text documents in English).
We have submitted the following four runs:
(
Since signature models wer obtained from a collection of texts in English, the aim of runs 3 and 4 is to measure the quality of signatures build from an English corpus to lter tweets in Spanish.
We present the obtained results in terms of the following measures: Precision, Recall, F1(Precision,Recall), Reliability, Sensitivity and F1(Reliability,Sensitivity). The last three of these measures were considered as the o cial mesaures of RepLab 2013 [1].
Tables 1 and 2 sumarize the results obtained by our four runs. As it can be appreciated, our run relative good values of Precision, Recall and F1 if we take into account that our signature models are base only in English texts. The fact that recall values are lower than those of precision can be explained since the proposed contextualization is base on a completely di erent collection of documents and in this way the learned models focus on modeling domain aspects rather than typical aspects observed in tweets such as irony and sarcasm-based aspects related to entities.
It is worth mentioning that our runs are completely unsupervised, and do not use the training data provided to this task at all.
Results obtained by runs 1 and 2 can not be compared to those ones obtained by runs 3 and 4 because they are applied to di erent input data.
Interestingly, when regarding the o cial RepLab 2013 measure of Reliability, which is a precision-based measure, it can be seen that the results obtained by runs 3 and 4 are smaller than those obtained by runs 1 and 2 even when the values of Precision of runs 1 to 4 are similar. Besides, these values of Reliability are smaller than those of Sensitivity, which is a recall-based measure, while Recall values are always smaller than Precision values.
In this paper, we have described the UNED-READERS system that participated in the ltering task at RepLab 2013. This knowledge-based system relies on a new way of contextualizing entity names from relative large and broad collections of texts using probabilistic signature models. The learned contexts can be shown useful to model domain vocabulary if we consider the obtained values of Precision and Recall. The submitted system is completely unsupervised. It does not use the training set provided by the competition.
Acknowledgments. This work has been partially supported by the Spanish Ministry of Science and Innovation, through the project Holopedia (TIN201021128-C02), and by the Spanish Government (MINECO) in the framework of CHIST-ERA program (READERS project).