Using Twitter as an effective marketing tool has become a gold mine for companies interested in their online reputation. A quite significant research challenge related to the above issue is to disambiguate tweets with respect to company names. In fact, finding if a particular tweet is relevant or irrelevant to a company is an important task not satisfactorily solved yet; to address this issue in this paper we propose a Wikipedia-based two-pass algorithm. The experimental evaluations demonstrate the effectiveness of the proposed approach.
Twitter1 is a microblogging site which currently2 ranks 8th world wide in total
traffic according to Alexa3. This huge popularity has turned Twitter into an
effective marketing platform with almost all the major companies maintaining
Twitter accounts. Moreover, Twitter users often express their opinions about
companies via 140-character long Twitter messages called tweets [
Our approach consists in a two-pass filtering algorithm that makes use of Wikipedia as an external knowledge resource in the first pass, and a concept
2 Data as of July 31, 2012 3 http://www.alexa.com/siteinfo/twitter.com term score propagation mechanism in the second pass. The first step is precisionoriented, where the aim is to keep the noisy tweets to a minimum. The second step enhances the recall via a score propagation technique, and by making use of other sources of evidence. Our technique shows high accuracy over the RepLab2012 dataset.
The rest of the paper is organized as follows: section 2 presents a more detailed description of the considered problem. Section 3 describes the proposed methodology in detail. Section 4 gives presents the experimental evaluation of our method, and Section 5 summarizes the related work. Finally section 6 concludes the paper. 2
In this section we give a brief overview of the RepLab2012 filtering task. We were given company names and a set of tweets obtained via issuing a query (i.e. the name of the company) to Twitter Search4. Due to the issue of noise in Twitter, the tweets obtained via the query may or may not be relevant to the company. Our task addresses the problem of distinguishing the tweets relevant from those non-relevant to a company. 3
This section describes the proposed filtering method in detail. We first explain how we use Wikipedia as an external knowledge resource: we use only portions of a company’s Wikipedia page, as only some of the portions are meaningful for filtering purposes. Next, we explain the two steps of our algorithm. 3.1
As Meij et al. point out in [
Finally, the concept terms extracted from the various portions of Wikipedia are split into single terms. These are then used for matching against the terms in the tweets for the task of filtering. 3.2
After collecting all the extracted concept terms from Wikipedia we order them by their specificity in the Wikipedia article thereby giving a score to each term. We then check for the occurrence of these concept terms in the tweets, and the number of occurrences per term is multiplied by the score of that particular concept term to constitute a score for the tweet. Tweets that have a score above a certain threshold are considered to be relevant. The intuition behind the use of Wikipedia concept terms for the first phase is to keep the precision as high as possible, and to get relevant tweets only. 3.3
The second pass makes use of the idea of concept term score propagation in order to discover more tweets relevant to a particular company i.e. to increase the recall. The score propagation technique is based on the intuition that terms colocated with significant concept terms may have some relevance to that concept. The scores for concept terms in a relevant tweet obtained from the first pass are redistributed among co-located terms. This in turn gives some score to the nonconcept terms, and by using the scores of these non-concept terms we perform a second computation to obtain the scores of tweets. Moreover, this phase uses more sources of evidence, these are: – POS tag of the company name occurring within the tweets – URL occurring within the tweets – Twitter username occurring within the tweets – Hashtag occurring within the tweets
The score propagation technique as well as the extra sources of evidence mentioned above enable us to extract more tweets relevant to the company, thus increasing the recall. We show the results of the evaluation metrics in the next section. 4
As mentioned in section 2 the task comprises binary classification and hence we report the effectiveness of our algorithm through the standard evaluation metrics of precision and recall. We were given a very small trial dataset (six companies) and a considerably larger test dataset (31 companies). We report our results for the test dataset. Table 1 shows the precision and recall figures after the application of the first pass, and after the application of the second pass of our algorithm.
First Pass Second Pass Precision 0.84827 0.81129
Recall 0.16307 0.76229 As Table 1 shows the first pass yields a high precision but an extremely low recall. The application of the second pass increases the recall by a large degree, while not overly reducing the precision. The significantly large increase in recall proves the effectiveness of the score-propagation technique combined with the use of multiple sources of evidence.
The RepLab2012 filtering task used the measures of Reliability and
Sensitivity for evaluation purposes, these are described in detail in [
Named entity recognition in tweets has been an area of active research over
the past few years with a focus on semi-supervised learning techniques [
Understanding and analysing the content of tweets is another significant
research direction where the goal is to extract keyphrases from the content of tweets
[
Semantic enrichment of microblog posts has also been studied with the aim
of determining what a tweet is about [
We proposed a two-pass algorithm for company name disambiguation in tweets. Our algorithm makes use of Wikipedia as a primary knowledge resource in the first pass of the algorithm, and the tweets are matched across Wikipedia terms. The matched terms are then used for score propagation in the second pass of the algorithm that also makes use of multiple sources of evidence. Our algorithm showed competitive performance and demonstrates the effectiveness of the techniques employed in the two passes. As a future work, we aim to refine the score propagation technique of the second pass by taking into account better features for effective scores computation.