The online reputation management systems help to the consumers to make buying decisions looking for opinions in the web about many products o ered by companies, also interested in the same opinions. This paper presents the system developed by the SINAI research group at the WEPS-3 task, called Online Reputation Management. Given a Twitter entry and a company name, the goal is to decide if the entry talks about this company. Our system is based on the use of linguistic information in Twitters entries for extracting information and creating an XML data collection about companies referred in the Twitter entries. This XML collection is lled extracting information from Internet web pages, like Wikipedia, and the use of the DBpedia ontology. Using this collection and some logical rules we have developed a promising system, with a very good precision and easily extensible with the addition of new rules and other Internet resources.
blogs like Twitter or Facebook can be detrimental to your brand, especially when competitors are standing close by to snatch up customers. One way to combat that threat is through a reputation management strategy with the so called On-line Reputation Management (ORM) systems.
A rst step in a ORM system is the detection of the company given a set of opinions. The ambiguity of names is an important bottleneck for the experts who want to analize these opinions.
This paper presents the system developed by the SINAI research group at the WEPS-3 task, the ORM task.
The main goal of this task is to discover if a Twitter entry belongs to a given company. As the organization said, Twitter has been chosen as target data because it is a critical source for real time reputation management and also because ambiguity resolution is challenging: tweets are minimal and little context is available for resolving name ambiguity.
Existing ORM systems[
Our system works with linguistic rules[
The following section describes the system developed. In Section 3, we explain the experiments and obtained results. Finally, conclusions are presented in Section 4. 2
A Twitter entry may contain multiple entities and proper nouns that refers to a company. Given an entry and a company name, our objective is to determine the company that each entity or proper noun refers to decide if the entry talks about this company. We use some linguistic rules to perform this task.
The rst module of the system analize the Twitter entry, extracting each pair entity-type found. The Name Entity Recognizer (NER) used was the module included in GATE1. GATE (General Architecture for Text Engineering) is a stable, robust, and scalable infrastructure which allows users to build and customise language processing components.
The second module create for each organization a complete XML le with the following data: Organization name.
Webhome. The organization of WePS-3 delivered a le with all the organizations, where each organization appears with its webhome and several data. The webhome was extracted and preprocessed (html tags and stopwords removal).
Wikipedia page. For each company the data of its wikipedia2 page, if exists, has been extracted and added to its XML le.
DBpedia abstract. The DBpedia Ontology3 is a shallow, cross-domain ontology, which covers over 259 classes, described by 1,200 di erent properties, forming a subsumption hierarchy. For each company the data of the abstract property, if exists, has been added to the XML le. The abstract property in DBpedia describes the company and its main features. DBpedia products. For each company the data of the products property, if exists, has been added to the XML le. The products property includes the products that this company manufactures.
The following text shows an example of the XML le created for each company. 1available at http://gate.ac.uk/ 2available at http://www.wikipedia.org/ 3available at http://dbpedia.org/ <doc> <field name="id">Starbucks</field> <field name="webhome" url="http://www.starbucks.com/">Starbucks Coffee Company skip to Main Navigation...</field>
<field name="DBpediaAbstract">Starbucks Corporation is an international coffee and coffeehouse...</field>
<field name="DBpediaProducts">Merchandise Baked goods Smoothies Frappuccino beverages Bottled beverages Made to order beverages Boxed tea Whole bean coffee Alcoholic Beverages</field> </doc>
The third module applied the manual rules generated. These rules are the following: (sinai 1) The name of the company appears in the Twitter entry. (sinai 2) The name of the company appears in the entities detected in the Twitter entry, and the entity recognized is the type \organization". (sinai 3) At least one the entities detected in the Twitter entry appears in the organization webhome. (sinai 4) At least one the entities detected in the Twitter entry appears in the wikipedia page of the organization. (sinai 5) At least one the entities detected in the Twitter entry appears in the DBpedia abstract or DBpedia products of the organization.
The result of this third module for each Twitter entry and its possible organization is a TRUE/FALSE value, that con rm if the entry belongs to this organization. 3
We carried out experiments using the framework given by the WePS-3 organizers. The data set contains 47 organizations, with roughly 500 Twitter entries for each organization, 22490 entries in total.
Applying the linguistic rules described, most of the tags are true. The distribution of tags obtained is presented in the Table 1. The sum of the tagged entries is 22013 because one of the organizations cause an error in the system and its entries were not tagged. 1. The best result has been obtained with the rst experiment (sinai 1), with a 0,63 value of accurary. Almost the 80% of the tags were tagged as FALSE, and the precision value for positive tags achieves a 0,84, so this simple rule works well for the positive tags, and not bad for negative tags. 2. The second experiment (sinai 2) tagged all the entries as FALSE, because the NER module did not work well to classify the entities, because of the length and format of the Twitter entries. The result obtained is not relevant. 3. Webhome information contains general text with no relevant information in the most of cases, so a lot of the entries were tagged as TRUE, and the results were poor. Only we can emphasize that the recall for positive tags achieves a value of 0,7. 4. Wikipedia contains relevant and summarized information about a company. This information was not available for all the organizations, and in some cases if the organization name is also a common word, like apple, obtained a wrong Wikipedia webpage. The result was good in terms of accuracy, 0,61, and also in terms of precision for positive and negative. We emphasize the recall value obtained for negative tags, 0,72. 5. The results obtained using DBpedia information (abstract and products) are not relevant.
Only for a low percentage of the organization we oatained this information (in some cases our system did not nd this information or the name of the organization in DBpedia was di erent than our query). In other cases, if the organization name is also a common word, like apple, the information obtained was wrong. 4
In this paper we have presented the experiments carried out for the CLEF 2010 WePS task (Online Reputation Management). We conducted diagnostic experiments applying di erent logical rules over the data o ered by CLEF. This rules used an XML information collection built from the Wikipedia y DBpedia and the behavior of the system has been very promising. We have obtained a 0.63 value of accuracy of the best rule, obtaining a precision of 0.84 for the positive tags and 0.72 for the negatives.
Nevertheless, the nature of task and the training data give us a guide to improve the results. The absence of information for some companies in Wikipedia and the low relevance of DBpedia encourage us in the use of more speci c information sources in Internet. Like the information o ered by the sources used is so sparse, we think that a combined system will improve the precision, e.g. weighting the rules using some kind of learning algorithm based on the information given of each company.
This work has been partially supported by a grant from the Spanish Government, project TEXTCOOL 2.0 (TIN2009-13391-C04-02), a grant from the Andalusian Government, project GeOasis (P08-TIC-41999), and a grant from the University of Jaen, project RFC/PP2008/UJA-08-16-14 and project UJA2009/12/14.