This paper reports on our ongoing work regarding opinion mining from Web-based discussion forums in the realm of the Understanding Advertising (UAd) project. Our approach to opinion mining is to first RDFise discussion forums in SIOC, and in a second phase to interlink the so created data with linked datasets such as DBpedia. We are confident that this should allow a market researcher to formulate queries using domain semantics and hence understand what people think about a certain product or service. The system's architecture, preliminary results, and the current available demonstrator are discussed in this work.
The analysis performed in UAd is twofold, (i) by visual interpretation of advertisements (from print media, Web and TV), and (ii) by using information available on the Web. Fig. 1 depicts the overall UAd system architecture, consisting of (i) the UAd Analyser (the front-end for the end-user), (ii) the “Public Knowledge Interface”, and (iii) the Visual Analysis module. Information about products and services are gathered from the Web through the so called UAd “Public Knowledge Interface” (PKI). We have developed three methods converting plain (HTML) Web content into structured data represented in RDF allowing us to be both flexible and comprehensive: 1. Plain old screen scraping (in the so called UAd Harvester/Mapper module); 2. Pattern-based RDFising and Interlinking for online discussions (the UAd
Discussion Tracer); 3. Schema-based a-priori RDFising and Interlinking (for statistical data from
Eurostat; described elsewhere [
This paper is structured as follows: First, we review related work in section 2. Then, in section 3 we discuss our approach representing discussions and opinions. In section 4 we present the system’s architecture, discuss the data acquisition and the market researcher’s interface. We present preliminary results in section 5. Finally, we discuss our findings and highlight future work in section 6.
Recent research on opinion mining has focused on sentiment analysis, simple
“pro” and “cons” classification [
Motivated by earlier experiences [
We aim at orienting the opinion holder context on domain semantics [
To the best of our knowledge there exists no other work in the area of opinion mining that deals with explicitly modelled opinions along with linked data sets for its domain knowledge.
The basic idea of linked data was outlined by Sir Tim Berners-Lee [
To support a market researcher in analysing a certain market, one of the sources used in the UAd PKI are Web-based discussion forums. For enabling structured
queries and browsing it is necessary to represent the discussions in a machineinterpretable way and enhance it with domain semantics. Web-based discussion forums offer a well-structured source for this purpose, hence the idea to exploit them along with linked datasets.
Our goal is it to explicitly model the opinions in a discussion being compliant to the Web of Data. We decided to reuse an existing vocabulary to represent the discussions rather than reinventing the wheel. Due to its popularity and wide-spread use, the Semantically-Interlinked Online Communities (SIOC) vocabulary4 has been selected to represent discussion threads and posts.
However, in case of explicitly representing opinions we did not manage to find an appropriate vocabulary. Although one could for example use a review vocabulary5 as a base and extend it, we found it better suited to define a dedicated vocabulary for this task.
Our “Opinion Mining Core Ontology”6 (cf. Fig. 2) basically defines the following classes and properties: – opm:DiscussionOpinion, the central hub that connects discussion threads with opinions about a certain entity; – opm:Opinion, an abstract representation of an opinion; – opm:Topic, a proxy concept to trigger aspects of a certain topic.
We use skos:Concept of SKOS [
It has to be noted that opm:Opinion is currently deliberately underspecified.
We intend to extend and refine this part to the ontology based on our experiences
with the system and regarding earlier work from [
The introduced lightweight ontology above plays a decisive role in our opinion mining process. In order to achieve better scalability and reusability, it acts as a nexus between the domain of concern and the RDFised data. This is why it makes no difference for our opinion mining model if there is the DBpedia categorisation behind or some other domain specific ontology. Therefore, our approach offers flexibility by choice of domain and yields a generic opinion creation. 4
In the process of discussion tracing in UAd, two major components are involved (Fig. 3), namely (i) the UAd Data Acquisition (highlighted), where Web-based discussions are harvested, RDFised and interlinked, and (ii) the UAd Analyser, allowing to query and access the data. 4.1
The data acquisition in UAd is performed in three phases; in a first step the most common data in a Web-based discussion forum, such as title, author, creation date, etc. is RDFised using SIOC. In a second phase the entities occurring in the discussion posts are identified and interpreted regarding a certain domain (in our demonstrator this domain is “cars”). This second step involves the interlinking to linked datasets such as DBpedia or instances of some other domain specific ontology. For our purposes DBpedia offers enough adequate instances and well formed domain model respectively area of interest. However, as mentioned in Section 3 this is not mandatory and DBpedia can be easily replaced by any other domain ontology. Currently, interlinking with DBpedia is done manually, however in the final version we are aiming to automate this task. In a third phase the (subjective) statements of participants are analysed, and further added to the knowledge base. This is mainly achieved by the creation of opm:DiscussionOpinion instances and their respective properties. To this end, we use a manually pre-configured list of possible topics, that is instances of opm:Topic to trigger the creation of opinions.
We have implemented a client/server system (Fig. 3, left and bottom) to perform the data acquisition in UAd. Within the scope of our research we support RDFising popular discussion forum types7 such as vBulletin or phpBB. Data extraction occurs automatically using extracion profiles, manually defined for several forum types; a single acquisition task represents a single job on the server. The server has been implemented using a Java application server (Tomcat) along with a Jena 2/PostgreSQL RDF store taking care of the scheduling and execution of the acquisition tasks.
At the client side, a Firefox plug-in (Fig.4) allows a user to define, control and monitor the tracing tasks. The plug-in has been developed in JavaScript and XUL8. A user typically adds the link of a discussion forum and selects the forum type. Currently, only entire forums can be extracted. We plan to support the selection of sub-forums independently from each other for the extraction task. The user can also specify time parameters for the acquisition tasks, for example how often per week a job should be triggered to update the store. 4.2
The UAd Analyser is a Web Application allowing a market researcher to examine the data gathered by the UAd Acquistion Server. In Fig. 5 the current state of the implementation (implemented with the Google Web Toolkit9) is depicted. The user can limit the data by selecting certain car classifications and issues
as well as by restricting the time period. The queried data is visualised with a Simile Timeplot10 module, displaying the time on the X-axis and the number of discussion posts in the Y-axis. Discussion threads are illustrated as red vertical lines; the users may retrieve detailed information by clicking on the line and browse to the discussion thread where the matching post is located.
The post count, respectively a single time unit, in the X-axis reflects the occurrence frequency of topic. Additionally information about, diversity of authors who posted that day can be explored. The knowledge about authors diversity can be used to underline for instance how reliable or unreliable is the sentiment in chosen posts. The most important contribution of this visualisation is to offer an overview on diverse discussion forums regard a topic of interest. 5
In order to assess our opinion mining system, a baseline-evaluation using two standard information retrieval measures (precision and recall) has been performed. We have compared our approach to a full-text index (Lucene11). The domain is currently limited to “cars” (as we have mostly advertisements for the visual analysis available) although we note that the methodology is expected to yield similar results for other domains. The flexibility of our approach is mainly determined by the availability of appropriate instances from DBpedia. 10 http://simile.mit.edu/timeplot/ 11 http://lucene.apache.org/java/docs/ Our reference data set contains approximately 1000 posts that have been extracted from a single discussion forum12, focusing on the content of three subforums including all threads and posts about certain car types. Two of the extracted sub-forums contain discussions about cars belonging to the mid-sized car class according to categorisation from DBpedia13. The working data set includes 60 representative posts (20 per car type). We have manually selected posts containing discussion on topics such as “performance and problems” and “popularity”.
The extracted posts were firstly used to generate opinions on the discussion topics, and secondly for the initialisation of the index over the reference test data (for Lucene). We have converted each of them into a single file containing information on the posting date, author, post URI, the content and the title of the thread the post belongs to, allowing to create an index searchable by Lucence. The Lucene index contains the fields author, title, summary, content and link to post corresponding to the properties in RDFised data and with the intention to provide as similar as possible initial point to RDFised data, for comparison and measurement of results.
Prior to the manual creation of the triggers for discussion topics we have
analysed the initialised fields of the Lucene index for occurrence frequency of
specific keywords and the “Zipf” distributions [
" problem " .
Listing 1.1. Sample discussion topic snippet. 1 @prefix : < http :// sw . joanneum . at / uad / cars / opinions # > . 2 @prefix utop : < http :// sw . joanneum . at / uad / cars / topics # > . 3 @prefix opm : < http :// sw . joanneum . at / uad /u - opm / schema / core -u - opm . rdf # > . 4 5 : do11 a opm : DiscussionOpinion ; 6 opm : about < http :// dbpedia . org / resource / Alfa_Romeo_156 >; 7 opm : in < http :// www . automotiveforums . com / vbul ... php ?t =173469 >; 8 opm : onTopic utop : performance_and_problems .
Listing 1.2. Sample generated discussion opinion. 5.2
For the evaluation we have compared our method with the standard Lucene retrieval results of simple queries. Additionally we had a look at extended Lucenequeries; these extended queries have been used to decrease the influence of a single trigger. Listing 1.3 shows a sample SPARQL query we have used for our approach.
1 prefix owl : < http :// www . w3 . org /2002/07/ owl # > 2 prefix utop : < http :// sw . joanneum . at / uad / cars / topics # > . 3 prefix opm : < http :// sw . joanneum . at / uad /u - opm / schema / core -u - opm . rdf # > 4 5 SELECT * FROM < http :// sw . joanneum . at / uad > 6 WHERE { 7 ? do a opm : DiscussionOpinion ; 8 opm : about ? about ; 9 opm : in ? in ; 10 opm : onTopic utop : performance_and_problems . 11 ? about owl : sameAs < http :// dbpedia . org / resource / Alfa_Romeo_156 > . 12 }
Listing 1.3. Sample opinion mining SPARQL query.
From table 5.2 we learn that regarding recall our method unsurprisingly seems to outperform simple full-text indexing. Even in the extended mode Lucene’s precision and recall values are below our approach.
Precision Recall
Lucene UAd Analyser “performance “popularity” “performance “popularity” and problems” and problems” simple 0.4 1 0.76 0.86 extended 0.2–0.62 0.56–0.86 simple 0.1 0.05 0.95 extended 0.05–0.8 0.3–0.7 Table 1. Results from the Evaluation (Lucence vs. UAd).
Although we have used a rather limited working set in this evaluation we are optimistic that the results scale well both regarding size and other domains; further evaluations are in the scope of our current research. 6
In this paper we have proposed a novel approach to opinion mining on the Web by using Web of Data technologies and linked datasets. Our goal is to explicitly model opinions found in discussions on the Web; we have developed an according vocabulary to represent these opinions formally (in RDF) and have reported on an implementation of this approach.
We contemplate on using GoodRelations14—an ontology for linking product descriptions and business entities on the Web—in order to more accurately describe the target of an discussion in our realm.
To increase the precision we ponder about extending our opinion mining core
mechanism with Natural Language Processing techniques and/or use neural
networks to categorise topics automatically. As a part of the sentiment classification
we aim to use SentiWordnet [
Currently, we summarise results visually respectively topics, identities, time and occurrence frequency to mirror the sentiment intention in opinions environment. However, currently we do not dive into sentiment interpretation of opinions. Considering the visual analysis, it is important to mention that sentiment interpretation underlies the judgement of end user and his observation standpoint. Anyway, objective parameters such as time period, identities, number of posts etc. can be evaluated independent of matter of particular interest. For further evaluations, user annotated content like reviews or similar will be used additionally. 14 http://www.heppnetz.de/projects/goodrelations/ 15 P stands for “Positive” and N for “Negative” in this context.
The research reported in this paper has been carried out in the “Understanding Advertising” (UAd) project, funded by the Austrian FIT-IT Programme. The authors would like to thank their colleagues Magdalena Lauber, Wolfgang Weiss, and Werner Bailer for their support and valuable comments.