-Making sense of text is a challenge for computers particularly with the ambiguity associated with language. Various annotators continue to be developed using a variety of techniques in order to provide context to text. In this paper, we describe Denote - our annotator that uses a structured ontology, machine learning, and statistical analysis to perform tagging and topic discovery. A short screencast for the curious is also available at http://youtu.be/espItTRQVzY as well as demonstration links provided in the conclusion.
The availability of structured link open data, through
initiatives such as the “Linked Open Data (LOD)” project1,
has given rise to a new class of annotators for unstructured
text. Annotators like TagME [
II.
Denote searches its ontology for similar concepts to the input text by performing keyword extraction then calculating a weighted Jaacard coefficient on resource descriptions. This provides a measure of text similarity. For each resource, its known categories (defined in the ontology) are subjected to a Bayesian filter to exclude those resources and categories that do not appear relevant. This provides a measure of semantic similarity. The surviving resources are then used for the annotations. Denote’s output is in the form of a synopsis whose lexicon is given in Table I. The output is a single sentence per annotation with a set of relevant URIs sorted in order of likelihood with confidence and available support statistics.
“Text” [Is_A {}] [[[With_Value •] Of_Units •] |
Acting_As {}] [Cat_Of {}]
Fig. 1. The output of an annotated text.
Denote uses a database of linked open data, represented
in the form of n-triples (<subject><predicate><object>), to
perform annotations, similarity identification,
1 http://linkeddata.org/
2 http://www.alchemyapi.com/
disambiguation and topic categorization. Denote's database is
DBPedia [
DENOTE’S ANNOTATION LEXICON EXPLAINED
Is_A {} Acting_As {} With_Value • Of_Units • Cat_Of {}
"is a", "is an", "is used by". Asks: What is it? Context/role. Asks: How is it used? If number, Asks: What is the number value? If number, Asks: What is the units of measure? Asks: What relevant topic categories? III.
Table II demonstrate Denote’s capabilities when compared to TagME and DBPedia Spotlight using the same input text of: “BLT. The sub that proves great things come in threes. In this case, those three things happen to be crisp bacon, lettuce and juicy tomato. While there's no scientific way of proving it, this BLT might be the most perfect BLT sandwich in existence. The default configuration for Denote, TagME and Spotlight were unchanged. Spotlight does not perform category analysis. TagMe gives a topic listing but this list is simply the annotated text rather than a separate categorization. Consequently, the [Cat_Of] portion of Denote’s synopsis was omitted and left for part C.
DBPedia Spotlight was the least effective with the fewest annotations and an incorrect disambiguation of BLT as a “Bizarre Love Triangle”. TagME performed well with numerous annotations with few mistakes (incorrectly tagged words “crisp” and “juicy”. Both Denote and TagME shared similar annotations but it is through Denote’s [Acting_As] vocabulary that provided context information. For example, both correctly annotated “lettuce” to its surface form, but it was Denote that identified that lettuce was acting as a main ingredient. Similarly, Denote linked the phrase “bacon, lettuce, and juicy tomato” as an alias or alternate name.
ANNOTATION OF “BLT. THE […] IN EXISTENCE.” WITH DENOTE, TAGME AND DBPEDIA SPOTLIGHT.
Denote (DBPedia)
TagME (Wikipedia) Is_A {/BLT} Acting_As {/name} Is_A {/BLT} Acting_As {/name} /BLT
DBPedia
Spotlight (DBPedia/Wiki
pedia) /Bizarre_Love _Triangle /Sandwich /Existence
Is_A {/Bacon_sandwich, Bacon,Side_bacon} Acting_As {/mainIngredient, /ingredient}
Is_A {/Lettuce} Acting_As {/mainIngredient, /ingredient}
Is_A {/Tomato} Acting_As {/mainIngredient, /ingredient}
Acting_As {/alias, /alternateName} /Submarine_
sandwich /Potato_chip /Bacon /Lettuce /Juice /Tomato /Scientific_m ethod BLT sandwich sandwich in existence sub crisp bacon lettuce juicy tomato bacon , lettuce and juicy tomato scientific way
The number annotator is unique with respect to other annotators in that Denote attempts to identify text that is normally associated with a numerical value. Using statistical analysis on the Jaacard/Bayes-discovered list of similar concepts, Denote attempts to match up number values with annotated text. Figure 2 demonstrates on the input text “The radio shack color computer has only 16 kb of memory”.
“memory” With_Value 16 Of_Units #int Cat_Of {/Home_Computers, TRS-80_Color_Computer}
Denote has access to over 656,000 categories defined in DBPedia’s <dcterm:subject> ontology. A Bayesian filter is used on each similar concept in order to determine if the subject(s) of which the concept belongs to is contextually related to the text being annotated. DBPedia Spotlight demo does not perform topic category determination. TagME’s demo performs topic categorization by simply listing its annotated text in a cloud-tag structure rather than a defined set of category topics. Consequently, we compare Denote’s output with Alchemy. The Alchemy annotator can perform named entity extraction from a list of 200+ defined (sub)entities. In this comparison, the “storyline” of The Godfather movie was retrieved from the Internet Movie Database (IMDb) and annotated. Table III gives the results.
Alchemy results were limited to primitive named entity types of city and person with the exception of an incorrect categorization of “television show”. In contrast, Denote tagged text into rich categories that include “ItalianAmerican novels”, “organized crime novels”, and “Godfather characters ”.
In this paper we demonstrated Denote – a semantic annotator based on the DBPedia ontology and compared its features with that of same-class text taggers. Denote’s middleware engine demo is available at http://ls3.rnet.ryerson.ca/annotator while a developerfriendly demo is at http://inextweb.com/denote_demo. Denote’s annotation capabilities are wrapped around a RESTful interface allowing for 3rd-party developers to create their own semantic-aware applications. The result, we hope, is an improvement in information search and retrieval for the end user. Our future work involves parallelisation to scale the service for a large number of concurrent clients. We are also developing proof-of-concept demonstrations including a semantic movie recommender whose database will be included as a data-set to the LOD project.