In this decade the amount of textual content stored on the web became enormous, but the basic structure of web documents remained unchanged: a mixture of text and markup. When creating a document, the user rarely has the possibility of embedding semantic annotation into the content, because editor applications do not have such a feature or they are too difficult to use. We think that the creation of semantically rich documents can be best facilitated by a content editor with text mining technology running in the background. In this paper some of these technologies are brought into spotlight.
On leave from Dept. of Telecommunications and Media Informatics, Budapest University of Technology and Economics cial markup language. This requires certain skills from the user and has a negative impact on the size of the potential audience. Another approach is to annotate the text off-line, after the content has been created, without human supervision. To achieve this natural language processing (NLP) technology, namely information extraction (IE) tools are required. Considering the huge amount of textual data already present on the Web, these researches are very important.
Undoubtedly, it would be fruitful to provide tools for the average user to ease the creation of semantic annotations when editing web content. The Docuphet [9] project aims at discovering and experimenting with the possible solutions of the problem. In our view the web content creation should be a continuous, mutual dialogue between human and machine rather than a simple one-time input process.
This paper is organized as follows. In the following section we describe the motivations and goals behind Docuphet and the main components of the system. In Section 3 the technologies of the implemented components are detailed. Section 4 presents two example applications of the system. In Section 5 we discuss our experiences, and propose some requirements on dialogue-assisted semantic annotators. Section 6 reviews the related work in the field of semantic annotation software. Section 7 concludes the paper and discusses the future work.
The main goal of the Docuphet project is to create a system that enables the user to produce semantic annotation easily. This is achieved via an intuitive user interface that is supported by text processing algorithms in the background. We intend to capture the meaning of the currently edited content by addressing simple questions to the user. While the user types, the available text is processed by IE applets in the background. Then extracted facts are formulated as statements that are conveyed to the user in the form of closed-ended questions. If the user confirms a statement, the system automatically embeds the appropriate semantic annotation into the text. The annotator software does not require any special technical knowledge or skill from the user, it is only assumed that she knows the particular content that she is editing, and hence she is able to decide if a related closed-ended question is true or not. Another property of the system is that it stores text and annotation together. This integration has many advantages. For instance when the text changes, no extra look-up operations are required to transfer the changes into the corresponding semantic annotations. If the annotations were stored separately, the maintenance of extra identifiers would be necessary to link text and annotation. To make it accessible and runnable without installation, the client part of the system runs in an ordinary web browser.
It is a natural requirement against every system to support multiple languages. This requirement was considered when developing every component of the system. However, many IE techniques are language specific.1 For the first applications, the Hungarian was selected as primary language.
Docuphet focuses only on the creation of annotations. The further use the semantically annotated content, such as semantic search and retrieval, or machine reasoning is currently out of the scope of the project. We rendered our efforts to the design and the validation of our concept, therefore some components of the prototypical applications are not yet optimized for the efficiency under heavy load.
The user interacts with the Docuphet Content Editor’s (DCE) web interface. The web application forwards the created content to the server via AJAX calls. On the server the content is distributed to various IE modules. The modules suggests annotation suggestions. Each suggestion comprises a textual statement or question, a numeric confidence level, one or more possible answers to the question, and semantic annotations (one per each positive answer). The confidence level is a real number of the unit interval, which specifies the validity of the suggestion. Under a certain (configurable) level suggestions are automatically disregarded. The module’s suggestions are collected on the server and are sent back to the client, where they are presented to the user in the form of pop-up closedended questions. If the user confirms a statement the system inserts the corresponding annotation into the content. When the user finishes editing, the full annotated document is sent back to the server and saved there. Figure 1 provides an overview of the system. In the next section we discuss the components of Docuphet: the content editor, the annotation storage, the text processing and information extraction applets.
There is an excess of tools for creating content on a computer. These can be categorized in many ways. One aspect is the mode of editing. There are WYSiWYG editors like desktop word processors. Other, structured editors have two views: one for editing and one for viewing the documents. At structured editors, the user handles objects like section, title, paragraph. This category includes wiki editors, publishing tools, DocBook editors and scientific editors for LATEX . In these applications the final formatting is done with style class files or style sheets.
Another aspect is the technology of the editor. There are two main
categories: the more function-rich desktop applications that need
to be installed on the client, and the web-based editors [
The special requirements against DCE led to the development of a completely new solution. The DCE is an easy to use WYSiWYN2 editor, which is capable of editing the structure of a document without the need of learning a markup language. DCE also handles the communication with the server, the representation of suggestions and the integration of annotations (see also Figures 4–5 for screenshots).
For selecting the best of the plethora of content storing formats we set up the following requirements: 1. simplicity to allow the implementation as web editor; 2. standard, stable and free to use; 3. proper support (documentation, examples, templates, editors, tools); 4. extendibility to carry annotations; 5. support of the following formatting: paragraphs, sections and titles; lists, program listings, emphasizes, images, tables, links.
Many options were evaluated: RTF , texinfo [
Many possible technologies were evaluated for storing semantic
annotations in a DocBook document: HTML Metadata, RDF XML
[
The RDFa [
The trouble with Bob </title> <para id="ch1" property="dc:creator">
Alice </para> <para about="#ch1" property="dc:creator">
Tom </para> ... </div> RDFa was also favoured because it is easy to integrate with other XML namespaces, like Docbook. RDFa allows to annotate every part of a document, while it is still relatively easy to retrieve the RDF triples from the XML. To conform DocBook with RDFa, we extended the Docbook RNG schema to carry RDFa annotations, and we termed this DocBook profile DocBook/RDFa.
3.4.1 Named Entity Recognition A named entity (NE) is a natural textual identifier of an object, such as e.g. person names, names of companies, locations, names of products, addresses, telephone numbers, email addresses. Named entity recognition (NER) is an NLP task, which aims at identifying and classifying NEs in the text.
The difficulty of the recognition task depends on the type of NEs. Telephone numbers, e-mail addresses can be easily recognized with simple regular expressions. The recognition of personal names and locations is more difficult, but it can be effectively supported with appropriate vocabularies. The recognition of company or product names can be much harder, because essentially no constraint applies on their surface form. NER is primarily performed by analysing some features of the candidate NEs, e.g. surface clues (capitalization, numbers, special symbols), the frequency, the insentence, in-paragraph, and in-document positions, whereas grammatical and morphological analysis may also be applied. The Docuphet framework contains a general purpose named entity recognizer, the JNER, implemented in Java. This component comprises several modules, all of them analysing the text with a different technique. Many of them use vocabularies, e.g., for given names, company suffixes, locations. Others are based on regular expressions. It is possible to plug in external tools, such as a stemmer or a morphological parser. Other external tools can serve as connectors to databases (i.e IMDB, DMOZ) or to search engines (google, wikia search).
Although usually not considered as NEs, the recognition of professions (like painter, composer, engineer) and human properties (blond, tall) are also supported in JNER.
Conforming with the philosophy of Docuphet (ask relevant questions from the user), in this system the NER can also be assisted by asking closed-ended questions from the user.
Let us define an information frame (IF) as an RDF triple (we use Notation 3 syntax of RDF in this article for brevity) <subject> <predicate> <object> . in which at least one value is missing and thus substituted by a variable name. The class of the missing component(s) may be known. An example IF: <X (an instance of the person class)> <location of birth> <Y (an instance of the location class)> . The information frame recognition (IFR) means the recognition of instances of an IF in the text by identifying the missing components of the triple. For instance in the sentence “Pat Nixon was born in Nevada in 1912”, we can recognize an instance of the above IF: <Pat Nixon> <location of birt> <Nevada> .
IFs can be defined in various ways. One method is to derive them
from “semantic frames” which are used in the Berkeley FrameNet
project [
BUILDING(=WORK)), scenes(DESIGNER desings BUILDING) The frame elements are referring to certain semantic roles of actors and objects present in the scene. In the FrameNet project, human annotators mark the occurrences of the frame elements in texts. Evidently, to find the possible semantic role of a text element, it is very helpful to have the named entities and their types identified beforehand. In some cases additional rules may also be useful, i.e. specifying constraints on the relative position of the element of an IF (in-sentence, in-paragraph). While limiting the number of potentially identifiable IFs, this constraint has many practical advantages: it enables to start the IFR process before the whole content is available, and narrows significantly the search space thus reducing the computation time.
JFrame is the IFR component of the Docuphet framework, written in Java. IFs can be defined and the corresponding recognition rules can be given in JFrame. The input of the module is a token stream, in which the recognized NEs and their types are already marked. A JFrame IF definition may contain rules related to the class and lemmas of NEs in the token stream and have conditions on their order. JFrame also provides a confidence level for every recognized IF instance in accordance to the concept of Docuphet’s workflow. Currently in Docuphet, the IFR rules are defined by hand after experimentation. This is necessary because:
There isn’t enough properly annotated text which can be used as training data.
The annotation method in Docuphet is based on dialogues. Therefore, for each IF a simple function must be defined, which generates the corresponding closed-ended questions.
To support the recognition of IFs, a sentence segmenter was
developed, named as JSentence. The component implements a
modified version of the algorithm described in [
In this section two demo applications of Docuphet are presented.
BioBase is a web site to collect biographies of known people
(similar to the ones in [
In BioBase a two layered IF recognition has been implemented. In the first layer NEs are identified as follows: 1. JNER analyses the text configured with all the NER rules available for Hungarian language. Currently this includes person name recognition (male and female distinguished) based on regular expressions and given name vocabulary, location, nationality, profession, education, residence, family status, social relations (friends/other related people) recognition based on vocabularies, email and phone number recognition based on regular expressions and date recognition based on a custom Java component, regular expressions and a list of month names. 2. JNER assigns a confidence level to every recognized NE, which is calculated by summing the pre-defined values of matching rules.3 Over confidence level 0.95, an annotation suggestion is created with the following RDF triple: <article id> <related to [NE type]> <[NE value]>. where NE type and NE value are substituted with the actual values. The annotation suggestions also have level of confidence4 that is set to 0.95. DCE accepts every annotation above 0.9 without asking a confirming question from the user, in order to avoid of flushing the user with questions.5 The target of the annotations (the location where the annotation is placed) is the node before the given paragraph. From these annotations, a tag cloud can be generated with typed (see also Figure 4). 3. For the top three person and location NEs with confidence level between 0.8 and 0.95, an annotation suggestion of value 3The creator of a rule can set both positive and negative confidence coefficients. 4This is based on the NE confidence level, but can be weighted, e.g. when a candidate person NE shares the surname with an already recognized NE. 5Every annotation can be easily removed by deletion or with the undo action. 0.8 is created with the question “This article is related to the person/location (value)” with the same target and RDF triple as in the previous step.
Based on the NEs recognized, the following IE attempts are made: 1. First, the person is identified whom the article goes. This is done by creating suggestions on the first person NE found (in the title or in the text) with the triple <article id> <describes the life of> <[NE value]>. where the target is the beginning of the article, the confidence level is 0.8. This is repeated with the subsequent person NEs until a positive answer is given by the user. (In our experience the article is nearly always about the first mentioned person). 2. If the subject of the article is known, the date and place of birth and death are attempted to be identified next. This is done by analysing the date and location tokens in range of a centered window of 15 tokens around the occurrences the identified person. Extra confidence is added if the verb is in past tense form “szu¨letett” (was born), “elhunyt j meghalt” (died) around a particular date or location (these are often omitted however). 3. Nationality, profession, education, residence, friends and parent’s name are identified in a similar way as described in the previous point, involving certain terms (his mother j father) where appropriate.
Using these annotations the persons can be categorized by the era,
which of they lived or live in, the profession, the nationality or
location. All the RDF properties used in the process are in BioBase’s
namespace, but if required, they can be easily mapped into other
namespaces, like FOAF [
Our experience with BioBase shows that Docuphet is able to capture the basic biographic information about a person. When creating a 500 word long biography (see Figure 4), the system pops up 10–15, mostly adequate questions. One direction of further work could be the to addition of new event IFs based on the review of biographies.
IFR performed in FlatBase also in two layers, but in a completely different way than in BioBase. On the first level, the main informaOn both levels, specially configured JNER instances are used. The configuration initially includes a shorter list of locations, regular expression based rules to recognize price, size, and contact information. IFR is then performed based on already known NEs and certain trigger words related to building materials, heating types, etc. When some information is extracted, the JNERs may be reconfigured accordingly (e.g. loading the corresponding quarters when a district of Budapest is found).
When some of the main information are missing from the ad by saving, the user is asked to complete.6 If this happens again, a limited number of direct questions are formulated related to the missing information, every question only once. At the end, the ad is saved even if some information are still missing.
FlatBase has proven to be very effective because flat advertisements are usually short, very similar to each other, and their vocabulary is limited. As a result, Flatbase is capable of extracting nearly all (usually not more than 5–10, see Figure 5) important information from an advertisement. This suggest that FlatBase may become a user-friendly input-assisting tool for advertisement portals.
6But the actual content is stored anyway.
Bodain and Robert composed five requirements on the static
properties of semantic annotations [
For dialogue-assisted annotators, such as Docuphet, most of the
above requirements can be carried over7, but as an outcome of our
experimentation, we can now formulate additional requirements on
the semantic annotation creation:
1. A particular question must never be asked twice to avoid of
the discontent of the user. This implies that:
2. Every suggestion must be stored in the document, regardless
from the answer, if any. Per-session storage is not sufficient
since the document can be edited in several sessions. Further
research is needed to find out whether suggestions should be
stored on a per-document or a per-user basis.
3. There are two main types of suggestions depending on the
positive/negative answer of the user. As a future work it
should be investigated how negative answers can be exploited
at annotation.
4. According to the “handling dynamic updates” rule described
in [
Limited scope: We have defined two types of annotations: basic and derived. Basic annotations regard only to a specific text scope (a title, a paragraph, list item). We assume that changes outside the scope do not affect them. To fulfil this assumption, basic annotations are very simple e.g. “This paragraph is related to Budapest” when the token Budapest is present. These annotations are usually retrieved by NER as described in Section 4.1, and re-validated upon text change in their scope.
Dependencies: We define a dependency graph of annotations. Derived annotations depend on basic or other derived annotations. The dependency is tracked with a list of annotation IDs. Every time an annotation changes, its dependants should be re-validated. If an annotation is deleted, the derived annotations must be deleted as well. Furthermore, a derived annotation may require a re-validation when non-annotated parts of the text change, since some derived annotations may depend on the characteristics of the text or on missing annotation. For example if the user accepts a new “main category” annotation, the old one must be re-validated or simply deleted. 5. The number of questions asked together has to be limited.
This is important because if the user pastes in a larger piece 7although we applied a pre-defined semantic vocabulary because of the nature of the system
In the last 15 years, an excess of semantic annotation tools had been developed. Here we recall and compare the most important ones (see also Table 1).
We can divide the semantic annotators into two main groups:
Semantic Wikis: One form of inserting semantic
annotations into documents is via semantic wikis, such as Semantic
Mediawiki [
Desktop ontology builders and annotators: This group
contains some feature-rich desktop annotators for authoring
semantically annotated documents. Prote´ge´ [
S-CREAM [
COHSE [
Docuphet has many in common in the visualization of annotations
with SWEDT [
Like Melita [
Like COHSE and Magpie, Docuphet is based on pre-defined concepts and relations that are termed information frames. The targeted audience of unskilled users and the NLP- and IE-based dialogue-assisted annotation creation render our solution rather unique. Our approach is comparable to the question-sequence-based guidance provided by some complex installation wizards, where the questions are based on the information gathered earlier. In Docuphet the set of IFR elements represents the knowledge base, which provides a sophisticated, flexible and order-independent solution.
Docuphet is dialogue-assisted semantic text annotator. The computer–human dialogue is facilitated by IE techniques: named entity recognition and information frame extraction. In Docuphet— unlike some semantic wikis—it is not possible to annotate the text aIt collects all named entities as potential instances of ontology classes and build the corresponding ontology simultaneously. Therefore Docuphet is only capable of handling pre-defined RDF information triples, which limits the flexibility of the system. On the other hand, this very property allows to compose easy-to-understand questions about the known triples, as the questions are defined together with the corresponding IFs. This way it is easy to create annotations even for the completely uninitiated users.
Given these properties, Docuphet is most useful when the domain of the text is known in advance. Two exemplary applications were presented in Section 4. Other possible applications include annotation of economic or sport news, product reviews or geolocation reviews. In these cases the set of appropriate IFs and corresponding IFR rules have to be created in advance.
Despite these limitations, there is a basic functionality available
without specific domain knowledge. Docuphet is capable of
recognizing NEs in an arbitrary text, and formulating questions about
the NE candidates. This makes it a very useful tool for building
NE databases and for disambiguation applications. Another
possible application area is the assistance of context sensitive browser
applications tools, such as In4’s iGlue [
As for the future work, we intend to enable Docuphet to access and edit wikipedia articles via the interface provided by the MediaWiki’s public API. As wikipedia uses the wikitext format, being very different from Docbook/RDFa, apparently the most problematic is the conversion of the articles, and the placement of the the annotations in wikitext. We also plan to integrate Docuphet with large public databases like IMDB, to facilitate disambiguation and named entity recognition.
We think that in machine understanding, bidirectional communication — questions and answers — is a key element — just like in human understanding. However, we admit that if the questions are not relevant enough, this proactive behavior probably causes discontent on the user’s part. To find out more about the users’ reactions when using our system, we intend to conduct experiments and surveys with many users.
The relevance of the questions can be improved if topical category
labels are available for the documents. Therefore we plan to
prepare the Docuphet to collaborate with document classifiers, such as
e.g. the hitec3 framework [
Domonkos Tikk was supported by the Alexander von Humboldt Foundation. [7] R. Corderoy. troff. www.troff.org/. [8] DFKI Knowledge Management. Kaukolu.
www.dfki.de/web/forschung/km/. [9] The Docuphet project. www.docuphet.net. [11] FCKEditor. www.fckeditor.net/. [42] wikitext. en.wikipedia.org/wiki/Wikitext.