Annotation is a key way in which documents grow and increase in value. This paper explores the possibility to use concepts extracted from documents by using a Natural Language Processing tool to characterize the content of digital theses. Then, using the results of the study, the paper explores the use of annotated theses in order to access to pertinent information stored in these documents and to extract knowledge by an “intelligent” search system.
The growth of the World Wide Web and of the corpus of documents it covers has increased the necessity to propose solutions to improve information retrieval. Our proposition is based on a specific semantic annotation process of the documents, made during the writing step and explored during the search session. These semantic annotations also allow knowledge extraction from the documents and lead to an intelligent information processing. Accessing and extracting knowledge from online documents is crucial to develop advanced knowledge services for the Semantic Web.
The scientific library of Doc’INSA sets up since 1997 a project named CITHER, which makes possible the diffusion and the access of scientific theses through Internet. Currently, a user can get the contents of only one thesis at the same time without being able to select relevant extracts corresponding to a unit of corpus finer than the chapter. This is the result of: the use of an inadequate format, such as PDF (Portable Document Format), the description of the contents by only the keywords added outside the documents, and the use of the tags proposed by the Dublin Core metadata which bring general information of the thesis.
Our research focus is based on the opportunity to use the domain concepts to build the users requests and to organize the document structure. We propose to the users to build a semantic structure for the documents by using a Natural Language Processing (NLP) tool that extracts concepts and by using a base of concepts of the domain field. The selected concepts are stored in the documents as semantic tags. Then, a user’s query generates a web access to a page that contains pertinent information. The discovery of relevant contents is done by matching the user’s query with the Embedded Semantic Tags (EST’s). Once the desired information is found the user is able to read only the pertinent fragments and he can select the right documents, in our case scientific theses.
While recent research efforts seek to add relevant markups to the content of the web
pages [
Generating Adaptive Annotation to Structure Documents
The Semantic Web aims to create contents that can be manipulated by humans but
also by machines [
The HyperText Markup Language (HTML) was the initial language used to display
documents on the web. The main drawback of HTML is its inability to represent
semantic contents. This led to the Extensible Markup Language (XML) [
Following a meticulous work of extraction of concepts [
We have built a knowledge base with the concepts extracted from a corpus of
theses. This base must be regularly updated with the new concepts extracted from new
theses stored in the digital library. By making some experiments, we have evaluated
that the number of the concepts brought by each new thesis of a specified domain does
not increase infinitely; it quickly tends towards a constant stabilization. The number
of new concepts becomes to be very weak after the evaluation of about 15 to 17 theses
of the same domain [
Our proposal for adaptive semantic annotation can be characterized by the following elements: •
The PhD student is assumed to write his thesis by making semantic annotations. These annotations are generated in XML format. To simplify the student task, a concept extraction tool can be called after the selection of a written fragment, section or chapter. The NLP tool, named Nomino, proposes pertinent concepts and the user can accept or deny then for an insertion in the document • Another way to add concepts is by selecting them from the base of concepts. In the base, hierarchies according to the computer field order the concepts. • While the new tags are currently employed in the document the annotation system is transparent to the student and it is not necessary to know how works XML.
Our proposed annotation system involves the addition of a schema that defines the structure of the document (the thesis). In the next section, we describe the schema used in order to validate well-structured documents. 3
Validation of the Structure by Using XML Schema
A thesis is based on several logical entities, like the introduction, the conclusion, the
chapters, the sections, the subsections, the paragraphs and the blocks of text that are
the finest textual logical entities. These entities can be “tagged” or “not tagged”. A set
of “tagged” paragraphs can constitute a “BlockParagraphe”. The “tagged” state results
from the presence of metadata (concepts) surrounding the “BlockParagraphes”. At the
beginning of the block we find the heading of the metadata “EnteteMetadata”, and at
the end, the “PiedMetadata” (see Fig. 1). The existence of elements such as the
heading of metadata, the lists of paragraphs and the foot of metadata is necessary in a
“tagged” block, so the minimum cardinality of each element is one. In the same way,
the entities named “EnteteMetadata” and “PiedMetadata”, all the elements such as the
opening or closing tags, the list of the concepts and the boolean variables “Precedent”
(before) and “Suivant” (following) must be initialized. The overlap between different
elements is not allowed in XML Schema [
In this section we illustrate the use of EST’s to identify and extract responses to queries by using concepts names. A digital thesis search tool is used to parse the theses and extract the required response. Using narrower and broader concepts found in the base of concepts expands the user request, composed by keywords or concepts-words. These new concepts are proposed to the user in order to clarify his main idea and to reformulate the query by using adequate concepts. The search tool is able to provide the fragments where the concepts of the query physically appear and the fragments surrounded by the pertinent semantic tags even, if the concepts are not explicitly written in the fragments. For example, if we have the following XML paragraph “<Internet> <Semantic_Web>The Semantic Web provides a common framework that allows data to be shared and reused across application, enterprise, and community boundaries. <Internet/> </Semantic_Web>” and the user is searching for all the fragments containing the concept “Internet”, by using any research system he is not going to obtain the paragraph presented above. Instead, by using our system, even if the “Internet” word is not written in the paragraph, by using our XML tags the user is going to find this paragraph.
XML Schema
Thesis Metadata
Logic structural elements
Title Section Chapter Paragraph …
Semantic annotation Semantic structural elements
Digital library Methology Resource metadata Metadata schema …
Thesis repository
The thesis produced by the student is composed by metadata coming from the logical structure (chapter, section, paragraph, image, etc.,) and metadata coming from the semantic structure (paragraph about “architecture”, “model” or “prototype”, etc.). The metadata used for the semantic structure are very powerful because they give specific information about the fragment or the associated concepts. For example, we can have concepts like: “digital library”, “Fourier model”, etc. Thanks to these two kinds of metadata the information retrieval tool is able to explore the tags in order to know which chapter is talking about a specific concept (see Fig. 1). A typical user interaction with the search system consists in the insertion of a query. If there are more concepts closer to those used in the query then the concepts base will propose to the user other concepts in order to expand the query. The user has the option to select the most adequate concepts to expand his request. Finally, once the user has selected the concepts, the system searches the right information in the thesis repository and then it shows the pertinent fragments to the user. In this paper we present an approach to find pertinent information by using information retrieval tools in a digital library context. We have defined a specific structure for the digital document during its creation. According to this point of view, we have defined a semantic structure of the document by integrating new metadata in significant parts of the corpus. This makes possible to identify semantic segments of the scientific theses stored in our digital library: CITHER. In a search session based on keywords or concepts, the system will compare them with the semantic metadata (delimiting the semantic segments) and with the keywords describing the thesis. Thanks to this approach the user can get pertinent fragments of one or several theses.