Recycling an Information Extraction system to automatically produce Semantic Annotations for the Web Thierry Poibeau Alexandre Arcouteil Cyril Grouin Thales et LIPN INALCO INALCO Domaine de Corbeville 2, rue de Lille 2, rue de Lille F-91 404 Orsay, France F-75007 Paris, France, F-75007 Paris, France, thierry.poibeau@thalesgroup.com alexandre.arcouteil@inalco.fr cgrouin@msh-paris.fr Abstract This paper is intended to show how an Information extraction system can be recycled to produce RDF schemas for the semantic web. We show that this kind of systems has to respect operational constraints like the fact that the information produced must be highly relevant (high precision, possibly low recall). We conclude in reconsidering some tasks like Question Answering (Q/A): the production of explicit structured data on the web will lead a better relevance of information retrieval engines. 1 Introduction 2 Related work Information Extraction (IE) is a technology The bases of IE as defined in the introduction dedicated to the extraction of structured are exposed in (Pazienza, 1997). IE is known information from texts. This technique is used to have established a now widely accepted to highlight relevant sequences in the original linguistic architecture based on cascading text or to fill pre-defined templates (Pazienza automata and domain-specific knowledge 1997). (Appelt et al, 1993). Several papers mentioned With the development of the semantic web, current limitations of MUC-like systems in such tools appear to be very interesting to terms of adaptability and studied the resource automatically extract semantic information development cost (Grishman & Sundheim, from existing web pages. In this paper, we will 1996). Event’99 was a task intended to not focus on the analysis of semi-structured evaluate event-level indexing into news stories documents by means of wrappers. Even if (Hirschman et al., 1999). The idea is “to HTML is a semi-structured format, most of the minimize the number of event-specific rules” information available on the web is located to be produced and to favor a light generic inside unstructured and untagged paragraphs. template, informally called a “templette” This paper is intended to show how an (Event 99). This paper is based upon the same Information extraction system can be recycled idea applied to the AFP newswire. to produce RDF schemas for the semantic web. Different systems tried to extract We will see that this kind of systems has to information in analyzing the structure of respect operational constraints like the fact that different kind of texts. For example, (Lacroix the information produced must be highly et al. 98) presents a system able to extract relevant (high precision, possibly low recall). information from the structure of HTML We conclude in reconsidering some tasks like pages. This kind of application will increase Question Answering (Q/A): the production of with the development of more structured explicit structured data on the web will lead a document (like XML documents). Wrapper better relevance of information retrieval factories go one step beyond, in connecting engines. together distant pieces of texts and in extracting information from poorly structured documents (Sahuguet and Azavant 98). To address the problem of portability, a state toolbox Intex to design dictionaries recent research effort focused on using and automata (Silberztein 1993). machine learning throughout the IE process (Muslea, 1999). A first trend was to directly apply machine learning methods to replace IE components. For example, statistical methods have been successfully applied to the named- entity task. Among others, (Bikel et al., 1997) learns names using a variant of hidden Markov models. However, a 90% success rate is reached at the cost of tagging manually about half a million words. (Cucchiarelli & Velardi, 1999) propose a more interesting approach: they adopt a hybrid approach mixing a core generic system extended with some learning Figure 2: The named entity recognizer mechanisms. Their system is able to learn new proper names by generalizing the data extracted by the basic rule-based system. We • The third stage performs text will adopt a very similar approach, except the categorization from “semantic fact that we want to stay in a symbolic signatures” automatically produced from framework, mainly for readability reasons. a rough semantic analysis of the text. We use an external industrial system 3 Inf ormation extraction system implementing a vector space model to The architecture consists in a multi-agent categorize texts (the Intuition™ system platform. Each agent performs a precise from the French company Sinequa, cf. subtask of the information extraction Salton (1988)). process. A supervisor controls the overall • The fourth stage extracts specific process and the information flow. The information (most of time, specific overall architecture is presented in (Poibeau, relationships between named entities). It 2001). can be for example the number of The system can be divided into five parts: victims of a terrorist event. This step is information extraction from the structure of achieved in applying a grammar of the text, the module for named entity transducers (extraction patterns) over the recognition (location, dates, etc), semantic text. filters, modules for the extraction of specific domain-dependent information and modules • The next stage links all these information for the filling of a result template. together to produce one or several result • Some information is extracted from the template(s) that present(s) a synthetic view structure of the text. Given that the AFP of the information extracted from the text. newswire is formatted, some wrappers The template corresponding to the text is automatically extract information about chosen among the set of all templates, the location and the date of the event. according to the identified category of the This non-linguistic extraction increases the quality of the result by providing text (registered by the system at the third 100% good results. It is also accurate analysis step). A specific template is when one thinks of the current produced only if some main slots are filled development of structured text (HTML, (the system distinguished among XML) via the web and other corporate obligatory and optional slots). networks. • The second stage is concerned with the Partial templates produced by different recognition of relevant information by sentences are merged to produce only one means of a linguistic analysis. This stage template per text. This merging is done under allows the recognition of various named constraints on what can be unified or not. The entities (person names, organizations, locations and dates) of the text. New results are then stored in a database, which kinds of named entities can be defined exhibit knowledge extracted from the corpus. according to a new domain (for examples, gene names to analyze a genome database). We use the finite- 4 Application overview : know ledge 5 Semantic annotations and other extraction f rom various domains outputs Various applications have been developed The system currently produces various kinds of using the above architecture, to ensure its output, for example: genericity. They concerned different domains: − XML(/HTML) tagged texts for named − Event-based extraction and indexing of the entity highlighting in texts. AFP newswire. This multi-domain extraction system is currently running in real time, on the AFP newswire. About 15 templates have been defined that cover about 30% of the stories. From the remaining 70%, the system only extract surface information, especially thanks to the wrappers. The performances are between .55 and .85 P&R, if we do not take into account the date and location slots that are filled by means of wrappers. New extraction templates are defined to prove system scalability. − Event-based extraction from financial news Figure 3: A news story from FirstInvest annotated stories (FirstInvest, a French financial with XML tags website). This application is very close to the previous one. − Extraction of gene interactions from the − Event database for the analysis of the AFP genomics database Flybase. This kind of newswire. A new template is produce for bases are structured by gene description, each new event. but researchers want to find relations among genes. In this context, the IE engine is intended to automatically produce a knowledge base about gene interaction, from the analysis of free texts. − Customer Request Management application (extracting information from emails relating software problems). This last case poses the problem of email analysis and management. The language used in such texts is not as correct as it can be in news stories. Specific grammar and orthographic relaxations must be applied to achieve relevant results. Figure 4: Event-base AFP indexing: each text fills a specific template, given its topic The last three applications concern texts from the Internet. FirstInvest is an electronic financial newswire available on the Web. − A dynamic knowledge base for gene Flybase, like other electronic databases in interaction (a query-able knowledge base genomics, is a collection of public data freely made of Prolog-like terms) available for researchers. The CRM application concerns a currently very popular area, which activation(1.28,Dfd) is also related to Knowledge Management. activation(5-HT1A,C) activation(ac,E) activation(Ac13E,G) activation(Dfd,1.28) interaction(2R-F,mys) interaction(2R-L,mys) Figure 5: A part of the knowledge base generated from the analysis of Flybase The range of performance is generally located possible to semi-automatically adapt the system so that precision is very high. This point is crucial to produce high quality data for lower recall than classical IE tools (recall we often see a precision above .95 with a recall of .15, which is not a problem as such since genomics databases are highly redundant. Of course, an effort is made to produce data with 6 Information Extraction and RDF Expressing structured information using the RDF syntax should provide interoperability two kinds of information should be produced − a structure analysis inferring a RDF schema statement; schema. 1.1 Structure analysis An IE system isolates semantic groups from which a RDF schema is built. As an example, in the case of Flybase, the IE system identifies 7 Structured data interactions between genes, the list of genes According to the RDF schema, the gene and and the list of interactions. A RDF schema interaction description in Flybase is defines an object class and the relations that represented ina new RDF file. Let’s take the could exist between several objects from these example of two genes, abd-A and trx: classes. In our case, these classes are the class “gene” and the class “interaction”: + Then, the system defines statements about The interaction between the two genes is existing constraints on the classes and the represented as follows: properties, according to the syntactic analysis of the document: P&R is the harmonic means of recall and precision. This metric is classical to measure the performance of filtering and extraction systems. http://www.itl.nist.gov/div894/894.01/
Lacroix Z., Sahuguet A. and Chandrasekar R. 1998. Information Extraction and Database Techniques: A User-Oriented Approach to Querying the Web. Conference on Advanced 8 Conclusion Information Systems Engineering, 1998. In this paper we have shown that a versatile IE Muslea I. 1999. Extraction patterns for system is very appropriate to automatically Information Extraction tasks: a survey, analyze unstructured texts from the web and AAAI’99 (available at: http://www.isi.edu/ produce semantic annotations. Some ~muslea/RISE/ML4IE/) researches still need to be done to produce more robust IE tools that will be able to deal Pazienza M. T. ed. 1997. Information with various kind of texts. We have proposed extraction (a multidisciplinary approach to an some methods (Poibeau, 2001), but large emerging information technology), Springer experiments still need to be done. In particular, Verlag (Lecture Notes in Computer Science), it is necessary to mix NLP approach with Heidelberg, Germany. wrappers to make good use of semi-structured Poibeau T. 2001. “Deriving a multi-domain texts. information extraction system from a rough These systems should change a bit the face ontology”. In Proceeding of the 17th of the Web. Given that more and more International Conference on Artificial structured and semantically annotated data will Intelligence, Seattle, USA. pp. 1264–1270. be available, Question Answering systems should give more accurate answers to user Sahuguet A. and Azavant F. (1998) W4F: a requests, for example. In this sense, IE systems WysiWyg Web Wrapper Factory, Technical allow to really extract and structure the report from the Penn Database Research semantic of the Web. Group, University of Pennsylvania. Bibliography Salton G. 1988. Automatic Text Processing. Addison-Wesley, Reading, MA. Appelt D.E, Hobbs J., Bear J., Israel D., Kameyana M. and Tyson M. 1993. FASTUS: a Silberztein M. 1993. Dictionnaires finite-state processor for information extraction électroniques et analyse automatique des from real-world text. Proceedings of IJCAI’93, textes. Masson, Paris. Chambéry, France, pp. 1172—1178. W3C. 1999. Resource Description Framework Bikel D., Miller S., Schwartz R. and (RDF) Model and Syntax, W3C Weischedel R. 1997. Nymble: a high Recommendation, 22 February 1999 performance learning name-finder, Proceeding (http://www.w3.org/TR/REC-rdf- of the fifth Conference on Applied Language syntax/} Processing, Washington, USA. Cucchiarelli A. and Velardi P. 1999. Adaptability of linguistic resources to new domains: an experiment with proper noun dictionaries, Proceedings of the Vextal Conference, Venice, Italy, pp. 25—30. Grishman R. and Sundheim B. (1996) Message understanding conference-6, a brief history. Proceedings of Computational Linguistics (COLING’96), Copenhagen, Denmark, August 1996. Hirschman L., Brown E., Chinchor N., Douthat A., Ferro L., Grishman R., Robinson P. and Sundheim B. 1999. Event99: a proposed event indexing task for broadcast news, available at