Information Extraction transforms unstructured or semi-structured text into structured data that can be understood by machines. It is a crucial technique towards realizing the vision of the Semantic Web. Wrapper Induction (WI) is the task of automatically learning wrappers (or extraction patterns) for a set of homogeneous Web pages, i.e. pages from the same website, generated using consistent templates 1 . WI methods [1,2] learn a set of rules enabling the systematic extraction of specific data records from the homogeneous Web pages. In this paper we adopt a user driven paradigm for IE and we perform on demand extraction on entity-centric webpages. We adopt our WI method [2,3] developed within the LODIE (Linked Open Data for Information Extraction) framework [4]. The main advantage of our method is that does not require manually annotated pages. The training examples for the WI method are automatically generated exploiting Linked Data.
Using WI to extract information from structured Web pages has been studied extensively. Early studies focused on the DOM-tree representation of Web pages and learn a template that wrap data records in HTML tags, such as [1,5,6]. Supervised methods require manual annotation on example pages to learn wrappers for similar pages [1,7,8]. The number of required annotations can be drastically reduced by annotating pages from a specific website and then adapting the learnt rules to previously unseen websites of the same domain [9,10]. Completely unsupervised methods (e.g. RoadRunner [11] and EXALG [12]) do not require any training data, nor an initial extraction template (indicating which concepts and attributes to extract), and they only assume the homogeneity of the considered pages. The drawback of unsupervised methods is that the semantic of produced results is left as a post-process to the user. Hybrid methods [2] intend to find a tradeoff with these two limitations by proposing a supervised strategy, where the training data is automatically generated exploiting Linked Data. In this work we perform IE using the method proposed in [2,3] and follow the general IE paradigm from [4].
In LODIE we adopt a user driven paradigm for IE. As first step, the user must define her/his information need. This is done via a visual exploration of linked data (Figure 1). The user can explore underlying linked data using the Affective Graphs visualization tool [13] and select concepts and properties she/he is interested in (a screenshot is shown in Figure 1). These concepts and properties get added to the side panel. Once the selection is finished, she/he can start the IE process. The IE starts with a dictionary generation phase. A dictionary d i,k consists of values for the attribute a i,k of instances of concept c i . Noisy entries in the dictionaries are removed using a cleaning procedure detailed in [3]. As a running example we will assume the user wants to extract title and author for the concept Book. We retrieve from the Web k websites containing entity-pages of the concept types selected by the user, and save the pages W ci,k . Following the Book example, Barnes&Noble2 or AbeBooks3 websites can be used, and pages collected in W book,barnesandnoble and W book,abebooks .
For each W ci,k we generate a set of extraction patterns for every attribute. In our example we will produce 4 sets of patterns, one per each website and attribute. To produce the patterns we (i) use our dictionaries to generate bruteforce annotations on the pages in W ci,k and then (ii) use statistical (occurrence frequency) and structural (position of the annotations in the webpage) clues to choose the final extraction patterns.
Briefly, a page is transformed to a simplified page representation P ci : a collection of pairs 〈xpath4 , text value〉. Candidates are generated matching the dictionaries d i,k against possible text values in P ci (Figure 2).
/HTML [1]/BODY [1]/DIV [2]/DIV [2]/DIV [2]/DIV [1]/H2 [1]/text() [1] breaking dawn /HTML [1]/BODY [1]/DIV [2]/DIV [2]/DIV [2]/DIV [4]/DIV [1]/H2 [1]/EM [1]/text() [1] breaking dawn /HTML [1]/BODY [1]/DIV [2]/DIV [2]/DIV [2]/DIV [4]/TABLE [10]/TBODY [1]/TR [1]/TD [3] Final patterns are chosen amongst the candidates exploiting frequency information and other heuristics. Details of the method can be found in [2,3]. In the running example, higher scoring patterns for extracting book title from AbeBooks website are shown in Figure 3. All extraction patterns are then used to extract target values from all W ci,k . Results are produced as linked data, using the concept and properties initially selected by the user for representation, and made accessible to the user via an exploration interface (Figure 4), implemented using Simile Widgets 5 .
A video showing the proposed system used with the running Book example can be found at http://staffwww.dcs.shef.ac.uk/people/A.L.Gentile/ demo/iswc2014.html.
In this paper we describe the LODIE approach to perform IE on user defined extraction tasks. The user is prompted a visual tool to explore available linked data and choose concepts for which she/he wants to mine additional material from the Web. We learn extraction patterns to wrap relevant websites and return structured results to the user.