In this paper, we present MantisTable V, a novel unsupervised and automatic approach for the Semantic Table Interpretation. The approach is performed against DBpedia and Wikidata, and it can be easily adapted to any other Knowledge Graph. Moreover, we provide a tool (LamAPI) that allows to e ciently fetch data needed for Semantic Table Interpretation tasks from the Knowledge Graph dumps. The approach is manageable through a User Interface (tUI), a separated tool, which allows the visualisation and modi cation of table data and semantic annotations.
The Semantic Table Interpretation (STI) is a research eld in continuous evolution with increasing interest over time, also considering the great di usion of tabular data on the web. The input of STI is: i) a well-formed and normalised relational table (i.e., a table with headers and simple values, thus excluding nested and gure-like tables), as the one in Fig. 1, and ii) a Knowledge Graph (KG) which describes real-world entities in the domain of interest (i.e., a set of concepts, datatypes, predicates, instances, and the relations among them), as the example in Fig. 2. The output returned is a semantically annotated table, as shown in Fig. 3.
Moreover, the STI process is composed of the following main annotation steps: i) semantic classi cation of columns, which takes into account the values of a column to mark it as Literal column (L-column) if values are datatypes (e.g., strings, numbers, dates, etc., such as 2015, 10/04/1983, etc.), or as Named-Entity column (NE-column) if values are concepts (e.g., Film, Director, etc., such as Jurassic World, Colin Trevorrow, etc.); ii) detection of the subject column (Scolumn), which identi es the main column (the one all the others are referring to) among the NE-columns identi ed in the previous step (e.g., the Title column in Fig. 3); iii) concept and datatype annotation, which associates NE-columns with a concept in the KG (e.g., the column Title is associated with Film in Wikidata1), and L-columns with a datatype in the KG (e.g., the column Year is of type date); and iv) predicate annotation, which identi es the relations between the S-column and the other columns (e.g., Film publication date Year).
Each of the above steps is obtained by annotating column values referring to existing KGs. For example, in Fig. 3 if the majority of entities in the Title column is associated with Film, these entities are of type Film. Similarly, publication date can be identi ed as the predicate connecting entities in the Title column with datatypes of type date of the Year column.
Unfortunately, explicit situations like the ones in the example are not so common, therefore we need to set up strategies and algorithms to address several issues.
This work is an improvement and extension of MantisTable (seManti cs Table)
[
iterative, to improve the cell disambiguation process. Compared to MantisTable SE, this new version of the approach can consider di erent types of tables (concerning the number of columns) and uses new algorithms to identify and classify L-columns. It also manages ambiguity in annotations optimally, as will be analysed in the following sections. Our approach uses DBpedia and Wikidata as the rst matching Knowledge Graphs (KGs) because they are the richest data sources with ground truths available. MantisTable V can be easily adapted to be used with any KG, through the use of a new tool, called LamAPI (Label matching API ). Thanks to LamAPI and its index systems, it is possible to efciently search a particular entity by its ID or full-text search. Together with MantisTable V and LamAPI, we have developed a user interface (tUI) that allows the management of the tables and the annotations, as well as the update. tUI (t able U ser I nterface) is a fully con gurable tool, which can be used with any STI approach.
The main contributions of this paper are: (i) MantisTable V, a comprehensive approach which deals with all phases of the STI process, (ii) LamAPI, an open-source tool to e ciently manage and retrieve data of KGs, (iii) tUI, a fully con gurable open-source UI to manage and display and update tables and semantic annotations. All tools have been encapsulated in Docker containers to facilitate the deployment and scalability by replication.
The remainder of the paper is organised as follow: in Section 2 we describe the functionalities of the LamAPI tool while in Section 3 MantisTable V is described. Details on tUI are depicted in Section 4. Section 5 introduces the Gold Standards and discusses the evaluation results. Finally, conclusions and pointers, are presented in Section 6. 2
As seen in Section 1, to obtain the STI of tabular data, it is required to link
elements of the table with the elements of a KG. The elements in the KGs (e.g.,
DBpedia or Wikidata) are frequently stored in Resource Description Format (RDF)
format, so to access these elements, it is necessary to query a SPARQL endpoint.
For instance, the most popular way to access DBpedia dumps is by using
OpenLink Virtuoso, a row-wise transaction-oriented RDBMS with a SPARQL query
engine to access to RDF graph store2. Wikidata instead uses Blazegraph3 that is
a high-performance graph database supporting RDF/SPARQL APIs. The issue
faced with these solutions is the time required for importing the data.
Wikidata 2019 dump requires some days to set up4. Another problem is given by
the amount of information present in a KG; for instance, the Wikidata dump is
about 1.1TB (uncompressed). The English version of DBpedia instead is split
into multiple les of the size of 26GB, which leads to high computation times
to obtain a complete STI (e.g., TableMiner+ according to the author in [
Therefore, in order to obtain an e cient approach, it is necessary to identify
other ways for querying KG. To do that, :n the state of the art, two works can
be identi ed, FactBase [
The open-source tool that provides these APIs is called LamAPI (Label matching API )7 and provides the following services: 1. Lookup: given a free text (in this case a data inside the table cell), it retrieves the entities with the greatest similarity, using the IB similarity8 scoring algorithm of ElasticSearch which combines di erent search strategies (i.e., full-text search based on tokens, on n-grams and fuzzy search). The ElasticSearch contains an index of the KG entities to improve the performance. Considering a table cell containing \Jurassic World" the result returned is shown in Listing 1.1 or Listing 1.2; 2. Concepts: given an entity it retrieves all its concepts as shown in Listing 1.3. This service can extend automatically the list of concepts associated with a given entity through the use of vector similarity measures between the di erent concepts in the KG. Thanks to this functionality, it is possible to extend the candidates associated with a cell. 3. Literals: given an entity it retrieves all the related literals values and predicates as shown in Listing 1.3; 2 virtuoso.openlinksw.com 3 blazegraph.com
similarity.html 4. Predicates: given two entities it retrieves all the predicates between them; considering the entity \Jurassic World" and the entity \Colin Trevorrow", the list of predicates is shown in Listing 1.4; 5. Objects: given an entity it retrieves all the related objects and predicates; for example, with the entity \Jurassic World" the result is the one shown in Listing 1.4.
of large dimensions10. For Wikidata, we had to make a di erent design decision in order to support multi-language.
This new way to access DBpedia/Wikidata provided by LamAPI overcame the limitations of SPARQL endpoints such as: { SPARQL endpoint response times are directly proportional to the size of the returned data. In this context, sometimes it is not even possible to get a result because of the endpoint returning a timeout; { the volume of requests per second is limited (online endpoint) or computationally expansive (a local endpoint requires at least 64GB of ram and tons of CPU cycles);
10 dumps.wikimedia.org/wikidatawiki/entities/ { there are some intrinsic limits in the SPARQL language expressiveness (i.e., the full-text search capability that is useful for label matching is possible to be obtained only with extremely slow \contains" or \regex" queries11).
LamAPI is developed using ElasticSearch, MongoDB and Python12. This Section will focus on the algorithmic process of MantisTable V13. The process is organised into eight phases as follows: i) Data Preparation and Normalisation, ii) Column Analysis and Subject Detection, iii) Cell Entity Annotation (CEA), iv) Column Predicate Annotation (CPA), v) Column Type Annotation (CTA), vi) Revision and viii) Export.
The process was designed to retrieve the candidate for a given cell only once,
especially if the content of that cell is repeated across multiple tables. This
allows the approach to avoid repeating queries for the same content and saves
network time. Every phase must be completed for all the tables in a dataset
before going to the next one. This does not preclude running the tool against
only one table, but when running with multiple tables, the execution time will be
sharply reduced (for the exact text in a cell, we have the same candidate entities
that will be sorted considering the content of every table. This will be explained
more in detail in Paragraph 3). To describe each phase of the STI approach,
consider Table 1, which lists some movies with additional information, such as
director and release year.
11 docs.openlinksw.com/virtuoso/rdfsparqlrulefulltext/
12 lamapi.ml
13 bitbucket.org/disco unimib/mantistable-v/
title director release year domestic distributor length in min worldwide gross
Jurassic World Colin Trevorrow 2015 Universal Pictures 124 1,670,400,637
Superman Returns Bryan Singer 2006 Warner Bros. 154 391,081,192
Batman Begins Christopher Nolan 2005 Warner Bros. 140 371,853,783
Avatar James Cameron 2009 Twentieth Century Fox 162 2,744,336,793
i. Data Preparation and Normalisation. During this phase, all tables' cells are
analysed using a tokeniser managing special characters and additional spaces.
For each normalised cell, the candidate entities are retrieved from the Lookup
service of LamAPI. The obtained candidates will be ranked during the next
phases.
ii. Column Analysis and Subject Detection. During Column Analysis we
identify literal columns (L-column) by using a set of Regextypes [
All the values are normalised in [
For L-columns, the con dence score is computed as follows: { for L-columns with numeric datatype ( oat and integer Regextypes): all the numeric values (object of RDF-triples) linked to candidate entities are taken using the Literal service of LamAPI. The con dence score is calculated as of the formula in Equation 2, where lit(ei;j ) is the numerical values associated to the candidate entity ei;j .
jtx(i; j) lit(ei;j )j max(jtx(i; j)j; jlit(ei;j )j; 1) (2) { for L-columns with string datatype: the con dence score is computed using the Jaccard distance. We change the similarity measure, particularly for the long string, because the number of edits required to change a long string into another one is not necessarily signi cant (Edit distance). The Jaccard instead considers n-grams. { for L-column with date datatype, the dates are considered as sortable numeric values in the format YYYYMMDDHHmmSS. The con dence score is computed as described for the numeric datatype.
Cells with the same content in di erent tables start considering the same set of candidate entities, but it will be sorted di erently concerning the entire table's contents in the next phases. As an example, we consider the cell containing \superman returns" in Table 1. In this case, it is referred to the movie, but if we consider Table 2, it is referred to the video game.
videogame publisher release date superman returns electronic arts 2006 pokemon white nintendo 2010 call of duty activision 2003
Considering the Table 1 and the cell \Superman Returns", the candidate entities are associated to the ontology concept \ lm" and \video game" (Listing 1.5). In the CTA phase, where the approach extracts the types (concepts) of entities, all entities associated with the concept \video game" are penalised because the most frequent concept is \ lm".
Instead, when we consider the video games in Table 2 the CPA phase, where the approach extracts the relationship between entities, allowed us to penalise all entities with the concept \ lm" with the same name because it has few or no relationship with the rest of the content of the row.
iv. Column Predicate Annotation (CPA) Considering that all the necessary information is gathered in the previous phase using LamAPI, the CPA is a relatively fast process. All the predicates previously identi ed for each column are sorted by their relative frequency to the entire column. The predicate with the greatest frequency will be ranked rst. This process allows reducing the number of candidate entities for every cell. Con dence scores for the predicates of the director column are shown in Listing 1.8.
When we consider the video games in Table 2, the CPA phase allowed us to penalise the \ lm" with the same name. This is because it does not have any relationship with the rest of the content of the table (the lm does not have anything to do with \electronic arts", while the video game has a property with exact match).
Listing 1.8. Example for CPA.
" P57 ": 2 "" lcaobneflid"e:nc"ed"i:rec1t.o0r " 4 " P58 ":
" label ": " screenwriter " 6 " P162""c:onfidence ": 0.75 8 "" lcaobneflid"e:nc"ep"r:odu0c.e6r25" 10 " P161 ":
" label ": " cast member " 12 " confidence ": 0.25 v. Column Type Annotation (CTA) To get the CTA annotation, we collect the types/concept of every ei;j resulting from the CEA. The concept with maximum frequency has been selected for the CTA annotations.
For every column, we collect the frequencies of the concepts, as shown in Listing 1.9 for Wikidata and in Listing 1.10 for DBpedia. 2
Listing 1.9. Example of the structure storing the most frequent concepts. " movie_table ": "0": " Q229390 (3 D film )": 1, " Q11424 ( film )": 1, "1":" Q25110269 ( live - action / animated film )": 0.33 "3":" Q5 ( Human )": 1 " Q1762059 ( film production company )": 1, " Q375336 ( film studio )": 0.5 , " Q1107679 ( animation studio )": 0.5 , " Q18127 ( record label )": 0.5 , " Q4830453 ( business )": 0.5 , " Q10689397 ( television production company )": 0.25
Listing 1.10. Example of the structure storing the most frequent concepts. " movie_table ": "0": " Film ": 1 " Work ": 1 "1":
" Person ": 1 "3": " Company ": 1 " Organisation ": 1
If the system returns many annotations for one column (e.g. column 0 in Listing 1.9, 1.10), the approach randomly selects one of them as the nal annotation. vi. Revision The revision phase analyses all the information gathered in the previous phases to do a nal reordering of the candidates. In particular, this allows for correcting CEA entities previously selected: every entity have to be coherent with the rest of the concepts and predicates selected in every column. Moreover, predicates are also re-ranked. vii. Export The MantisTable V approach previously described keeps the candidates coming from each phase. It is possible to apply some thresholds during the export phase to balance annotation quality and the number of annotations provided. The export threshold can have a signi cant role in evaluation metrics for every gold standard.
MantisTable V is developed using Python. 4
User Interface of tUI tUI14 is a Web application that aims to provide a visualisation tool for STI approaches; it can work with any backend that provides API endpoints to retrieve data. Endpoints are stored in a YAML con guration le: UI will display data and functionalities based on which APIs are available. tUI consists of three main parts: i) the view listing the datasets, ii) the view listing the tables contained in each dataset (Fig. 6), iii) the view showing the table data and the semantic annotations (Fig. 7). Regarding the annotated table view, tUI supports all the three main tasks of the STI (CTA, CPA, CEA): annotations can be viewed directly 14 bitbucket.org/disco unimib/tui/ inside the table. Concerning the CEA task, if multiple candidates are retrieved, all of them can be shown in the UI. Endpoints to retrieve the datasets list, the tables list and the table data (with or without annotations) are mandatory to ensure the basic functionality of the tool and display the data correctly. Other non-mandatory endpoints that may be provided will enable the following features: i) export of annotations in any format (Fig. 9, multiple exports supported, e.g., SemTab CSV, JSON-LD, RDF/XML, RDF/N-Triple, R2RML), ii) editing and saving annotation (Fig. 8), and iii) global search (into datasets or tables). tUI is developed using React and Typescript15.
To evaluate our approach on a large dataset and compare it with other
state-ofthe-art approaches, we tested our approach against di erent datasets and gold
standards. In particular we consider T2Dv2 Gold Standard16 and the three most
complex rounds of the various versions of the international challenge SemTab17
[