In this paper we present AMALGAM, a matching approach to annotate tabular dataset with the use of a knowledge graph, developed in the context of the Semantic Web Challenge on Tabular Data to Knowledge Graph Matching (SemTab 2020). The ultimate goal is to provide fast and e cient approach to annotate tabular data with entities from a background knowledge. The approach combines lookup and ltering services combined with text pre-processing techniques. Experiments conducted in the context of SemTab 2020 with both Column Type Annotation and Cell Type Annotation tasks showed promising results.
Making web data complying with the FAIR principles [
In order to perform a more ne-grained analysis of the SemTab challenge
tasks, we categorize them as two main tasks e.g., structure, and semantic
annotation. Structure annotation, deals with various tasks including data type
prediction and table header annotation [
Three tasks that are organised into several evaluation rounds are de ned in
SemTab 2020: (i) assigning a semantic type (e.g., a KG class) to a column (CTA);
(ii) matching a cell to a KG entity (CEA); (iii) assigning a KG property to the
relationship between two columns (CPA). The most popular approaches to deal
with these three tasks are a supervised learning setting, where entities candidate
are selected by a classi cation model [
To address the above mentioned tasks of the SemTab challenge, AMALGAM is designed according to the work ow depicted in Fig. 1. There are three major phases which consist in, respectively, Pre-Processing, Annotation context and Tabular data to KG matching. The rst two steps of the Work ow are identical for the two tasks.
To describe each phase of the AMALGAM approach we consider Table 1, which lists Alberta region towns with additional information, country, elevation above sea level, etc. Tables Pre-Processing. As the Table 1 shows, the content of each table can have di erent types (string, date, oat, etc.). The aim of this pre-processing step is to ensure that the process of loading each table happens without any error. For example, a problem of textual encoding where some characters are loaded as noisy sequences, text eld with an unescaped delimiter, will cause the record being processed to have an extra column, etc. Loading incorrect encoding might strongly a ect the lookup performance. Therefore, we used the Pandas2 library to x all noisy textual data in the tables.
Annotation context. We consider a table as a two-dimensional tabular structure (see Fig. 2) that is composed of an ordered set of x rows and y columns. Each intersection between a row and a column determines a cell cij with the value vij where 1 i x; 1 j y. To identify the attribute label of column also called header detection (CTA task), the approach consists in annotating all the items of the column using entity linking. Then, the attribute label row detection is estimated using the random entity linking. In this use case, the annotation context is represented by the list of items in the column. For example, the context of the rst row in the Fig. 2 is: [Grande Prairie, Sundre, Peace River, Vegreville, Sexsmith]. According to the same logic, we consider that all cells in the same row describe the same context. More precisely, the rst cell of the row describes the entity and the following cells the associated properties. For example, the context of the rst row in the Fig. 2 is: [Grande Prairie, city in Alberta, Canada, Sexsmith, 650, Alberta].
Assigning a semantic type to a column (CTA). The CTA task can be performed by exploiting the process described in Fig. 2. The Wikidata API allows to look up a Wikidata item3 according to the title of its corresponding page on a given Wikipedia page, or other Wikimedia family site. In our case, the main information needed from the entity is a list of the instances of (P31), subclass of (P279) and part of (P361) statements. To do so, a parser is developed to retrieve this information from the Wikidata built request. For example, "Grande Prairie" provides the following results: [list of towns in Alberta:Q15219391, village in Alberta:Q6644696, city in Alberta:Q55440238]. To achieve this, our methodology combines wbsearchentities and parse actions provided by the API. It could be observed that in this task, there were many items that have not been annotated. This is because tables contain incorrectly spelled terms. Therefore, before implementing the other tasks, a spell check component is required.
As per the literature [
3 https://www.wikidata.org/w/api.php 4 https://textblob.readthedocs.io/en/dev/ 5 https://nlp.johnsnowlabs.com/
Gurunudi6, Wikipedia api7, Pyspellchecker8, Serpapi9. A comparison of these approaches could be found in Table 2.
Our choice is oriented towards Gurunudi and the Wikidata API with a postprocessing step consisting in validating the output using fuzzywuzzy10 to keep only the results whose ratio is greater than the threshold of 90%. For example, let's take the expression \St Peter's Seminarz" after using the Wikidata API we get \St Peter's seminary" and the ratio of fuzzy string matching is 95%.
We are now able to perform the CTA task. In the trivial case, the result of an item lookup is equal a single record. The best matching entity is chosen as a result. In the other cases, where the result is more than one, no annotation is produced for the CTA task. Finally, if there is no result after the lookup, another one is performed using the output of the spell check produced by the item. At the end of these lookups, the matched couple results are then stored in a nested dictionary [item:claims]. The most relevant candidate, counting the number of occurrences, is selected.
Matching a cell to a KG entity (CEA). The CEA task can be performed by exploiting the process described in Fig. 3. Our approach reuse the process of the CTA task and made necessary adaptations. The rst step is to get all the statements for the rst item of the list context. The process is the same as CTA, the only di erence is where the result provides more than one record. In this case, we create nested dictionary with all candidates. Then, to disambiguate the candidates entities, we use the concept of the column generated with the CTA task. Next, a lookup is performed by using the other items of the list context in the claims of the rst item. If the item is found, it is selected as the target entity; if not, the lookup is performed with the item using the Wikidata API (if the result is empty, no annotation is produced).
7 https://wikipedia.readthedocs.io/en/latest/code.html 8 https://github.com/barrust/pyspellchecker 9 https://serpapi.com/spell-check 10 https://github.com/seatgeek/fuzzywuzzy
With this process, it is possible to reduce errors associated with the lookup.
Let's take the value \650\ in row 0 of the table Fig. 3 for instance. If we lookup
directly in Wikidata, we can get many results. However, if we check rst in the
statements of the rst item of the list, \Grande Prairie\, it is more likely to
successfully identify the item.
This section reports the overall results (The Primary Score) of AMALGAM for the
two matching tasks in the four rounds of SemTab 2020 [
The primary results (F1 score and Precision) of AMALGAM for CTA and CEA in the four rounds are presented in Tables 3, 4, 5 and 6. In the rst round, AMALGAM achieved the results in Table 3. It could be observed that AMALGAM handles properly the two tasks, in particular in the CEA task. Regarding the CTA task, these results can be explained according to a new revision created in the item revision history and there are probably spelling errors in the contents of the tables. For example, "rural district of Lower Saxony" after 16 April 2020 became "district of Lower Saxony". A possible solution is to retrieve the history of revisions (parsing Wikidata data history dumps) to use them. This could be a perspective to this work.
In Round 2 we particularly focused on the spell check of items to improve the results of the CEA and CTA tasks. Clearly, our choice tends to use two API services, Wikipedia and Gurunudi, for spelling correction. It could be observed that the achieved results are better than the previous round both in terms of precision and F1-Score. However, these results may be improved too. From previous rounds, we noted that one single term is ambiguous as it refers to more than one entity. In Wikipedia, there is only one article for each concept. However, there can be many equivalent titles for a concept due to the existence of synonyms, etc. For example, the term \Paris" may refer to many concepts such as \the capital and largest city of France\, \son of Priam, king of Troy\, \county seat of Lamar County, Texas, United States\, etc. For the next rounds, disambiguation item is required.
In Round 3 and 4, to overcome the issue related to disambiguation, we have updated our approach by integrating the concept of the column obtained in CTA in the linking phase. We showed that the two tasks can be performed relatively successfully with AMALGAM, achieving higher than 0.86 in precision and recall values. However, the automatic disambiguation of items proved to be a more challenging task.
A second evaluation is performed with the Tough Tables (2T) dataset which
is designed to evaluate table annotation approaches in solving the CEA and CTA
tasks [
Its advantage is that it allows to perform CTA and CEA tasks in a timely manner. This may be accomplished through the combination of a lookup services and a spell check techniques. It is the rst participation of the AMALGAM system which is still in the early stages, so there are still rooms from some improvements. However, the experimental results show that our approach achieved promising results.
Our ndings during this rst participation suggest that the matching process is very sensitive to errors in spelling. Thus, as of future work, an improved spell checking techniques will be investigated. To process such errors the contextual based spell-checkers are needed. Often the string is very close in spelling, but context could help reveal which word makes the most sense. Further more, we will improve the approach by nding a trade-o between e ectiveness and e ciency.