SemTab 2019 [4, 3] was the first edition of the Semantic Web Challenge on Tabular Data to Knowledge Graph Matching, successfully collocated with the 18th International Semantic Web Conference (ISWC) and the 14th Ontology Matching (OM) Workshop: http://www.cs.ox.ac.uk/isg/challenges/sem-tab/ The Challenge Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 1AIDA project: https://www.turing.ac.uk/research/research-projects/ artificial-intelligence-data-analytics-aida
Tabular data in the form of CSV files is the common input format in a data analytics pipeline. However a lack of understanding of the semantic structure and meaning of the content may hinder the data analytics process. Thus gaining this semantic understanding will be very valuable for data integration, data cleaning, data mining, machine learning and knowledge discovery tasks. For example, understanding what the data is can help assess what sorts of transformation are appropriate on the data.1
Tables on the Web may also be the source of highly valuable data. The addition of semantic information to Web tables may enhance a wide range of applications, such as web search, question answering, and knowledge base (KB) construction.
Tabular data to Knowledge Graph (KG) matching is the process of assigning semantic tags from Knowledge Graphs (e.g., Wikidata or DBpedia) to the elements of the table. This task however is often difficult in practice due to metadata (e.g., table and column names) being missing, incomplete or ambiguous.
Tabular data to KG matching tasks typically include (i) cell to KG entity matching, (ii) column to KG class matching, and (iii) column pair to KG property matching.
There exist several approaches that aim at addressing one or several of above tasks
and datasets with ground truths that can serve as benchmarks. Despite this significant
amount of work, there was a lack of a common framework to conduct a systematic
evaluation of state-of-the-art systems. The creation of SemTab aims at filling this gap
and becoming the reference challenge in this community, in the same way the OAEI is
for the Ontology Matching community.
The SemTab 2019 challenge started in mid April 2019 and closed in mid October
2019. It was organised into four evaluation rounds where we aimed at testing different
datasets with increasing difficulty. Table 1 shows the participation per round. We had
a total of 17 systems participating in Round 1. Round 2 had a reduction of
participating systems (from 17 to 11), which helped us identify the core systems and groups
actively working in tabular data to KG matching. Round 3 and Round 4 preserved
the 7 core participants across rounds and all three tasks. SemTab 2019 core
participants: MTab [
SemTab 2019 was successful as we managed to (i) create a small community
around the challenge, (ii) advance the state of the art, (iii) gather feedback from the
evaluation to improve future editions of the challenge, and (iv) release 4 generated
benchmark datasets with ground truths [
Please refer to [
The results of the challenge were presented during ISWC 2019. Four participating teams also presented their systems.
els by Phuc Nguyen, Natthawut Kertkeidkachorn, Ryutaro Ichise and Hideaki Takeda.
(Tabularisi) by Avijit Thawani, Minda Hu, Erdong Hu, Husain Zafar, Naren Teja Divvala, Amandeep Singh, Ehsan Qasemi, Pedro Szekely and Jay Pujara.
(Team sti) by Marco Cremaschi, Roberto Avogadro, and David Chieregato.
tion System by Yoan Chabot, Thomas Labbe, Jixiong Liu and Raphae¨l Troncy.
We also had a devoted session during the Ontology Matching workshop where we described the challenge and had a system presentation:
dewiele, Bram Steenwinckel, Filip De Turck, Femke Ongenae.
Slides and photos are available on the challenge website: http://www.cs.ox. ac.uk/isg/challenges/sem-tab/ SIRIUS2 and IBM Research3 sponsored the prizes for the best systems in the challenge. This sponsorship was important not only for the challenge awards, but also because it shows a strong interest from industry.
1st Prize (CTA, CEA and CPA): MTab Team. 2nd Prize (CTA, CEA and CPA): IDLab Team. 3rd Prize (CTA, CEA and CPA): Tabularisi Team. 3rd Prize (CEA): ADOG Team.
Kavitha Srinivas (IBM Research): Kavitha.Srinivas@ibm.com
Udayan Khurana (IBM Research) Erik Bryhn Myklebust (University of Oslo) Monika Solanki (Agrimetrics) Ole Magnus Holter (University of Oslo) Pedro Szekely (University of Southern California) Basil Ell (University of Bielefeld; University of Oslo) Marco Cremaschi (University of Milano - Bicocca)
Asan Agibetov (Medical University of Vienna) 2SIRIUS: Norwegian Centre for Research-driven Innovation: https://sirius-labs.no 3https://www.research.ibm.com/ We would like to thank the challenge participants, the ISWC & OM organisers, the AIcrowd team, and our sponsors (SIRIUS and IBM Research) that played a key role in the success of SemTab. This work was also supported by the AIDA project (Alan Turing Institute), the SIRIUS Centre for Scalable Data Access (Research Council of Norway), Samsung Research UK, Siemens AG, and the EPSRC projects AnaLOG, OASIS and UK FIRES.