The present research investigates the use of Naive Bayes classifiers to match knowledge graphs and tabular data, with particular emphasis on Column Type Annotation, Cell Entity Annotation, Column Property Annotation and Table Topic Detection. Using feature extraction techniques such as number of co-occurrences and term frequency, the study evaluates the efectiveness and performance of Naive Bayes classifiers on a variety of datasets. The proposed method is straightforward and generic, making a contribution to the field of knowledge graph matching and demonstrating the potential of Naive Bayes classifiers for the integration and interoperability of tabular data and knowledge graphs.
The exponential growth of digital information has made both structured and unstructured
data increasingly prevalent. Among structured data, tabular datasets play a crucial role in
organizing and presenting information in a structured format across various domains such as
digital libraries [
Naive Bayes classifiers are proven to be efective in various classification tasks due to their
simplicity, eficiency, and robustness [
The rest of this paper is organized as follow: Section 2 presents some related work on table annotations, Section 3 presents Naive Bayes classifier, Section 4 gives an overview of the research methodology, Section 5 presents how we processed to solve the diferent tasks of the challenge, Section 6 presents the results and finally, Section 7 conclude the paper.
The SemTab Challenge is an annual competition that evaluates table annotation systems. It
requires understanding the semantics of tabular data and knowledge graphs. Its previous
editions introduced three tasks: CTA Task, which assigns a semantic type from a KG to a
table column; CEA Task, which matches a cell of a given table to a KG entity; and CPA Task,
which assigns a KG property to the relationship between two columns. These tasks have been
addressed by diferent systems using various approaches, including:
• bbw (boosted by wiki) [
It improves matching by using multiple services like DBpedia Lookup, DBpedia
endpoint, Wikipedia, Wikidata, and a cross-lingual matching strategy, enhancing the overall
eficiency.
• DAGOBAH [
Despite the large number of annotation systems available, they are often complicated to implement. This year, we focus on implementing a highly flexible machine learning method, exploring Naive Bayes classification on table to KG matching problems. 3https://github.com/fokobrice3/STProbClass/tree/main/MNB_2023 4https://github.com/fokobrice3/STProbClass/blob/main/GUIDE.pdf
A Naive Bayes classifier is a simple probabilistic classification method based on Bayes’ theorem, which calculates the probability of a specific class based on observed features. For any occurrences of A and B, the Bayes’ theorem asserts the rule given by the equation 1. (|) = (|) * () () (1) • P(A|B): this is the probability of event A given that event B has occurred. • P(B|A): this is the likelihood of observing evidence B if the event A is true. • P(A): this is the initial belief or knowledge about the probability of A before considering any evidence. • P(B): represents the overall probability of observing B, regardless of the occurrence of event A.
Bayes’ theorem is useful for inferring causes from their efects, as it simplifies determining
the likelihood of an efect based on its presence or absence [
This section describes the research methodology employed in this work. We developed the
research methodology by relying on what we know in empirical research in software engineering
[
The research question: "How to use Naive Bayes classifiers to match tabular datasets to knowledge graph?" was used as the guideline of this work. To reply to this question, one should provide a system that accepts a tabular dataset and a set of labels (classes or properties from a knowledge graph) as inputs, and produces the annotated dataset with these labels. To this end, the following questions should be replied: • How can a column of tabular data be classified using a knowledge graph class? This task is known as CTA and the fundamental query is "Which features must be used to classify a column with a label?" • How can data from a tabular data cell be classified using knowledge graph entities? The CEA is presented here. The fundamental query is "Which features must we use to classify a cell with a label?" • How can a relationship between two columns of tabular data be classified using knowledge graph property? This task is known as CPA. The fundamental query is "Which features must we use to classify a relationship with a label?" • How can knowledge graph class be used to classify the topic of tabular data ? This is the TTD. The fundamental query is "Which features must we use to identify a topic with a label?"
The research methodology combines case study research, action research, and experimental
research, three empirical research techniques used in software engineering [
To enable the proposed solution to be applied in any situation, it is important to gain a deeper understanding of the tabular data used in the knowledge graph matching problem through the study of these case studies. We used the Scrum process, we ran, and improved our system using the Sprint, iterative and incremental practices.
The exploration of the use of Naive Bayes classifiers to solve the SemTab challenges tasks allowed us to come up with a generic pipeline presented by Fig 1. This pipeline involves data pre-processing, feature extraction, classification of known labels and prediction of new labels components. In the following paragraphs, we present the diferent components of this pipeline and the implementation of the system.
The first step in our approach is data pre-processing, which prepares the raw datasets for training and matching. For each dataset (test and train), the pre-processing steps consist of:
2. Set characters to lowercase, 3. Remove stopping word,
Once processed, the dataset contains cleaned data that can be used for feature extraction. The training datasets were provided by the SemTab organizers and are presented in the table 1:
5.2.1. Column Type Annotation For the CTA task, features are extracted from the table columns, including column headers when taken into account, and column descriptions. During the training phase, these features will be connected to the annotation (class/label) that was taken from the training data as presented by Fig. 2.
• Column Headers: names of the columns, • Column Descriptions: a bag of words/tokens related to the column content. 5.2.2. Cell Entity Annotation Concerning the CEA task, features are extracted from both the tabular data and the knowledge graph to capture the relevant information for aligning each cell with the appropriate entity. These features include cell contents, entity labels and contextual information that provide additional context for matching as presented by Fig. 3. 5.2.3. Column Property Annotation For the CPA task, features are extracted to capture the relationships or properties between two columns in the tabular data. These features provide insights into the associations or connections that exist between the columns. Examples of features for CPA (presented by Fig. 4) may include: • Co-occurrence count: which is the number of times specific values or combinations of values appear together in the two columns. This gives us information about the likelihood or frequency of values appearing in both columns at the same time. Only the tokens with a co-occurrence >= 1 are taken into account. • Relationship: relationships are obtained by linking pairs of tokens together. This consists of combining tokens in the diferent cells of the target columns in the training dataset for the CPA task. 5.2.4. Table Topic Detection Concerning the TTD task, features related to the overall content and structure of the table are extracted. They help determine the primary topic or subject matter of the table. Examples of features for TTD (presented by Fig. 5) may include: • Table Headers: the names of the columns provide valuable information about the table’s content. These headers can be extracted as textual features that contribute to the classification of the table topic when they are available. • Key Terms: relevant tokens from the table content can serve as features for the TTD task. These terms are extracted using Term Frequency (TF) algorithms 8.
This Section presents how we implemented the Naives Bayes classification for each task using the label from the KG inside the train datasets and the extracted features. 5.3.1. Goal The objective is to build the learning function ()− > • is one of the classes/labels in the training dataset(e.g Number, Boolean, Food, Person,
Hotel, P18, P651, Q625, Q31, Q54389 etc.). • = (1, 2, ..., ) is the representation of the data that comes in a bag of tokens/words.
Given that we have a set of classes/labels 1, 2, ..., , and we want to determine the probability of an instance to belong to each class/label. The Bayesian probability of a class/label is obtained using the equation 2.
(|) = (|) * () () (2) • (|): this is the conditional probability to be calculated. • (|): this is the likelihood of the features of under the assumption that it belongs to class/label , (|) = (1|) * (2|) * ... * (|) • (): represents the initial belief or knowledge about the probability of an instance belonging to class/label before considering any evidence. • (): represents the overall probability of observing instance , regardless of its class/label.
To classify an instance , we compute (|) for each class and select the class with the highest probability. The implementation is tailored to the specific requirements of each dataset. The following paragraphs give an overview. 5.3.2. WikidataTables Dataset This dataset consists of tables in CSV format. The CTA, CEA, and CPA targets have to be classified using Wikidata’s classes and properties. Three Naive Bayes classifiers were trained for each task, using the labeled dataset on train. The labels, provided by this dataset, represent the ground truth annotations obtained from the Wikidata KG. The entity, semantic type, and relationship predictions for new, unseen tabular data are then predicted using the three trained classifiers. 5.3.3. tFood Dataset The tFood dataset consists of tables in CSV format. There, the Wikidata class and properties have to be used to classify CTA, CEA, CPA, and TTD targets. Using the same process on WikidataTables dataset, four Naive Bayes classifiers are trained using the labeled dataset of the tFood training data. The first column of a table is ignored as it lacks relevant information for the TTD task (e.g., prop0, prop1, prop2, etc.). Since no corpus is provided for each table, we also adjust the TF-IDF to term frequency. Due to time-consuming training, a limit of 50 cells from each CSV file are randomly extracted per table during training. 5.3.4. SOTAB Dataset The SOTAB dataset tables are provided in GZ-compressed JSON files. The task was to classify CTA and CPA with schema.org and DBpedia classes. Using the labeled dataset of the SOTAB training data, we trained six classifiers, including two in round 1 and four in round 2. We also limited the number of data elements that could be retrieved from each JSON file to 20 by file at random for each task because many JSON files contained a lot of data, which made the training too time-consuming. 5.3.5. Development environment The development environment was composed of VSCode as code editor, Node.js as the JavaScript runtime, and npm as the package manager. For feature extraction and data preprocessing, we implement the diferent utilities from scratch and we consider the Natural 9 npm package for Porter Stemming and building classifiers. We used a desktop with a Ryzen 1700 8 core processor and 16Gb RAM. We also consider multi-instance activity as performance optimization techniques to enhance the eficiency of the implementation.
The SemTab 2023 challenge consisted of two (02) rounds lasting from April 14 to June 22, 2023. The pipeline presented by Fig. 1 was applied to each dataset. The following paragraphs present the results provided by the SemTab organizers for the diferent datasets and a short discussion.
WikidataTables dataset was provided in Round 1 of the challenge. The test data are presented in Table 2 and the challenge’s CTA, CEA and CPA results on this dataset are shown in Fig. 6.
The tFood dataset was provided in Round 1 of the challenge. This dataset was divided into two main parts: tFood-horizontal and tFood-entity. The test data are presented in Table 3.
The tFood datasets revealed challenges in training and inference time, as well as the lack of a corpus to accurately extract the key terms for TTD task. In addition, we discovered that the table’s redundancies and unclear data were not appropriate to the approach presented in this paper. Unfortunately, to reduce training time issues, we took a subset in each table with a maximum of 50 rows per table. Typically, 6-8 hours were spent on training and 9-10 hours for inference per task. Fig. 7 presents the results of the challenge on these datasets.
The SOTAB dataset was provided in Rounds 1 and 2. An overview of the test data is presented in Table 4.
Due to the time required for training and inference, and the limited performance of our training environment, we just submitted to round 2 with this dataset.
The SOTAB dataset has a larger number of tables and rows in JSON format compared to other datasets. To reduce training and inference time, we significantly decreased the number of training tables , and the rows were reduced to 20 per table. We also point out that the redundancy and incorrect data in the tables were not helpful for the proposed approach. Each task took 10-12 hours for training and 8-9 hours for inference.
This paper outlines the method we propose for annotating tabular data with knowledge graph classes, entities, and properties using Naives Bayes multiclass classifiers. Data pre-processing aims to ensure the accuracy and integrity of the tabular data as well as to make the subsequent matching process easier. Feature extraction techniques such as the number of co-occurrences and the frequency of terms provide useful information to capture the semantic relationships between tabular data and knowledge graphs. Due to redundant and misleading data in the training dataset, the computation time, the approach was severely limited but we are exploring further solutions to improve our feature extraction and computation times.
We are grateful to SemTab organizers for having given us the opportunity to share this work with the community.