=Paper=
{{Paper
|id=Vol-3063/om2021_Tpaper4
|storemode=property
|title=A hybrid approach for large knowledge graphs matching
|pdfUrl=https://ceur-ws.org/Vol-3063/om2021_LTpaper4.pdf
|volume=Vol-3063
|authors=Omaima Fallatah,Ziqi Zhang,Frank Hopfgartner
|dblpUrl=https://dblp.org/rec/conf/semweb/FallatahZH21
}}
==A hybrid approach for large knowledge graphs matching==
https://ceur-ws.org/Vol-3063/om2021_LTpaper4.pdf
A Hybrid Approach for Large Knowledge
Graphs Matching
Omaima Fallatah1,2 , Ziqi Zhang1 , and Frank Hopfgartner1
1
Information School, The University of Sheffield, Sheffield, UK
{oafallatah1,ziqi.zhang,f.hopfgartner}@sheffield.ac.uk
2
Department of Information Systems, Umm Al Qura University, Saudi Arabia
oafallatah@uqu.edu.sa
Abstract. Matching large and heterogeneous Knowledge Graphs (KGs)
has been a challenge in the Semantic Web research community. This
work highlights a number of limitations with current matching methods,
such as: (1) they are highly dependent on string-based similarity mea-
sures, and (2) they are primarily built to handle well-formed ontologies.
These features make them unsuitable for large, (semi-) automatically
constructed KGs with hundreds of classes and millions of instances. Such
KGs share a remarkable number of complementary facts, often described
using different vocabulary. Inspired by the role of instances in large-scale
KGs, we propose a hybrid matching approach. Our method composes
an instance-based matcher that casts the schema matching process as
a two-way text classification task by exploiting instances of KG classes,
and a string-based matcher. Our method is domain-independent and is
able to handle KG classes with unbalanced population. Our evaluation
on a real-world KG dataset shows that our method obtains the highest
recall and F1 over all OAEI 2020 participants.
Keywords: Knowledge Graphs · Machine Learning · Schema Matching.
1 Introduction
In recent years, many public Knowledge Graphs (KGs) have been developed
and shared, e.g., DBpedia [1] and NELL [2]. Common KGs are often domain-
independent and semi-automatically constructed. Common KGs are highly com-
plementary, therefore, they are often integrated in several web applications such
as reasoning and query answering.
KGs have gained more attention in the Semantic Web, which facilitates shar-
ing and reusing knowledge such as those annotated in ontologies. Similar to on-
tologies, KG entities are highly heterogeneous, since many real-word entities can
be described using different vocabulary. Nevertheless, while ontologies primar-
ily focus on modelling the schema of a specific domain, cross-domain KGs are
known for describing numerous instances. Due to their nature of being largely
Copyright © 2021 for this paper by its authors. Use permitted under Creative Com-
mons License Attribution 4.0 International (CC BY 4.0).
2 O. Fallatah et al.
generated in a semi-automated manner, KGs are less well-formed compared to
manually created and well-designed ontologies.
The problem of ontology matching has been well studied, and matching sys-
tems are annually evaluated through the Ontology Alignment Evaluation Initia-
tive (OAEI 3 ). A new track for matching KGs has been introduced to OAEI in
2018, where ontology matchers are evaluated on the tasks of matching classes,
properties and instances. By design, KGs are known for their large number of
instances (ABox). Therefore, the majority of current matchers focus on match-
ing their instances. However, recent studies have shown that the problem of
matching KGs schema (TBox) remains a challenging task [4]. Moreover, many
KG matchers exploit class matches to generate and refine instance matches [9].
With current matching solutions mainly focusing on well-formed ontologies,
the problem of matching automatically curated and large KGs remains signifi-
cant. While the majority of the state-of-the-art methods are highly dependent
on string/language and structural-based techniques [27], KGs often lack some
textual descriptions, e.g., comments, required by these methods. In terms of
structural-based similarity measures, despite that some KGs lack the schematic
information required by such methods, they can be error-prone [19]. This justifies
their high-level of dependency on string-based matchers’ results.
This work proposes a novel method for mapping classes in large KGs by
combining string-based measures with an instance-based method. The latter
only uses annotated instance names to generate similar class pairs. Our domain-
independent method utilizes the large number of instances in KGs, and is able
to cope with unbalanced population of KG classes. This is particularly useful
in scenarios of large KGs with rich populated instances, such as DBpedia that
is the central linking dataset in the current linked data cloud, and NELL that
creates a large-scale KGs in a never-ending machine reading fashion. In addition
to OAEI KG benchmark, we conduct an experiment to evaluate the performance
of our method on a real-world KG benchmark [4]. We compare the results of our
proposed approach against the systems participated in the KG track in OAEI
2020 and show that our method obtains the highest recall and F1 measure in
the task of matching common KGs classes.
The remainder of this paper is structured as follows. An overview of the
related work is provided in Section 2; Section 3 describes the details of the
proposed matcher; Section 4 describes our experiments and 5 discuses the results,
followed by a conclusion and future work discussion in Section 6.
2 Related Work
Ontology matching systems often combine different matching techniques [21].
Element-level matchers discover similar entities by utilizing the textual annota-
tions defined in the ontology’s entities, e.g., URIs, labels, and comments. Other
methods leverage lexical databases, such as WordNet4 , as background knowledge
3
http://oaei.ontologymatching.org/
4
https://wordnet.princeton.edu/
A Hybrid Approach for Large Knowledge Graphs Matching 3
to discover semantic similarity. However, recent studies, such as [20], highlights
that WordNet lacks sufficient coverage in comparison to word embedding based
similarity measures. It is difficult for semantic-based techniques to outperform
string-based ones, therefore, combining both measures is a common strategy [9].
Matchers such as the well-known AML [5] and LogMap [13] employ element-
level techniques. Both matchers make use of background knowledge bases in
order to match biomedical ontologies. Some recent OAEI KG track participants
have been utilizing other resources. For example, Wiktionary [24], which is an
element level-matcher that uses an online lexical resource known as Wiktionary.
Similarly, ALOD2Vec [23] utilizes WebIsALOD5 , an automatically generated
RDF dataset of hypernym relations, as background knowledge.
In terms of Structural-level matchers, they exploit structural information
available in well-formed ontologies like disjoint axioms to refine element-level
alignments, such as in AML and LogMap matcher family. ATBox [9] is an OAEI
2020 participant which uses similar techniques to filter mappings initially dis-
covered by a string-based matcher. Such an approach requires a well-formed
ontology which is not the case in the context of common KGs that lack the
schematic richness due to their automatically generated nature [4].
The final matcher category is Extensional or instance-based matchers that
use instances data to generate schema level alignments. The intuition of such a
method is that similar classes or properties shares a substantial overlap of their
instances. However, according to [12], it is significantly challenging to measure
the extension of such an overlap. Previous works that incorporate this method are
predominantly domain-dependent [26]. Zhang et al.[27] introduced an instance-
based approach that only matches the properties within single LOD datasets,
which include some KGs such as DBpedia. Their results are encouraging to
apply instance-based methods on cross-dataset settings, particularly with LOD
datasets and KGs sharing many similar characteristics.
3 Approach
3.1 Overview
Our matching approach can be formalized as: the Input takes two KGs, O
and O0 , where O contains a set of classes O = {CO 0
, CO1 i
, .., CO }, and each class
contains a set of instances COi
= {ei0 , ei1 , ..., ein }. Similarly, O0 contains a set
j j j j
of classes such that O0 = {CO0 1 j
0 , CO 0 , .., CO 0 } where CO 0 = {e0 , e1 , ..., em }. Our
method is composed of an instance-based matcher and a name matcher.
The architecture of the proposed method is illustrated in Figure 1.
The workflow starts with parsing the two input KGs and applying general
text preprocessing (Section 3.2). The second component of the method is the
matching process which starts with an instance-based matcher (Section 3.3).
This matcher is performed in two stages and will result in two directional align-
ment sets, denoted AO→O0 which is a set of correspondences between classes
5
http://webisa.webdatacommons.org/
4 O. Fallatah et al.
Fig. 1: The architecture of the proposed matcher
from O and O0 respectively, and AO0 →O which is a set of correspondences in
the opposite direction. Section 3.3 explains the process of aggregating the two
directional alignments in order to obtain one alignment set for this matcher
Ainstance . The second matcher is based on string/semantic similarity, which be-
longs to the element-level matchers category. This matcher uses class labels to
generate equivalent class pairs denoted as Aname (Section 3.3). The process of
selecting the final alignments A is described in Section 3.4 (Post Processing
in fig. 1).
3.2 Pre-processing
The matcher starts by parsing the two input KGs in order to separately index
their lexical data structure. Given a KG, we create an index of its classes by
following the standard free text indexing approach for search engines. Here, each
class is treated as a document, and the text content of that ‘document‘ is the
concatenation of the labels of all the class’s instance names. In order to obtain
a cleaner version of the datasets, standard text preprocessing techniques are
applied. All entity labels are transferred into lowercase and all stopwords and
non-alphanumeric characters are removed. Finally, we replace all underscores
characters, which are often used to separate multi-word entity labels, with a
space character. KGs class names can also be described with multiple words,
such as placeofworship or by using a camel case (e.g.,ReligiousBuilding).
Therefore, a word segmentation process which utilizes a dictionary is applied to
infer the spaces between words while the camel case is replaced with a white
space as well.
3.3 The matching process
Instance-based Matcher . This matcher uses a self-supervised approach to
map KG classes based on their shared instances. The matching process is divided
into a two-way classification process where a KG classifier is trained with one
KG’s instances, and then used to classify a given instance into one of the classes
from that KG. As illustrated in Figure 1, this matcher starts by applying an exact
A Hybrid Approach for Large Knowledge Graphs Matching 5
name filter then undersamples the datasets from the two KGs in preparation for
the training phase. After the training process, the classifier trained on O (i.e.,
CLSO ) is used to classify instances from O0 . The classification results are then
used to elicit the directional alignments AO→O0 . The second alignment election
process is similar to the first one, except that the two KGs roles are reversed to
generate AO0 →O . Finally, the candidate class pairs for this matcher is generated
based on the two directional alignments.
Fig. 2: The distribution of instances across classes in two common KGs
Exact Name Filtering. We start by filtering classes in both KGs with exact
names. Therefore, if a class label exists in both input KGs, both classes will be
excluded from the instance based matching process. Our goal here is to use the
instance-based matcher to leverage the final alignments with class pairs that are
likely to not be discovered by simple string matchers. Further, this also serves
as a blocking step which reduces the search space for the matcher, as large KGs
can have hundreds of classes to be matched.
Undersampling. Typical large KGs are often very imbalanced. For example,
Figure 2 shows the distribution of classes in two common KGs, that we will dis-
cuss later in Section 4, i.e., DBpedia6 and NELL7 . While some classes in NELL
have over 20,000 instances, other classes have less than 10 instances. This imbal-
ance problem can detrimentally affect the learning process and therefore, a sam-
pling process can be useful. The problem of learning from imbalanced datasets
has been a thoroughly studied field where different solutions have been devel-
oped and analyzed [22]. Solutions often require targeting the majority classes,
i.e, classes with large number of data points, and the minority classes, i.e, classes
with few data points [14].
While random undersampling/oversampling [15] are common practices in ma-
chine learning applications, they still carry some major limitations. Randomly
undersampling the majority classes can result in losing relevant information in
the eliminated samples. In contrast, random oversampling, which generates du-
6
https://wiki.dbpedia.org/develop/datasets/dbpedia-version-2016-10,visited
on 14-2-2020
7
http://rtw.ml.cmu.edu/rtw/resources, iteration number 1115, visited on 22-2-
2020
6 O. Fallatah et al.
plicate data points in minority classes, can ultimately result in model overfitting
as a result of having the multiple samples of same data points. According to [6],
datasets with severe class imbalance can be very challenging to train machine
learning models and will often require specialized resampling techniques as op-
posed to generalized solutions.
Our sampling strategy aims to balance KGs instance population by under-
sampling the majority classes. Here, we define a Majority class as a class with a
number of instances which exceeds the average number of instances per class in
that particular KG. Due to their automated generation process, large KGs often
suffer from redundant information in their instances [8]. Our goal is to obtain
a smaller yet an indicative instance samples in order to limit the effect of the
sparsity problem on the training/learning process.
To help identify a set of indicative instance names, we deploy a TF-IDF [25]
based method to resample KG instances. TF-IDF is widely used to evaluate the
relevance of words to a collection of documents by weighting their occurrences.
In this task, we calculate TF-IDF of tokens for each class. Hence, the weight of a
token here represents how relevant a token is to a particular class in comparison
to other classes in the KG. Consequently, for each majority class, we use the top k
words in terms of TF-IDF score to undersample its instance names. To illustrate,
i
assuming that CO is a majority class in O, and W = {w0 , w1 , w2 ..wk−1 }, is a
i
list of tokens with the top k TF-IDF score in CO . Then, we discard instance
names that do not contain one of the words in W . As a result, we have a set of
indicative instance names that will be used to train CLSO . The same process
will be applied to the classes in the other input KG, i.e., O0 to train CLSO0 .
Training KG classifiers. A KG classifier CLSO will be trained using the pre-
viously undersampled data. We utilize pre-trained word embeddings as features.
Word Embeddings (WE) are considered one of the most effective approaches to
capture the semantic similarity of words, unlike traditional feature representa-
tion methods. As classification method we experiment with two SoA methods
(1) a model that uses pre-trained BERT model [16], and (3) a Deep Neural
Network (DNN). The architecture of this model is inspired by previous high
performing models in different NLP tasks such as in [17] and [3].
KG1 alignment elicitation is the process of eliciting candidate directional
alignment between a class in one KG (i.e., O) and classes in another KG (e.g.,
j
O0 ), based on the output of CLSO . To perform the elicitation given CO 0 =
j j j j
{e0 , e1 , ..., em } we apply CLSO to classify each instance of CO0 . As a result,
we classify each instance name into a class in O. A correspondence between
j i
CO 0 and CO is added to the candidate alignments set AO→O 0 if the majority of
j i
CO0 instances were classified as instances of CO . To generate a similarity value
between [0,1], we use the percentage of instances that voted for the majority
j i
class. For example, if 700 out of 1000 instances in CO 0 were classified as, CO the
similarity measure of that pair will be 0.7.
KG2 alignment elicitation. Similar to KG1 alignment elicitation, we reverse
the roles of the two KGs to generate AO0 →O . Candidate alignments are generated
based on the output of the classifier trained on O0 , i.e., CLSO0 .
A Hybrid Approach for Large Knowledge Graphs Matching 7
Similarity computing. This phase of the matcher aims to (1) combine the
two directional alignments resulted from the two KGs alignment elections (i.e,
AO→O0 and AO0 →O ), and (2) select the final alignments for the instance-based
matcher. In order to combine both alignments sets, each directional alignment
will be first stored into an alignment matrix of a dimension of |O|.|O0 |. An
alignment matrix contains the correspondences for all class pairs from the two
input KGs. To aggregate the two matrices, we take the average of the similarity
4 5 5 4
value of each pair. For example, if (CO ,CO 0 ,0.69) in AO→O 0 and (CO 0 ,CO , 0.73)
in AO0 →O ) their aggregated similarity value will be 0.71.
Consequently, the final alignment Ainstance is generated by following the au-
tomated alignment selection approach introduced in [7]. Given an alignment
matrix and a threshold t, this method goes through each row at a time and
selects the maximum correspondence in each row, if the similarity value for that
correspondence is beyond t (e.g., bold text in fig. 3). When a class is involved
3
in two correspondences (e.g., CO 0 in row 1 and 7), only the one with the higher
6 3
similarity is retained (e.g., (CO , CO 0 , 0.92)), and the previously selected corre-
spondence is deleted. This process takes places iteratively until all classes are
selected with a correspondence and no changes are to be made.
Fig. 3: An example of calculating the final alignment using the method in [7]
Name matcher . This matcher calculates the similarity of KG classes based
on the string and the semantic similarity of their names. Given a set of all pos-
sible correspondences between classes in O and O0 , generated with an exclusive
pairwise comparison, we measure the word embedding similarity and the edit
distance similarity of the two class names. We only apply the two similarity
measures to KGs class names, as not all KGs provides other longer descriptions
such as comments. For the edit-distance similarity, we calculate the normalized
levenshtein distance for each class pairs. This method normalizes the edit dis-
tance value by the length of the longer string to get a value between [0.0, 1.0].
In terms of the word embedding similarity, a Google pre-trained word2vec
model is used to represent class names and measure their cosine similarities in the
Vector Space Model where semantically similar words are represented closer to
each other. Following the same approach in Section 3.2, concatenated strings such
as awardtrophytournament are segmented into multiple words. Thus, in the case
8 O. Fallatah et al.
of a multi-word class name, the matcher aggregates the vector representation of
each word composing the class name by taking an element-wise average of the
vectors of each composing word.
We then choose the maximum of the two similarity measures, if the similarity
scores are higher than a threshold tn . To illustrate, assuming that a pair of the
two classes RailwayStation and TrainStation where their word embedding
similarity is 0.83 and their edit distance is 0.56 we select the maximum simi-
larity value, i.e., the word embedding similarity which is also higher than the
tn . However, if the two similarity scores of a pair are lower than the tn , then
that pair will not be added to the candidate alignment set. The output of this
matcher is a candidate alignment set Aname to be combined with the instance
matcher alignments (i.e., Ainstance ) to be detailed in the next section.
3.4 Final alignment selection
The above-explained similarity measures will both result in an alignment set. The
goal of this final stage of the matching approach is to combine the two match-
ers results, as well as to select the final alignments for the complete matcher.
Given Aname and Ainstance , we follow the same method as explained before in
Section 3.3 for creating Ainstance out of the two directional alignment.
4 Experiments
The aim of this experiment is to test our matching approach on the task of
matching large KGs and to compare it to OAEI participants. In addition to
OAEI participants, we also tested the performance of the baseline matcher KG-
baselineLabel8 . Similar to OAEI, we use precision, recall, F-measure metrics to
evaluate the accuracy of the matching methods against the gold standard. The
Matching Evaluation Toolkit (MELT) [11] was used to perform this evaluation.
Our matcher is implemented with python and was wrapped using MELT exter-
nal matchers wrapping tool. The evaluation was executed on a VM with 128GB
of RAM, 16 vCPUs (2.4 GHz), and 12GB GPU.
4.1 Datasets
The datasets used for evaluation are: (1) The NELL-DBpedia dataset is cre-
ated from the schema of two large public KGs [4]. The gold standard consists of
129 true positive class alignments between NELL and DBpedia. To the best of
our knowledge, this dataset is the largest available benchmark for matching KGs
classes. This dataset is domain-independent and offers a substantial number of
instances, which allow for evaluating instance-based matchers. (2) The OAEI
Knowledge Graphs Benchmark, which offers five test cases generated from
eight different KGs. The largest test cases in terms of class matching have 15
and 14 positive class alignments [10]. Similar to OAEI, in this work, we evaluate
and share the average results of the five tasks.
8
http://oaei.ontologymatching.org/2020/results/knowledgegraph/index.html
A Hybrid Approach for Large Knowledge Graphs Matching 9
4.2 Method’s parameters
Our method has the following parameters: (1) k which is the number of words
ranked by their TF-IDF scores in a class to be used for the undersampling
process. In the reported results, we set k to 10. However, we have created three
different configurations based on k = 5, 10, 20. We discuss the impact of k value
in section 5.1. (2) For the threshold value of the name matcher tn , we set tn to
0.8 which is inline with previous element-level methods that combines multiple
similarity measures such as [9,20]. (3) In the final alignment process, we apply
a threshold (t) of 0.22, the value recommended for this method in [7].
5 Results and discussion
As shown Table 1, on the NELL-DBpedia dataset, one can observe that our
matcher achieves a recall and F1 that are clearly higher than all OAEI 2020
participants. Here, we report the highest result out of the three k value configu-
rations, i.e., 10. However, our matcher outperforms all OAEI matchers, despite
what k value we use. AML is the best performing matcher on the task of match-
ing classes in OAEI KG track. However, on the task of matching common KG
classes, our matcher outperforms AML with 0.06 in F1, and 0.12 in recall. In
terms of OAEI KG dataset, our matcher does not perform as well, i.e., the F1
is 0.77. We argue that the systems that outperform our matcher on this task im-
plement a variety of matchers that target not only the labels of KG entities, but
also consider other entities’ metadata. For example, AML incorporates 9 differ-
ent matchers including structural matchers, and other filters to further improve
the quality of the matching results.
Table 1: Performance of the OAEI 2020 participants and the proposed approach
on two benchmarks. The best results on each dataset are marked in bold.
Matcher NELL-DBpedia OAEI benchmark
P R F1 P R F1
ALOD2Vec [23] 1.00 0.79 0.88 1.00 0.67 0.8
AML [5] 1.00 0.8 0.89 0.98 0.81 0.89
ATBox [9] 1.00 0.79 0.88 0.97 0.79 0.87
LogMap [13] 0.99 0.79 0.88 0.95 0.76 0.84
LogMapBio 0.99 0.79 0.88 0.95 0.76 0.84
LogMapKG 0.99 0.79 0.88 0.95 0.76 0.84
LogMapLt 1.00 0.59 0.74 0.8 0.43 0.56
Wiktionary [24] 1.00 0.79 0.88 1.00 0.67 0.8
DESKMatcher [18] 0.0 0.0 0.0 0.76 0.66 0.71
KGbaselineLabel 1.0 0.61 0.76 1.00 0.59 0.74
proposed matcher 0.98 0.92 0.95 0.88 0.73 0.77
Other systems also utilize external background knowledge resources such
as ALOD2Vec and Wiktionary. However, our matcher only uses the labels of
entities to produce matching class alignments. Moreover, the majority of OAEI
10 O. Fallatah et al.
systems incorporate multiple string-based techniques such as n-gram, prefixes
and suffixes. For instance, one of the string processes implemented by ATBox
matcher aimed at finding ontology specific stopwords, which are words that often
appear in a certain KG or ontology. It considers such words as stopwords to be
removed prior to applying further string matching techniques. This allows to
discover the similarity of hsidebar starship, starshipi and hsidebar novel,
noveli, if sidebar was a corpus specific stopword. However, those classes will
not be matched by word embedding models or by an edit-distance similarity
with a high threshold such as the one we apply, i.e., 0.8.
Another contribution to the different performance of our matcher across the
two datasets is the datasets’ nature. While the first dataset is constructed from
common KGs where classes annotate complementary real-world entities, the
OAEI datasets have very different nature. Moreover, they are restricted to a
single domain (entertainment) where distinguishing entities of book, music and
movie can be a difficult task. This is due to the usage of words in naming such
entities, which can be very heterogeneous and inconsistent. To illustrate, fig. 4
shows the difference in the performance of our KG classifier when it is trained to
classify NELL instances, as opposed to when it is trained to classify the Memo-
ryAlpha KG instances in the OAEI benchmark dataset. Clearly, it was easier to
classify instances from multi-domain large KGs like NELL.
(a) (b)
Fig. 4: The instance classification report of a 20 randomly sampled classes from
the OAEI KG MemoryALpha in (a) and NELL in (b). Note that y-axis numbers
indicate class IDs.
5.1 The impact of components and parameters
In this section, we discuss the impact of several components and parameters of
the proposed method. First, in addition to the BERT-based classifier, we also
tested our method with another model, based on a simple DNN with 4 density
A Hybrid Approach for Large Knowledge Graphs Matching 11
connected layers. Although both models outperform all OAEI participants on
the task of matching common KGs, we use BERT model as it is shown to achieve
slightly better F1 (by 0.01). This shows that our approach is generalizable to
other machine learning algorithms for the KG classifier. Second, in terms of the
threshold k, we tested three configurations, 5, 10, and 20. We noticed identical
results with k=10 and 20, while slightly lower F1 when k=5, i.e., 0.94 which
still significantly outperforms all OAEI participants. Further, setting k to 10
compared to 20 led to a significant reduction in runtime due to more aggressive
undersampling (from 55 minutes on the common KGs dataset to 29 minutes).
Finally, the exact name filtering had a positive effect on the overall performance
of our method, by increasing the F1 by 0.02 on the common KGs dataset.
6 Conclusion and future work
This work reports an ongoing study of utilizing instances to match KGs classes.
We proposed a novel domain independent approach for matching classes in large
KGs. To the best of our knowledge, our matcher includes the first instance-
based matcher for matching KG classes with the ability to handle unbalanced
populations. Our findings suggest that a hybrid approach that composes an
instance-based matcher can be very effective for matching common KG classes.
In future versions of our matcher, we aim to further study the thresholds focusing
on the possibility to automate that decision, and to further improve our matcher
combination to be able to address different matching tasks. We also aim to extend
our matching method to match all KGs entities.
References
1. Bizer, C., Lehmann, J., Isele, R., Jakob, M., Jentzsch, A., Kontokostas, D., Mende,
P.N., Hellmann, S., Morsey, M., van Kleef, P., Auer, S., Bizer, C.: DBpedia – A
Large-scale, Multilingual Knowledge Base Extracted from Wikipedia. Semantic
Web pp. 1–5 (2012)
2. Carlson, A., Betteridge, J., Kisiel, B., Settles, B., Hruschka, E.R., Mitchell, T.M.:
Toward an architecture for never-ending language learning. In: Twenty-Fourth
AAAI Conference on AI (2010)
3. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., Kuksa, P.:
Natural language processing (almost) from scratch. Journal of machine learning
research 12(ARTICLE), 2493–2537 (2011)
4. Fallatah, O., Zhang, Z., Hopfgartner, F.: A gold standard dataset for large knowl-
edge graphs matching. In: Ontology Matching 2020: Proceedings of the 15th Inter-
national Workshop on Ontology Matching co-located with (ISWC 2020) (2020)
5. Faria, D., Pesquita, C., Santos, E., Palmonari, M., Cruz, I.F., Couto, F.M.: The
agreementmakerlight ontology matching system. In: OTM Confederated Interna-
tional Conferences. pp. 527–541. Springer (2013)
6. Fernández, A., Garcı́a, S., Galar, M., Prati, R.C., Krawczyk, B., Herrera, F.: Learn-
ing from imbalanced data sets, vol. 11. Springer (2018)
12 O. Fallatah et al.
7. Gulić, M., Vrdoljak, B., Banek, M.: Cromatcher: An ontology matching system
based on automated weighted aggregation and iterative final alignment. Journal of
Web Semantics 41, 50–71 (2016)
8. Hertling, S., Paulheim, H.: Dbkwik: A consolidated knowledge graph from thou-
sands of wikis. In: 2018 IEEE International Conference on Big Knowledge (ICBK).
pp. 17–24 (2018)
9. Hertling, S., Paulheim, H.: Atbox results for oaei 2020. In: CEUR Workshop Pro-
ceedings. vol. 2788, pp. 168–175. RWTH (2020)
10. Hertling, S., Paulheim, H.: The knowledge graph track at oaei. In: European Se-
mantic Web Conference. pp. 343–359. Springer (2020)
11. Hertling, S., Portisch, J., Paulheim, H.: MELT - matching evaluation toolkit. In:
Semantic Systems. The Power of AI and Knowledge Graphs - 15th International
Conference. pp. 231–245 (2019)
12. Isaac, A., Van Der Meij, L., Schlobach, S., Wang, S.: An empirical study of instance-
based ontology matching. In: The Semantic Web, pp. 253–266. Springer (2007)
13. Jiménez-Ruiz, E.: Logmap family participation in the oaei 2020. In: Proceedings
of (OM 2020). CEUR-WS (2020)
14. Ksieniewicz, P.: Undersampled majority class ensemble for highly imbalanced bi-
nary classification. In: Proceedings of the Second International Workshop on Learn-
ing with Imbalanced Domains: Theory and Applications. pp. 82–94 (2018)
15. Liu, A.C.: The effect of oversampling and undersampling on classifying imbalanced
text datasets. The University of Texas at Austin p. 67 (2004)
16. Maiya, A.S.: ktrain: A low-code library for augmented machine learning. arXiv
preprint arXiv:2004.10703 (2020)
17. Minaee, S., Kalchbrenner, N., Cambria, E., Nikzad, N., Chenaghlu, M., Gao, J.:
Deep learning based text classification: A comprehensive review. arXiv preprint
arXiv:2004.03705 (2020)
18. Monych, M., Portisch, J., Hladik, M., Paulheim, H.: Deskmatcher. In: CEUR Work-
shop Proceedings. vol. 2788, pp. 181–186 (2020)
19. Ngo, D., Bellahsene, Z., Todorov, K.: Opening the black box of ontology matching.
In: Extended Semantic Web Conference. pp. 16–30. Springer (2013)
20. Nkisi-Orji, I., Wiratunga, N., Massie, S., Hui, K.Y., Heaven, R.: Ontology align-
ment based on word embedding and random forest classification. In: Joint Euro-
pean Conference on Machine Learning and Knowledge Discovery in Databases. pp.
557–572. Springer (2018)
21. Otero-Cerdeira, L., Rodrı́guez-Martı́nez, F.J., Gómez-Rodrı́guez, A.: Ontology
matching: A literature review. Expert Systems with Applications pp. 949–971
(2015)
22. Padurariu, C., Breaban, M.E.: Dealing with data imbalance in text classification.
Procedia Computer Science 159, 736–745 (2019)
23. Portisch, J., Hladik, M., Paulheim, H.: Alod2vec matcher results for oaei 2020. In:
CEUR Workshop Proceedings. vol. 2788, pp. 147–153. RWTH (2020)
24. Portisch, J., Paulheim, H.: Wiktionary matcher results for oaei 2020. In: CEUR
Workshop Proceedings. vol. 2788, pp. 225–232. RWTH (2020)
25. Schütze, H., Manning, C.D., Raghavan, P.: Introduction to information retrieval,
vol. 39. Cambridge University Press Cambridge (2008)
26. Thor, A., Kirsten, T., Rahm, E.: Instance-based matching of hierarchical ontolo-
gies. Datenbanksysteme in Business, Technologie and Web (2007)
27. Zhang, Z., Gentile, A.L., Blomqvist, E., Augenstein, I., Ciravegna, F.: An unsuper-
vised data-driven method to discover equivalent relations in large Linked Datasets.
Semantic Web pp. 197–223 (2017)