Embedding-Assisted Entity Resolution for Knowledge Graphs

Abstract

Entity Resolution (ER) is a main task for integrating di erent knowledge graphs in order to identify entities referring to the same real-world object. A promising approach is the use of graph embeddings for ER in order to determine the similarity of entities based on the similarity of their graph neighborhood. Previous work has shown that the use of graph embeddings alone is not su cient to achieve high ER quality. We therefore propose a more comprehensive ER approach for knowledge graphs called EAGER (E mbedding-Assisted Knowledge Graph E ntity Resolution) to exibly utilize both the similarity of graph embeddings and attribute values within a supervised machine learning approach and that can perform ER for multiple entity types at the same time. Furthermore, we comprehensively evaluate our approach on 19 benchmark datasets with di erently sized and structured knowledge graphs and use hypothesis tests to ensure statistical signi cance of our results. We also compare our approach with state-of-the-art ER solutions, where EAGER yields competitive results for shallow knowledge graphs but much better results for deeper knowledge graphs.

Keywords

• Knowledge Graph
• Knowledge Graph Embedding
• Entity Resolution

-

Knowledge Graphs (KGs) store real-world facts in machine-readable form. This is done by making statements about entities in triple form (entity; property; value). For example the triple (Get Out, director, Jordan Peele) tells us that the director of the movie "Get Out" is "Jordan Peele". Such structured information can be used for a variety of tasks such as recommender systems, question answering and semantic search. For many KG usage forms including question answering it is bene cial to integrate KGs from di erent sources. An integral part of this integration is entity resolution (ER), where the goal is to nd entities which refer to the same real-world object.

Existing ER systems mostly focus on matching entities of one speci c entity type (e.g. publication, movie, customer etc.) and assume matched schemata for this entity type. This proves challenging when trying to use these systems for ER in KGs typically consisting of many entity types with heterogeneous attribute (property) sets. See Figure 1 for an example showcasing the many di erent challenges of this task, such as heterogeneous date representations ("1979-02-21" vs. "21 Febuary 1979"), di ering URIs ("dbr:Jordan Peele" vs. "wd: Q3371986"), schemata ("dbo:birthdate" vs. "wdt:P569") and overall information contained.

DBpedia "Get Out"^^xsd:String dbo:Film

rdf:type rdfs:label

dbr:Get_Out dbo:starring

dbo:director dbr:Allison_Williams rdfs:label "Allison Williams (actress)"

rdfs:label dbo:birthDate "Jordan

Peele" 1979-0221^^xsd:date

Wikidata wd:Q11424 wdt:P31

rdfs:label wd:Q25136235 wdt:P57 wdt:P161

rdfs:label wdt:P569 "Jordan

Peele" 21 February 1979 "Get Out"^^xsd:String wd:Q510970 rdfs:label "Daniel

Kaluuya" dbr:Jordan_Peele wd:Q3371986

We observe there are entities of di erent types ( lm, director, actor) and di erent attributes with heterogeneous value representations (e.g., birth date values "1979-02-21" in DBpedia and "21 Febuary 1979" in Wikidata for two matching director entities). Moreover, we see that matching entities such as the movie "Get Out" have di erent URIs and di erently named edges referring to properties and related entities, e.g. rdf:type vs. wdt:P31. These aspects make a traditional schema (property) matching as a means to simplify ER very challenging so that entity resolution for KGs should ideally not depend on it. Given that URIs and property names may not show any similarity it becomes apparent that the graph structure and related entities should be utilized in the ER process, e.g., to consider the movie label and director to match movies.

A promising way to achieve this in a generic manner, applicable to virtually any entity type, is the use of graph embeddings. By encoding the entities of the KGs into a low-dimensional space such approaches alleviate the obstacles posed by the aforementioned KG heterogeneities. Capturing the topological and semantic relatedness of entities in a geometric embedding space enables the use of these embeddings as inputs for machine learning (ML) algorithms. The performance of graph embedding approaches for ER has been recently studied by Sun et. al [ 28 ]. However, as they point out, most approaches focus on re ning the embedding process, while ER mostly consists of nding the nearest neighbors in the embedding space. Hence, the use of graph embeddings has to be tailored to the ER task for good e ectiveness. We build on the ndings of [ 28 ] and investigate the usefulness of learned graph embeddings as input for ML classi ers for entity resolution. While there are di erent settings for KG integration, such as enhancing a given KG or KG fusion, we focus here on the simple ER setting, i.e., nding matching entities in two data sources. The resulting match mappings can then be used for applications such as question answering or as input for KG fusion.

In this paper, we propose and comprehensively evaluate the rst (to our knowledge) graph embedding supported ER system named EAGER: Embedding Assisted Knowledge Graph Entity Resolution. It uses both knowledge graph embeddings and attribute similarities as inputs for an ML classi er for generic entity resolution with several entity types. EAGER utilizes di erent kinds of graph embeddings, speci cally the ones that performed best in [ 28 ], as well as di erent ML classi ers. We comprehensively evaluate the match e ectiveness of EAGER with using graph embeddings and attribute similarities either alone or in combination for 19 datasets of varying size and structure. Some of these ER tasks with multiple entity types from the movie domain have been newly developed for this work. The evaluation also includes a comparison of EAGER with stateof the-art ER approaches, namely Magellan [ 16 ] and DeepMatcher [ 19 ]. All our results are analyzed using hypothesis tests to ensure statistical signi cance of our ndings.

We rst discuss related work followed by an overview of EAGER. Section 4 describes the used datasets including the new benchmarks from the movie domain. Our evaluation is presented in Section 5 and we conclude in Section 6. 2

Related Work

Entity resolution has attracted a signi cant amount of research, sometimes under di erent names such as record linkage [ 8,10 ], link discovery [ 29,26 ] or deduplication [ 25 ]. We focus on the discussion of the most related ER approaches. and refer to surveys and books such as [ 11,20,6 ] for a more thorough overview. Traditional ER approaches rely on learning distance- or similarity-based measures and then use a threshold or classi er to decide about whether two entities are the same. These classi ers can be unsupervised [ 22,23 ], supervised [ 26,14 ] or employ active learning [ 25,21 ]. For example the Magellan Framework [ 16 ] provides supervised ML classi ers and provides extensive guides for the entire ER process. Recently, deep learning has seen some success in certain settings. DeepER [ 9 ] and DeepMatcher [ 19 ] provide a variety of di erent architectures and among other aspects, such as attribute similarities, use word embeddings as inputs for these networks. Both frameworks have shown that especially for unstructured textual data deep learning can outperform existing frameworks.

Collective ER approaches try to overcome the limitations of the more conventional attribute-based methods. This paradigm uses the relationships between entities as additional information and in some cases even considers previous matching decisions in the neighborhood. Bhattacharya and Getoor [ 3 ] show that using the neighborhood of potential match candidates in addition to attribute-based similarity is especially useful for data with many ambiguous entities. SiGMa [ 17 ] uses an iterative graph propagation algorithm relying on relationship information as well as attribute-based similarity between graphs to integrate large-scale knowledge bases. Pershina et al. [ 24 ] propagate similarities using Personalized PageRank and are able to align industrial size knowledge graphs. Zhu et al. [ 32 ] reformulate entity resolution as multi-type graph summarization problem and use attribute-based similarity as well as structural similarity, i.e. connectivity patterns in the graph.

More recently the use of graph embeddings has been shown promising for the integration of KGs. An overview of currently relevant approaches that solely rely on embedding techniques can be found in [ 28 ], some of these techniques have been used in this work and will be discussed in more detail in Section 3.3. Knowledge graph embedding (KGE) models typically aim to capture the relationship structure of each entity in latent vector representations in order to be used for further downstream applications. For an overview of current knowledge graph embedding approaches we refer the reader to a recent survey from Ali et al. [ 1 ].

EAGER aims to combine the two generally separate ER approaches of graph embedding techniques and traditional attribute- based methods for the integration of KGs with multiple entity types without relying on additional schema matching or any structural assumptions about the entities. Our extensive evaluation for a large spectrum of KGs demonstrates the viability of the proposed approach. 3

Overview of EAGER

In this section we present an overview of the EAGER approach for ER in knowledge graphs and the speci c approaches and con gurations we will evaluate. We start with a formal de nition of the ER problem and an overview of the EAGER work ow. Subsequently we explain how we create the input vector for our ML classi ers and conclude with a discussion of the prediction step. KGs are constructed by triples in the form of (entity; property; value), where property can be either a attribute property or a relationship and value a literal or another entity, respectively. Therefore, a KG is a tuple KG = (E ; R; A; L; T ), where E is the set of entities, A the set of attribute properties, R the set of relationship properties, L the set of literals and T is the set of triples. We distinguish attribute triples TA and relationship triples TR, where TA : E A L are triples connecting entities and literals, e.g. (dbr:Jordan Peele, dbo: birthDate, "1979-02-21") and TR : E R E connect entities, e.g. (dbr: Get Out, dbo:director, dbr:Jordan Peele) as seen in Figure 1. Our goal is to nd a mapping between entities of two KGs. More formally, we aim to nd M = f(e1; e2) 2 E1 E2je1 e2g, where refers to the equivalence relation. Furthermore, we assume we are provided with a subset of the mapped entities MT M as training data, which is also sometimes referred to as seed alignment in the literature. The remaining chapter is dedicated to illustrate how our approach tackles entity resolution in heterogeneous KGs. A schematical overview can be found in Figure 2. Given two KGs KG1; KG2 and a set of initial matches MT we create a feature vector for each match (e1; e2) 2 MT to train a machine learning classi er. Additionally to the positive matches provided in MT we sample negative examples by sampling random pairs (e1; e2) 2= MT to create a balanced set of positive and negative examples. After the training step the classi er then acts as an oracle to answer speci c alignment queries, i.e. entity pairs, in order to make a prediction. In the following we present our approach in more detail. Since schemata across di erent KGs may di er wildly, creating a schema matching before ER in heterogeneous KGs is di cult and can introduce additional sources for error. Keeping the focus on the matching process, we chose to concatenate all attribute values of each entity into a single string and used 3 similarity measures for comparisons: Levenshtein, Generalized Jaccard with an Alphanumeric Tokenizer, which returns the longest strings of alphanumeric characters, The code for EAGER and our experiments can be found in https://github.com/ jonathanschuchart/eager and Trigrams with the Dice coe cient. The second part of the input vector consists of KGEs. Given that the focus of this study lies not on the creation of embeddings itself, our approach can take any entity embeddings that are embedded in the same space. Since most KG embedding frameworks are not specialized for ER, we use OpenEA which was developed by Sun et al. for their 2020 benchmark study[ 28 ]. It o ers a variety of embedding approaches and embeds entities into the same space. Speci cally, we chose three of the best approaches of said study, namely BootEA, MultiKE and RDGCN: BootEA Sun et al. in 2018 [ 27 ] based their approach on the TransE model and combined it with elaborate bootstrapping and negative sampling techniques to improve performance. TransE aims to nd an embedding function that minimizes jj (eh) + (r) (et)jj for any (eh; r; et) 2 TR. Bootstrapping is done by additionally sampling likely matching entities (resampled every few epochs based on the current model) in order to increase the e ective seed alignment size. Additionally, negative relationship tuples are sampled and resampled every few epochs based on the current model in order to improve the distinction between otherwise similar entities. Since TransE is an unsupervised model, Sun et al. proposed a new objective function which incorporates both the original objective function of TransE and the likelihood of two entities from di erent KGs matching.

MultiKE In order to also incorporate more than just relational information, Zhang et al. [ 31 ] proposed a exible model which combines di erent views on each entity. Here, the name attribute, relations and all remaining attributes are embedded separately, using pre-trained word2vec word embeddings [ 18 ] for names and a variation on TransE for relations. Attribute embeddings are obtained by training a convolutional neural network taking the attribute and attribute value as input. All three embedding vectors are then combined into a single uni ed embedding space. In this approach the two knowledge graphs are treated as one combined graph where entities from the seed alignment are treated as equal. RDGCN Di erent to the aforementioned approaches, Wu et al. [ 30 ] proposed a new technique using two constructed conventional graphs and the GCN model by Kipf and Welling with highways. Instead of learning embeddings for entities and relations within one graph, RDGCN constructs a primary entity graph and a dual relationship graph in order to alternate the optimization process between the two. That way, the relationship representations from the dual graph are used to optimize the entity representations from the primal graph and vice versa by applying a graph attention mechanism. As the actual neigborhood information of each entity is not fully exploited in this case, Wu et al. showed that feeding the resulting entity representations into a GCN can help signi cantly improve the overall embedding quality.

https://github.com/nju-websoft/OpenEA As the aim of our study is to investigate to what degree combining entity embeddings with attribute similarities is superior to using either on their own, we present three di erent variants of our approach, that only di er in the construction of their input vector: EAGERE contains solely the embeddings, EAGERA consists exclusively of the attribute similarities and nally EAGERAkE which is a concatenation of entity embeddings and attribute similarities. The respective input vector is given to a classi er along with the seed alignment. In the evaluation we achieved the best results using either a Multilayer Perceptron (MLP) [ 13 ] or Random Forest (RF) [ 4 ], but any classi er can be used. The trained classi er is presented with alignment queries, i.e. pairs of entities that it will have to classify as match or non-match. Choosing these pairs is a non-trivial question since exploring all possible pairs would lead to a quadratic number of alignment queries relative to the KG size, which is not scalable to large datasets. Traditionally, blocking strategies are used to reduce the number of pairs by a linear factor. Due to the heterogeneous nature of KGs new strategies for this problem have to be found. An alternative could be to use the embeddings to nd a number of nearest neighbors, which is a scalable solution since the triangle inequality in metric spaces can be exploited to reduce the number of comparisons for the neighborhood search. Finding a good solution for this problem is however out of scope for our study and in the experiments we therefore use the test data to create prediction pairs, sampling negative examples randomly as done in the training step. More on our experimental setup can be found in Section 5.1. 4

Datasets

To evaluate our approach we use multiple datasets that can generally be put into two categories: rich and shallow graph datasets. The rich graph datasets were presented in [ 28 ] and consist of samples from DBpedia (D), Wikidata (W) and Yago (Y). Given their origin in web-scale KGs they o er a wide range of relationships as well as entity types. The linking tasks include KG samples of di erent density, size, as well as covering cross-lingual settings (EN-DE & ENFR).

To investigate how the interplay of attribute similarities and graph embeddings fares in settings with less dense KGs we created a new benchmark dataset with multiple entity types. These KGs are taken from the movie domain, where the gold standard was hand-labeled for the ve entity types Person, Movie, TvSeries, Episode, Company. The movie datasets were created from three sources containing information about movies and tv series: IMDB, TheMovieDB https://www.imdb.com/ https://www.themoviedb.org/ and TheTVDB. We make the movie datasets publicly available for future research at https://github.com/ScaDS/MovieGraphBenchmark. More details on each of the datasets can be found in Table 1 and Table 2 respectively. KD-Y YDGB 12685 23557 3206,,269318 17312,,711164 15,000 2712 2900 6608,,907603 16351,,110501 15,000 15EN-DE EN 215 286 47,676 83,755 81,988 DE 131 194 50,419 156,150 15,000 169 171 84,867 96 116 92,632 186,335 15,000 EN-FR EFRN 221607 430048 4407,,836344 6773,,116271 15,000 119636 128291 9860,,311182 6686,,787999 15,000 D-W WDBD 246113 847943 225913,,790980 648571,,806101 100,000 233198 736208 568186,,240537 847687,,211093 100,000 KD-Y YDGB 23827 33789 249040,,158188 572439,,076827 100,000 23310 23767 587665,,524675 584575,,012661 100,000 100EN-DE DENE 318916 425512 333356,,325490 575126,,765105 100,000 312730 312869 662229,,538985 576903,,274170 100,000 EN-FR EFRN 430000 456169 320598,,620875 449276,,762792 100,000 327897 346648 654691,,930921 540331,,932729 100,000 5

Evaluation

We discuss our results on the presented datasets, starting with a description of the experiment setup, followed by the results on the shallow and rich graph datasets, with a focus on investigating whether the use of attribute similarities https://www.thetvdb.com/ in combination with knowledge graph embeddings is bene cial for the respective setting. Furthermore we compare our approach with state-of-the-art frameworks. 5.1

Setup For the evaluation we use a 5-fold cross validation with a 7-2-1 split in accordance with [ 28 ]: For each dataset pair the set of reference entity matches is divided into 70% testing, 20% training and 10% validation. For each split we sample negative examples to create an equal share of positive and negative examples. The entire process is repeated 5 times to create 5 di erent folds.

For the OpenEA datasets the graph embeddings were computed using the hyperparameters given by the study of [ 28 ]. For all other datasets the *-15K parameter sets were used. For the classi ers, mostly scikit-learn's default parameters were used, though Random Forest Classi er was used with 500 estimators and MLP used two hidden layers of size 200 and 20. Furthermore, MLP was trained using the Adam [ 15 ] optimizer with = 10 5. 5.2

Results

The results for all datasets are displayed in Table 3. In the bottom row we display the average rank of each combination of input variant, embedding approach and classi er which is a number between 1 and 14 (since there are 14 possible combinations), where 1 would mean this combination achieves the best result for each dataset.

For the movie datasets we can see that EAGERAkE with MultiKE performs best, especially with the MLP classi er. This suggests that even for datasets with relatively few relational information it can be bene cial to use knowledge graph embeddings. However, it seems very dependent on the way these embeddings are constructed in order for them to be useful with MultiKE being the only one of the three embedding approaches to explicitly incorporate attribute data. This also becomes apparent when comparing the results of EAGERE and EAGERA where MultiKE performs best for EAGERE but is still clearly outperformed by EAGERA.

imdb-tmdb imdb-tvdb tmdb-tvdb

Metric fm prec rec

Looking at the movie datasets in more detail as shown in Figure 3, we can see that there is a di erence in performance depending on the entity type. In most cases, EAGERAkE reaches an F-measure of over 90% for all entity types showing that the approach is generic and able to achieve good match quality for multiple heterogeneous entity types. Still there are some di erences between the entity types. TVShows and Films generally perform worse than TVEpisodes and Persons with especially the precision for Film standing out negatively. This is especially pronounced in the IMDB-TMDB and IMDB-TVDB datasets. This might be attributed to di erent sets of attributes between those datasets, e.g. as IMDB does not contain full-length descriptions of lms and tv shows whereas TMDB and TVDB do. Interestingly, Films/TVShows with very dissimilar titles due to di erent representations of non-English titles can be matched using the KGEs. For example the soviet drama "Defence Counsel Sedov" has the romanized title "Zashchitnik Sedov" in IMDB, while TMDB has either the translated "Defence Counsel Sedov" or the cyrillic "Защитник Седов". These entity pairs are correctly matched in the EAGERAkE variant.

Looking at the rich datasets it is again evident that EAGERAkE achieves the best results. Overall it can solve the diverse match tasks including for multilingual KGs and larger KGs very well with F-Measure values between 96% and 99% in most cases. As before MultiKE with the MLP classi er performs the best

E, RDGCN

E, BootEA

A k E, BootEA

A

A k E, MultiKE

E, MultiKE

A k E, RDGCN

out of all graph embedding approaches, which is due to the fact that it explicitly takes advantage of attribute information of each entity, as opposed to BootEA and RDGCN.

Comparing the performances between the datasets we see that on the variants with richer graph structure (V2) the results are better than on (V1) for the respective datasets. There is also a di erence when contrasting the di erent sizes of the datasets. While EAGERAkE with BootEA and MultiKE generally seem to achieve better results on the larger 100K datasets compared to their 15K counterparts, this is less true for RDGCN.

To properly compare the performance of the approaches across all approaches we used the statistical analysis presented by Demsar [ 7 ] and the Python package Autorank [ 12 ], which aims to simplify the use of the proposed methods by Demsar. The performance measurement for each dataset and classi er are our paired samples. Given that we have more than two datasets simply using hypothesis tests for all pairs, would result in a multiple testing problem, which means the probability of accidentally reporting a signi cant di erence would be highly increased. We therefore use the procedure recommended by Demsar: First we test if the average ranks of algorithms are signi cantly di erent using the Friedman test. If this is the case we perform a Nemenyi test to compare all classi ers and input combinations.

The null hypothesis of the Friedman test can be rejected (p = 6:572 10 25). A Nemenyi test is therefore performed and we present the critical distance diagram in Figure 4. The axis shows the average rank of the input/embedding combination. Groups that are connected are not signi cantly di erent at the signi cance level of 0.05, which is internally corrected to ensure that all results together ful ll this. Approaches that have a higher di erence in average rank than the critical distance (CD) are signi cantly di erent.

We can see that EAGERAkE with MultiKE signi cantly outperforms all other variants. This is evidence that the combination of attribute similarities and embeddings is preferable to using attribute similarities or embeddings on their own for the task of entity resolution in rich knowledge graphs, even if the embedding approaches already incorporate attribute information. We compare our approach to the state-of-the-art ER frameworks Magellan [ 16 ] and DeepMatcher [ 19 ]. Magellan is an ER framework that allows the use of ML classi ers for ER. We present the best performing classi ers XGBoost [ 5 ] and Random Forest (RF). DeepMatcher provides several deep learning solutions for ER, we employ the hybrid variant which uses a bidirectional recurrent neural network with a decomposable attention-based attribute summarization module. To avoid any decrease in performance due to blocking we provide both frameworks with respective training or test entity mappings directly. Because such a setup is not possible for the approaches discussed in [ 28 ], which mostly use resolution strategies based on nearest neighbors, we cannot fairly compare our approach with theirs and therefore refrain from this comparison here.

We start with the comparison for the shallow datasets. Since both Magellan and DeepMatcher expect matched schemata we align the attributes by hand where necessary. We report F-measure (fm), Precison (prec) and Recall (rec) averaged over the 5 folds. For the comparison with other approaches we use EAGERAkE with MultiKE and for brevity we will refer to it simply as EAGER. The results are shown in Table 4. All frameworks perform very well with almost all F-measure values over 0:95.

For all three movie datasets Magellan RF outperforms all other approaches in terms of F-measure.

5

Magellan RF

Magellan XGBoost

DeepMatcher

EAGER MLP

EAGER RF

For the rich graph datasets the heterogeneity of the di erent KGs was a problem for Magellan and DeepMatcher since they both expect perfectly matched schemata. This was manageable for the smaller datasets, were this can be done by hand. In order to use Magellan and Deepmatcher on the rich graph datasets we did the same as for EAGER and concatenated all entity attributes into a single attribute. We can see that EAGER using MLP outperforms all other approaches except on D-W (V1) and D-Y (V1) for the 100K sizes, where DeepMatcher performs best. Magellan is outperformed on all datasets by EAGER and DeepMatcher.

The Friedman test shows a signi cant di erence (p = 8:675 10 07). Looking at the critical distance diagram in Figure 5 we can see that EAGER MLP does not signi cantly outperform EAGER RF or DeepMatcher, but it is the only approach that signi cantly outperforms both Magellan approaches. While there is no signi cant di erence between EAGER MLP and DeepMatcher, EAGER does not depend on the provision of schema matching. 6

Conclusion & Future work

We explored the combination of knowledge graph embeddings and attribute similarities for entity resolution in knowledge graphs with multiple entity types. These approaches are included in a new learning-based ER system called EAGER. We tested our approach on a range of di erent datasets and showed that using a combination of both graph embeddings and attribute similarities generally yields the best results compared to just using either one. We showed that our approach yields competitive results that are on par with or signi cantly outperform state of the art approaches. The approach is generic and can deal with several entity types without prior schema matching.

Future work will investigate blocking strategies utilizing both embeddings and attribute information, as well as smarter attribute combination strategies (e.g. using property matching[ 2 ]). Unsupervised and active learning in this context should be explored to alleviate the di culty of obtaining training data. Acknowledgments. This work was supported by the German Federal Ministry of Education and Research (BMBF, 01/S18026A-F) by funding the competence center for Big Data and AI "ScaDS.AI Dresden/Leipzig". Some computations have been done with resources of Leipzig University Computing Center.

References

1. Ali , M. , Berrendorf , M. , Hoyt , C.T. , Vermue , L. , Galkin , M. , Sharifzadeh , S. , Fischer , A. , Tresp , V. , Lehmann , J.: Bringing Light Into the Dark: A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Uni ed Framework pp. 1 { 40 ( 2020 ), http://arxiv.org/abs/ 2006 .13365

2. Ayala , D. , Hernandez , I. , Ruiz , D. , Rahm , E.: Leapme: Learning-based property matching with embeddings . arXiv preprint arXiv:2010 . 01951 ( 2020 )

3. Bhattacharya , I. , Getoor , L. : Collective entity resolution in relational data . IEEE Data Eng. Bull. 29 , 4 { 12 ( 2006 )

4. Breiman , L. : Random forests . Machine Learning 45(1) , 5 { 32 ( 2001 ). https://doi.org/10.1023/A:1010933404324, http://dx.doi.org/10.1023/A% 3A1010933404324

5. Chen , T. , Guestrin , C. : XGBoost: A scalable tree boosting system . In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining . pp. 785 { 794 . KDD '16, ACM ( 2016 ). https://doi.org/10.1145/2939672.2939785, http://doi.acm. org/10 .1145/ 2939672.2939785

6. Christen , P. : Data Matching: Concepts and Techniques for Record Linkage , Entity Resolution, and Duplicate Detection. Springer Publishing Company, Incorporated ( 2012 )

7. Demsar , J.: Statistical comparisons of classi ers over multiple data sets . Journal of Machine Learning Research 7 , 1 { 30 ( 2006 )

8. Domingos , P.: Multi-relational record linkage . In: In Proceedings of the KDD-2004 Workshop on Multi-Relational Data Mining . pp. 31 { 48 ( 2004 )

9. Ebraheem , M. , Thirumuruganathan , S. , Joty , S.R. , Ouzzani , M. , Tang , N.: Deeper - deep entity resolution . CoRR abs/1710 .00597 ( 2017 ), http://arxiv.org/abs/ 1710.00597

10. Elfeky , M.G. , Elmagarmid , A.K. , Verykios , V.S.: Tailor: A record linkage tool box . In: ICDE ( 2002 )

11. Elmagarmid , A.K. , Ipeirotis , P.G. , Verykios , V.S.: Duplicate record detection: A survey . IEEE Transactions on Knowledge and Data Engineering 19 , 1 { 16 ( 2007 )

12. Herbold , S. : Autorank: A python package for automated ranking of classi ers . Journal of Open Source Software 5 ( 48 ), 2173 ( 2020 ). https://doi.org/10.21105/joss.02173, https://doi.org/10.21105/joss.02173

13. Hinton , G.E.: Connectionist learning procedures . Artif. Intell . 40 ( 1 {3), 185 {234 (Sep 1989 ). https://doi.org/10.1016/ 0004 - 3702 ( 89 ) 90049 - 0 , https://doi.org/10. 1016/ 0004 - 3702 ( 89 ) 90049 - 0

14. Isele , R. , Bizer , C. : Learning expressive linkage rules using genetic programming . PVLDB 5 , 1638 { 1649 ( 2012 )

15. Kingma , D.P. , Ba , J.: Adam: A method for stochastic optimization ( 2017 )

16. Konda , P. , Das , S. , C. , P.S.G. , Doan , A. , Ardalan , A. , Ballard , J.R. , Li , H. , Panahi , F. , Zhang , H., Naughton , J.F. , Prasad , S. , Krishnan , G. , Deep , R. , Raghavendra , V. : Magellan: Toward building entity matching management systems . Proc. VLDB Endow . 9 ( 12 ), 1197 { 1208 ( 2016 ). https://doi.org/10.14778/2994509.2994535, http://www.vldb.org/pvldb/vol9/p1197-pkonda.pdf

17. Lacoste-Julien , S. , Palla , K. , Davies , A. , Kasneci , G. , Graepel , T. , Ghahramani , Z. : Sigma: simple greedy matching for aligning large knowledge bases . In: KDD ( 2013 )

18. Mikolov , T. , Chen , K. , Corrado , G. , Dean , J.: E cient estimation of word representations in vector space ( 2013 )

19. Mudgal , S. , Li , H. , Rekatsinas , T. , Doan , A. , Park , Y. , Krishnan , G. , Deep , R. , Arcaute , E. , Raghavendra , V. : Deep learning for entity matching: A design space exploration . In: Proceedings of the 2018 International Conference on Management of Data, SIGMOD Conference 2018 . pp. 19 { 34 . ACM ( 2018 ). https://doi.org/10.1145/3183713.3196926, https://doi.org/10. 1145/3183713.3196926

20. Nentwig , M. , Hartung , M. , Ngomo , A.C.N. , Rahm , E.: A survey of current link discovery frameworks . Semantic Web 8 , 419 { 436 ( 2017 )

21. Ngomo , A.C.N. , Lyko , K. : Eagle: E cient active learning of link speci cations using genetic programming . In: ESWC ( 2012 )

22. Ngomo , A.C.N. , Lyko , K. : Unsupervised learning of link speci cations: deterministic vs. non-deterministic . In: OM ( 2013 )

23. Nikolov , A., d'Aquin , M. , Motta , E.: Unsupervised learning of link discovery conguration . In: ESWC ( 2012 )

24. Pershina , M. , Yakout , M. , Chakrabarti , K. : Holistic entity matching across knowledge graphs . 2015 IEEE International Conference on Big Data (Big Data) pp. 1585 { 1590 ( 2015 )

25. Sarawagi , S. , Bhamidipaty , A. : Interactive deduplication using active learning . In: KDD ( 2002 )

26. Sherif , M.A. , Ngomo , A.C.N. , Lehmann , J.: Wombat - a generalization approach for automatic link discovery . In: ESWC ( 2017 )

27. Sun , Z. , Hu , W. , Zhang, Q. , Qu , Y. : Bootstrapping entity alignment with knowledge graph embedding . IJCAI International Joint Conference on Arti cial Intelligence 2018-July , 4396 { 4402 ( 2018 ). https://doi.org/10.24963/ijcai. 2018 /611

28. Sun , Z. , Zhang , Q. , Hu , W. , Wang , C. , Chen , M. , Akrami , F. , Li , C. : A benchmarking study of embedding-based entity alignment for knowledge graphs . Proceedings of the VLDB Endowment 13 ( 11 ), 2326 { 2340 ( 2020 ), http://www.vldb. org/pvldb/vol13/p2326-sun.pdf

29. Volz , J. , Bizer , C. , Gaedke , M. , Kobilarov , G.: Silk - a link discovery framework for the web of data . In: LDOW ( 2009 )

30. Wu , Y. , Liu , X. , Feng , Y. , Wang , Z. , Yan , R. , Zhao , D. : Relation-aware entity alignment for heterogeneous knowledge graphs . Proceedings of the Twenty-Eighth International Joint Conference on Arti cial Intelligence (Aug 2019 ). https://doi.org/10.24963/ijcai. 2019 /733, http://dx.doi.org/10.24963/ ijcai. 2019 /733

31. Zhang , Q. , Sun , Z. , Hu , W. , Chen , M. , Guo , L. , Qu , Y.: Multi-view knowledge graph embedding for entity alignment . In: IJCAI . vol. 2019-Augus, pp. 5429 { 5435 . IJCAI (jun 2019 ). https://doi.org/10.24963/ijcai. 2019 /754, http://arxiv. org/abs/ 1906 .02390

32. Zhu , L. , Ghasemi-Gol , M. , Szekely , P.A. , Galstyan , A. , Knoblock , C.A. : Unsupervised entity resolution on multi-type graphs . In: ISWC ( 2016 )