Entity Resolution (ER) is a critical data integration task that is addressed through the Filtering-Verification framework. Filtering leverages unsupervised techniques for reducing the computational cost to the most similar candidate pairs, whereas Verification typically uses supervised learning techniques to distinguish between matching and non-matching pairs of entities. The state-of-the-art Verification methods leverage Deep Learning in combination with language models. Despite their high accuracy, their applicability is limited, due to their requirement for a large number of labelled instances from the dataset at hand. To address this issue, we propose a novel methodology for end-to-end Deep ER that waives the need for labelled instances. In essence, it associates the pair of data sources at hand < 1, 2 > with the another pair of data sources of known groundtruth, < 1, 2 >, whose similarity distributions after Filtering are very close. The Verification method is then trained on the candidate pairs of < 1, 2 > and applied to the candidate pairs of < 1, 2 >, linking their records without any labelled instances. Through a thorough experimental study that involves six datasets from two diferent domains, we demonstrate the high efectiveness of our approach, which outperforms established methods that require no labelled instances, while approximating the performance of the state-of-the-art Verification algorithms.
To address these challenges, we propose an end-to-end ER pipeline that integrates Filtering and Verification, which applies both Matching and Clustering. This pipeline is designed to be adaptable: The Matching algorithm can be trained on datasets with ground truth and then applied to diferent datasets, while Filtering and Clustering can be applied using default values, requiring no labelled instances. This allows the full pipeline to generalise efectively to datasets without labelled data, providing a robust foundation for entity resolution tasks across diverse datasets.
In this work, we focus on finding the overlap between two datasets, which are duplicate free. As shown in Figure 1, our approach identifies the pair of datasets from a set of dataset pairs < ,1, ,2 >∈ D with known ground truth which is most similar to the input pair of datasets without labelled instances, < 1, 2 >. We denote the dataset pair most similar to < 1, 2 > by < ′1, ′2 >. This is achieved by comparing the similarity distributions after Filtering. The set of candidate pairs from < ′1, ′2 > is then used to train the Matching algorithm, which is subsequently applied to the candidate pairs of < 1, 2 >. Finally, we employ clustering algorithms to refine the pairs generated by the Matching algorithm. Our approach aims to enhance the scalability and applicability of ER systems, making them more versatile and eficient in real-world applications. Through an extensive experimental study involving six datasets from two distinct domains, we demonstrate the high efectiveness of our approach. It approximates the performance of state-of-the-art Verification algorithms, which require labelled training instances, while outperforming established methods that require no labelled instances.
We define as an entity profile with id or just entity as a uniquely identified set of name-value pairs: =< , {(name, value)} >. Note that this plain data model accommodates data formats of any structuredness, from structured records (in relational databases or CSV files) to semi-structured entity descriptions (in RDF) and to free-text. A set of entity profiles of this form is called data source.
ER typically comes in one of two forms: (i) Record Linkage, also known as Clean-Clean ER, receives as input two individually duplicate-free data sources, 1 and 2, which are overlapping, i.e., sharing some of their entity profiles. (ii) Deduplication, also known as Dirty ER, receives as input a single dataset, , with duplicates in itself. In both cases, the output comprises the matching entity profiles.
In this context, we formally define the task we are tackling in this work as follows: Given two data sources < 1, 2 >, which lack labelled entity pairs, along with a set of data source pairs D, each with a known groundtruth, detect the matching entity profiles in < 1, 2 > by leveraging the training instances provided by D so as to maximize the efectiveness in terms of recall, precision and F-Measure.
This definition applies to Record Linkage, but can be easily extended to Deduplication. It focuses on end-to-end ER pipelines that perform both Filtering and Verification, as shown in Figure 1. Filtering receives two datasets, 1 and 2, and outputs the candidate pairs . Verification involves two steps: (i) Matching estimates the matching likelihood between the entities of each candidate pair. (ii) Clustering partitions the similarity graph into the set of matching entities, resolving the conflicts that may arise after Matching (e.g., an entity from 1 is matched with two or more entities from 2).
Our goal is to apply end-to-end ER pipelines that combine the state-of-the-art DL-based matching algorithms with an unsupervised state-of-the-art filtering method. We can waive the need to train the matchers on a large set of labelled instances by using other datasets with known-groundtruth as training sets. Based on Problem 1, which dictates that the input comprises a pair of data sources with no labelled instances, < 1, 2 >, along with a set of data source pairs D, each with a known groundtruth, our solution consists of the following steps:
1) We apply a predetermined Filtering method with specific parameter configuration to each labelled data source pair < ,1, ,2 >∈ D, yielding a set of candidate pairs .
2) For each labelled data source pair < ,1, ,2 >∈ D, we convert the entities and in each candidate pair , ∈ into the embedding vectors and using a pre-trained SBERT model. Then, we compute the cosine similarity of the resulting vectors, i.e., (, ), yielding a similarity distribution per data source pair < ,1, ,2 >.
3) We apply the same filtering method with the same configuration to the unlabelled data source pair, < 1, 2 >, yielding the set of candidate pairs .
4) We estimate the similarity distribution , by converting all pairs of entities in into pairs of SBERT embedding vectors and computing their cosine similarity.
5) Using KL-divergence, we estimate the similarity distribution ′ of the data source pairs in D that is closer to . The selected data source pair is denoted by ′.
6) We train the selected matching algorithm on all candidate pairs extracted from ′, ′ , using the respective groundtruth for assigning their labels.
7) We apply the trained matching algorithm on the candidate pairs of the given unlabelled data source pair, . As a result, each candidate pair in is associated with a matching probability, yielding a weighted undirected similarity graph .
8) We remove any conflicts from the matching outcomes, by applying a clustering algorithm to the similarity graph to produce the final output.
To put this methodology into practice, we need to specify the algorithms that are used for Filtering, Matching and Clustering.
Starting with the Filtering algorithm, we selected kNN Join, which converts each entity into a sparse
-gram vector and then identifies its nearest neighbors using Cosine, Dice or Jaccard similarity.
kNN Join consistently achieves very high efectiveness, while being easily configured [
For Matching, there is no clear winner among DITTO [
Finally, for Clustering, the best balance between efectiveness and time eficiency is achieved by
UMC, which excels in balanced datasets, where there is a relatively equal number of duplicates in 1
and 2, and EC, which excels in all other types of datasets [
In this context, it is worth delving into the exact configuration parameters of each considered
algorithm. kNN-Join is configured by: (i) Prep, a binary parameter that specifies whether pre-processing
(i.e., stemming and stop-word removal) is applied or not, (ii) Indexing, a binary parameter that specifies
whether the smallest data source is used for indexing and the largest one for querying or vice versa,
(iii) Q, a numeric parameter that determines the size of the q-gram, with ∈ [
For the Matching algorithms, the number of epochs is the only hyperparameter that we fine-tune, with a maximum of 40 epochs. Preliminary experiments indicated that a suitable default value is 25 epochs – in many cases, the considered matching algorithms achieved their best performance with lower epochs, while a few reached their best performance after more than 30 epochs. For all other hyperparameters, we use default values determined by their open-source implementation.
For the Clustering algorithms, the only configuration parameter is the similarity threshold, ∈ [0, 1), which prunes the edges with a similarity score lower than . The default value of has been set to 0.5, as a matching probability lower than 0.5 indicates pairs of entities that most likely are not matching.
Experimental Setup. The complete code (in Python) and data are available on GitHub1. We selected
six commonly used datasets [
DITTO
EMTransformer
GNEM
ZeroER
DADER
Unicorn
DITTO
EMTransformer
GNEM
ZeroER
DADER
Unicorn
DITTO
EMTransformer
GNEM
ZeroER
DADER
Unicorn
DITTO
EMTransformer
GNEM
ZeroER
DADER
Unicorn
DITTO
EMTransformer
GNEM
ZeroER
DADER
Unicorn
DITTO
EMTransformer
GNEM
ZeroER
DADER
Unicorn
(f) Performance of the end-to-end pipelines on 6.
specifically designed to be applied to datasets without labelled instances, namely ZeroER [
Research Question 1: How does our methodology compare to ZeroER, DADER and Unicorn, the main end-to-end ER pipelines that also require no labelled instances? Looking into Table 3, we observe that our methodology ofers significant advantages over the other state-of-the-art approaches that require no labelled datasets. Combined with GNEM, it outperforms ZeroER, DADER and Unicorn, on average, provided that there is suficient GPU RAM to support labelled datasets with large sets of candidate pairs. As an alternative, EMTransformer trades higher scalability and robustness for slightly lower efectiveness – only Unicorn achieves higher F-Measure, on average, because it requires a large amount of training data to achieve this performance on unlabelled datasets.
Research Question 2: How does our methodology compare to the traditional DL-based settings, which require a large dataset of labelled instances of the dataset at hand? We compare the performance on the “Labelled Dataset” columns of Table 3 with the adjacent ones of the “Same Dataset”. DITTO’s efectiveness raises significantly when trained on instances from the same dataset: it underperforms by 19% and 31% under the fine-tuned and the default configurations, respectively, regardless of the clustering algorithm. This applies to all datasets except 6.
EMTransformer also underperforms the traditional settings in all cases, without any single exception (unlike DITTO). In half the datasets, though, its performance is very close to the traditional settings under the fine-tuned settings. The diference between the two approaches is <4%, <3% and <6% in 3, 4 and 6, respectively, for both clustering algorithms. As a result, EMTransformer lies much closer to the traditional approach than DITTO.
The best performance, by far, is exhibited by GNEM. Under the fine-tuned configurations, it is very close to the traditional settings in 3 and 4, underperforming by just <4% and <1%, respectively, regardless of the clustering algorithm. In 6, its performance even outperforms the traditional settings by 2% and 5% for UMC and EC, respectively. Only in 5, it underperforms the traditional settings by >15%. However, it sufers from instability, given that it does not scale to 1 and 2. This applies to the default configurations, too, where it does not scale in 3 and 4.
Overall, GNEM constitutes the most efective choice for our approach in case there are no hardware limitations, exhibiting a performance very close or even better than the traditional settings. Te most robust solution for our approach is EMTransformer, at the cost of a slightly lower efectiveness.
We presented a novel methodology for applying deep ER end-to-end pipelines without requiring any labelled instances from the input data sources, 1 and 2. It leverages the Filtering-Verification framework, using a high-end unsupervised technique for the former step. The same Filtering technique is applied to a set of labelled dataset pairs D, generating candidate pairs with high recall and low precision. The candidate pairs of < 1, 2 > are compared with those of all dataset pairs in D and the one with the closest similarity distribution is used as a training set for the DL-based matching algorithm. The resulting trained matcher together with a clustering algorithm form the Verification step. Our experimental analysis demonstrates the high efectiveness of our methodology, which outperforms most relevant techniques from the literature. The best results are achieved when using GNEM as the matching algorithm, with EMTransformer following in close distance. In the future, we plan to investigate ways of improving the selection of the labelled dataset and perhaps of the instances extracted from it. Acknowledgements. This work was partially funded by the EU Horizon project RECITALS (Grant Agreement 101168490).
The authors have not employed any Generative AI tools.