This paper introduces WYM (Why do You Match?), an intrinsically explainable model designed for Entity Matching (EM). WYM is built upon decision units, which are basic information units formed by either pairs of similar terms belonging to diferent entity descriptions, or unique terms present in only one of the descriptions. Decision units enable the definition of a new feature space that can compactly and meaningfully represent pairs of entity descriptions. By training an explainable binary classification model on these features, WYM generates customized and efective explanations for EM datasets.
explanations can result in usability issues. Specifically, records may contain a significant number
of features, leading to complex explanations that are challenging to manage and comprehend
for users [
To address the challenges of EM model interpretability, we proposed WYM [
This paper is an extended abstract of paper [
The WYM functional architecture for generating intrinsically interpretable predictions for entity descriptions is shown in Figure 1. It comprises four main components: the Decision unit generator, the Relevance scorer, the Explainable matcher, and the Explanation analysis tool. In the following, we suppose that entity descriptions share the same schema.
This component implements three main functionalities. Firstly, a word-piece tokenization with stop word removal is applied. Then we apply the BERT language model (fine-tuned for the task at hand) to encode the meaning of the entity descriptions into contextualized embeddings. Finally, embeddings from an entity description are possibly paired with the ones of the second description, thus forming paired decision units. For this last operation, we leverage the schema of the dataset, if any, to reduce the alignment space where an adaptation of the Gale–Shapley implementation of the Stable Marriage algorithm is applied.
Example. Let us consider the dataset iTunes-Amazon dataset from the Magellan benchmark.
x2cu everything is 4
pop music 2015 warner bros record inc.
Optimized Stable Marriage on BERT word embeddings x2cu jason derulo everything is 4 clean pop $1.29 2011 because music
Decision unit generator
Paired
x2cu everything is 4
pop music 2015 warner bros. records inc.
Counterfactual Explanation
x2cu everything is 4 clean
pop 2011 because music
The goal is to understand when descriptions refer to the same song and the impact of the decision units in the prediction. The Decision unit generator takes the descriptions in the dataset as input as shown in the left part of Figure 1 and generates paired and unpaired decision units. The Figure shows two examples of paired and unpaired units. Among them, [x2cu,x2cu], the name of the song, is an example of paired decision unit, [clean], a word that is part of the album title in the second description, is an example of an unpaired decision unit.
To assign relevance scores to decision units in the matching process, WYM utilizes a supervised regression model trained using a dataset where each entry represents a decision unit, and the target class is estimated by applying a heuristic rule, which relies on the class to which the decision unit belongs and the frequency of co-occurrence of the decision unit to the target class computed on the entire dataset.
Example. The relevance score assigned to the paired decision unit [x2cu,x2cu] is 0.822, thus pushing WYM to consider the descriptions as matching. The score of the unpaired decision unit [clean] is -0.149, thus pushing the classifier to consider the descriptions as non-matching.
The relevance scores assigned to the decision units provide an estimate of their contribution to the overall prediction. We enhanced this estimation by incorporating contextual and structural knowledge. We introduced three types of knowledge by aggregating features and scores based on attribute, entity description, and record. To accomplish this, we employed various statistical operators such as max, min, and count, which were applied to the decision units. The new dataset is used to train a binary classifier (a logistic regression classifier) that infers if the pairs of entity descriptions refer to the same real-world entity. Finally, we leverage the interpretability of the selected binary classifier to estimate the efect that each decision unit has on the prediction. We begin by extracting the learned coeficients from the classifiers, which indicate the importance of each generated feature. Next, we employ an inverse feature engineering transformation to identify the units that contribute to each feature and associate them with the impact score. (a) The explanation computed by WYM.
(c) Counterfactual explanation (matching). (b) Comparison with token-based explanation.
(d) Counterfactual explanation (non-matching).
Example. The classifier generates the prediction (a match in the example of Figure 1, with a
probability score of 0.999) and the application of inverse transformations to its output generates
the explanation, i.e. the impact of each decision unit in the prediction. We observe that
the units that mostly contribute to the predictions are [x2cu,x2cu] and [5:13,5:14] and
that, for example, the term ‘because’, with a score of -0.578, pushes the classifier to a
nonmatching prediction. Figure 2a shows the WYM GUI with the impact scores, which represent the
contribution score of each decision. Note that if we sum all them along with the intercept of the
LR and we apply the sigmoid function we obtain the exact prediction of WYM. In other words,
the impact score is the impact of a decision unit as part of the EM record before the application
of the sigmoid. Figure 2b ofers a comparison with the explanation computed by LIME coupled
with DITTO. Even if our experiments in Section 3 show that DITTO achieves better accuracy
than WYM in the dataset, the explanation provided by LIME is less interpretable[
Tools for the analysis of the explanations allows the users to analyze the importance of the decision units (also comparing them to token-based explanations generated for the same entity descriptions) and to generate counterfactual and exemplary explanations. Counterfactual explanations allow users to identify key features for the matching decision and to improve the accuracy of the EM model. The technique for computing counterfactual explanations changes in case of predictions of matching / non-matching entities. A simple heuristic for matching entity descriptions is to remove decision units from the explanation in descending order by impact and generate new prediction scores. The process iterates until the prediction changes to non-matching. Counterfactual explanations may lack meaning when applied to non-matching entities, particularly when the entities are vastly dissimilar, and the inclusion of additional features to establish a match would result in a significant alteration of the original description.
WYM implements the following heuristics to generate counterfactual explanations for matching entity descriptions: MoRF : the Most Relevant Features are removed first, thus allowing the users to identify the features needed for the explanation. A probabilistic variation of the MoRF heuristic is also implemented to generate many counterfactual explanations for the same record; LeRF : the Least Relevant Features are removed first, thus allowing the users to identify the features which are suficient for the explanation. A probabilistic variation of the LeRF heuristic, generating many explanations for the same record, is also implemented; random: the features are randomly removed, thus generating reference baselines; manual: the user selects the decision units to remove.
Example. Figure 2c shows a counterfactual example generated by WYM with the MoRF strategy from the same prediction introduced in Scenario 1. The left part of the Figure shows the original pair of entity descriptions. On the right, we show the counterfactual explanation where the title and the duration of the song are removed. This means that selected units in the title and duration assume paramount importance in the prediction. The user can select other heuristics and the results are shown with the same tabular representation.
To compute counterfactual explanations for a pair of non-matching entity descriptions, WYM: 1) extracts from it the positive explanation, which only includes the “positive” decision units making the entity descriptions refer to the same real-world entity; 2) injects “negative” decision units into the positive explanation until the prediction changes again to non-matching. Since the goal is to identify the features that maximize the diversity between the descriptions, the most negative decision units are injected firstly.
Example 3. Figure 2d shows a counterfactual example for a non-matching entity prediction. The removal of unpaired decision units from the song title of both descriptions and from the genre of the first entity description makes WYM change the prediction class. This counterfactual explanation is consistent with the previous (and it is something somewhat expected from our domain knowledge): the terms in the song title are the ones that lead the model to understand if descriptions refer to the same real song.
Finally, the WYM system ofers a feature that automatically identifies exemplary explanations
from the ones generated for the entire dataset. These explanations are evaluated based on their
ability to jointly satisfy three key metrics: explanation entropy, prediction-relevant units, and
explanation overlap. Firstly, the explanation entropy metric is used to gauge the balance of token
impacts in an explanation. An explanation with a low entropy (i.e., an unbalanced distribution of
token impacts) is more intriguing than one with a high entropy (i.e., a near-uniform distribution),
as it helps users identify the most significant units for prediction more easily. Secondly, the
prediction-relevant units metric is used to evaluate the relevance of the decision units for the
prediction, with a preference for a lower percentage of units that are relevant for prediction,
as this provides a more concise and usable interpretation of the model’s behavior. Lastly,
the explanation overlap metric is used to examine sets of explanations with complementary
characteristics, such as explanations with diferent decision unit impact distributions at the
attribute level, to achieve a more comprehensive interpretation of the EM model’s behavior.
The user can specify the weight of each metric in the selection of emblematic explanations, and
the Smooth Local Search technique [
In this reduced extended abstract, the experimental evaluation focuses on demonstrating 1)
how efective is WYM in solving EM tasks (Section 3.1) and 2) if the impact scores provide a
reliable interpretation of the EM predictions (Section 3.2). Interested readers can refer to the
paper [
Datasets. The experiments are performed against 12 datasets provided by the Magellan library1 which are usually considered the reference benchmark for the evaluation of EM tasks. In Table 1, we show some of their descriptive statistics: the total number of records representing matching entities in the fourth column and the percentage of records associated with a matching label in the last column. For the purposes of the experimental evaluation, each dataset is divided into training, validation, and test set which were created with 60-20-20 proportions.
The efectiveness of WYM against the datasets in the benchmark in terms of F1 score is computed.
The results are compared with the results achieved by DeepMatcher+2 (DM+) , one of the
pioneering EM systems based on Deep Learning, AutoML [
To evaluate the contribution of the impact scores assigned to the decision units to the overall accuracy of WYM, this experiment perturbs the dataset records by removing selected decision units and analyzing the performance variations on these synthetic datasets. We experiment with three techniques for the removal of the decision units applied to the datasets: 1) MoRF, where we eliminate for each record the k decision units that contribute most to the prediction (i.e. units with high positive impact in records describing matching entities and units with high negative impact for non-matching), 2) LeRF, where the k decision units that contribute less to the prediction are removed (i.e. high negative impact in case of entity matches and high positive impact / in case of non-matches), and 3) Random, where k random decision units are removed. We expect that when we remove the most relevant decision units (MoRF) from records describing matching entities, the efectiveness (F1 score) will decrease; on the other hand, the model should not be afected by the removal of the least relevant units first (LeRF). The results of the experiment are shown in Figure 3, where, for each dataset, the F1 score generated by WYM as the removal technique varies, is reported.
Discussion. Analyzing the results we observe how impact scores assigned by WYM reflect the real importance of each token on the prediction. By perturbing the data with the MoRF strategy, WYM performance drops drastically (up to 60% in some datasets). The phenomenon is mostly marked as the number of removed units increases, however, in some datasets (such as Abt-Buy, Amazon-Google, and the two versions of Walmart-Amazon) the performance drops after the removal of a single unit. Moreover, the LeRF perturbation does not produce substantial variations in performance, which in most of the datasets slightly improves.
We presented WYM, i.e. an approach for performing interpretable entity matching that predicts
if a pair of entity descriptions refer to the same real-world entity, and provides the terms (i.e.,
the decision units) that mainly led to the decision. As already pointed out in the literature [