3While deep learning approaches have shown promising results in Natural Language Processing and Computer Vision domains, they have not yet been able to achieve impressive results in Ontology Alignment, and have typically performed worse than rule-based approaches. Some of the major reasons for this are: a) poor modelling of context, b) over tting of standard DL models, and c) dataset sparsity, caused by class imbalance of positive alignment pairs wrt negative pairs. To mitigate these limitations, we propose a dual-attention based approach that uses a multi-faceted context representation to compute contextualized representations of concepts, which is then used to discover semantically equivalent concepts.
Deep Learning models proposed so far [
In an e ort to mitigate the above-mentioned challenges, we propose VeeAlign[
VeeAlign[
Context Representation In VeeAlign, in the case of concepts, its neighboring concepts form its context. Each neighboring concept has a role and exerts a
Fig. 1: VeeAlign Architecture
separate in uence on the concept alignment, therefore we categorize neighboring
concepts into four facets: ancestor nodes, child nodes, nodes connected through
a datatype property and nodes connected through an object property. Here, we
de ne parent and child nodes as those connected to the central node through
Subclass-of relations. Sifting through several ontologies and their reference
alignments, we observed that two concepts align not just based on their one-hop
neighbours, but also on the basis of similarity of "ancestral nodes". Ancestral
nodes of a concept are de ned as the nodes that lie on the path all the way
from the root to the parent node of the concept, and this path is referred to as
a "lineage path". Thus, given a concept, apart from its one-hop neighbours we
also enumerate all its lineage paths, consider them as part of context and use
them while computing alignments. Since only ancestors will have such "lineage
paths" and not one-hop neighbours (children, datatype property neighbours and
object property neighbours), we consider these one-hop neighbours as paths of
length 1 in order to maintain consistent terminology. The context of properties
is modelled separately, by considering their domain and range as context.
Dual Attention Attention [
Thus, path level attention aims to nd the most important paths for each contextual facet. This involves computing the attention weight of each node in each path with respect to the main concept, and then adding and normalizing these weights. When done for each lineage path, it yields the relative importance of that path. Given the relative importance of each path, a max-pool layer is then applied over each of these path weights to yield the most important path. After path-level attention, the next step is node-level attention. This is achieved by computing the attention weights of each node in the most important path. These weights are then used to take a weighted linear combination of the node embeddings available in the path embedding.
Model Training As shown in Figure 1, the training process involves
computing the representations of all four types of context i.e., ancestral context, child
context, data property neighbour context and object property neighbour
context. The context representation computation involves applying the path-level
attention followed by the node-level attention. The child, data properties and
object properties context have paths of only length one, therefore there is only
path level attention, not the node level attention. These four context
representations are combined through a weighted linear combinations to get the nal
context representation. This context vector is then concatenated with the
semantic representation of the central concept, and the combined representation
is input to a linear layer for dimensionality reduction in a lower dimension space.
Since a candidate alignment pair consists of elements (concepts or properties)
from both source and target ontologies, we perform the above computations for
both source and target elements to get the contextualized representations by
passing both through a Siamese Network [
Our system requires training data in the form of positive and negative alignment pairs. Although many tracks in OAEI have reference alignments that contain positive example pairs, as per the rules, one could not use them for training. So, we resorted to using pseudo-training data, i.e the output of the highest-performing system, AML, as an approximation of the reference alignments. However, we found that AML was only able to identify concepts whose names had some sort of string similarity, but not concepts that had di erent names but similar contexts. Our approach of modelling structural context and then attending on it thrives on identifying not just the former using semantic similarity, but also the latter using structural similarity. Since this was missing in AML's output, there were a large number of False Negatives (FNs). In addition, there were a few False Positives (FPs) with similar names but di erent contexts. Thus instead of using AML's output directly, we decided to manually correct AML's output as a preliminary stage. In order to discover FPs, we simply validated AML's output and discovered 12 FPs. To discover FNs, we took a cartesian product of all concept pairs, sorted them based on context similarity and annotated the top 1000 pairs, discovering a total of 35 FNs. Our annotations process included three annotators where the nal decision was taken using majority voting algorithm.
As part of our submission, we targeted two tracks i.e. Conference and Multifarm. We performed the above exercise for all ontology pairs in the Conference track. For Multifarm, due to lack of time, we could not train a separate model or integrate a translator, but instead used the pre-trained Conference model, since both tracks share similar ontological structure and are comprised of the same ontologies, albeit in di erent languages. In place of a translator, we merely substituted the initial Universal Sentence Encoder4 model with its multilingual variant5.
Our Deep Learning model requires several parameters and hyperparameters, which we optimized through a grid-search algorithm. The model was converged using MSE loss and Adam optimizer with a learning rate of 0.001, after training for 50 epochs with a batch size of 32. The maximum number of paths considered for each node is set to 21, and the maximum length of the path is taken as 8. All randomizations including the ones in PyTorch and Numpy are done by setting 0 as the manual seed. The entire codebase of our system is available on GitHub6 and so is the con guration parameters le7.
Table 1 shows the results of the conference track of OAEI 2020. There are in total 9 matching tasks. There are three di erent reference alignments: ra1 (original), ra2 (consistency violation-free) and rar2 (consistency & conservativity violationfree), each of which contain class matching, property matching and hybrid (both class and property) matching tasks respectively. VeeAlign achieves state-of-theart results in all the entity matching tasks, 3rd position in all the property matching tasks and top 2 positions in all the hybrid matching tasks, sometimes losing out to AML by a narrow margin of 0.01 points in F1-score. In rar2, which 4 https://tfhub.dev/google/universal-sentence-encoder-large/5 5 https://tfhub.dev/google/universal-sentence-encoder-multilingual/3 6 https://github.com/Remorax/VeeAlign 7 https://github.com/Remorax/VeeAlign/blob/master/src/con g.ini has been considered to be the main set of reference alignments this year (since it is both consistency and conservativity violation-free), VeeAlign tops the table.
VeeAlign has certainly produced some very encouraging results. In the conference track, it achieves state-of-the-art results in entity matching, which indicates that our two-step attention model that adopts a multifaceted approach to context representation, is able to outperform systems that use handcrafted rules and manually assigned parameter values. VeeAlign's performance dips when it comes to property matching, indicating that our approach of modelling property similarity as a weighted sum of property, domain and range similarity is not good enough and needs more work. Nevertheless, our current approach ensured we achieve state-of-the-art results in hybrid-matching tasks, a sizeable feat in just the debut run.
In the multifarm track, unsurprisingly VeeAlign found itself being unable to compete with other SOTA systems, that, unlike VeeAlign, used an integrated translator. However despite this and the lack of a separately trained model, it was still able to use ontological structure to produce moderately decent results with low recall but high precision. 3.2
While current OAEI evaluation measures are de nitely thorough and multifaceted, a possible improvement would be to incorporate K-fold sliding window evaluation. Here, (K-1) folds of the reference alignments are provided as input (if the system needs it for training) and the last fold is used for testing. This is done for every possible fold, and an average of the performance on each fold is taken to yield the nal performance. This data split into K-folds can occur either at the ontology-pair level (possible in tracks where there are a large number of ontology pairs in the reference alignments, such as conference or multifarm tracks) or at the "concept-pair" level (possible in tracks where there are fewer number of ontology pairs, but the ontologies are larger in size). In case of the former, (K-1) folds of ontologies are provided for training and 1 fold for testing. In the conference track, for instance, which has 21 ontology pairs , if we take K=7, 6 folds (i.e 18 ontology pairs) are provided for training and the last fold (i.e 3 ontology pairs) are tested on. This is repeated for every 3 pairs of ontologies, and an average is taken.
In case of the latter, a "concept-pair" split could be achieved by taking a cartesian product of the concepts in both ontologies, splitting it into K folds, taking K-1 for training and in each test fold, the concept pairs which are part of the alignment output by a system can be marked as True, and the rest as False. The same can be done for the ground truth alignments, and the corresponding True Positives (TPs), False Positives (FPs) and True Negatives (TNs) can be calculated to yield precision, recall and F1-scores. When evaluating by "conceptpair" split, the input for non-supervised systems remains the same, and is still the entirety of the ontology. For supervised systems, it would be the (K-1) folds of concept pairs, with pairs present in reference alignments being marked as True and the rest as False. Such a le could be dynamically created and input to the system. However, while evaluating the alignments generated by each system, the procedure remains the same, i.e calculating TPs, FPs and TNs on the last fold.
Both of these proposed methods of evaluation are in compliance with OAEI rules, and the training and testing data are clearly separated (removing any chance of over- tting) and an average is taken to remove any chance of bias. Moreover, K-fold sliding window evaluation is widely accepted in the AI community as a fair mode of evaluation. Going ahead, these measures would give supervised systems a fair chance to compete against systems that use handcrafted rules, would encourage more DL-based systems in the future and thus allow Deep Learning to thrive in Ontology Alignment as well. 4
As part of OAEI 2020, we introduced VeeAlign, an Ontology Alignment that uses a supervised Deep Learning approach to discover alignments. In particular, it uses a two-step model of attention combined with multi-facted context representation to produce contextualized representations of concepts, which aids alignment based on semantic and structural properties of an ontology. VeeAlign achieves state-of-the-art results in Conference track beating AML, LogMap and other mature systems in just its debut run, which is certainly encouraging and indicative of the validity and e ectiveness of our approach. In multifarm track, VeeAlign produces decent results using solely ontological structure. The results have high precision but low recall due to lack of incorporation of a translator, which we plan to x in future editions of OAEI. In addition, we plan on improving property matching to further improve our performance on the Conference track. Lastly and mostx importantly, we plan on targeting the biomedical tracks (such as Anatomy, LargeBio and Biodiversity) and adapting VeeAlign to use biomedical background knowledge, if available.