This paper presents the results obtained by the Holontology (Holistic ontology matcher) system in the OAEI 2018 evaluation campaign. We describe here the results in the Anatomy, Conference, Taxon and Knowledge Graph tracks. We report a general discussion on the results and on the future improvements of the system.
1.1
Holontology is a modular system based on the LPHOM system (Linear Program for
Holistic Ontology Matching) [
The system treats the ontology matching problem, at schema-level, as a combinatorial optimization problem. The problem is modelled through a linear program extending the maximum-weighted graph matching problem with linear constraints (matching cardinality, structural, and coherence constraints). 1.2
The way the system works is detailed in the following steps: 1. The first step of Holontology is to load the ontologies and translate them into an internal structure, which can be annotated and edited, so that later treatment is done in an efficient way. Each ontology is loaded independently, and its hierarchical structure is conserved in a format based on the three possible basic blocks of ontologies: classes and properties (object and data). Each of them is represented as a Node, and depending on their subtype, they are subclassed into a ClassNode, ObjectPropertyNode or DataPropertyNode. These nodes are then enriched with AnnotationData. 2. The second step consists in a pre-processing step. Here, we expand camel case and title case into proper names. As the loading of the ontologies is done only once, we take advantage of Java ways of storing references, so we have no need to translate and cut matrices. We can use our hierarchies both for storage and computation. 3. The third step computes a cartesian product between all the entities of same type (classes and properties) in order to build a similar linear program using the same framework as LPHOM, as described below. 4. The fourth step leads a second wave of pre-processing, in such a way that new relations are created. For example, this is the step where we compute tense similarity (seeing whether or not an object property is passive or active towards its classes). More precisely, for each property, we compute its polarity using tense analysis, e.g. Author reviews Paper (active) vs. Paper reviewed by Author (passive). This allows reviews matching reviewed by reversed. 5. The fifth step applies a combination of different similarity measures (exact match, Levenstein, Jaccard, and Lin), drawing both from the AnnotationData build in the previous steps, and the constraints based on ontologies themselves.We have tested the combination of similarity measures and the results reported here are in terms of ‘exact match’ over the pre-processed entity naming and annotations (considering the exact match between tokens). As expected, we obtain good values in terms of precision, as better discussed below. 6. The sixth step converts the given constraints in a form that can be used by a solver.
We choose CPLEX for its ability to perform optimizations, and we manage probable thresholds if needed. Unlike LPHOM, we provide all the constraints, and we do not apply any cutting in this step. 7. In the seventh step, we take the given result from the solver and convert it in an alignment that can be exported in RDF. 8. In the (optional) eight step, we combine the alignment result and run step five to seven if needed to reinforce the obtained results. 1.3
Due to a huge incompatibility between some libraries used in the SEALS client and ours, we had to create a fully executable jar in the conf/ directory, and call this jar from the SEALS bridge. The bridge grabbed the URI of the two ontologies passed as arguments, and wrote them to a file called “bridge-ontologies.url”, then called the holontology.jar in the conf directory, waiting for its full execution. holontology.jar opens the created “ontologies.url”, reads the two URIs, and does the processing. The result is written in the “result.alignment” file. The bridge then reads the result file and returned it to the SEALS client. 1.4
Holontology, as its predecessor LPHOM, is not available as an open-source. The version we present here is meant to be more modulable than LPHOM, and handles the problem differently, by insisting on annotating data and using structures instead of matrices. The .zip for the SEALS jar can be retrieved at https://cloud.irit.fr/index. php/s/gReZo8yaRDqdmjk
The generated alignments are available at https://cloud.irit.fr/index. php/s/hv3oALXN6fHuZWi. 2
The reader can refer to the OAEI web pages for the results of Holontology in the Anatomy, Conference, Taxon and Knowledge Graph tracks as well a comparison with other participants. Here, we provide a first discussion and comments on our results. 2.1
Our results for the Anatomy track are summarised in Table 2.1. Compared to the evaluation of LPHOM in OAEI 2016 1, we observe that globally the quality of results decreases, for instance the F-measure looses 0.3 points. These results can be explained by two choices in Holontology. First we only use exact match (we observe that Holontology returns only 456 alignments compared to LPHOM which returns 1555 alignments). Second we do not cut results according to that (i.e, we do not apply any threshold). However, we observe that Holontology is 8 times faster that LPHOM. Our results for the Conference track are summarised in Table 2.2. Contrarily to the anatomy track, the results of Holontology are better than the results of LPHOM for the conference track. Globally, Holontology gains in recall for the different tasks in this track. We can explain that by the different pre-processing strategies that have been implemented in Holontology compared to LPHOM. By comparing the different tasks, the tool needs additional efforts to handle data and object properties that occur in the M2 tasks. 1 http://oaei.ontologymatching.org/2016/results/anatomy/index.html
Our results for the Taxon task are summarised in Table 2.3. We have been obtained intermediate results, with a precision up to 0.22. However, for the set of given queries to be translated with the help of the generated alignments, our alignments were not useful, QWR (Query Well Rewritten) measure of 0.
Time (s) output corres. eval. corres. correct corres. Global Prec. Average Prec. (1:1) (1:n) (m:n) QWR 965 44 13 3 0.23 0.22 44 0 0 0.00 Our results for the Knowledge graph track are summarized in Tables 2.4 and 2.4. For this track, Holontology proceeded faster than the other systems (including the baseline). However, it has not be able to deal with properties, probably, as the track organisers explained, because all properties are typed as rdf:Property and not subdivided into owl:DatatypeProperty and owl:ObjectProperty). Despite the fact that Hontology is an extended version of LPHOM that has participated in OAEI 2016, for its first participation Hontology has relative intermediate results. Table 3 summarises the performance of Hontology in terms of ranking of the best systems in each task. Track Time Size Prec. F-m. Rec. darkscape oldschoolrunescape 23 42 0.75 0.22 0.13 runescape darkscape 38 62 0.93 0.25 0.14 runescape oldschoolrunescape 35 47 0.62 0.22 0.13 heykidscomics dc 67 4 1.00 0.10 0.05 marvel dc 26 4 1.00 0.21 0.12 marvel heykidscomics 61 4 1.00 0.11 0.06 memory-alpha memory-beta 26 2 0.00 0.00 0.00 memory-alpha stexpanded 21 2 0.00 0.00 0.00 memory-beta stexpanded 21 2 0.00 0.00 0.00
Table 5. Track results for Knowledge graph track.
With respect to LPHOM, Hontology is a modular system that optimises the ontology structures in memory. As for LPHOM, we model and express the matching problem through a set of constraints (cardinality, structural, and coherence constraints) applied on the results of a pre-processing and exact matching steps. We do not have applied any threshold on the generated alignments. As expected, using an exact match on preprocessed entity naming and comments may improve precision in detriment of recall. With respect to these aspects, we plan to improve the criteria of selection of similarity measures and thresholds for our future participation, in particular with the aim of improving recall.
Other points include the fact that Hontology is a system designed to deal with holistic ontology matching at schema-level. Hence, it was not able to generated alignments for the tasks involving instance matching. We plan to implement instance matching strategies in future versions of the system. Furthermore, despite our optimisation over LPHOM, our system was not able to deal at all with the large ontologies in the LargeBio and Phenotype tasks. We note, however, that we could deal with the Complex Taxon task in terms of volume because we do not treat the instances. We plan to address these points in the future.
With respect to the OAEI procedure, we focus on the tracks based on SEALS. However, as stated above, we have encountered problems for dealing with the incompatibilities of package versions in the SEALS dependencies. We have implemented a non-ideal solution and hope for the next evaluation this kind of issue will be fixed.
Finally, Hontology has been initially designed to deal with hoslistic ontology matching. However, there is no track in the campaign proposing the evaluation of such kind of matching approaches. In the future, it could be interesting to have a dedicated holistic track. 4
This paper has introduced the Hontology system and discussed the main points on the results of its first participation in the OAEI campaigns. We have as well pointed out some directions for future improvements.
Cassia Trojahn has been partially supported by the French CIMI Labex project IBLiD (Integration of Big and Linked Data for On-Line Analytics).