=Paper=
{{Paper
|id=Vol-3063/oaei21_paper9
|storemode=property
|title=Lily results for OAEI 2021
|pdfUrl=https://ceur-ws.org/Vol-3063/oaei21_paper9.pdf
|volume=Vol-3063
|authors=Shiyi Zou,Jiajun Liu,Zherui Yang,Yunyan Hu,Peng Wang
|dblpUrl=https://dblp.org/rec/conf/semweb/ZouLYHW21
}}
==Lily results for OAEI 2021==
https://ceur-ws.org/Vol-3063/oaei21_paper9.pdf
Lily Results for OAEI 2021
Shiyi Zou 1 , Jiajun Liu 2 , Zherui Yang 3 ,Yunyan Hu 2 , Peng Wang 1, 2, 3 †
1
Southeast University - Monash University Joint Graduate School, China
2
School of Computer Science and Engineering, Southeast University, China
3
Chien-Shiung Wu College, Southeast University, China
{shiyizou, jiajliu, zherui_yang, yunyhu, pwang} @ seu.edu.cn
Abstract. This paper presents the results of Lily in the ontology align-
ment contest OAEI 2021. As a comprehensive ontology matching system,
Lily is intended to participate in three tracks of the contest: anatomy,
conference, and spimbench. The specific techniques used by Lily will be
introduced briefly. The strengths and weaknesses of Lily will also be
discussed.
1 Presentation of the system
With the use of hybrid matching strategies, Lily, as an ontology matching sys-
tem, is capable of solving some issues related to heterogeneous ontologies. It can
process normal ontologies, weak informative ontologies [1], ontology mapping de-
bugging [2], and ontology matching tunning [3], in both normal and large scales.
In previous OAEI contests [4–11], Lily has achieved preferable performances in
some tasks, which indicated its effectiveness and wideness of availability.
1.1 State, purpose, general statement
The core principle of matching strategies of Lily is utilizing the useful clues
correctly and effectively. Lily combines several effective and efficient matching
techniques to facilitate alignments. There are five main matching strategies: (1)
Generic Ontology Matching (GOM) is used for common matching tasks with
normal size ontologies. (2) Large scale Ontology Matching (LOM) is used for
the matching tasks with large size ontologies. (3) Instance Ontology Matching
(IOM) is used for instance matching tasks. (4) Ontology mapping debugging is
used to verify and improve the alignment results. (5) Ontology matching tuning
is used to enhance overall performance.
The matching process mainly contains three steps: (1) Pre-processing, when
Lily parses ontologies and prepares the necessary information for subsequent
Copyright © 2021 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
This work is supported by National Key R&D Program of China (2018YFD1100302)
and 13th Five-Year All-Army Common Information System Equipment Pre-Research
Project (No.31511110310).
†
Corresponding author pwang@seu.edu.cn (Peng Wang)
2 Shiyi Zou 1 , Jiajun Liu 2 , Zherui Yang 3 ,Yunyan Hu 2 , Peng Wang 1, 2, 3 †
steps. Meanwhile, the ontologies will be generally analyzed, whose characteris-
tics, along with studied datasets, will be utilized to determine parameters and
strategies. (2) Similarity computing, when Lily uses special methods to calculate
the similarities between elements from different ontologies. (3) Post-processing,
when alignments are extracted and refined by mapping debugging.
In this year, some algorithms and matching strategies of Lily have been
modified for higher efficiency.
1.2 Specific techniques used
Lily aims to provide high quality 1:1 concept pair or property pair alignments.
The main specific techniques used by Lily are as follows.
Semantic subgraph An element may have heterogeneous semantic interpre-
tations in different ontologies. Therefore, understanding the real local meanings
of elements is very useful for similarity computation, which are the foundations
for many applications including ontology matching. Therefore, before similarity
computation, Lily first describes the meaning for each entity accurately. However,
since different ontologies have different preferences to describe their elements, ob-
taining the semantic context of an element is an open problem. The semantic
subgraph was proposed to capture the real meanings of ontology elements [12].
To extract the semantic subgraphs, a hybrid ontology graph is used to repre-
sent the semantic relations between elements. An extracting algorithm based on
an electrical circuit model is then used with new conductivity calculation rules
to improve the quality of the semantic subgraphs. It has been shown that the
semantic subgraphs can properly capture the local meanings of elements [12].
Based on the extracted semantic subgraphs, more credible matching clues can
be discovered, which help reduce the negative effects of the matching uncertainty.
Generic ontology matching method The similarity computation is based
on the semantic subgraphs, which means all the information used in the simi-
larity computation comes from the semantic subgraphs. Lily combines the text
matching and structure matching techniques.
Semantic Description Document (SDD) matcher measures the literal simi-
larity between ontologies. A semantic description document of a concept con-
tains the information about class hierarchies, related properties and instances,
and external knowledge sources. A semantic description document of a prop-
erty contains the clues about hierarchies, domains, ranges, restrictions and re-
lated instances. In addition, WordNet [13] and domain-specific ontologies (the
UBERON [14] Ontology for the Anatomy track) are exploited as external re-
sources to find synonyms and cross-references between entities. Indeed, we ex-
plore the property ”hasDbXref”, which is mentioned in almost every class of
Uberon. This property references the classes’URI of some external ontologies
such as the human and mouse of the Anatomy track. Consequently, we align
Lily Results for OAEI 2021 3
every two entities of the Anatomy track in case if they are both referenced in a
single class of Uberon.
For the descriptions from different entities, the similarities of the correspond-
ing parts will be calculated. Finally, all separated similarities will be combined
with the experiential weights.
Matching weak informative ontologies Most existing ontology matching
methods are based on the linguistic information. However, some ontologies may
lack in regular linguistic information such as natural words and comments [15].
Consequently the linguistic-based methods will not work. Structure-based meth-
ods are more practical for such situations. Similarity propagation is a feasible
idea to realize the structure-based matching. But traditional propagation strate-
gies do not take into consideration the ontology features and will be faced with
effectiveness and performance problems. Having analyzed the classical similar-
ity propagation algorithm, Similarity Flood, we proposed a new structure-based
ontology matching method [1]. This method has two features: (1) It has more
strict but reasonable propagation conditions which lead to more efficient match-
ing processes and better alignments. (2) A series of propagation strategies are
used to improve the matching quality. We have demonstrated that this method
performs well on the OAEI benchmark dataset [1].
However, the similarity propagation is not always perfect. When more align-
ments are discovered, more incorrect alignments would also be introduced by
the similarity propagation. So Lily also utilizes a strategy to determine when to
use the similarity propagation.
Large scale ontology matching Matching large ontologies is a challenge due
to its significant time complexity. We proposed a new matching method for large
ontologies based on reduction anchors [16]. This method has a distinct advantage
over the divide-and-conquer methods because it does not need to partition large
ontologies. In particular, two kinds of reduction anchors, positive and negative
reduction anchors, are proposed to reduce the time complexity in matching.
Positive reduction anchors use the concept hierarchy to predict the ignorable
similarity calculations. Negative reduction anchors use the locality of matching
to predict the ignorable similarity calculations. Our experimental results on the
real world datasets show that the proposed methods are efficient in matching
large ontologies [16].
Ontology mapping debugging Lily utilizes a technique named ontology map-
ping debugging to improve the alignment results [2]. Different from existing meth-
ods that focus on finding efficient and effective solutions for the ontology mapping
problems, mapping debugging emphasizes on analyzing the mapping results to
detect or diagnose the mapping defects. During debugging, some types of map-
ping errors, such as redundant and inconsistent mappings, can be detected. Some
warnings, including imprecise mappings or abnormal mappings, are also locked
4 Shiyi Zou 1 , Jiajun Liu 2 , Zherui Yang 3 ,Yunyan Hu 2 , Peng Wang 1, 2, 3 †
by analyzing the features of mapping result. More importantly, some errors and
warnings can be repaired automatically or can be presented to users with revising
suggestions.
Ontology matching tuning Lily adopted ontology matching tuning this year.
By performing parameter optimization on training datasets [3], Lily is able to
determine the best parameters for similar tasks. Those data will be stored. When
it comes to real matching tasks, Lily will perform statistical calculations on the
new ontologies to acquire their features that help it find the most suitable con-
figurations, based on previous training data. In this way, the overall performance
can be improved.
Currently, ontology matching tuning is not totally automatic. It is difficult
to find out typical statistical parameters that distinguish each task from oth-
ers. Meanwhile, learning from test datasets can be really time-consuming. Our
experiment is just a beginning.
2 Results
2.1 Anatomy track
The anatomy matching task consists of two real large-scale biological ontologies.
Table 1 shows the performance of Lily in the Anatomy track on a server with
one 3.46 GHz, 6-core CPU and 8GB RAM allocated. The time unit is second
(s).
Table 1. The performance in the Anatomy track
Matcher Runtime Precision Recall Recall+ F-Measure
Lily 430 0.901 0.902 0.747 0.901
This year, We use parallel computing to futher reduce the time consumption.
Compared with the result in OAEI 2020 [5], there are some improvements in
runtime. However, as can be seen in the overall result, there are still some gaps
compared with the state-of-art system which indicates it is still possible to make
further progress. The further exploration of external knowledge will be leveraged
in the future for the better results.
2.2 Conference track
In this track, there are 7 independent ontologies that can be matched with one
another. The 21 subtasks are based on given reference alignments. As a result of
heterogeneous characters, it is a challenge to generate high-quality alignments
for all ontology pairs in this track.
Lily Results for OAEI 2021 5
Lily adopted ontology matching tuning for the Conference track this year.
Table 2 shows its latest performance.
Table 2. The performance in the Conference track
Test Case ID Precision Recall F.5-Measure F1-measure F2-measure
ra1-M1 0.67 0.57 0.65 0.62 0.59
ra1-M3 0.67 0.47 0.62 0.55 0.5
ra2-M1 0.67 0.49 0.62 0.57 0.52
ra2-M3 0.63 0.42 0.63 0.57 0.50
rar2-M1 0.62 0.52 0.6 0.57 0.54
rar2-M3 0.62 0.43 0.57 0.51 0.46
Average 0.65 0.48 0.62 0.57 0.52
Compared with the result in OAEI 2018 [5], there is one very slightly im-
proved its precision but decreased its recall and F1-measure. All the tasks share
the same configurations, so it is possible to generate better alignments by as-
signing the most suitable parameters for each task. The performance of Lily was
even worse than StringEquiv in some tasks. We will further analyze this task
and our system to find out the reason later.
2.3 Spimbench track
This tack is an instance-mactching tack which aims to match instances of cre-
ative works between two boxes. And ontology instances are described through
22 classes, 31 DatatypeProperty and 85 ObjectProperty properties.
There are about 380 instances and 10000 triples in sandbox, and about 1800
CWs and 50000 triples in mainbox.
Table 3. Performance in the spimbench task
Track Matcher Precision Recall F-Measure Time
AML 0.8349 0.8963 0.8645 6446
FTRLIM 0.8543 1.000 0.9214 1525
SANDBOX
LogMap 0.9383 0.7625 0.8413 7483
REMiner 1.0 0.9967 0.9983 7284
Lily 0.9836 1.000 0.9917 2050
AML 0.8386 0.8835 0.8605 38772
FTRL-IM 0.8558 0.9980 0.9215 2247
MAINBOX
LogMap 0.8801 0.7095 0.7856 26782
REMiner 0.9986 0.9966 0.9977 33966
Lily 0.9908 1.000 0.9954 3899
6 Shiyi Zou 1 , Jiajun Liu 2 , Zherui Yang 3 ,Yunyan Hu 2 , Peng Wang 1, 2, 3 †
Lily utilized almost the same startegy to handle these two different size tasks.
We found that creative works in this task was rich in text information such as
titles, descriptions and so on. However, garbled texts and messy codes were mixed
up with normal texts. And Lily relied too much on text similarity calculation
and set a low threshold in this task, which accounted for the low precision.
As is shown in Table 3, Lily outperforms most the others in sandbox and
mainbox. And the results of Lily and REMiner are close, but the running time of
Lily is comparative. Meanwhile, experiments shows that simple ensemble meth-
ods and a low threshold contribute to increase of matching efficiency. Neverthe-
less, compared with FTRL-IM, there is still potential for Lily to speed up in
process of matching.
2.4 Large Biomedical Ontologies
This track consists of finding alignments between the Foundational Model of
Anatomy (FMA), SNOMED CT, and the National Cancer Institute Thesaurus
(NCI). These ontologies are semantically rich and contain tens of thousands of
classes. Table 4 shows its latest performance.
Table 4. The performance in the Largebio track
Track type Precision Recall F-Measure Time
small fragments 0.9055 0.8276 0.8648 452
FMA-NCI
large fragments -1 - - -
small fragments 0.6211 0.5304 0.5721 2340
FMA-SNOMED
large fragments - - - -
small fragments - - - -
SNOMED-NCI
large fragments - - - -
1
For specific data, please refer to the website
http://oaei.ontologymatching.org/2021/results/
This year, for the first time, Lily system participates in the Large Biomed-
ical Ontologies track. Although its performance is lower than some of systems,
Lily achieved competitive results in terms of recall. In addition, its runtime is
very close to those excellent systems. However, it still falls short of the best
systems in terms of runtime and performance, and we will continue to consider
improvements in the future.
3 General comments
In this year, some modifications were done to Lily for both effectiveness and
efficiency. Especially, we participate in the Large Biomedical Ontologies track
and had a good result. The performance has been improved as we have expected.
The strategies for new tasks have been proved to be useful.
Lily Results for OAEI 2021 7
On the whole, Lily is a comprehensive ontology matching system with the
ability to handle multiple types of ontology matching tasks, of which the results
are generally competitive. However, Lily still lacks in strategies for some newly
developed matching tasks. The relatively high time and memory consumption
also prevent Lily from finishing some challenging tasks.
4 Conclusion
In this paper, we briefly introduced our ontology matching system Lily. The
matching process and the special techniques used by Lily were presented, and
the alignment results were carefully analyzed.
There is still so much to do to make further progress. Lily needs more opti-
mization to handle large ontologies with limited time and memory. Thus, tech-
niques like parallelization will be applied more. Also, we have just tried out
ontology matching tuning. With further research on that, Lily will not only
produce better alignments for tracks it was intended for, but also be able to
participate in the interactive track.
References
1. Wang, P., Xu, B.: An effective similarity propagation model for matching ontologies
without sufficient or regular linguistic information. In: The 4th Asian Semantic
Web Conference (ASWC2009), Shanghai, China (2009)
2. Wang, P., Xu, B.: Debugging ontology mappings: a static approach. Computing
and Informatics 27(1), 21–36 (2012)
3. Yang, P., Wang, P., Ji, L., Chen, X., Huang, K., Yu, B.: Ontology matching tuning
based on particle swarm optimization: Preliminary results. In: Chinese Semantic
Web and Web Science Conference. pp. 146–155. Springer (2014)
4. Wu, J., Pan, Z., Zhang, C., Wang, P.: Lily results for OAEI 2019. In: Proceedings of
the 14th International Workshop on Ontology Matching co-located with the 18th
International Semantic Web Conference (ISWC 2019), Auckland, New Zealand,
October 26, 2019. vol. 2536, pp. 153–159. CEUR-WS.org (2019)
5. Tang, Y., Wang, P., Pan, Z., Liu, H.: Lily results for OAEI 2018. In: Proceedings of
the 13th International Workshop on Ontology Matching co-located with the 17th
International Semantic Web Conference, OM@ISWC 2018, Monterey, CA, USA,
October 8, 2018. pp. 179–186 (2018)
6. Wang, P., Wang, W.: Lily results for OAEI 2016. In: Proceedings of the 11th
International Workshop on Ontology Matching co-located with the 15th Interna-
tional Semantic Web Conference (ISWC 2016), Kobe, Japan, October 18, 2016.
pp. 178–184 (2016)
7. Wang, W., Wang, P.: Lily results for OAEI 2015. In: Proceedings of the 10th Inter-
national Workshop on Ontology Matching collocated with the 14th International
Semantic Web Conference (ISWC 2015), Bethlehem, PA, USA, October 12, 2015.
pp. 162–170 (2015)
8. Wang, P.: Lily results on SEALS platform for OAEI 2011. In: Proceedings of the
6th International Workshop on Ontology Matching, Bonn, Germany, October 24,
2011 (2011)
8 Shiyi Zou 1 , Jiajun Liu 2 , Zherui Yang 3 ,Yunyan Hu 2 , Peng Wang 1, 2, 3 †
9. Wang, P., Xu, B.: Lily: Ontology alignment results for OAEI 2009. In: Proceedings
of the 4th International Workshop on Ontology Matching (OM-2009) collocated
with the 8th International Semantic Web Conference (ISWC-2009) Chantilly, USA,
October 25, 2009 (2009)
10. Wang, P., Xu, B.: Lily: Ontology alignment results for OAEI 2008. In: Proceedings
of the 3rd International Workshop on Ontology Matching (OM-2008) Collocated
with the 7th International Semantic Web Conference (ISWC-2008), Karlsruhe,
Germany, October 26, 2008 (2008)
11. Wang, P., Xu, B.: LILY: the results for the ontology alignment contest OAEI 2007.
In: Proceedings of the 2nd International Workshop on Ontology Matching (OM-
2007) Collocated with the 6th International Semantic Web Conference (ISWC-
2007) and the 2nd Asian Semantic Web Conference (ASWC-2007), Busan, Korea,
November 11, 2007 (2007)
12. Wang, P., Xu, B., Zhou, Y.: Extracting semantic subgraphs to capture the real
meanings of ontology elements. Tsinghua Science and Technology 15(6), 724–733
(2010)
13. Miller, G.A.: Wordnet: A lexical database for english. Communications of the ACM
38(11), 39–41 (1995)
14. Mungall, C.J., Torniai, C., Gkoutos, G.V., et al.: Uberon, an integrative multi-
species anatomy ontology. Genome biology 13(1), R5 (2012)
15. Wang, P., Xu, B.: Matching weak informative ontologies. Sci China Inf Sci 64(12),
229101 (2021)
16. Wang, P., Zhou, Y., Xu, B.: Matching large ontologies based on reduction anchors.
In: IJCAI 2011, Proceedings of the 22nd International Joint Conference on Ar-
tificial Intelligence, Barcelona, Catalonia, Spain, July 16-22, 2011. pp. 2343–2348
(2011)