Matching Ontologies for Air Tra c Management: A Comparison and Reference Alignment of the AIRM and NASA ATM Ontologies

Abstract

Air tra c management (ATM) relies on the timely exchange of information between stakeholders to ensure safety and e ciency of air tra c operations. In an e ort to achieve semantic interoperability within ATM, the Single European Sky ATM Research (SESAR) program has developed the ATM Information Reference Model (AIRM), which individual information exchange models should comply with. An OWL representation of the AIRM { the AIRM Ontology (AIRM-O) { facilitates applications. Independently from the European e orts, the NASA Air Tra c Management Ontology (ATMONTO) has been developed as an RDF/OWL ontology representing ATM concepts to facilitate data integration and analysis in support of NASA aeronautics research. Conceptualization mismatches between the AIRM-O and ATMONTO ontologies { mostly due to di erent design decisions, but also as a consequence of the di erent regulatory systems and philosophies underlying ATM in Europe and the United States { pose a challenge to automatic ontology matching algorithms. In this paper, we describe mismatches between AIRM-O and ATMONTO, evaluate performance of automatic matching systems over these ontologies, and provide a manual reference alignment.

-

Modern air tra c management (ATM) employs standardized models for the exchange of information required for seamless air tra c operations. Each exchange model has a di erent focus. The Aeronautical Information Exchange Model (AIXM) [ 1 ], for example, facilitates the representation of messages for pilots and air tra c controllers notifying of important events such as temporary runway closures and malfunctions of navigation aids. The exchange models are Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). subject to constant evolution in various standards working groups. In this regard, maintaining consistent co-evolution of the di erent exchange models is a necessity not only to guarantee e ciency of operations { by ensuring interoperability of systems { but also for safety reasons.

Recognizing the necessity of a common reference for the constantly evolving exchange models, the Single European Sky ATM Research (SESAR) program established the ATM Information Reference Model (AIRM) [ 25 ], developed under supervision of EUROCONTROL in an e ort with industry and academia but meanwhile also adopted by the International Civil Aviation Organization (ICAO). The individual exchange models must ensure compliance with AIRM.

The AIRM Ontology (AIRM-O) [ 21 ] is an OWL ontology derived from the UML representation of AIRM in an e ort to facilitate operationalization of AIRM. In this regard, previous work has investigated automatic compliance validation between exchange models and AIRM [ 22 ] as well as the annotation of ATM data sources with a semantic description of the contents [ 15 ].

The NASA Air Tra c Management Ontology (ATMONTO) [ 12, 13 ] supports NASA's aeronautics research activities by facilitating integration of data from various sources for analysis purposes. Developed independently from AIRM with a di erent purpose and under a di erent regulatory system { the United States instead of Europe { the question arises to what extent ATMONTO is actually compatible with AIRM-O.

In order to link AIRM-O and ATMONTO, we manually produced a reference alignment between these ontologies. In the course of the alignment process, we identi ed di erent types of mismatches between AIRM-O and ATMONTO, which we relate to existing mismatch classi cations from literature. During the manual mapping process, we also experimented with state-of-the-art ontology matching systems. Some of the encountered mismatches pose a serious challenge for automatic ontology matching systems. According to the results from some of the benchmarks organised by the Ontology Alignment Evaluation Initiative (OAEI), the performance of ontology matching systems has improved signi cantly over recent years [ 7 ]. In some tracks, several of the competing systems achieve close to perfect F-measure [ 5 ], i.e., they are able to identify almost all relations in the track's ground truth alignment without producing false positives. Matching the two ATM ontologies, however, proved somewhat di cult for these systems. Some of the tested systems identi ed very few but correct relations whereas others identi ed a couple of more correct relations, but included too many incorrect relations. The reference alignment between ATMONTO and AIRM-O may serve the ontology matching community as a gold standard for improving and evaluating matching algorithms.

The remainder of this paper is organized as follows. In Sect. 2 we present relevant background information about the investigated ATM ontologies. In Sect. 3 we introduce a reference alignment between ATMONTO and AIRM-O. In Sect. 4 we identify mismatches between the ontologies. In Sect. 5 we evaluate performance of automatic matching systems. In Sect. 6 we review related work. We conclude with a summary and an outlook on future work.

Ontologies for Air Tra c Management

The AIRM addresses the issue of semantic interoperability between ATM systems through harmonized and agreed upon de nitions of the information being exchanged in ATM [ 25 ]. The exchanged ATM information must comply with the AIRM de nitions, the individual exchange models are aligned with the AIRM. AIRM is de ned in UML, the various diagrams falling into the following subject elds: AirTra cOperations, Aircraft, AirspaceInfrastructure, BaseInfrastructure, Common, Environment, Flight, Meteorology, Stakeholders, and Surveillance. The subject elds represent speci c concerns of ATM.

In order to facilitate application of AIRM in practice, the SESAR exploratory research project BEST6 developed the AIRM Ontology (AIRM-O) [ 21 ]. AIRM-O has been semi-automatically derived from the XML Metadata Interchange (XMI) representation of the AIRM UML diagrams using manual preprocessing and XSL Transformation (XSLT) scripts to obtain an OWL ontology. The transformation of the AIRM UML diagrams into an OWL ontology follows the Object Management Group's guidelines from the Ontology De nition Metamodel [ 17 ].

Independently from AIRM, ATMONTO was developed in the context of NASA's aeronautics research activities as a facilitator for data integration and analysis. ATMONTO supports semantic integration of ATM data being collected and analyzed at NASA for research and development purposes. The ontology functions as an integrative superstructure upon which to overlay data from multiple stove-piped aviation data sources, thus enabling cross-source queries that would be otherwise time-consuming and costly. ATMONTO includes a wide range of classes, properties, and relationships covering aspects of ight and navigation, aircraft equipment and systems, airspace infrastructure, meteorology, air tra c management initiatives, and other areas.

Development of ATMONTO followed a classic knowledge modeling approach. First, domain experts identi ed a core set of aviation data sources to be integrated. After an analysis of these sources, a proposed set of ATM concepts, properties, and relations was developed and presented to the experts for critique. The corresponding revisions led to an initial version of ATMONTO. Since this version was built in a bottom-up fashion driven by a need to accommodate the core data sources, the initial ontology did not represent the full complexity of the ATM domain. Gradually, additional data sources were incorporated, thereby revising and extending ATMONTO's set of concepts, properties, and relations. By the end of the development process, more than ten di erent data sources were covered by the ontology, and ATMONTO's structure had been generalized well beyond those sources. Although a general model of the ATM domain, ATMONTO's development was heavily driven by application requirements. In turn, AIRMO's scope is overall broader than ATMONTO's since AIRM has been subject to a more coordinated standardization and governance process inside SESAR, harmonizing the various ATM information exchange models. 6 Achieving the Bene ts of SWIM by Making Smart Use of Semantic Technologies, https://project-best.eu/ In order to develop a reference alignment between AIRM-O and ATMONTO, a panel of six experts, each having experience within the ATM domain and knowledge of semantic technologies, collaboratively produced a mapping between concepts of the two ontologies. All the experts were asked to match each of the 157 classes in ATMONTO to corresponding classes in the larger AIRM-O { see Table 1 for statistics about the size of the ontologies { by making use of the experts' own domain knowledge as well as all available input, including descriptive class and property annotations in the ontologies and informative web resources such as Skybrary7.

ATMONTO AIRM-O

In addition to identifying equivalence classes, each expert also indicated subsumption relationships between concepts as well as potential mismatches of varying degree (see Sect. 4). After the initial matches were compiled, two of the ve experts in the panel reviewed the matches for each ATMONTO class and produced a consensus mapping holding equivalence relations between classes from the ontologies. With the consensus mapping as a starting point, the reference alignment was developed using the following approach: 1. Develop equivalence reference alignment. The consensus mapping described above is formatted in RDF/XML according to the Alignment Format [ 3 ]. 2. Develop subsumption reference alignment. Here, the same procedure as in the OAEI 2011 edition [ 4 ] was followed: The two source ontologies were merged into one single ontology in Protege. Then OWL equivalentClass axioms consistent with the mapping described above were manually added between the corresponding classes in the merged ontology. An automated reasoner (HermiT) performed subsumption reasoning over the classes in the merged ontology in order to infer subsumption relations. In addition, subsumption mappings that were discovered in the manual mapping process but not identi ed by the reasoner were included in the reference alignment. 3. Evaluate reference alignments. Once both reference alignments were complete they were manually inspected for errors and inconsistencies.

The reference alignment between ATMONTO and AIRM-O [ 20 ] comes as two separate alignment les, one holding only equivalence relations and the other holding only subsumption relations. The equivalence reference alignment 7 https://www.skybrary.aero/ contains 32 relations in total and the subsumption reference alignment contains 83 subsumption relations. Only direct subsumption relationships were considered in the subsumption reference alignment, following the convention used during the development of the reference alignment for the Oriented Matching track arranged in OAEI 2011 [ 4 ]. 4

Mismatches between AIRM-O and ATMONTO

In the course of conducting the manual alignment of ATMONTO and AIRM-O (see Sect. 3), several of the identi ed candidate equivalence relations were considered \light matches" at rst. In these cases, an equivalence relation between the classes was often deemed too strong { despite lexically similar class names hinting at a relation { given that the experts performed poorly on the alignment task { as judged by the two reviewing experts. Extensive discussions among the experts involved in the matching exercise revealed that similar class names were no guarantee of a correct match. In fact, in approximately 25% of the identi ed exact-match pairs in the nal reference alignment, the class names did not have any words in common whereas in approximately 40% of the identi ed \light-match" candidate equivalence relations the class names did have words in common. This may explain partly why automated alignment techniques focusing on class name similarity did not perform particularly well (see Sect. 5).

The initially identi ed \light matches" between ATMONTO and AIRM-O actually represent ontology mismatches. Multiple classi cation systems for mismatchs with varying degrees of detail and often considerable overlap exist in literature. Figure 1 shows a classi cation of mismatch types synthesized from Klein [ 14 ] and Visser et al. [ 23, 24 ] along with mismatch types encountered during the manual matching between ATMONTO and AIRM-O. Notwithstanding the di erences between classi cation systems, there seems to be consensus that the development of an ontology involves two separate processes and, correspondingly, two broad categories of mismatches can be distinguished [ 23, 24 ]. First, conceptualization mismatches are the result of di erent interpretations of the represented domain, leading to di erent classes, individuals, and relations being modeled in di erent ontologies for the same domain. Explication mismatches, on the other hand, are the result of di erent speci cations of domain interpretations in form of di erent terms, modeling styles, and encodings being employed.

One category of conceptualization mismatches concerns di erences in model coverage and scope between ontologies from the same domain, which occur when two ontologies cover di erent parts of that domain or the same part at different levels of detail. In this regard, a structure mismatch occurs when two ontologies distinguishing the same set of classes di er in how they are structured through relations; we could not nd a clear case of structure mismatch between ATMONTO and AIRM-O. A mismatch concerning di ering levels of detail occurs when one class is modeled in more depth and with greater delity than the other. The ASP M eteorologicalCondition class from ATMONTO and AerodromeCondition from AIRM-O, for example, both represent meteorological

Conceptualisation

Explication

Ontology Mismatches Structure Mismatch

Differing Level of Detail Differing Intended Use

Differing Scope Categorization Mismatch Aggregation-level Mismatch

Differing Level of Abstraction Concept Scope

Homonyms Model Coverage and Granularity

Terminological

Modeling Style

Encoding Synonyms

Concept Description

Attribute-type Mismatch Terms Mismatch Terms & Definiens Mismatch Concept & Definiens Mismatch Concept Mismatch

Differing Word Senses

Paradigm

Attribute Assignment Mismatch

Differing Standards Definiens Mismatch Differing Representation information. ASP M eteorologicalCondition, however, is more detailed, comprising all aspects of sky, wind, visibility, and weather whereas AerodromeConditon is limited to sky conditions. Di erent properties and relations of similar classes may also re ect di erences in how the classes are to be used in the context of a domain application (di ering intended uses). For example, ReRouteSegment in ATMONTO describes an alternative air route option for contingency planning purposes, whereas RouteSegment describes an actual portion of a route being own. Eventually, the di ering scope of ontologies may result in a class from the source ontology lacking a matching class in the target ontology because the class from the source ontology lies outside the de ned scope of the target ontology. An example of a di ering scope is the missing equivalent in AIRM-O for the class DelayM odel in ATMONTO, which speci es a numerical model of airspace delay under speci c tra c conditions. There is no matching class in AIRM-O because modeling concerns fall outside the scope of this ontology.

A concept scope conceptualization mismatch occurs when two classes seem to represent the same concept, yet do not cover exactly the same instances, although the classes intersect. Categorization mismatches and aggregation-level mismatches fall into the concept scope mismatch category. A categorization mismatch occurs when two ontologies include the same class, but each ontology decomposes the class into di erent subclasses. ATMONTO's Airport is equivalent to AIRM-O's Aerodrome, however due to di erent geographical and applicationwise scope Airport includes the subclasses U Sairport and InternationalAirport whereas Aerodrome has no such subclasses. An aggregation-level mismatch occurs when two ontologies de ne the same underlying concept using classes at di erent levels of abstraction. A di ering level of abstraction is encountered when the matched classes intersect but some instances are outside the intersection. Consider, for example, AviationIndustryM anuf acturer in ATMONTO and AerospaceM anuf acturer in AIRM-O. In this case, the term \Aerospace" has a broader meaning than \Aviation", hinting at a subsumption relation.

The class of explication mismatches encompasses terminological, modeling style, and encoding mismatches. In this regard, an encoding mismatch relates to how the ontologies employ di erent formatting when describing instances, e.g., describing an instance either in miles or kilometres [ 14 ]; we omit this mismatch type in the remainder of this analysis. More relevant for our analysis are the terminological and modeling-style mismatches identi ed by Visser et al. [ 23, 24 ], which occur due to di erent knowledge de nitions used in the ontologies and their associated concepts.

The category of terminological mismatches comprises mismatches related to synonyms and homonyms. The synonym mismatch as explained by Klein [ 14 ] refers to two lexically di erent terms in fact meaning the same thing (e.g. `Airport/Heliport' versus `Aerodrome'), so we do not consider this a real mismatch in our analysis. Term mismatches as well as terms-and-de niens mismatches de ned by Visser et al. [ 23, 24 ] belong to the synonym mismatches. A term mismatch occurs when the de nitions share the same concept and the same de niens, but the terms are di erent. Correspondingly, a term-and-de niens mismatch occurs when the de nitions refer to the same underlying concept, but the terms and de niens are di erent. The relation between Airport in ATMONTO and Aerodrome in AIRM-O could also be considered a terms-and-de niens mismatch.

Mismatches related to homonyms occur when the meaning of two identical terms is di erent (e.g. the term `Conductor' has a di erent meaning in music than in electrical engineering). We refer to homonym mismatches proper as di ering word senses. There were a few incidents of homonymy that complicated the alignment process for ATMONTO and AIRM-O. For example, the term \Flow" had a slightly di erent meaning in ATMONTO and AIRM-O. In AIRM-O, a ow is a tra c pattern, while in ATMONTO ow is a concrete measurement of the number of aircraft per time unit traversing a volume of airspace.The classes have an exact or close lexical match, but the two classes correspond to two di erent word senses.

Modeling style mismatches are further decomposed into concept description and paradigm mismatches. A concept description mismatch occurs when two similar concepts are modelled di erently, e.g., that the same intention is modelled through the use of properties in one ontology and by using distinct sub-classes for the same target values in the other ontology [ 6 ]. A speci c type of concept description mismatch between ATMONTO and AIRM-O is classes with similar names de ning di erent versions of the same concept based on di ering technical standards adopted by ontology developers, e.g., by FAA and EUROCONTROL. Finally, paradigm mismatches refer to how di erent paradigms can be used to represent concepts such as time, action, plans, causality, propositional attitudes, etc. For example, one ontology might use temporal representations based on interval logic, while another might use a representation based on points [ 6 ]. Paradigm mismatches relate to what we call \di ering representation", and one example of such a mismatch is between P lannedF lightRoute in ATMONTO and T rajectory in AIRM-O. These two classes are used to represent the planned aircraft trajectory (or ight plan). In AIRM-O, the planned trajectory is composed of a sequence of trajectory points, elements, segments, and constraints. In ATMONTO, the ight plan is speci ed using a hierarchically decomposable route structure. These are fundamentally di erent methods of representing a planned route, based on di erent conceptual models of what constitutes a route. 5

Performance of Automatic Matching Systems

We challenged three matching systems that normally rank highly on several tracks of the OAEI campaigns on the equivalence reference alignment: { AgreementMakerLight (AML) [ 9 ]. We ran AML using the GUI version from 20168 and the \Automatic Match" mode, letting AML handle the con guration of individual matching algorithms and external background sources (e.g. WordNet). AML includes terminological, structural and lexical matchers and uses WordNet as a general-purpose lexical resource as well as the Doid and Uberon ontologies for matching of biomedical ontologies. Property relations included in the produced alignment were disregarded when evaluating the performance of AML. { LogMap [ 11 ]. We used the latest available standalone distribution of LogMap9 with default matching parameters. LogMap combines terminological matching with capabilities for diagnosing and repairing incoherent alignments. Optionally, LogMap can also employ external resources such as WordNet. As with AML there were some property relations included in the produced alignment, which we do not consider in the evaluation. { YAM++ [ 16 ]. YAM++ is provided as a web application10. We used the default matcher parameters, which included both an element-level and a structure-level matching algorithm.

The evaluation results from running the matching systems on the equivalence reference alignment are shown in Figure 2. As the gure shows, all three systems manage to avoid many false positives, especially LogMap which obtains perfect precision with no false positives. All three systems obtain a recall of 0.31. The results reveal that all three matching systems are able to correctly detect the true positive relations where the source and target classes are exact string matches. All three matchers also capture one relation where the source class (SID ) is an acronym of the target class (StandardInstrumentDeparture) due to the fact that \Standard Instrument Departure" is expressed in the label of the source class. The remaining relations in the reference alignment are not detected by these systems.

A closer inspection of the alignments produced by these three matching systems with respect to the equivalence reference alignment reveals that the following factors contribute to making this a challenging dataset: 8 There was an issue with the dependency to the Gephi Toolkit that prevented us from using the most recent version of AML. 9 https://sourceforge.net/projects/logmap-matcher/ les/ 10 http://yamplusplus.lirmm.fr/index { Domain-speci c and technical terminology. Most of the classes in both ontologies describe aviation-speci c concepts and technical terms. Often the class names and their natural language de nitions include acronyms and abbreviations used only in aviation. Considering that typically used lexical resources (such as the aforementioned WordNet) have low coverage of technical terminology, this constitutes a challenge for matching systems. { Compound class names. Several of the classes involved in the relations represented in the reference alignment contains equal substrings, a feature often exploited by string-matching techniques. However, in most relations one or both class names are compound words, such as PhysicalRunway - Runway or AircraftModel - AircraftMakeModelSeries, resulting in a low similarity scores for algorithms based on basic substring analysis. Here, a more comprehensive string-based analysis is required to identify such relations, possibly resulting in the unwanted e ect that additional false positive relations are being included in the computed alignment as well. { Synonymy, homonymy and polysemy. The two ontologies use synonymous terms for concepts with the same meaning (e.g. Airport vs. Aerodrome). Synonymy can often be resolved using lexicons or other external sources (e.g. other ontologies). Homonymy and polysemy are more of a challenge to solve. Some of the class names in these two ontologies can have a di erent meaning outside the ATM domain. Examples of this are Gate, Taxi or Star (which is short for Standard Terminal Arrival Route in the ATM world) and such challenges are not addressed through the use of lexicons such as WordNet.

Evaluation of equivalence reference alignment 1,00 0,90 0,80 0,70 0,60 0,50 0,40 0,30 0,20 0,10 0,00

AML

Precision

LogMap Recall F-measure

YAM++ Fig. 2. Performance of selected state-of-the-art matchers over ATMONTO and AIRM-O Evaluation datasets that include reference alignments declaring the correct set of mappings between ontologies are important for the continued improvement of ontology matching techniques. The OAEI provides an annual standardised evaluation process for matching system. However, with only a few exceptions over the years, the OAEI tracks mainly involve one-to-one equivalence relations, neglecting other semantic relations and complex correspondences whose identi cation is important for more profound integration processes [ 8, 18 ]. One of these OAEI tracks is the Conference Track, a widely used benchmark for ontology matching systems, that since its inception in 2005 has been subject to many revisions [ 26 ]. This track now includes 16 ontologies describing conference organization and there are two versions of reference alignments, all holding one-to-one equivalence relations. The rst version is referred to as \crisp" alignments where all con dence values are 1.0. The second version is referred to as an \uncertain" version of the reference alignment where the con dence values re ect the opinion from a group of human experts [ 7 ].

For the 2018 OAEI campaign, a complex alignment track was launched, o ering reference alignments holding complex relations in four di erent datasets. One of the datasets included complex reference alignments for some of the ontologies in the Conference Track [ 19 ]. The other datasets represented real-world ontologies from the domains of hydrography, plants and species, and geoscience. Having real-world ontologies in benchmarks is important because such ontologies may expose issues arising in practice which may be overlooked by the developers of (semi-)arti cial benchmarks [ 27 ]. 7

Summary and Future Work

We contrasted AIRM-O with the ATMONTO. Mismatches between these ontologies coupled with the complex and diverse nature of the ATM domain, which covers many technical subject elds, renders automatic ontology matching di cult. The presented manual alignment of AIRM-O and ATMONTO potentially facilitates integration of datasets in di erent formats, e.g., NASA aeronautics research data with ATM information in the operational System Wide Information Management (SWIM) network. As a byproduct, the ontology matching community gains access to a reference alignment for two complex real-world ontologies from the ATM domain. We refer to a separate publication [ 10 ] for a more detailed comparison of AIRM-O and ATMONTO from an ATM perspective.

Future work will investigate the potential for complex reference alignments between AIRM-O and ATMONTO beyond simple equivalence and subsumption relations. using the Expressive and Declarative Ontology Alignment Language (EDOAL) [ 2 ]. During the manual mapping process, we identi ed a large number of complex relations, e.g., class-to-property relations and many-to-many relations, which additional reference alignments can be developed from. In this regard, complex matching represents an area with a potential for signi cantly advancing the state-of-the-art in ontology matching. Acknowledgments. We thank Scott Wilson from EUROCONTROL and Joe Gorman from SINTEF for their contributions to the reference alignment. Part of this work was conducted as part of the BEST project. This project received funding from the SESAR Joint Undertaking under grant agreement No 699298 under the European Union's Horizon 2020 research and innovation program. This work was also supported by the NASA Airspace Operations and Safety Program. The views expressed in this paper are those of the authors.

References

1. Aeronautical Information Exchange Model. http://aixm.aero/ (Accessed: 26 August 2019 )

2. EDOAL: Expressive and Declarative Ontology Alignment Language . http://alignapi.gforge.inria.fr/edoal. html (Accessed: 26 August 2019 )

3. A format for ontology alignment . http://alignapi.gforge.inria.fr/format. html (Accessed: 26 August 2019 )

4. Ontology Alignment Evaluation Initiative: Oriented Matching 2011 . http://oaei.ontologymatching.org/2011/oriented/index. html (Accessed: 26 August 2019 )

5. Achichi , M. , et al.: Results of the Ontology Alignment Evaluation Initiative 2017 . In: Shvaiko, P. , Euzenat , J. , Jimenez-Ruiz , E. , Cheatham , M. , Hassanzadeh , O . (eds.) Proceedings of the 12th International Workshop on Ontology Matching. CEUR Workshop Proceedings , vol. 2032 , pp. 61 { 113 . CEUR-WS.org ( 2017 )

6. Chalupsky , H.: OntoMorph: A translation system for symbolic knowledge . In: Cohn, A.G. , Giunchiglia , F. , Selman , B . (eds.) Proceedings of the Seventh International Conference on Principles of Knowledge Representation and Reasoning (KR 2000 ). pp. 471 { 482 . Morgan Kaufmann

7. Cheatham , M. , Hitzler , P. : Conference v2. 0: An uncertain version of the OAEI conference benchmark . In: Mika, P. , Tudorache , T. , Bernstein , A. , Welty , C. , Knoblock , C.A. , Vrandecic , D. , Groth , P.T. , Noy , N.F. , Janowicz , K. , Goble , C.A. (eds.) ISWC 2014 . LNCS , vol. 8797 , pp. 33 { 48 . Springer ( 2014 )

8. Cruz , I.F. , Palmonari , M. , Caimi , F. , Stroe , C. : Building linked ontologies with high precision using subclass mapping discovery . Arti cial Intelligence Review 40 ( 2 ), 127 { 145 ( 2013 )

9. Faria , D. , Pesquita , C. , Santos , E. , Palmonari , M. , Cruz , I.F. , Couto , F.M.: The agreementmakerlight ontology matching system . In: OTM Confederated International Conferences" On the Move to Meaningful Internet Systems" . pp. 527 { 541 . Springer ( 2013 )

10. Gringinger , E., Keller, R.M., Vennesland , A. , Schuetz , C.G. , Neumayr , B. : A comparative study of two complex ontologies in air tra c management . In: Proceedings of the AIAA/IEEE 38th Digital Avionics Systems Conference (DASC) ( 2019 ), in press

11. Jimenez-Ruiz , E. , Cuenca Grau , B. : LogMap: Logic-based and scalable ontology matching . In: ISWC 2011. LNCS , vol. 7031 , pp. 273 { 288 . Springer ( 2011 )

12. Keller, R.M.: The NASA Air Tra c Management Ontology . https://data.nasa.gov/ontologies/atmonto (March 2018 )

13. Keller, R.M.: Building a knowledge graph for the air tra c management community . In: Companion of The 2019 World Wide Web Conference . pp. 700 { 704 ( 2019 )

14. Klein , M.: Combining and relating ontologies: an analysis of problems and solutions . In: Proceedings of the IJCAI-01 Workshop on Ontologies and Information Sharing ( 2001 )

15. Neumayr , B. , Gringinger , E. , Schuetz , C.G. , Schre , M. , Wilson, S. , Vennesland , A. : Semantic data containers for realizing the full potential of system wide information management . In: Proceedings of the 36th IEEE/AIAA Digital Avionics Systems Conference (DASC) ( 2017 )

16. Ngo , D. , Bellahsene , Z. : Overview of YAM++ { (not) yet another matcher for ontology alignment task . Web Semantics: Science, Services and Agents on the World Wide Web 41 , 30 { 49 ( 2016 )

17. Object Management Group: Ontology De nition Metamodel v1. 1 ( 2014 ), https://www.omg.org/spec/ODM/1.1/

18. Spiliopoulos , V. , Vouros , G.A. , Karkaletsis , V.: On the discovery of subsumption relations for the alignment of ontologies . Journal of Web Semantics 8 ( 1 ), 69 { 88 ( 2010 )

19. Thieblin , E. , Haemmerle , O. , Hernandez , N. , Trojahn , C. : Task-oriented complex ontology alignment: Two alignment evaluation sets . In: European Semantic Web Conference . pp. 655 { 670 . Springer ( 2018 )

20. Vennesland , A. , Keller, R., Gringinger , E. , Schuetz , C.G. , Neumayr , B. , Wilson, S. , Gorman , J.: ATMONTO2AIRM A reference alignment between the NASA ATM Ontolgy and the ATM Information Reference Model Ontology . https://w3id.org/airm-o/atmonto2airm/ ( 2019 )

21. Vennesland , A. , Neumayr , B. , Schuetz , C.G. , Savulov , A. , Wilson, S. , Gringinger , E. , Gorman , J.: AIRM-O { ATM Information Reference Model Ontology . https://w3id.org/airm-o/ontology/ ( 2017 )

22. Vennesland , A. , Gorman , J. , Wilson, S. , Neumayr , B. , Schuetz , C.G. : Automated compliance veri cation in ATM using principles from ontology matching . In: Aveiro, D. , Dietz , J.L.G. , Filipe , J . (eds.) Proceedings of the 10th International Joint Conference on Knowledge Discovery , Knowledge Engineering and Knowledge Management, IC3K . pp. 39 { 50 . SciTePress ( 2018 )

23. Visser , P.R.S. , Jones , D.M. , Bench-Capon , T.J.M. , Shave , M.J.R.: An analysis of ontology mismatches; heterogeneity versus interoperability . In: Proceedings of the AAAI 1997 Spring Symposium on Ontological Engineering . pp. 164 { 72 ( 1997 )

24. Visser , P.R.S. , Jones , D.M. , Bench-Capon , T.J.M. , Shave , M.J.R. : Assessing heterogeneity by classifying ontology mismatches . In: Guarino, N. (ed.) Proceedings of the 1st International Conference on Formal Ontology in Information Systems . IOS Press ( 1998 )

25. Wilson, S. , Suzi , R., Van der Stricht, S.: The SESAR ATM information reference model within the new ATM system . In: Proceedings of the 2014 Integrated Communications, Navigation and Surveillance Conference (ICNS) Conference . pp. L3{1{L3{13 ( 2014 )

26. Zamazal , O. , Svatek , V. : The ten-year ontofarm and its fertilization within the onto-sphere . Journal of Web Semantics 43 , 46 { 53 ( 2017 )

27. Zhou , L. , Cheatham , M. , Krisnadhi , A. , Hitzler , P.: A complex alignment benchmark: GeoLink dataset . In: Vrandecic, D. , Bontcheva , K. , Suarez-Figueroa , M.C. , Presutti , V. , Celino , I. , Sabou , M. , Ka ee , L., Simperl , E. (eds.) ISWC 2018 . LNCS , vol. 11137 , pp. 273 { 288 . Springer ( 2018 )