Matching Ontologies for Air Traffic Management: A Comparison and Reference Alignment of the AIRM and NASA ATM Ontologies Audun Vennesland1,2 , Richard M. Keller3 , Christoph G. Schuetz4(B) , Eduard Gringinger5 , and Bernd Neumayr4 1 Norwegian University of Science and Technology audun.vennesland@ntnu.no 2 SINTEF, Trondheim, Norway 3 Intelligent Systems Division, NASA Ames Research Center, Moffett Field, CA, USA rich.keller@nasa.gov 4 Johannes Kepler University Linz, Linz, Austria {christoph.schuetz,bernd.neumayr}@jku.at 5 Frequentis AG, Vienna, Austria eduard.gringinger@frequentis.com Abstract. Air traffic management (ATM) relies on the timely exchange of information between stakeholders to ensure safety and efficiency of air traffic operations. In an effort to achieve semantic interoperability within ATM, the Single European Sky ATM Research (SESAR) program has developed the ATM Information Reference Model (AIRM), which individual information exchange models should comply with. An OWL representation of the AIRM – the AIRM Ontology (AIRM-O) – facili- tates applications. Independently from the European efforts, the NASA Air Traffic Management Ontology (ATMONTO) has been developed as an RDF/OWL ontology representing ATM concepts to facilitate data integration and analysis in support of NASA aeronautics research. Concep- tualization mismatches between the AIRM-O and ATMONTO ontologies – mostly due to different design decisions, but also as a consequence of the different regulatory systems and philosophies underlying ATM in Europe and the United States – pose a challenge to automatic ontology matching algorithms. In this paper, we describe mismatches between AIRM-O and ATMONTO, evaluate performance of automatic matching systems over these ontologies, and provide a manual reference alignment. 1 Introduction Modern air traffic management (ATM) employs standardized models for the exchange of information required for seamless air traffic operations. Each ex- change model has a different focus. The Aeronautical Information Exchange Model (AIXM) [1], for example, facilitates the representation of messages for pilots and air traffic controllers notifying of important events such as temporary runway closures and malfunctions of navigation aids. The exchange models are Copyright c 2019 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 2 A. Vennesland et al. subject to constant evolution in various standards working groups. In this regard, maintaining consistent co-evolution of the different exchange models is a necessity not only to guarantee efficiency of operations – by ensuring interoperability of systems – but also for safety reasons. Recognizing the necessity of a common reference for the constantly evolving exchange models, the Single European Sky ATM Research (SESAR) program established the ATM Information Reference Model (AIRM) [25], developed under supervision of EUROCONTROL in an effort with industry and academia but meanwhile also adopted by the International Civil Aviation Organization (ICAO). The individual exchange models must ensure compliance with AIRM. The AIRM Ontology (AIRM-O) [21] is an OWL ontology derived from the UML representation of AIRM in an effort to facilitate operationalization of AIRM. In this regard, previous work has investigated automatic compliance validation between exchange models and AIRM [22] as well as the annotation of ATM data sources with a semantic description of the contents [15]. The NASA Air Traffic Management Ontology (ATMONTO) [12, 13] supports NASA’s aeronautics research activities by facilitating integration of data from various sources for analysis purposes. Developed independently from AIRM with a different purpose and under a different regulatory system – the United States instead of Europe – the question arises to what extent ATMONTO is actually compatible with AIRM-O. In order to link AIRM-O and ATMONTO, we manually produced a reference alignment between these ontologies. In the course of the alignment process, we identified different types of mismatches between AIRM-O and ATMONTO, which we relate to existing mismatch classifications from literature. During the manual mapping process, we also experimented with state-of-the-art ontology matching systems. Some of the encountered mismatches pose a serious challenge for automatic ontology matching systems. According to the results from some of the benchmarks organised by the Ontology Alignment Evaluation Initiative (OAEI), the performance of ontology matching systems has improved significantly over recent years [7]. In some tracks, several of the competing systems achieve close to perfect F-measure [5], i.e., they are able to identify almost all relations in the track’s ground truth alignment without producing false positives. Matching the two ATM ontologies, however, proved somewhat difficult for these systems. Some of the tested systems identified very few but correct relations whereas others identified a couple of more correct relations, but included too many incorrect relations. The reference alignment between ATMONTO and AIRM-O may serve the ontology matching community as a gold standard for improving and evaluating matching algorithms. The remainder of this paper is organized as follows. In Sect. 2 we present relevant background information about the investigated ATM ontologies. In Sect. 3 we introduce a reference alignment between ATMONTO and AIRM-O. In Sect. 4 we identify mismatches between the ontologies. In Sect. 5 we evaluate performance of automatic matching systems. In Sect. 6 we review related work. We conclude with a summary and an outlook on future work. Matching Ontologies for Air Traffic Management 3 2 Ontologies for Air Traffic Management The AIRM addresses the issue of semantic interoperability between ATM sys- tems through harmonized and agreed upon definitions of the information being exchanged in ATM [25]. The exchanged ATM information must comply with the AIRM definitions, the individual exchange models are aligned with the AIRM. AIRM is defined in UML, the various diagrams falling into the following subject fields: AirTrafficOperations, Aircraft, AirspaceInfrastructure, BaseInfrastructure, Common, Environment, Flight, Meteorology, Stakeholders, and Surveillance. The subject fields represent specific concerns of ATM. In order to facilitate application of AIRM in practice, the SESAR exploratory research project BEST6 developed the AIRM Ontology (AIRM-O) [21]. AIRM-O has been semi-automatically derived from the XML Metadata Interchange (XMI) representation of the AIRM UML diagrams using manual preprocessing and XSL Transformation (XSLT) scripts to obtain an OWL ontology. The transformation of the AIRM UML diagrams into an OWL ontology follows the Object Management Group’s guidelines from the Ontology Definition Metamodel [17]. Independently from AIRM, ATMONTO was developed in the context of NASA’s aeronautics research activities as a facilitator for data integration and analysis. ATMONTO supports semantic integration of ATM data being collected and analyzed at NASA for research and development purposes. The ontology functions as an integrative superstructure upon which to overlay data from multiple stove-piped aviation data sources, thus enabling cross-source queries that would be otherwise time-consuming and costly. ATMONTO includes a wide range of classes, properties, and relationships covering aspects of flight and navigation, aircraft equipment and systems, airspace infrastructure, meteorology, air traffic management initiatives, and other areas. Development of ATMONTO followed a classic knowledge modeling approach. First, domain experts identified a core set of aviation data sources to be integrated. After an analysis of these sources, a proposed set of ATM concepts, properties, and relations was developed and presented to the experts for critique. The corresponding revisions led to an initial version of ATMONTO. Since this version was built in a bottom-up fashion driven by a need to accommodate the core data sources, the initial ontology did not represent the full complexity of the ATM domain. Gradually, additional data sources were incorporated, thereby revising and extending ATMONTO’s set of concepts, properties, and relations. By the end of the development process, more than ten different data sources were covered by the ontology, and ATMONTO’s structure had been generalized well beyond those sources. Although a general model of the ATM domain, ATMONTO’s development was heavily driven by application requirements. In turn, AIRM- O’s scope is overall broader than ATMONTO’s since AIRM has been subject to a more coordinated standardization and governance process inside SESAR, harmonizing the various ATM information exchange models. 6 Achieving the Benefits of SWIM by Making Smart Use of Semantic Technologies, https://project-best.eu/ 4 A. Vennesland et al. 3 Reference Alignment In order to develop a reference alignment between AIRM-O and ATMONTO, a panel of six experts, each having experience within the ATM domain and knowledge of semantic technologies, collaboratively produced a mapping between concepts of the two ontologies. All the experts were asked to match each of the 157 classes in ATMONTO to corresponding classes in the larger AIRM-O – see Table 1 for statistics about the size of the ontologies – by making use of the experts’ own domain knowledge as well as all available input, including descriptive class and property annotations in the ontologies and informative web resources such as Skybrary7 . Table 1. Ontology Statistics Classes Object Properties Data Properties ATMONTO 157 126 189 AIRM-O 915 1761 494 In addition to identifying equivalence classes, each expert also indicated subsumption relationships between concepts as well as potential mismatches of varying degree (see Sect. 4). After the initial matches were compiled, two of the five experts in the panel reviewed the matches for each ATMONTO class and produced a consensus mapping holding equivalence relations between classes from the ontologies. With the consensus mapping as a starting point, the reference alignment was developed using the following approach: 1. Develop equivalence reference alignment. The consensus mapping described above is formatted in RDF/XML according to the Alignment Format [3]. 2. Develop subsumption reference alignment. Here, the same procedure as in the OAEI 2011 edition [4] was followed: The two source ontologies were merged into one single ontology in Protégé. Then OWL equivalentClass axioms consistent with the mapping described above were manually added between the corresponding classes in the merged ontology. An automated reasoner (HermiT) performed subsumption reasoning over the classes in the merged ontology in order to infer subsumption relations. In addition, subsumption mappings that were discovered in the manual mapping process but not identified by the reasoner were included in the reference alignment. 3. Evaluate reference alignments. Once both reference alignments were complete they were manually inspected for errors and inconsistencies. The reference alignment between ATMONTO and AIRM-O [20] comes as two separate alignment files, one holding only equivalence relations and the other holding only subsumption relations. The equivalence reference alignment 7 https://www.skybrary.aero/ Matching Ontologies for Air Traffic Management 5 contains 32 relations in total and the subsumption reference alignment contains 83 subsumption relations. Only direct subsumption relationships were considered in the subsumption reference alignment, following the convention used during the development of the reference alignment for the Oriented Matching track arranged in OAEI 2011 [4]. 4 Mismatches between AIRM-O and ATMONTO In the course of conducting the manual alignment of ATMONTO and AIRM-O (see Sect. 3), several of the identified candidate equivalence relations were con- sidered “light matches” at first. In these cases, an equivalence relation between the classes was often deemed too strong – despite lexically similar class names hinting at a relation – given that the experts performed poorly on the alignment task – as judged by the two reviewing experts. Extensive discussions among the experts involved in the matching exercise revealed that similar class names were no guarantee of a correct match. In fact, in approximately 25% of the identified exact-match pairs in the final reference alignment, the class names did not have any words in common whereas in approximately 40% of the identified “light-match” candidate equivalence relations the class names did have words in common. This may explain partly why automated alignment techniques focusing on class name similarity did not perform particularly well (see Sect. 5). The initially identified “light matches” between ATMONTO and AIRM-O actually represent ontology mismatches. Multiple classification systems for mis- matchs with varying degrees of detail and often considerable overlap exist in literature. Figure 1 shows a classification of mismatch types synthesized from Klein [14] and Visser et al. [23, 24] along with mismatch types encountered during the manual matching between ATMONTO and AIRM-O. Notwithstanding the differences between classification systems, there seems to be consensus that the development of an ontology involves two separate processes and, correspondingly, two broad categories of mismatches can be distinguished [23, 24]. First, conceptu- alization mismatches are the result of different interpretations of the represented domain, leading to different classes, individuals, and relations being modeled in different ontologies for the same domain. Explication mismatches, on the other hand, are the result of different specifications of domain interpretations in form of different terms, modeling styles, and encodings being employed. One category of conceptualization mismatches concerns differences in model coverage and scope between ontologies from the same domain, which occur when two ontologies cover different parts of that domain or the same part at dif- ferent levels of detail. In this regard, a structure mismatch occurs when two ontologies distinguishing the same set of classes differ in how they are struc- tured through relations; we could not find a clear case of structure mismatch between ATMONTO and AIRM-O. A mismatch concerning differing levels of detail occurs when one class is modeled in more depth and with greater fidelity than the other. The ASP M eteorologicalCondition class from ATMONTO and AerodromeCondition from AIRM-O, for example, both represent meteorological 6 A. Vennesland et al. Ontology Mismatches Conceptualisation Explication Modeling Style Encoding Model Coverage Terminological and Granularity Attribute-type Mismatch Structure Mismatch Synonyms Concept Description Differing Level of Detail Terms Mismatch Attribute Assignment Mismatch Differing Intended Use Terms & Definiens Mismatch Differing Standards Differing Scope Paradigm Homonyms Concept Scope Definiens Mismatch Categorization Mismatch Concept & Definiens Mismatch Differing Representation Aggregation-level Mismatch Concept Mismatch Differing Level of Abstraction Differing Word Senses Fig. 1. Classification of ontology mismatches, synthesized from Klein [14] (white) and Visser et al. [23, 24] (light grey), extended with mismatch types encountered when mapping ATMONTO to AIRM-O (dark grey). information. ASP M eteorologicalCondition, however, is more detailed, compris- ing all aspects of sky, wind, visibility, and weather whereas AerodromeConditon is limited to sky conditions. Different properties and relations of similar classes may also reflect differences in how the classes are to be used in the context of a domain application (differing intended uses). For example, ReRouteSegment in ATMONTO describes an alternative air route option for contingency planning purposes, whereas RouteSegment describes an actual portion of a route being flown. Eventually, the differing scope of ontologies may result in a class from the source ontology lacking a matching class in the target ontology because the class from the source ontology lies outside the defined scope of the target ontology. An example of a differing scope is the missing equivalent in AIRM-O for the class DelayM odel in ATMONTO, which specifies a numerical model of airspace delay under specific traffic conditions. There is no matching class in AIRM-O because modeling concerns fall outside the scope of this ontology. A concept scope conceptualization mismatch occurs when two classes seem to represent the same concept, yet do not cover exactly the same instances, although the classes intersect. Categorization mismatches and aggregation-level mismatches fall into the concept scope mismatch category. A categorization mismatch occurs when two ontologies include the same class, but each ontology decomposes the class into different subclasses. ATMONTO’s Airport is equivalent to AIRM-O’s Aerodrome, however due to different geographical and application- wise scope Airport includes the subclasses U Sairport and InternationalAirport whereas Aerodrome has no such subclasses. An aggregation-level mismatch occurs when two ontologies define the same underlying concept using classes at different levels of abstraction. A differing level of abstraction is encountered when the matched classes intersect but some instances are outside the intersection. Consider, for example, AviationIndustryM anuf acturer in ATMONTO and Matching Ontologies for Air Traffic Management 7 AerospaceM anuf acturer in AIRM-O. In this case, the term “Aerospace” has a broader meaning than “Aviation”, hinting at a subsumption relation. The class of explication mismatches encompasses terminological, modeling style, and encoding mismatches. In this regard, an encoding mismatch relates to how the ontologies employ different formatting when describing instances, e.g., describing an instance either in miles or kilometres [14]; we omit this mismatch type in the remainder of this analysis. More relevant for our analysis are the terminological and modeling-style mismatches identified by Visser et al. [23, 24], which occur due to different knowledge definitions used in the ontologies and their associated concepts. The category of terminological mismatches comprises mismatches related to synonyms and homonyms. The synonym mismatch as explained by Klein [14] refers to two lexically different terms in fact meaning the same thing (e.g. ‘Air- port/Heliport’ versus ‘Aerodrome’), so we do not consider this a real mismatch in our analysis. Term mismatches as well as terms-and-definiens mismatches defined by Visser et al. [23, 24] belong to the synonym mismatches. A term mismatch oc- curs when the definitions share the same concept and the same definiens, but the terms are different. Correspondingly, a term-and-definiens mismatch occurs when the definitions refer to the same underlying concept, but the terms and definiens are different. The relation between Airport in ATMONTO and Aerodrome in AIRM-O could also be considered a terms-and-definiens mismatch. Mismatches related to homonyms occur when the meaning of two identical terms is different (e.g. the term ‘Conductor’ has a different meaning in music than in electrical engineering). We refer to homonym mismatches proper as differing word senses. There were a few incidents of homonymy that complicated the alignment process for ATMONTO and AIRM-O. For example, the term “Flow” had a slightly different meaning in ATMONTO and AIRM-O. In AIRM-O, a flow is a traffic pattern, while in ATMONTO flow is a concrete measurement of the number of aircraft per time unit traversing a volume of airspace.The classes have an exact or close lexical match, but the two classes correspond to two different word senses. Modeling style mismatches are further decomposed into concept description and paradigm mismatches. A concept description mismatch occurs when two similar concepts are modelled differently, e.g., that the same intention is modelled through the use of properties in one ontology and by using distinct sub-classes for the same target values in the other ontology [6]. A specific type of concept description mismatch between ATMONTO and AIRM-O is classes with similar names defining different versions of the same concept based on differing technical standards adopted by ontology developers, e.g., by FAA and EUROCONTROL. Finally, paradigm mismatches refer to how different paradigms can be used to represent concepts such as time, action, plans, causality, propositional attitudes, etc. For example, one ontology might use temporal representations based on interval logic, while another might use a representation based on points [6]. Paradigm mismatches relate to what we call “differing representation”, and one example of such a mismatch is between P lannedF lightRoute in ATMONTO 8 A. Vennesland et al. and T rajectory in AIRM-O. These two classes are used to represent the planned aircraft trajectory (or flight plan). In AIRM-O, the planned trajectory is com- posed of a sequence of trajectory points, elements, segments, and constraints. In ATMONTO, the flight plan is specified using a hierarchically decomposable route structure. These are fundamentally different methods of representing a planned route, based on different conceptual models of what constitutes a route. 5 Performance of Automatic Matching Systems We challenged three matching systems that normally rank highly on several tracks of the OAEI campaigns on the equivalence reference alignment: – AgreementMakerLight (AML) [9]. We ran AML using the GUI version from 20168 and the “Automatic Match” mode, letting AML handle the configuration of individual matching algorithms and external background sources (e.g. WordNet). AML includes terminological, structural and lexical matchers and uses WordNet as a general-purpose lexical resource as well as the Doid and Uberon ontologies for matching of biomedical ontologies. Property relations included in the produced alignment were disregarded when evaluating the performance of AML. – LogMap [11]. We used the latest available standalone distribution of LogMap9 with default matching parameters. LogMap combines terminological matching with capabilities for diagnosing and repairing incoherent alignments. Option- ally, LogMap can also employ external resources such as WordNet. As with AML there were some property relations included in the produced alignment, which we do not consider in the evaluation. – YAM++ [16]. YAM++ is provided as a web application10 . We used the default matcher parameters, which included both an element-level and a structure-level matching algorithm. The evaluation results from running the matching systems on the equivalence reference alignment are shown in Figure 2. As the figure shows, all three systems manage to avoid many false positives, especially LogMap which obtains perfect precision with no false positives. All three systems obtain a recall of 0.31. The results reveal that all three matching systems are able to correctly detect the true positive relations where the source and target classes are exact string matches. All three matchers also capture one relation where the source class (SID) is an acronym of the target class (StandardInstrumentDeparture) due to the fact that “Standard Instrument Departure” is expressed in the label of the source class. The remaining relations in the reference alignment are not detected by these systems. A closer inspection of the alignments produced by these three matching systems with respect to the equivalence reference alignment reveals that the following factors contribute to making this a challenging dataset: 8 There was an issue with the dependency to the Gephi Toolkit that prevented us from using the most recent version of AML. 9 https://sourceforge.net/projects/logmap-matcher/files/ 10 http://yamplusplus.lirmm.fr/index Matching Ontologies for Air Traffic Management 9 – Domain-specific and technical terminology. Most of the classes in both on- tologies describe aviation-specific concepts and technical terms. Often the class names and their natural language definitions include acronyms and abbreviations used only in aviation. Considering that typically used lexical re- sources (such as the aforementioned WordNet) have low coverage of technical terminology, this constitutes a challenge for matching systems. – Compound class names. Several of the classes involved in the relations repre- sented in the reference alignment contains equal substrings, a feature often exploited by string-matching techniques. However, in most relations one or both class names are compound words, such as PhysicalRunway - Runway or AircraftModel - AircraftMakeModelSeries, resulting in a low similarity scores for algorithms based on basic substring analysis. Here, a more comprehensive string-based analysis is required to identify such relations, possibly result- ing in the unwanted effect that additional false positive relations are being included in the computed alignment as well. – Synonymy, homonymy and polysemy. The two ontologies use synonymous terms for concepts with the same meaning (e.g. Airport vs. Aerodrome). Synonymy can often be resolved using lexicons or other external sources (e.g. other ontologies). Homonymy and polysemy are more of a challenge to solve. Some of the class names in these two ontologies can have a different meaning outside the ATM domain. Examples of this are Gate, Taxi or Star (which is short for Standard Terminal Arrival Route in the ATM world) and such challenges are not addressed through the use of lexicons such as WordNet. Evaluation of equivalence r eference alignment 1,00 0,90 0,80 0,70 0,60 0,50 0,40 0,30 0,20 0,10 0,00 AML LogMap YAM++ Precision Recall F-measure Fig. 2. Performance of selected state-of-the-art matchers over ATMONTO and AIRM-O 10 A. Vennesland et al. 6 Related Work Evaluation datasets that include reference alignments declaring the correct set of mappings between ontologies are important for the continued improvement of ontology matching techniques. The OAEI provides an annual standardised evalu- ation process for matching system. However, with only a few exceptions over the years, the OAEI tracks mainly involve one-to-one equivalence relations, neglecting other semantic relations and complex correspondences whose identification is important for more profound integration processes [8, 18]. One of these OAEI tracks is the Conference Track, a widely used benchmark for ontology matching systems, that since its inception in 2005 has been subject to many revisions [26]. This track now includes 16 ontologies describing conference organization and there are two versions of reference alignments, all holding one-to-one equivalence relations. The first version is referred to as “crisp” alignments where all confidence values are 1.0. The second version is referred to as an “uncertain” version of the reference alignment where the confidence values reflect the opinion from a group of human experts [7]. For the 2018 OAEI campaign, a complex alignment track was launched, offering reference alignments holding complex relations in four different datasets. One of the datasets included complex reference alignments for some of the ontologies in the Conference Track [19]. The other datasets represented real-world ontologies from the domains of hydrography, plants and species, and geoscience. Having real-world ontologies in benchmarks is important because such ontologies may expose issues arising in practice which may be overlooked by the developers of (semi-)artificial benchmarks [27]. 7 Summary and Future Work We contrasted AIRM-O with the ATMONTO. Mismatches between these ontolo- gies coupled with the complex and diverse nature of the ATM domain, which covers many technical subject fields, renders automatic ontology matching diffi- cult. The presented manual alignment of AIRM-O and ATMONTO potentially facilitates integration of datasets in different formats, e.g., NASA aeronautics research data with ATM information in the operational System Wide Information Management (SWIM) network. As a byproduct, the ontology matching commu- nity gains access to a reference alignment for two complex real-world ontologies from the ATM domain. We refer to a separate publication [10] for a more detailed comparison of AIRM-O and ATMONTO from an ATM perspective. Future work will investigate the potential for complex reference alignments between AIRM-O and ATMONTO beyond simple equivalence and subsumption relations. using the Expressive and Declarative Ontology Alignment Language (EDOAL) [2]. During the manual mapping process, we identified a large number of complex relations, e.g., class-to-property relations and many-to-many relations, which additional reference alignments can be developed from. In this regard, complex matching represents an area with a potential for significantly advancing the state-of-the-art in ontology matching. Matching Ontologies for Air Traffic Management 11 Acknowledgments. We thank Scott Wilson from EUROCONTROL and Joe Gorman from SINTEF for their contributions to the reference alignment. Part of this work was conducted as part of the BEST project. This project received funding from the SESAR Joint Undertaking under grant agreement No 699298 under the European Union’s Horizon 2020 research and innovation program. This work was also supported by the NASA Airspace Operations and Safety Program. The views expressed in this paper are those of the authors. References 1. Aeronautical Information Exchange Model. http://aixm.aero/ (Accessed: 26 August 2019) 2. EDOAL: Expressive and Declarative Ontology Alignment Language. http://alignapi.gforge.inria.fr/edoal.html (Accessed: 26 August 2019) 3. A format for ontology alignment. http://alignapi.gforge.inria.fr/format.html (Ac- cessed: 26 August 2019) 4. Ontology Alignment Evaluation Initiative: Oriented Matching 2011. http://oaei.ontologymatching.org/2011/oriented/index.html (Accessed: 26 August 2019) 5. Achichi, M., et al.: Results of the Ontology Alignment Evaluation Initiative 2017. In: Shvaiko, P., Euzenat, J., Jiménez-Ruiz, E., Cheatham, M., Hassanzadeh, O. (eds.) Proceedings of the 12th International Workshop on Ontology Matching. CEUR Workshop Proceedings, vol. 2032, pp. 61–113. CEUR-WS.org (2017) 6. Chalupsky, H.: OntoMorph: A translation system for symbolic knowledge. In: Cohn, A.G., Giunchiglia, F., Selman, B. (eds.) Proceedings of the Seventh International Conference on Principles of Knowledge Representation and Reasoning (KR 2000). pp. 471–482. Morgan Kaufmann 7. Cheatham, M., Hitzler, P.: Conference v2.0: An uncertain version of the OAEI conference benchmark. In: Mika, P., Tudorache, T., Bernstein, A., Welty, C., Knoblock, C.A., Vrandecic, D., Groth, P.T., Noy, N.F., Janowicz, K., Goble, C.A. (eds.) ISWC 2014. LNCS, vol. 8797, pp. 33–48. Springer (2014) 8. Cruz, I.F., Palmonari, M., Caimi, F., Stroe, C.: Building linked ontologies with high precision using subclass mapping discovery. Artificial Intelligence Review 40(2), 127–145 (2013) 9. Faria, D., Pesquita, C., Santos, E., Palmonari, M., Cruz, I.F., Couto, F.M.: The agreementmakerlight ontology matching system. In: OTM Confederated Interna- tional Conferences” On the Move to Meaningful Internet Systems”. pp. 527–541. Springer (2013) 10. Gringinger, E., Keller, R.M., Vennesland, A., Schuetz, C.G., Neumayr, B.: A com- parative study of two complex ontologies in air traffic management. In: Proceedings of the AIAA/IEEE 38th Digital Avionics Systems Conference (DASC) (2019), in press 11. Jiménez-Ruiz, E., Cuenca Grau, B.: LogMap: Logic-based and scalable ontology matching. In: ISWC 2011. LNCS, vol. 7031, pp. 273–288. Springer (2011) 12 A. Vennesland et al. 12. Keller, R.M.: The NASA Air Traffic Management Ontology. https://data.nasa.gov/ontologies/atmonto (March 2018) 13. Keller, R.M.: Building a knowledge graph for the air traffic management community. In: Companion of The 2019 World Wide Web Conference. pp. 700–704 (2019) 14. Klein, M.: Combining and relating ontologies: an analysis of problems and solutions. In: Proceedings of the IJCAI-01 Workshop on Ontologies and Information Sharing (2001) 15. Neumayr, B., Gringinger, E., Schuetz, C.G., Schrefl, M., Wilson, S., Vennesland, A.: Semantic data containers for realizing the full potential of system wide information management. In: Proceedings of the 36th IEEE/AIAA Digital Avionics Systems Conference (DASC) (2017) 16. Ngo, D., Bellahsene, Z.: Overview of YAM++ – (not) yet another matcher for ontology alignment task. Web Semantics: Science, Services and Agents on the World Wide Web 41, 30–49 (2016) 17. Object Management Group: Ontology Definition Metamodel v1.1 (2014), https://www.omg.org/spec/ODM/1.1/ 18. Spiliopoulos, V., Vouros, G.A., Karkaletsis, V.: On the discovery of subsumption relations for the alignment of ontologies. Journal of Web Semantics 8(1), 69–88 (2010) 19. Thiéblin, É., Haemmerlé, O., Hernandez, N., Trojahn, C.: Task-oriented complex ontology alignment: Two alignment evaluation sets. In: European Semantic Web Conference. pp. 655–670. Springer (2018) 20. Vennesland, A., Keller, R., Gringinger, E., Schuetz, C.G., Neumayr, B., Wil- son, S., Gorman, J.: ATMONTO2AIRM A reference alignment between the NASA ATM Ontolgy and the ATM Information Reference Model Ontology. https://w3id.org/airm-o/atmonto2airm/ (2019) 21. Vennesland, A., Neumayr, B., Schuetz, C.G., Savulov, A., Wilson, S., Gringin- ger, E., Gorman, J.: AIRM-O – ATM Information Reference Model Ontology. https://w3id.org/airm-o/ontology/ (2017) 22. Vennesland, A., Gorman, J., Wilson, S., Neumayr, B., Schuetz, C.G.: Automated compliance verification in ATM using principles from ontology matching. In: Aveiro, D., Dietz, J.L.G., Filipe, J. (eds.) Proceedings of the 10th International Joint Confer- ence on Knowledge Discovery, Knowledge Engineering and Knowledge Management, IC3K. pp. 39–50. SciTePress (2018) 23. Visser, P.R.S., Jones, D.M., Bench-Capon, T.J.M., Shave, M.J.R.: An analysis of ontology mismatches; heterogeneity versus interoperability. In: Proceedings of the AAAI 1997 Spring Symposium on Ontological Engineering. pp. 164–72 (1997) 24. Visser, P.R.S., Jones, D.M., Bench-Capon, T.J.M., Shave, M.J.R.: Assessing het- erogeneity by classifying ontology mismatches. In: Guarino, N. (ed.) Proceedings of the 1st International Conference on Formal Ontology in Information Systems. IOS Press (1998) 25. Wilson, S., Suzi, R., Van der Stricht, S.: The SESAR ATM information refer- ence model within the new ATM system. In: Proceedings of the 2014 Integrated Communications, Navigation and Surveillance Conference (ICNS) Conference. pp. L3–1–L3–13 (2014) 26. Zamazal, O., Svátek, V.: The ten-year ontofarm and its fertilization within the onto-sphere. Journal of Web Semantics 43, 46–53 (2017) 27. Zhou, L., Cheatham, M., Krisnadhi, A., Hitzler, P.: A complex alignment benchmark: GeoLink dataset. In: Vrandecic, D., Bontcheva, K., Suárez-Figueroa, M.C., Presutti, V., Celino, I., Sabou, M., Kaffee, L., Simperl, E. (eds.) ISWC 2018. LNCS, vol. 11137, pp. 273–288. Springer (2018)