Ontology reuse is a complex process that requires the support of methodologies and tools to minimise errors and keep the ontologies consistent. Although many efforts have investigated ontology reuse for different tasks and purposes, this body of work does not seem to translate to practice. The goal of this research is a comprehensive "Ontology Reusability Assessment", that builds on and extends the current state of the art. In this paper, we describe this overall aim and two preliminary results: 1) a community questionnaire to gain an insight on the gap between theory and practice, and 2) a case study to see if it is possible to identify similar functional requirements across different domains.
Ontologies are explicit specifications of conceptualised domains shared by a community of users, and play a significant role in conceptual data modelling and AI [4]. In the context of ontology engineering, reusing existing knowledge models is largely recognised as a key factor in the development of ontologies that are both cost-effective and of higher quality, by reusing components that have previously and independently been validated [14]. Despite this, ontology reuse does not seem a consolidated practice [8]: for instance, an analysis of 377 biomedical ontologies in BioPortal 1 concluded that reuse was very limited (<5%) [9].
From state of the art reviewed, it is clear that some ontologies are reused more than others, possibly because they documented when compared with others [13]. One of the aspects affecting the reusability of an ontology is the perception of its quality with respect to some evaluation criteria. However, although much work has addressed the evaluation of the quality of ontologies from the perspective of reuse, there are no definitive principles and practices, and especially, there is no practical mechanism for providing developers with a qualitative and quantitative assessment of reusability.
The PhD research presented in this paper, currently in its second year, aims to develop a comprehensive framework for assessing ontologies for reuse, capturing the requirements for reusing an ontology, identifying the modalities of reuse, the ontology features, and any additional information that ontology engineers need to be aware of when selecting a candidate ontology to reuse.
To achieve this objective, we analysed state of the art in a top-down and bottom-up manner: we reviewed the current literature on the theory of ontology reuse, whilst we compiled a questionnaire aimed at ontology practitioners to identify mismatches between theory and common practices.
In parallel with the questionnaire, we also started investigating how ontological requirements can help determine whether an ontology can be reused for a given purpose. Competency questions (CQs), the natural language questions outlining the scope of knowledge represented by an ontology and assisting in its development and maintenance [23], are often seen as a way of expressing functional ontological requirements that can be used to promote ontology reuse and that facilitate the identification of patterns that implement such requirements [6]. Our approach aims to contribute a qualitative estimator of the reusability of a given design pattern. We illustrate this through a case study that developes an ontology for the role game Dungeons and Dragons (DnD) 2 . In the case study we objectively assess the similarity between the DnD ontological requirements and those of candidate ontologies to reuse or of design patterns that could be reused.
This paper reports our preliminary results on these two directions of research. We start by defining the problem statement Section 2. In Section 3 we discuss state of the art on how ontology engineers/developers make use of known definitions and measures in reuse. We present the methodology Section 4, the questionnaire and its preliminary results in Section 5, and in Section 6 we present the case study assessing the similarity between ontological requirements. Finally, in Section 7 we conclude and discuss future research directions.
Ontology reuse is one of the fundamental phases in all ontology design methodologies, and it is deemed as crucial because it allows developing ontologies by saving effort and by reusing ontological fragments that have been independently validated [12]. However, a number of reasons can hamper the effective reuse of existing ontologies:
(i) possible deficiencies in the ontology's documentation make it difficult to find all the ontologies suitable for reuse [11]; (ii) the lack of a standard way to verify the accuracy of CQs against an ontology typically results in misconceptions in determining the reusability of a candidate ontology [23]; (iii) the lack of standardisation in designing an ontology through reuse can introduce future errors, for example, by failing to keep track of changes in the reused ontologies [8]; (iv) insufficient information about the requirements that ontology engineers aim to satisfy when assessing candidate ontologies for reuse [11,8] These research questions lead to the definition of the following four objectives that this research aims to address: A 1: Identify the reasons that hamper reuse (RQ 1.1) A 2: Develop methods for reusing an ontology by identifying: the requirements for reusing an ontology (RQ 1.1), and the ontology features and additional information that ontology engineers need to know to make an informed decision (RQ 1.3, RQ 1.4). A 3: Develop mechanisms to assess the reusability of an ontology against the requirements identified and summarise them into an assessment metric (RQ 1.1, RQ 1.2). A 4: Evaluate the proposed methodology and metrics with the community to estimate the prospective level of uptake (RQ 1).
From reviewing the extensive state of the art on ontology reuse, we have established that reuse is based on the ability to determine the scope and the purpose for which an ontology is built. Therefore, we start our investigation from the analysis of the role that CQs play to express ontology requirements, and we investigate whether the semantic similarity between CQs can be used as an indicator of reusability of a candidate ontology, which could be used to determine an initial set of reusable ontologies to be selected by ontology developers.
In selecting the literature to review to investigate our research questions we also included ontology evaluation papers, as the quality of an ontology is an influential factor in determining whether to reuse an ontology. The literature review provides us with a set of widely used metrics and techniques that determine the ontology quality from different perspectives, even though these efforts are not always explicitly tailored to the task of ontology reuse. Furthermore, whilst several metrics have been proposed to support ontology reuse, few efforts have attempted to assess how ontology reuse theories affect development practices [8], which is the aim of this research.
The assessment method we aim to develop as part of this PhD combines these metrics to understand the thinking behind ontology selection and reuse, with a particular interest in the practical way of assessing the reusability of an ontology. Our assessment metric is based on the combination of four main aspects: schema metrics, instance metrics, community and social metrics, and documentation metrics:
Schema metrics [21,19,13]: pertain to the quality of the design of the ontology schema.This can be assessed by indicating the richness, width, depth, and inheritance of an ontology schema.
Instance Metrics [21,19,13]: evaluate the way an ontology is populated. The placement and distribution of instance data can indicate the effectiveness of the ontology design and the knowledge represented by the ontology. Metrics related to Community and Social aspects [11,20]: capture the several social and community aspects that can affect the quality of an ontology in the evaluation process. Metrics Related to Documentation [11,7,20]: determine whether an ontology includes some specification of the requirements, for instance the Ontology Requirement Specification Document or CQs.
In addition to identifying these metrics, we also used the literature review to guide us in preparing a questionnaire aimed at ontology practitioners that validates the importance of the identified measures in common practices. The literature review also includes studies that investigate functional ontological requirements [2] and how these are represented by means of CQs, since these fundamental in defining the scope of an ontology. For example, an analytical study presented by [16] included a list of 30 methodologies to build ontologies that start by defining CQs.
The purposes of many studies of ontological functional requirements can be categorised into: (i) ontology evaluation, by evaluating CQs over an ontology to check whether knowledge is entailed [1,2], or measuring the quality and accuracy of the knowledge encoded in the ontology [15,3]. (ii) ontology verification, by comparing an ontology against the ontological requirements to ensure that an ontology is built correctly [18]. (iii) methodology definition, by defining a methodology that could consistently drive the ontology engineers in developing an ontology from scratch. This by using a goal modelling approach (Tropos process) to capture, model and reason CQs [18]. (iv) analysis of CQs, by analysing the structure of CQs and proposing some popular linguistic patterns that can be reused to specify requirements [10,23,15,3]. (v) CQs formalisation, by automatically fromulating CQs in a controlled natural language based on CQs patterns [10], and producing a glossary of terms for each domain [22].
All these studies can improve the requirement specification activity by identifying possible problems in the definition of requirements [6]. Our contribution proposes the assessment of the similarity between requirements to decide on the reusability of an ontology.
This section introduces the different methodologies used for investigating the main research question, and its sub-questions. RQ 1.1 3 has been addressed by a systematic review to determine the challenges to ontology reuse. We have then validated the identified challenges with a qualitative study through an online questionnaire aimed at ontology developers (Section 5), that assesses the extent to which these challenges hamper reuse, and therefore addresses both RQ 1.24 and RQ 1.3 5 . Based on the survey's initial analysis, we recognised the importance of ontological requirements, and added two further sub-questions: RQ 1.46 and RQ 1.5 7 , whose aim investigates the feasibility of (semi-) automatic similarity assessment of requirements. We conduct a case study (Section 6) that develops a Dungeons and Dragons (DnD)ontology covering the different characters and the possible relationships between the resources of the game. The ontology development is divided in three phases, and is described in Section 6: 1) analysis of the DnD requirements by identifying concepts and relationships that are relevant in the domain; 2) implementation of the requirements using CLaRO [10], a template-based Controlled Natural Language resource to author CQs; and 3) identification of similar requirements in the CORAL curated data set [6] of ontological requirements. In order to assess the similarity between requirements and CQs we adopted the BERTSimilarity method [5], that allows us to compute the similarity between complete requirements, taking into account their grammatical structure. Wordnet was also used to account for entity and propriety synonyms.
The overall assessment mechanism(RQ 1) is meant as a summary of the measures for reusing an ontology. It is intended to be a visual graphical summary for all reasons that justify why an ontology can be reused, and thus a visual representation of the ontology reusability with respect to the requirements defined by an ontology developer.
The review of state of the art has highlighted the existence of a gap between theory and practice. However, it is not clear what are the barriers to a more extensive uptake of the practice: it is essential, therefore, to go back to the community, and elicit the information directly from ontology developers.
We designed an online questionnaire to understand how ontologists and knowledge engineers in different domains search for, evaluate, and select an ontology for reuse. The questionnaire was disseminated during the period March to November 2020 through personal contacts, and relevant mailing lists and conferences (e.g. the 19th International Semantic Web Conference). In total, 54 eligible respondents participated in this study, but we excluded respondents who classed themselves as having no ontology engineering experience.
The analysis of the answers to the questionnaire is intended to inform how each metric contributes to the assessment mechanism, based on how much the respondents consider a given metric when deciding whether to reuse an ontology.
The initial findings can be accessed here 8 . The results are still being analysed; however, it is worth noting that a large proportion of respondents (95%) indicates that they are likely or very likely to consider any ontology documentation, including CQs, when deciding reuse. These results are the basis for the DnD case study, presented in Sectionn 6.
CQs are often used as a starting point in the ontology development process and capture the functional requirements of the ontology. It is, therefore plausible to consider first the reuse of ontologies that have similar functional requirements to the ontology being developed. In order to verify this hypothesis, we devised an experiment where we develop an ontology for the role game Dungeons and Dragons (DnD)9 using the NeOn methodology [17] and reusing other ontologies from disparate domains do not bear an immediate similarity with the DnD one. The underlying idea is that there could be ontologies that cover different domains from the one of the ontology being designed, but that could still provide terms or patterns that can be reused because their semantics are similar to those in the domain of interest. For instance, the notion of a boss in an organisation can be considered similar to the one of a dungeon master in the game. Therefore, we aim to assess whether the similarity between requirements can be used as an indicator of the reusability of an ontology. We verify this hypothesis by desining a case study, where we use the Corpus of Ontological Requirements Annotated with Lexico-syntactic patterns, CORAL [6], as a repository of requirements to be matched. CORAL is an openly available corpus of 834 ontological requirements extracted from ontologies modelling different domains, including video games, buildings and terrains and business organisation. We chose to model DnD because its rules and roles in the games are potentially similar to the requirements in CORAL.
We worked with a domain expert and we identified 45 CQs for DnD, which then were analysed through natural language processing to the main terms of interest, using the vocabulary agnostic patterns proposed by [23]. We then tested those terms against CORAL, and we identified five candidate ontologies to reuse (SAREF4ENV, OneM2M, SAREF, SAREF4BLD and OntoDT) [8] with 38 requirements matching the DnD CQs. The remaining 7 CQs from DnD remained unmatched. We then applied question relevance methods, investigated by the question answering community, to detect duplicate CQs and measure the similarity between requirements 10 . The underlying hypothesis in this case study is that the similarity scores between requirements can indicate the reusability of the part of the ontology modelling the requirement. For example, in DnD, we ask, "What does a campaign have?". The corresponding requirement in CLARO is "An organisation has users", and we notice that one is a generalised form of the other(that indicate an implicit relationship). We use WordNet (a large lexical database of English) and apply the Wu-Palmer Similarity11 to asses how similar the word senses of the terms used in the requirements are and the relative position of Synsets in the hypernym tree. This shows that some terms in CLARO are more general than the ones in DnD, and therefore we could reuse the pattern used to define the corresponding CQs in CLARO in DnD.
The ability to start selecting and identifing the relevant ontologies by analysing requirements has a clear potential and is worht exploring further. Applying this methodology to ontology repositories like BioPortal could enable ontology developers to identify ontologies to reuse or to model some of their requrements.
We argued that despite the many community efforts, and reuse being a key step in ontology development, the practice is not yet widespread. Therefore this PhD research aims to identify the barriers to a more extensive uptake of the practice and proposes an "Ontology Reusability Assessment" to provide an objective assessment to guide the reuse of an ontology. As part of this effort we sought feedback from the community through an online questionnaireaimed at understanding how ontologists and knowledge engineers select an ontology to reuse.
From an initial analysis of the responses we identified that the ability to analyse and match CQs is a potential good indicator of the reusability of an ontology. Choosing the reuse of a candidate ontology by exploring the similarity between a set of given requirements and those documenting existing ontologies could facilitate and support their reuse. As a next step, we aim to identify linguistic features in the requirements that can be an indication of whether an ontology could be reused. The proposed mechanism for reuse will be then presented back to the community for evaluation. An evaluation study including expert ontologist and community members will ensure that the reuse assessment is both practical and relevant.