The use of ontologies has been acknowledged as a successful approach to solving interoperability and knowledgerelated problems. They have also been recognized as valuable for explainability in AI-based systems. Developing ontologies involves a lot of tacit knowledge and requires tool support. Several tools are available and can be used for diferent purposes to aid ontology engineering (OE) activities. The use of a suitable set of tools contributes to increasing development process performance and improving ontology quality. Aiming at getting evidence about the use of tools to support OE, we conducted a survey with 74 ontology engineers. We investigated which tools have been used, their main characteristics, benefits and dificulties perceived when using them, and properties considered important in OE tools. The results indicate that tools have been widely adopted, but some issues, such as integration problems and scope limitations, still need to be addressed. This paper presents the survey, its results, and discusses the main findings.
Ontologies allow the formal representation of fundamental concepts interpreted by users, in a way
that facilitates communication and understanding of a given domain [
Developing ontologies is not trivial. Several ontology engineering (OE) methods have been proposed
to aid ontology engineers in developing high-quality ontologies (e.g., [
Currently, there is a wide range of tools to support OE. Some are devoted to OE activities (e.g.,
Protégé1), while others were not conceived for OE but can be used in this context (e.g., Visual Paradigm2).
Selecting the more suitable tools that provide the necessary features is an important decision in the OE
process and it may not be easy [
Although there has been an evolution in the functionalities of tools to support ontology development
and some tools have become very popular (e.g., Protégé) [
To take a step in this direction and gather evidence on this subject, we carried out a survey with 74 ontology engineers to investigate tools used to support OE. The results provide a preliminary picture of the state of practice. With that, we intend to share data and perceptions with other ontology engineers and shine a light on opportunities to advance the research topic.
In summary, the results reinforce the importance of using tools to aid OE. Tools have been widely
used, with a predominance of tools for conceptual model development and ontology implementation.
Both, OE-specific (e.g., ontology editors) and generic (e.g., text editors) tools have been used and there
has been a preference for free tools. Functional suitability, reliability, usability, and availability have
been considered the more relevant properties. Ontology development acceleration and ontology quality
improvement have been perceived as the main benefits, corroborating the literature [
In this paper, we provide an overview of our study, summarize the main findings, and discuss the results. The remainder of the paper is organized as follows: Section 2 presents the theoretical background for the paper; Section 3 presents the study protocol; Section 4 synthesizes the results; Section 5 discusses the results; Section 6 addresses the study limitations; Section 7 discusses related works; and Section 8 presents our final considerations.
An ontology is a formal and explicit specification of a shared conceptualization [ 22]. The
conceptualization is an abstract and simplified view of the portion of the world intended to be represented.
An important distinction sets apart ontologies as conceptual models, called reference ontologies, from
ontologies as computational artifacts, called operational ontologies [23]. A reference ontology is built
aiming at making the best possible description of the domain in reality, representing a model of
consensus within a community, regardless of its computational properties. Operational ontologies, in
turn, are machine-readable artifacts designed with the focus on guaranteeing desirable computational
properties [
Engineering ontologies is a complex and collaborative task, which is challenging for both newcomer
and experienced engineers [
Several tools have been developed specifically to support ontology development, such as Protégé,
OLED [29], and Onto4AllEditor [
According to Gómez-Pérez et al. [31] OE supporting tools can be grouped in six categories: development tools, including tools and integrated suites used to build a new ontology from scratch (usually support documentation, export and import to/from diferent formats and languages, graphical edition, library management); evaluation tools, used to evaluate the ontologies and their related technologies; merge and alignment tools support merging and aligning diferent ontologies in the same domain; annotation tools help add instances of concepts and relations and maintain (semi)automatically ontologybased markups in Web pages; querying tools and inference engines, allowing querying and performing inferences; and learning tools, which derive ontologies (semi)automatically from natural language texts, as well as semi-structured sources and databases, by means of machine learning and natural language analysis techniques.
The study consisted of a survey whose goal was to investigate the use of tools in OE to get an overview of this topic in practice. A survey aims to identify the characteristics of a broad population by generalizing data collected from a representative sample of individuals [32]. Surveys are conducted to produce a snapshot of the situation to capture the current status [33]. We chose this method to reach many individuals, which would be harder or unfeasible through interviews or case studies, for example. We followed the process defined in [ 33], which comprises five activities: scoping, when we delimit the scope of the study and establish its goals; planning, when the study design is determined; operation, which consists of data collection; analysis and interpretation, which involves analyzing data to draw conclusions about the research topic; and presentation and package, when the results are communicated.
Aligned with the study goal, we defined three main research questions: (RQ1) Which tools have been used and what are their characteristics?; (RQ2) What benefits and dificulties have been perceived when using the tools?; and (RQ3) Which properties have been considered important in OE tools?.
The instrument used in the study was a form created in Google Forms. It is composed of a consent term for participation in the study and three sections of questions. The first section has eight closed questions to characterize the participants. The second has 11 closed questions related to RQ1 and RQ2. One question allows the participant to complement their answer by providing information in text format. The third section is related to RQ3 and has one closed question plus two open questions to collect further information, comments, and suggestions. In some questions we used ordinal agreement scales but excluded the neutral option to capture meaningful information. The form is available in the study package [34].
The survey participants must be a sample of the target population. Therefore, we targeted ontology engineers with knowledge of and experience with ontology development and OE tooling support.
The procedure followed in the study consisted of three steps. In the first step, we carried out a small pilot to evaluate the form and the study protocol. We asked an ontology engineer to answer the questionnaire and report response time, problems, and suggestions. Taking his feedback into account, we made minor adjustments to the form. In the second step, we sent messages inviting people to take part in the study. We sent messages to research groups working with ontologies, mailing lists involving researchers and professionals in the field, contact networks at universities, public and private organizations, and authors of the articles selected in an ongoing systematic literature mapping we are performing on the topic. We also requested that those invited forward the invitation to other individuals whom they believed could participate in the study. The final step involved collecting data from the answered questionnaires, representing collected data in tables and graphs, and analyzing it.
Invitations to participate in the study were sent in two batches, the first in late December 2023 and the second in late January 2024. We contacted 227 people and received 74 answers until March 1st, 2023, which amounts to a response rate of 32.6%. In this section, we summarize data collected in the study. For questions in which the participants could choose more than one answer, the sum of the absolute values is higher than 74 and, thus, the sum of the rates exceeds 100%. For simplification reasons, we rounded the percentage values to the first decimal place. The complete set of collected data plus tables and graphs representing them is available in the study package [34].
The participants’ profile was identified through questions regarding their level and area of education,
knowledge of ontologies, and experience in OE. To better contextualize the results, we also asked about
the context in which they have worked with ontologies (academia or productive sector), the ontology
types they have developed (reference or operational), and how often they have used OE systematic
approaches (e.g., SABiO [
In the following, we synthesize collected data by grouping the questionnaire questions into four topics: Use of tools in OE, OE tool characteristics, Perceptions of using OE tools, and Properties of OE tools. The first two topics are related to RQ1, the third to RQ2, and the last one relates to RQ3.
Use of tools in OE: In this topic, we asked which tools the participants have used, how often they have used them, how they found the tools, and in which OE activities they have used OE-specific (i.e., tools addressing OE tasks, such as ontology editors) and generic (i.e., domain-independent tools that address general tasks, such as text editors) tools.
When asked about how often they use tools to support OE, the majority of participants (55.4%) informed that they always use tools, 39.2% indicated they use tools most of the time, and 5.4% rarely use them. Concerning the used tools, OE-specific tools have been predominant. Protégé (85.1%) has been the most used tool, followed by Visual Paradigm (43.2%), Astah (41.9%), and OntoUML-vp-plugin (36.5%). Protégé is a free, open-source platform that provides a graphical interface for developing ontologies and an architecture with a suite of tools to construct domain models and knowledge-based applications with ontologies [35]. Visual Paradigm and Astah are tools that support creating UML conceptual models and generate code from the models. OntoUML-vp-plugin,4 in turn, is a plugin designed for OntoUML modelers that adds features to Visual Paradigm to enable the use of OntoUML stereotypes in class diagrams, model checking, and transformation. Other OE-specific tools used by the participants have been OLED [29] (20.3%) and Menthor [36] (16.2%). Both are OntoUML editors that support model building and formal verification, incorporating support for a range of methodological resources for developing, evaluating, and implementing ontologies. Generic tools have also been used, mainly Electronic Spreadsheets (27%), Text Editors (27%), and Slide/Presentation Tools (24.3%). In addition to the tools listed in the figure, others were mentioned by the participants, such as OntoTrack, Onto4AllEditor, Diagram.io, Miro, Widoco, Astrea, OnToology, Hermit, Themis, CmapTools, ONTOFOX, Enterprise Architect, PlantUML, Coggle.it, and yEd Graph Editor.
As for how the participants discovered the tools, most of them (82.1%) received recommendations from other people (e.g., friends, work/study colleagues, teachers). 39.2% discovered the tools in courses or classes; 37.8% by searching the Internet; 36.5% have used tools employed where they work, 35.1% have used tools developed by the organization they work; and 17.6% declared to have used tools indicated in OE systematic approaches.
To investigate tooling support provided to ontology development phases, we asked the participants to indicate the phases that have been supported by OE tools and the type of used tools (OE-specific or generic). Requirements Specification has been supported mainly by generic tools (89.2% of the participants indicated that have used that kind of tool in this phase). The other phases have had a predominance of OE-specific tools: 64.9% in Ontology Conceptualization and Formalization; 71.2% in Ontology Design, 73% in Ontology Implementation, and 58.1% in Verification, Validation, and Testing. Figure 2 summarizes data about the use of tools in OE.
OE tools characteristics: In this topic, we asked the participants about the tools’ availability, features, and integrated support for OE. Figure 3 summarizes the results. Regarding availability, most participants (63.5%) declared that they have used only free tools, 33.8% informed that some of the tools they have used are free, and 2.7% of participants have not used free tools. Concerning features, we provided a list of options for the participants to indicate which ones have been ofered by the tools they have used. The features most indicated were ontology’s concept hierarchy representation (86.5%), ontology visualization (85.1%), diagram creation (82.4%), ontology verification (59.5%), operational ontology implementation (56.8%), and ontology validation (50%). We also asked the participants if the tools have provided integrated support for the OE process. 47.3% of participants declared that the tools partially provide integrated support; 41.9% informed that the tools do not provide integrated support, and 10.8% stated that the tools have provided integrated support for the OE process.
Perceptions of using OE tools: In this topic, we asked the participants about their perceptions of using OE tools regarding how useful the tools have been, provided benefits, and faced dificulties. Concerning tools usefulness, 43.2% of the participants stated that they have helped a lot; 39.2% answered that they have helped a little; 16.2% declared that they have helped very little; and one participant answered that tools have not helped at all. The participants were asked to justify their answers. We grouped the answers with the same meaning and categorized them. Regarding the reasons why participants have considered tools useful, they highlighted that the tools have helped simplify developing complex ontologies (13.5%), evaluate the ontology (12.2%), automate repetitive tasks (10.8%), visualize and understand the ontology as a whole (8.1%), increase productivity (8.1%), communicate and interact with stakeholders (2.7%), manage knowledge (2.7%) and reuse (2.7%). As for the reasons why participants have not considered the tools useful, they pointed out that the tools are too “fragmented” and the entire OE process cannot be supported by a single tool or a set of integrated tools (29.7%). Additionally, there has been a lack of support, documentation, tutorials, or other educational resources to help them use the tools (17.6%), automation has been limited (5.4%), there have been performance problems (5.4%), lack of free access (1.4%), and complex user interface (1.4%).
We also asked the participants to indicate the benefits and dificulties they have perceived when using the tools. As for benefits , they highlighted that the tools accelerate the ontology development process (90.5%) and improve the quality of the produced ontologies (85.1%). Facilitating reuse (66.2%) and improving work management (64.9%) have also been acknowledged, among others. As for dificulties , the ones more prominent were scope limitations (e.g., the tools support a limited set of OE activities or the provided functionalities have limitations) (62.2%) and integration problems (e.g., the tools do not allow integration with others, the result produced in one tool cannot be exported to another) (56.8%). Identifying and selecting the tools has been dificult for half of the participants. Figure 4 summarizes the participants perceptions.
Properties of OE tool: To investigate which properties ontology engineers expect to find in an OE supporting tool, we presented 10 properties (Functional Suitability, Eficiency, Usability, Reliability, Security, Maintainability, Portability, Availability of use, Collaboration, and Coverage) and asked the participants to indicate the level of relevance of each of them. The properties are based on ISO 25010 [37] (we adapted some of them to better fit OE tools). Figure 5 presents the results. Functional Suitability, Reliability, Usability, and Availability of Use were considered by the participants as the most relevant properties. In contrast, Maintainability and Security were considered less relevant (the number of “neutral” plus “irrelevant” plus “very irrelevant” answers is higher than in the other properties). The participants were also asked to indicate other properties they consider important in a tool to support OE. 31 participants answered this question. We grouped the equivalent properties. The main cited properties were Interoperability, Documentation & Support, Version Control, and Reuse.
Next, we discuss the results by considering the topics identified in Section 4.
Regarding the use of tools in OE, around 94.6% of the participants declared that have systematically used tools, which demonstrates the key role tools play in OE. The most used tools were Protégé (85.1%), Visual Paradigm (41.9%), and Astah (36.5%). They support implementing operational ontologies (Protégé) and creating ontology conceptual models (Visual Paradigm and Astah). These are the OE most specific activities and tooling support is crucial to perform them. Thus, it is not a surprise that the most used tools relate to these activities.
By analyzing the ontology types, we noticed that 79.7% of the participants have developed reference ontologies and 67.6% operational ontologies. Although there has been a predominance of reference ontologies, the number of participants that have used Protégé (85.1%) is higher than the ones using Visual Paradigm or Astah (59.5% of the participants have used only one of them or both). We believe that this is because Protégé is free, very popular among ontology engineers, and supported by a strong community of academic, government, and corporate users. In contrast, although Astah and Visual Paradigm are popular, they have similar features, which may divide users (e.g., 23% of the participants have used Visual Paradigm while 17.6% have preferred Astah), and are not free (Astah has a student version and Visual Paradigm has a community version with fewer features than the paid version). OntoUML-vpplugin was also cited by many participants (36.5%). It helps validate conceptual models created in Visual Paradigm and represented in OntoUML and generates code that implements the corresponding operational ontology. Thus, it provides integrated support to modeling and implementation activities and bridges the ontology conceptual model and its operational version. We believe that tools like this contribute to built-in quality and should be further explored by the OE community. Four participants (5.4%) declared that have used tools rarely. Given that developing ontologies without tool support can be unfeasible, we hypothesize that the participants may have thought that the question referred to OE-specific tools only. That was the first question about tools presented for the participants, thus they were not advised that both, generic and OE-specific tools, should be considered.
By looking at the 10 most cited tools, we notice that six of them are specific tools supporting conceptual modeling and ontology implementation, while four are generic tools. By analyzing in which OE phases the diferent types of tools have been used, we notice that the more specific the task is, the more specific the tool needs to be. Hence, there has been a predominance of generic tools to support Requirements Specification activities. Given that requirements are usually recorded in text format, generic tools such as text editors and spreadsheets can be helpful. Ontology conceptualization and formalization, design, implementation, and evaluation, in turn, have depended mainly on OE-specific tools. However, the use of generic tools to support ontology conceptualization and formalization (e.g., record concepts definitions and axioms) has also been expressive. Although generic tools have been useful, it may be hard to map information recorded in such tools (e.g., text editors) to the ontology stored in OE-specific tools (e.g., ontology editors). Thus, advances in OE-specific tools to support OE initial phases and connect them to other tools would be welcome.
By analyzing tool support for each OE phase disregarding the types of tools, we noticed that around 92% of the participants have used tools to support Requirements Specification, 86% to support Conceptualization and Formalization, 84% to support Design, 76% to aid in Implementation, and 66% in Verification, Validation, and Testing. It is possible to notice higher values in the three first phases when compared to the last two phases, even though tools are essential to implement operational ontologies. These results are probably because many participants (32.4%) have developed only reference ontologies and, thus, do not implement or test operational ontologies.
There has not been a common structured source of tools (e.g., a catalog, a repository). Most ontology
engineers (more than 80%) have discovered the tools mainly through recommendations from other
people. This suggests a lack of knowledge of the panorama of existing tools, their main characteristics,
features, strengths, weaknesses, and so on. Some ontology engineers (around 17%) have used tools
indicated by OE systematic approaches. Given that around 80% of the participants declared that have
used such approaches, this result suggests that OE methods have not suggested tools or have suggested
limited or unavailable ones, which is in line with observations made in [
Concerning OE tool characteristics, free tools have been predominant. This result corroborates the literature, which states that free access to the source code and cost savings are among the main characteristics desired in software tools [38, 39, 40]. The features ofered by the tools have been diverse. The six most cited features are directly related to OE core phases: diagram creation, concepts hierarchy representation, ontology visualization, ontology implementation, ontology verification, and validation. This demonstrates that the tools have been concerned with providing features to address core OE tasks. Other features have covered support activities, such as reuse, automatic documentation, and configuration management. Features related to requirements (competency questions definition and management) have been the ones less covered by the tools. This is consistent with the findings previously discussed, indicating a lack of support of OE-specific tools for requirements specification. Providing advances to support requirement specification activities and connecting the produced artifacts to outcomes of subsequent phases would be beneficial for the OE community.
Only around 10% of the participants informed that the tools they have used provide integrated
support to the OE process. This suggests that despite the range of existing OE tools, they still have not
supported the ontology lifecycle in a comprehensive and integrated way. In fact, most OE tools focus
on a few activities of the ontology lifecycle, mainly design and implementation [
As for the perceptions of using OE tools, around 82% of the participants perceive OE tools as
helpful. Although this result was in some way expected, almost half of these participants claimed that
the tools help just a little. Moreover, around 18% of the participants indicated that the tools help very
little or are not helpful. Participants declared that the main problems hindering tool usefulness have
been related to integration, reinforcing the issue discussed above, and documentation. Without a proper
integration among the several tools used to support OE activities and clear documentation and support
to use the tools, the advantages of using them are hampered [
The reasons why ontology engineers perceive OE tools as helpful (or not) were declared by the
participants in an open question. After that, they were asked about the benefits and dificulties of using
OE tools. In general, the results were consistent with the reasons presented by them, providing an
overview of the pros and cons of using OE tools. Accelerating the ontology development process and
improving the quality of the produced ontologies were the benefits perceived by more participants. These
benefits have also been advocated in the literature (e.g., [
Regarding properties of OE tools the four considered more relevant by the participants (Functional Suitability, Reliability, Usability, and Availability) show that ontology engineers expect tools with the capability of doing what they are set out to do (i.e., have the necessary functions to meet the target tasks and objectives [47]), producing correct and reliable results, communicating clearly with the users, and being easily accessed (e.g., free tools). In other words, the tools should correctly do what they are intended to do and it should be easy to access and use them. These properties are perceived mainly from the external perspective (i.e., when ontology engineers are the users of the tools) and are consistent with other results. For example, bugs, limited access, and usability problems were pointed out as dificulties faced by the participants when using the tools, indicating the need for improvements in reliability, availability, and usability and the relevance of such properties. Despite scope limitations having been pointed out by most of the participants as an obstacle to overcome when using the tools, Coverage was not among the top five properties. Security and Maintainability were the properties considered less relevant. Security refers to the ability of the tool to protect data and information. We believe that the participants perceived this property as less relevant because, in general, when developing ontologies, we do not record confidential or sensitive information in the tools. However, protecting the ontology models, concepts, instances, etc. is still necessary, which was recognized by 50% of the participants, who considered this property relevant or very relevant. Maintainability, in turn, concerns the internal perspective, i.e., from who develops the tool. Thus, its relevance would be perceived only by ontology engineers who interact with the tools by playing diferent roles, user (ontology engineer) and developer (software engineer). The results suggest that this is not the case for most of the participants.
The list of properties we provided to the participants was not exhaustive. To capture other relevant properties, we asked them to cite others they consider relevant. Interoperability, Documentation & Support, Version Control, and (support for) Reuse stand out. All these properties are related to aspects pointed out by the participants in other questions, which reinforces their relevance. Integration problems, lack of documentation & support, and version control issues have been indicated as dificulties faced when using the tools. Reuse, in turn, helps accelerate the development process, which is the main benefit of using tools perceived by the participants. To properly meet ontology engineers expectations and needs, tools should manifest properties referring to desirable quality attributes. The survey results indicate that some of them have not been adequately addressed by existing tools. Therefore, there is an opportunity to explore how these properties should be manifested and develop strategies to achieve them.
The validity of a study denotes the trustworthiness of the results. Every study has threats that should be addressed as much as possible and considered together with the results. We discuss threats using the classification presented in [33].
Threats related to the constructs involved in the study can afect the results and impact Construct Validity. The main threat is the participants misinterpreting the questions. To minimize this threat, after producing the questionnaire, we carried out an internal test in which one of the authors, who did not participate in the questionnaire creation, answered it and pointed out aspects to be improved. After improving the questionnaire, we ran a pilot with an ontology engineer external to our research group to evaluate the questionnaire and response time. Moreover, in the questionnaire, we provided examples and definitions for the participants to better understand how to answer the questions.
Another threat is related to the scale used in some questions. Given that we did not provide any common grounds, diferent participants may have interpreted terms (e.g., a lot, a little) in diferent ways. Moreover, the results might have been diferent if we had not excluded the neutral alternative in some questions. The alternatives of answers provided in each question also represent a threat because they may not cover all the relevant alternatives. To address this threat, when defining the questions and the respective alternatives of answers, we considered results from the systematic literature mapping about tools we are carrying out and, in addition, we included “Others” as an alternative the participants could choose and provide further information. Still regarding construct validity, it is important to be aware that the results reflect the participants’ personal experiences, interpretations, and beliefs, and the answers embed subjectivity that could not be captured through the questionnaire.
Concerning Internal Validity, which refers to the ability of a new study to repeat the behavior with the same participants and objects, the main threat is participants provide inaccurate answers because they think they could be evaluated, since we asked them to inform their email to participate in the study. To mitigate this threat, we informed participants that data would not be evaluated individually and the participants’ identities would not be revealed.
Regarding External Validity, i.e., to what extent it is possible to generalize the study results, the main threat is the study sample, which comprises 74 ontology engineers and most of them are Brazilians. Ideally, the sample should be larger and the geographic distribution of the study participants more diverse. To minimize this threat, we invited people from diferent countries and organizations and also the authors of papers selected in an ongoing systematic literature mapping we are carrying out. Moreover, we asked people to invite other people freely. Even so, the sample presents limitations. Given that the participants’ profile directs influence the results, new studies involving participants with diferent profiles might produce diferent results.
Finally, the decisions and data interpretations made by the researchers afect Reliability Validity, which regards the influence of the researchers on the results and conclusions. The classification schemas for categorizing data collected in open questions and the classification of data in such categories involve a lot of judgment, as well as analyzing data and getting conclusions. Therefore, other researchers could obtain diferent results. To minimize this threat, the collected data was initially discussed and analyzed by five authors. Data and analysis results were then presented to the other two authors (more experienced researchers) who reviewed them and discussed the results with the other researchers to refine the conclusions and reach a consensus.
In the literature there are several works addressing tools for OE. Some of them propose tools, such
as WIDOCO [48], a wizard for documenting ontologies, ROBOT [49], a tool to automate workflows,
and Chowlk [50], a drawing library and a converter to draw conceptual models and convert them into
operational ontologies. Other works compare tools. For example, in [
A few works have reported surveys investigating OE tools. We highlight the work by Khondoker and Mueller [53], who conducted a survey with 32 practitioners who were asked to choose a tool and perform a modeling task using it. After that, they provide feedback regarding the tool used, considering their perceptions of the tool’s user-friendliness, learning time, problems, and their level of satisfaction with the tool, among others. The results showed that although there was a wide range of free tools, the participants chose only a few of them, most OE-specific tools, with Protégé being the predominant.
From the works we found in the literature, we consider [53] the one most related to ours because it consists of a survey with OE practitioners aiming at getting their perceptions about OE supporting tools. In both works, Protégé is the predominant tool and there has been a preference for using OE-specific tools. However, there are some important diferences between the work by Khondoker and Mueller [ 53] and ours. The survey presented in [53] focused on the use of the tools in the conceptual modeling task at hand instead of on the aspects that led the participants to choose the used tools and how they use them. Moreover, in that work, only aspects relevant to the performed modeling task were investigated. In contrast, our work investigates a more comprehensive set of aspects, provides an overview of the use of tools to support OE practice, gathers evidence on this subject, and raises some issues that can be addressed in future research.
This paper presented a survey that investigated the use of tools to support OE. The survey aimed to answer three research questions: (RQ1) Which tools have been used and which are their characteristics?; (RQ2) What benefits and dificulties have been perceived when using the tools?; and (RQ3) Which properties have been considered important in OE tools?
In summary, concerning RQ1, there has been a predominance of OE-specific tools, mainly Protégé, Visual Paradigm, and Astah, which support modeling and implementation activities. Generic tools have been used mainly to support OE initial phases, mostly Requirements Specification. The tools have been discovered mainly through recommendations from other people and there has been a preference for free tools. Tools have not provided adequate integrated support for the OE process. As for RQ2, increasing development process performance and improving ontology quality have been acknowledged as the main benefits. In contrast, integration problems and scope limitations have been the main dificulties faced by ontology engineers. Regarding RQ3, Functional Suitability, Reliability, Usability, and Availability have been considered the most relevant properties.
These results provide a panorama of tool usage in OE and raise some issues that can be addressed in future research. By looking at the pros and cons of using the tools, their weaknesses, and strengths we can set a path to provide advances in the topic. For example, improving the support for ontology requirements specification activities and connecting the produced artifacts to other tools would benefit OE practice. This study can be replicated in other countries, allowing for data comparison and evaluation of its applicability in broader contexts. It is also necessary to produce solutions to enable integrated support for the OE process and deal with scope limitations. Moreover, there is a need to improve the quality of the tools by considering the properties ontology engineers expect to find in them. Therefore, investigating such properties and defining strategies to achieve them would be welcome.
Currently, we are carrying out a systematic mapping of the literature to provide a panorama of the state of the art about OE tools. We intend to analyze the survey and systematic mapping results to obtain a multivocal overview of tool usage in OE and shine a light on the road ahead. Considering the studies’ results, we intend to produce a solution (e.g., a catalog, a repository, guidelines) to help ontology engineers search and use tools.
We would like to thank everyone who took part in the survey and spread our questionnaire to help us with this study. This research is supported by Coordination for the Improvement of Higher Education Personnel - Brazil (CAPES) - Finance Code 001, and Espírito Santo Research and Innovation Support Foundation (FAPES) - Processes 2023-5L1FC, 2021-GL60J, 2022-NGKM5, and T.O. 1022/2022. of living and reused ontologies: status, trends, findings and recommendations, The Knowledge Engineering Review 35 (2020) e4. doi:10.1017/S0269888920000065. [22] R. Studer, V. R. Benjamins, D. Fensel, Knowledge engineering: Principles and methods, Data & knowledge engineering 25 (1998) 161–197. [23] G. Guizzardi, Conceptualizations, modeling languages, and (meta) models, in: Databases and Information Systems IV: Selected Papers from the Seventh International Baltic Conference, DB&IS’2006, volume 155, IOS Press, 2007, p. 18. [24] O. Noppens, T. Liebig, Ontology patterns and beyond: towards a universal pattern language, in: Proceedings of the 2009 International Conference on Ontology Patterns-Volume 516, 2009, pp. 179–186. [25] A. Stadnicki, F. F. Pietroń, P. Burek, Towards a modern ontology development environment,
Procedia Computer Science 176 (2020) 753–762. [26] M. Fernández-López, A. Gómez-Pérez, N. Juristo, Methontology: from ontological art towards ontological engineering, American Asociation for Artificial Intelligence (1997). [27] N. F. Noy, D. L. McGuinness, et al., Ontology development 101: A guide to creating your first ontology, 2001. [28] O. Corcho, M. Fernandez-Lopez, A. Gomez-Perez, Ontological engineering: what are ontologies and how can we build them?, in: Semantic web services: Theory, tools and applications, IGI Global, 2007, pp. 44–70. [29] J. Guerson, T. P. Sales, G. Guizzardi, J. P. A. Almeida, Ontouml lightweight editor: a modelbased environment to build, evaluate and implement reference ontologies, in: 2015 IEEE 19th international enterprise distributed object computing workshop, IEEE, 2015, pp. 144–147. [30] C. M. Fonseca, T. P. Sales, V. Viola, L. B. R. da Fonseca, G. Guizzardi, J. P. A. Almeida, Ontologydriven conceptual modeling as a service., in: JOWO, 2021. [31] A. Gomez-Perez, J. Angele, M. Fernandez-Lopez, V. Christophides, A. Stutt, Y. Sure, et al., A survey on ontology tools, OntoWeb deliverable 1 (2002). [32] S. Easterbrook, J. Singer, M.-A. Storey, D. Damian, Selecting empirical methods for software engineering research, Guide to advanced empirical software engineering (2008) 285–311. [33] C. Wohlin, P. Runeson, M. Höst, M. C. Ohlsson, B. Regnell, A. Wesslén, Experimentation in software engineering, Springer Science & Business Media, 2012. [34] C. A. D. Gomes, J. S. Monfardini, Glaice Kelly Q.and Salamon, R. d. S. Sangali1, I. S. R. Timoteo, M. P. Barcellos, V. E. S. Souza, Investigating tool usage in ontology engineering: a survey - protocol & data extraction (2024). doi:10.6084/m9.figshare.25927438. [35] M. Musen, The protégé project: A look back and a look forward, AI Matters 1 (2015). doi:10.
1145/2557001.25757003. [36] J. L. Moreira, T. P. Sales, J. Guerson, B. F. Braga, F. Brasileiro, V. Sobral, Menthor editor: an ontology driven conceptual modeling platform (2016). [37] I. O. for Standardization (ISO), Iso/iec 25010:2011 - systems and software engineering – systems and software quality requirements and evaluation (square) – system and software quality models, ISO Standards (2011). URL: https://iso25000.com/index.php/en/iso-25000-standards/iso-25010. [38] J. Sanders, Linux, open source, and software’s future, IEEE software 15 (1998) 88–91. [39] J. Feller, Perspectives on free and open source software, MIT Press, 2005. [40] B. Fitzgerald, Has open source software a future? (2005). [41] L. Zhang, Y. Yu, J. Lu, C. Lin, K. Tu, M. Guo, Z. Zhang, G. Xie, Z. Su, Y. Pan, Orient: Integrate ontology engineering into industry tooling environment, in: The Semantic Web–ISWC 2004: Third International Semantic Web Conference, Hiroshima, Japan, November 7-11, 2004. Proceedings 3, Springer, 2004, pp. 823–837. [42] A. Alobaid, D. Garijo, M. Poveda-Villalón, I. Santana-Perez, A. Fernández-Izquierdo, O. Corcho, Automating ontology engineering support activities with ontoology, Journal of Web Semantics 57 (2019) 100472. [43] G. Dziwis, L. Wenige, L.-P. Meyer, M. Martin, Ontoflow: A user-friendly ontology development workflow., in: SemIIM, 2022. [44] A. Katifori, C. Halatsis, G. Lepouras, C. Vassilakis, E. Giannopoulou, Ontology visualization methods—a survey, ACM Computing Surveys (CSUR) 39 (2007) 10–es. [45] M. Dzbor, E. Motta, C. Buil, J. Gomez, O. Görlitz, H. Lewen, Developing ontologies in owl: An observational study (2006). [46] A. Zouaq, R. Nkambou, A survey of domain ontology engineering: Methods and tools, in:
Advances in intelligent tutoring systems, Springer, 2010, pp. 103–119. [47] A. S. Puspaningrum, S. Rochimah, R. J. Akbar, Functional suitability measurement using goaloriented approach based on iso/iec 25010 for academics information system, Journal of Information Systems Engineering and Business Intelligence 3 (2017) 68–74. [48] D. Garijo, Widoco: a wizard for documenting ontologies, in: The Semantic Web–ISWC 2017: 16th International Semantic Web Conference, Vienna, Austria, October 21-25, 2017, Proceedings, Part II 16, Springer, 2017, pp. 94–102. [49] R. C. Jackson, J. P. Balhof, E. Douglass, N. L. Harris, C. J. Mungall, J. A. Overton, Robot: a tool for automating ontology workflows, BMC bioinformatics 20 (2019) 1–10. [50] S. Chávez-Feria, R. García-Castro, M. Poveda-Villalón, Chowlk: from uml-based ontology conceptualizations to owl, in: European Semantic Web Conference, Springer, 2022, pp. 338–352. [51] E. Alatrish, Comparison some of ontology, Journal of Management Information Systems 8 (2013) 018–024. [52] T. Slimani, Ontology development: A comparing study on tools, languages and formalisms, Indian
Journal of Science and Technology 8 (2015). [53] M. R. Khondoker, P. Mueller, Comparing ontology development tools based on an online survey, World Congress on Engineering 2010 (WCE 2010), London, UK, 2010.