Metrics offer an objective assessment of the inner fabrics of ontologies. They allow us to quickly understand graph properties, logical complexity, completeness of human-centered annotations, or degree of interconnection. When analyzed historically, ontology metrics tell much about development decisions and the impact of changes and can be used for quality control measures. The NEOntometrics software allows calculating evolutional ontology metrics for git-based repositories and implements the majority of literature-proposed metric frameworks. This paper presents the recently added visualization capabilities for visualizing the differences between ontologies in a repository, assessing the change impact of the most recent commit, and examining ontology evolution.
recommendation systems [
One way to gather these kinds of information is the calculation of metrics. Ontology metrics translate the structural attributes of ontologies into objective, reproducible measurements. The measures cover the use of RDF(S) and OWL formalisms, graph properties, or human-centered annotations.
With NEOntometrics [
This paper presents a new integrated visualization feature for NEOntometrics. The evolutional developments and the differences of ontologies in a repository can now be displayed using diagrams. A comparison view for the last two versions of an artifact visualizes changed measures, thus allowing a quick overview of impacted structural attributes. Integrating diagram capabilities into software for calculating ontology metrics eases the consumption and productive use of the measures and hopefully contributes to the broader dissemination of ontology metrics for quality control.
The paper is structured as follows: The next section recapitulates the state of the art regarding ontology evolution and visualization approaches. Section three presents NEOntometrics, including the newly implemented visualization features. The research concludes with an outlook and a discussion.
2023 Copyright for this paper by its authors.
[
#
[
In this related work, we aim to recapitulate the current state regarding software support for ontology
evolution and evolution visualization. [
Analyzing and visualizing ontology evolution is a multifaceted field with various manifold approaches. As such, there are already extensive literature reviews that collected the relevant state of the art:
Novais et al. [
Lambrix et al. [
between two ontology versions
collaborative environments
❌ ❌ ❌ ❌ ✅ ✅ ❌ ✅
❌ ✅2 ✅3 ❌ ✅4 ✅5 ❌ (✅)6 unstable ontology subparts. To instantly understand the impact of modeling decisions in the Protégé editor, [17] calculates a taxonomy of changes, and [16] calculates numerical differences between the saved version and currently performed changes.
While the visualization view of the new NEOntometrics features (cf. Figure 4) has some similarities to [16], our approach does not calculate the metrics instantly for a change but regards the last two committed versions in a git repository. NEOntometrics also implemented a more significant number of ontology metrics, allowing us to choose from ~160 metrics from different frameworks. Compared to the current state of the art regarding tool-supported ontology evolution, the NEOntometrics approach is strictly focused on visualizing ontology metrics. Further, it not only regards two versions but calculates and visualizes the overall version history, thus allowing conclusions on overall evolutional processes and design decisions.
NEOntometrics is a web-based application that calculates evolutional ontology metrics of git-based
repositories. It iterates through a repository and calculates the respective measures for every ontology
in every commit if the file has changed. It comes with an interactive help page Metric Explorer, is open
source7, and is available online8. For more information on NEOntometrics, cf. [
As calculating the version history of ontology repositories can take a considerable amount of time, NEOntometrics works asynchronously. A new calculation can be triggered by pasting the URL to a given repository in the bottom text field of the Calculation Engine (cf. Figure 1). If the repository is not yet known to the system, it can be put in the calculation queue. Afterward, a separate worker application retrieves the ontology from the git repository and starts the analysis.
The ontology metrics can be retrieved as soon as the calculation is finished. Currently, NEOntometrics supports OQuaRE [19], OntoQA [20], oQual [21], the cohesion metrics by Yao et al. [22], the good ontology metrics by Fernández et al. [23], Orme et al.’s evolutional metrics [24], and the complexity metrics by Yang et al. [25]. Metrics can be run on the ontology as it is or on the inferred graph. However, as the inference engine can take a considerable time, it is only advised for small ontologies. 7 https://github.com/achiminator/NEOntometrics h http://neontometrics.com
After selecting the desired metrics and frameworks and triggering the retrieval process, the software shows a paginated tabular view of the ontology metrics. The button “show the analytic” opens the visualization page. The measures shown in the diagrams are always congruent to the ones selected during the ontology retrieval process.
Three visualizations are available to examine the most recent changes, compare ontologies in a repository, and visualize the ontology evolution. The figures below exemplify these visualizations for the SciData ontology9 for interoperable scientific data exchange [26].
The first chart (cf. Figure 2) displays differences between the repositories’ ontologies in a bar diagram. The bar diagram presents the last version of each ontology. It allows a quick comparison of the available artifacts in a repository. The diagram can be scrolled by dragging the picture to either side of the frame. Hovering over a measure shows the detailed measures. Ontologies can be selected and deselected by clicking on their name below the chart. The scaling changes dynamically depending on the sizes of the bars visible in the frame.
The bar chart in the example visualizes the axioms and classes of the repositories’ ontologies. The thermo, scidata, and cao files have the most classes and axioms. However, cao has many more axioms than thermo. Thus, one can conclude that the thermo ontology is more driven by a taxonomical class structure, while scidata and cao have more additional axioms than thermo.
Further examination revealed that cao and scidata are indeed more logically interconnected. In the given example, the bar chart visualizes discrete count-based measures. However, the bar chart visualization works also for ratio-based formulas. For example, the OntoQA metrics shown in Table 2 and visualized in the line chart in Figure 3 could also be used in the bar diagram.
The second visualization (cf. Figure 3) contains an evolutional view of one ontology for the selected ontology metrics. It allows an understanding of the impact of changes on the structural attributes and can be used to understand and evaluate design decisions. The drop-down menu at the top of the window changes the ontology to be visualized. The user can activate and deactivate measures by clicking their name on the legend below the chart, and the diagram scales automatically, similar to the bar chart. Hovering over a data point gives further details on the displayed value.
The diagram in Figure 3 shows four OntoQA measures for the scidata ontology. At first, it is evident that relationship diversity is constantly at 0, originating from the fact that no object properties are declared on classes. The class utilization and average population increase early in the lifetime of the ontology. Otherwise, they stay reasonably consistent. No individuals nor classes were added in this ontology, and the changes in classes and the subclass structure are relatively subtle. On the opposite, 9 https://github.com/stuchalk/scidata - The authors of SciData and this paper are not affiliated with each other. the attribute richness increases heavily. As the classes are somewhat consistent, these changes are driven by data and object properties. Also, the ontology describes a relatively high number of data and object properties compared to declared classes.
Calculation
The last analysis shows which measures have changed and by how much in the recent commit. It thus provides a detailed view of the ontologies’ last two versions and shall allow a quick assessment of the impact of the most recent change to evaluate the usefulness and identify eventually unintended sideeffects. Similar to the previous analysis, the ontology can be selected in the drop-down menu at the top of the application. In contrast to the other visualizations, this view shows not only the selected metrics but every metric that has changed. The little icon on the left indicates an increase (⬆️ ) or decrease (⬇️ ). If a metric does not change, it is not on the list.
The evolutional analysis of ontology metrics offers an objective insight into the evolvement of their structural attributes and can tell much about underlying design decisions. With the software NEOntometrics, tool support is available for analyzing large quantities of ontologies. The new visualization capabilities presented in this paper extend the application for an easily consumable humanoriented metric interface. We hope they ease the consumption of ontology metrics and contribute to a broader dissemination of metrics for quality control.
Application for the ontology metrics are manifold. For example, the metrics allow the user to understand evolutional processes or differences between various ontologies. Ontology metrics can aid in making better-informed reusing and development decisions as they allow quickly grasping an ontology’s inner fabrics. One can set objective and reproducible goals for ontology developments and track their achievement through metrics.
Soon, we plan on integrating even more visualization capabilities to allow a better deep dive into the ontology development processes. Further on the NEOntometrics roadmap is integrating more ontology metrics, e.g., to assess SHACL-constructs and use of custom vocabularies. In that sense, we are interested in the functionalities that the community would like to see implemented. A potential evaluation of the usefulness of the visualizations and metrics is also desirable in future work.