We introduce a web-based tool for interactive monitoring of quality and structural properties of a set of ontologies. The tool performs custom SHACL-based data validation as well as computation of various structural metrics. These metrics are based on the OntoMetrics framework, allowing to detect the type of an OWL ontology based on its structural properties (like depth, width, sibling count, etc.). We present a prominent use case of OBO ontology catalog, allowing to dynamically index the catalog through time, detect version changes, compute structural properties and providing interactive visualizations over these data.
understanding. Various communities, like OBO Foundry [
However, these ontology catalogs focus mainly on provenance metadata (e.g. who is the author, when the dataset was updated) and much less on the actual content metadata) (e.g. how does the data schema, classes, properties, look like). Lack of such metadata makes it dificult to monitor the quality of the ontologies, as well as to understand the interrelationships between the ontologies.
In our previous work [
Section 2 presents OBO Foundry, as well as existing tools for validating and monitoring quality of a set of ontologies. After a brief introduction of the dashboard framework we present the overall architecture as well as quality metric computation implementation in section 3. Information about the OBO Foundry use case and evaluation is in section 4 and the paper is concluded in section 5.
The OBO Foundry [
CEUR
ceur-ws.org
documentation, documented plurality of users, commitment to collaboration, locus of authority, naming
conventions, notification of changes, maintenance, term stability and responsiveness. The OBO Foundry
presented its own dashboard solution based on the ROBOT tool, which tests the OBO Foundry ontology
catalog with a certain periodicity and provides a report including the number of violations (quality
check) and also compliance with these principles for each ontology. According to OBO Foundry, this
solution should help developers more easily identify problem areas to be improved. In our previous
work we already proposed generalization of the validation using SHACL rules [
Existing works on structural metrics exist, ranging from simple computations of basic statistics (e.g.
class/axiom counts), e.g. via ROBOT[
The core component is the backend indexing data into the ELK stack. The indexer component
is responsible for validating OWL ontologies and indexing the results to Elastics. The validation is
performed using SHACL (Shapes Constraint Language) as described in [
Our novel contribution are structural metrics that are computed and indexed by the Ontology metrics
component2. The component implements selected metrics[
The size and scope of an ontology can be initially assessed through the overall number of classes. Additionally, we distinguish specific types of classes, such as leaf classes—those without any subclasses, typically representing concrete concepts—and tangled classes, which inherit from more than one superclass.
Ontological hierarchies exhibit both vertical and horizontal complexity, measured by their: • Breadth — the number of classes at each hierarchical level, reflecting how concepts are grouped and diferentiated. • Depth — the number of subclassing steps from the ontology root to leaf classes. This metric helps identify how deep the ontology goes in terms of specialization, and whether concrete concepts are well-distinguished from abstract categories.
These metrics focus on the similarity of classes in terms of their structural context. Specifically, the sibling count captures how many classes share the same direct superclass with a given class. Meanwhile, the sibling group size measures the number of direct subclasses of each non-leaf class, providing insight into the ontology’s overall branching pattern and the distribution of conceptual categories.
The representation of the metric as well as validation results is unified using a part of the DQV (Data Quality Vocabulary) We specialized DQV with a custom ontology defining specific metrics – the OQO (Ontology Quality Ontology)3 serves as a structured vocabulary for describing a wide range of ontology quality metrics. Each metric OQO is modeled as an instance of dqv:Metric, with SKOS annotations such as skos:prefLabel, and skos:definition, also characterized by its dqv:expectedDataType. Metrics are grouped into semantic dimensions using dqv:inDimension property, which captures the aspect of quality the metric focuses on. All dimensions are grouped thematically under broader categories using dqv:inCategory property.
Each measured value is expressed as a dqv:QualityMeasurement. Unlike ontology validation—where each constraint violation is stored individually—ontology metrics are indexed collectively for each ontology procession.
Structural metrics can serve as feedback for developers to check if the design matches the intention of the developed ontology and reveal possible anti-patterns. They can also be used to recognize specific types of ontologies, such as glossary, taxonomy, thesauri, etc. by reaching the specific values for specific metrics. For example, if the ontology has most of the classes in one hierarchical level (max breadth / class count > 0.9) and low average depth, we would categorize it as a glossary - a simple set of terms. If it had a larger average depth (>4) and low tangled class ratio (tangled class count / class count < 0.1), we would assume it as taxonomy to categorize objects. As the average depth, ratio of tangled classes, and average sibling group size (number of subclasses for each parent class) rise, so does the complexity and richness of an ontology.
We explored possibilities of acquiring metrics by using the Apache Jena library - either by executing SPARQL queries or using the API itself. Not every ontology is materialized (reasoned before serialization), so the OWL API is included, ready to use well-known reasoners, such as Pellet, Hermit, and ELK. Large ontologies require a lot of memory and performance to process, so to prevent unnecessary loading, the ontology version extractor 4 retrieves the ontology header with the current version, and checks for existing metrics, avoiding duplicate procession.
As a use case, we created dashboard for the OBO Foundry ontology catalog. OBO Foundry also ofers its own dashboard, which provides basic information about ontologies such as: tests on OBO Foundry principles, violation reports, metrics, etc. The main limitation is that this dashboard does not provide the whole violation report, so that the user interested in this will have to use the ROBOT tool, which does not simplify but only complicate the process of browsing the necessary data about a ontology. Also an important limitation of the OBO Dashboard is the lack of any filtering of the data, e.g. to view violations of a specific level, or moreover to view all violations related to a particular subject in a ontology.
The dashboard shows validation results as well as computed metrics and their evolution in time. Our dashboard consists of five main sections: ”All ontologies”, ”Single ontology”, ”Specific ontologies”, ”Ontologies metrics” and ”Single ontology metrics”. Section ”All ontologies” provides general data and statistics over catalog, ”Single ontology” section shows detailed violation report and OBO metrics, and ”Specific ontologies” is used for comparing ontologies. ”Ontologies metrics” and ”Single ontology metrics” are relevant, but with our own metrics. ”Ontologies metrics” compares ontologies with a table and shows the distribution of metrics values over a catalog, such as class count 5.
This paper focused on augmenting our dashboard solution with structural ontological metrics which is yet to be evaluated by the community. The dashboard framework provides insights for ontology curators to the current status of ontologies and their evolution in time and our previous testing with OBO users revealed the need to still improve the understandability of the Kibana user interface for non-technical people. Apart from UX improvements, we would like to also work on traceability features, including providing details on to the axioms not passing the validation, as well as computing and showing changes between ontology versions.