There are numerous ontology visualization systems, however, the choice of a visualization system is non-trivial, as there is no method for evaluation and comparing them, except for empirical experiments, that are subjective and costly. In this research, we aim to develop non-empirical metrics for ontology visualizations evaluation and comparing. First, we propose several half-formal metrics that require expert evaluation. These metrics are completeness, semanticity, and conservativeness. We apply the proposed metrics to evaluate and compare VOWL and Logic Graphs visualization systems. And second, we develop a completely computable measure for the complexity of ontology visualizations, based on graph theory and information theory. In particular, ontology visualizations are considered as hypergraphs and the information measure is derived from the Hartley function. The usage of the proposed information measure is exemplified by the evaluation of visualizations of the sample of axioms from the DoCO ontology in Logic Graphs and Graphol. These results can be practically applied for choosing ontology visualization systems in general and regarding a particular ontology.
Visualization of an ontology improves comprehension of knowledge it contains. There
are numerous ontology visualization systems, the reviews are presented in [
Little researches consider the evaluation of visualization systems in general. In [
Therefore, we aim to develop formal metrics for ontology visualizations estimation. ’Formal’ means that they must be objective and computable. Some metrics we propose require external knowledge of the language being visualized, its semantics, and knowledge of other visualization systems, therefore, they are half-formal and require expert evaluation. Another criterion is based on graph theory and information theory and is fully computable.
The outline of the paper is as follows: in Section 2 we propose several metrics for expert evaluation and in Section 3 we derive the information measure for ontology visualizations complexity. 2
First, we propose to consider several features of visualization systems that, though related to the formal properties, like completeness, still require expert evaluation, as they involve external knowledge. 2.1
The most important property of a visualization system is its completeness with respect to the language being visualized, because if a visualization system can not represent some axioms of an ontology, the system can not be applied to the ontology. In addition, a common reference language serves as a common denominator for comparing different visualization systems.
Ontologies are denoted on the OWL language [
The formal foundation of OWL is description logics (DLs) [
We suppose, the advantage of a visualization with respect to a reference language is that it improves comprehension of a formula with representing its semantic. Therefore, we propose to evaluate the ability of a visualization system to represent semantics of expressions.
We consider a diagram of a visualization system for a logical relation as semantical,
if it represents the semantic of the relation. For example, compare the visualization of
conjunction from Graphol [
Non-Empirical Metrics for Ontology Visualizations Evaluation... 3
Finally, we suppose that it is important to use existing graphic primitives from
mathematical theories, as in the other case, i.e. introducing new graphic primitives, instead of
helping a user to understand an ontology it forces him or her to learn just one more
language. Considering again the example above, in Graphol a user has to learn that hexagon
denotes conjunction, while if the Venn diagram was used, the user familiar with Venn
diagrams would have understood the diagram without additional instructions.
Therefore, we consider a visualization system as conservative, if it uses the existing graphic
primitives.
We provide an evaluation of the VOWL [
Here we see that VOWL can represent only 12 of 15 entities of the SHOIN description logic, therefore, its completeness rate is 0.8. The diagrams for concepts, conjunction, disjunction, and equivalence are semantical and conservative, as they are based on Venn diagrams. The diagram for roles can be considered as graph-theory based, therefore, it is also semantical and conservative. The diagram for negation is conservative, as it uses the sign of negation from logic, but it is not semantical since it doesn’t represent the semantic of negation.
In addition, we provide an example of comparing of visualization systems,
performed by the authors. We compare VOWL with Logic graphs (LGs) [
In the previous section, we proposed several metrics for expert-based evaluation. Properties like completeness and conservativeness are important for visualization systems evaluation, but it is hard to imagine that they would be fully computable. Thus, we propose one more approach to ontology visualization systems evaluation intended to be completely formal. This approach is based on information measuring.
N 6 Erexsitsrtiecntitoianl 7 rUesntirvicetrisoanl 8 Transitive role 9 Inverse role 10 Role hierarchy 11 reNsutrmicbtieorn 12 Nominal 13 Funrcotlieonal 14 iCncolnucseiopnt 15 eqCuoivnacleepntce 1 1 0 1 1 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 1
The intended information measure should estimate not the content of the ontology, as it is the same for each visualization, but the complexity of its form. It implies the following requirement for the intended information measure: – as it should estimate the complexity of an ontology visualization, it should depend on the complexity of its structure, in other words, on the number of nodes, edges, and types of edges; – it should be normalized, as visualizations of one and the same ontology in different visualization systems can have a different number of nodes and edges; – as it should measure the visualization complexity, a visualization with a greater number of nodes, edges, or edges types should have a higher value of the measure.
Non-Empirical Metrics for Ontology Visualizations Evaluation... 5 6 Erexsitsrtiecntitoianl 7 rUesntirvicetrisoanl 8 Transitive role 9 Inverse role 10 Role hierarchy 11 reNsutrmicbtieorn 12 Nominal 13 Funrcotlieonal 14 iCncolnucseiopnt 15 eqCuoivnacleepntce 1 0.8 0.93 3.1
Before defining the information measure, we have to define the formal framework. We propose to consider ontology visualization as a hypergraph. Simple graphs are not suitable for our goals as many ontology visualization systems use edges connecting more than two nodes. A hypergraph can be represented as an incidence matrix, therefore, an ontology visualization can be represented as an incidence matrix as well.
Consider the following axiom from the Document Components Ontology (DoCO)
[
chapter v 9contains:paragraph t section and its visualization in the Graphol system, see Fig. 3. Here edges denoting disjunction connect three nodes: ’graphol.paragraph’, ’graphol.section’ and ’or’, therefore, it is the hypergraph. The incidence matrix for this hypergraph is Table 4. Each node of the diagram corresponds to a row of the matrix and each relation – to a column. As nodes ’graphol.paragraph’, ’graphol.section’ and ’or’ are connected with the edge ’or’, the corresponding cells have value 1. In this research, we ignore the direction of edges for simplicity. (1) (2) (3) (4)
Non-Empirical Metrics for Ontology Visualizations Evaluation... 7 (5) (6) 3.2
There are researches on graph information measuring. The survey of graph entropy
measures is in [
First, we define the set of all different values of an incidence matrix A: Definition 1. For a given incidence matrix A the set of all values is fag. For undirected graph fag = f0; 1g, for directed fag = f 1; 0; 1g, for directed graph with loops fag = f 1; 0; 1; 2g and so on.
Now we define the set of all possible edges E : Definition 2. For a hypergraph H with a given set of nodes X the set of all possible edges E = fagX .
We are ready to define the information measure for hypergraph complexity
estimation by deriving it from the Hartley function [
logbjAj; where A is an arbitrary set and b – an arbitrary number. We substitute the number of edges jEj as jAj and the number of all possible edges jE j as b.
Definition 3. For a hypergraph H with a given set of nodes X, a given set of edges E and a set of corresponding incidence matrix values fag, the information I(H) is following:
I(H) = logjEjjEj = 1 jXj logjfagjjEj
Consider several simple graphs for illustration, see Fig. 4, its information estimation is at the Table 5. As wee see, H2 has a more complex structure comparing to H1 and, therefore, its information value is higher. H3 is directed and each directed edge contains less information, therefore, with the same number of edges its information value is lower compared to H1. Summing up, the information measure satisfies the desired properties.
H1 x1 e1 e2 x2 x3
H2 x1 e1 e2 x2 x3 e3
H3
x1
e1
e2
x2
x3
We provide an example of comparing ontology visualizations with the developed
information measure. Unlike expert-based evaluation, where we compared visualization
systems itself, for information measure we have to compare visualizations of a
particular ontology. We use the DoCO ontology as it is a real ontology, used in different
applications, and it contains nontrivial axioms. We visualized some axioms of this
ontology in Graphol and Logic Graphs (LGs) [
The example of an incidence matrix for Graphol was provided in Table 4. Now consider the example of the incidence matrix for Logic Graphs. The incidence matrix for the axiom 1 in Logic graphs is Table 7.
We compare LGs with Graphol by measuring information of the corresponding visualizations for the sample of axioms, presented in Table 6. The result is in Table 8. As we see, the average information of LGs on this sample is higher, than of Graphol. 4
In this research, we proposed several non-empirical metrics for ontology visualization evaluation and comparing. These metrics are divided into two groups. The first group includes three metrics: completeness, semanticity, and conservativeness. These metrics require expert evaluation and, therefore, they are half-formal. As an example, we compared two ontology visualization systems: VOWL and Logic Graphs.
The second group consists of the completely computable information measure, derived from the Hartley formula, that allows normalized measuring complexity of ontology visualizations, represented as hypergraphs with incidence matrices. As an example, we compared Logic Graphs with Graphol by measuring average information of visualizations of the sample of axioms from the DoCO ontology. 1 2 3 4 5 6 7 8 9
Non-Empirical Metrics for Ontology Visualizations Evaluation... 9 abstract v (chapter t section) u (9ispartof:bodymatter t f rontmatter) af terword v section u 9ispartof:backmatter) appendix v (section u headedcontainer) u (9ispartof:backmatter) backmatter v discourseelement u container
chapterlabel v :sectionlabel chaptersubtitle v 9ispartof:chapter
f igure v mata t milestone glossary v section u (9ispartof:backmatter t f rontmatter)
These results can be practically applied for choosing ontology visualization systems in general and regarding a particular ontology. Considering the presented examples, it is recommended to use LGs rather than VOWL in general, as it has higher scores of completeness, semanticity, and conservativeness, and for visualizing the mentioned fragment of the DoCO, as LGs has higher informativeness.