One of the main features of ontologies is to enable humans and machines to communicate through semantic formalization. With this purpose, ontologies are used in many areas of Computer Science, such as artificial intelligence, database and software engineering. It has been also the key component for information systems in a increasing variety of domains, well-known examples are the areas of biology, geography and law.
The Semantic Web Project contributed for popularization of ontology as an artifact to build the knowledge representation, shared and reused. With the reuse of ontologies costs can be reduced, mainly in development phase.
As presented in [9], the reuse of ontologies can be seen from two main points: one is to create, extend, specialize and adapt an ontology to build a new one; the other is to combine different ontologies in a single one.
A first problem for reuse is ontology search. Usually, repositories offer poor navigation interfaces [2]. The lack of good quality documentation about available ontologies also contributes to this difficulty. The Metadata models that describe ontologies have been proposed to deal with these problems [6]. The use of a standard vocabulary for ontologies is understood as a basic requisite for a good description of ontologies on the Web.
In this paper we extend and evaluate the previously proposed OMV metadata model through its application to a portal for Portuguese ontologies. The motivation is to help users searching for ontologies and ontology research related resources.
The paper is organized as follows: in Section 2 we present our extension which we call OMV-R; in Section 3 we describe a set of ontology metrics that we use in the description of the ontologies in our portal; in Section 4 we discuss the evaluation of the model; finally, in Section 5 we show the conclusion and future works.
Our extension of the OMV model and its application to a portal aims to assist users in the process of finding out ontologies for reuse.
The original model proposed by [8] does not include information about ontology evaluation and evolution. We included those elements in the model and we also propose more elaborated ways for describing ontology metrics.
According to [13], metrics are useful to help evaluate ontologies during the build and application phase, enabling a quick and simple understanding about what are being modeled through these structures and facilitating the control of its future evolution. This also holds for ontology reuse.
In [10] a study is performed about the maturity of the current process, methodologies and tools focused on ontology reuse. Their strengths and requirements are identified.
Specifically on the evaluation of ontologies, [16] presents a methodology called Requirements-Oriented Methodology for Evaluation Ontologies (ROMEO). This methodology propose a set of criteria for ontology evaluation. Inspired by these previous works, we considered that ontology evaluation is another important feature in the ontology selection process.
As OMV does not provide such information so important to enable reuse, we extended it with new classes as shown in Figure 1.
This section will present the classes, properties and relations such as were added to metadata model OMV.
OntologyEvaluation Class: this class represents the evaluation of an ontology. The properties of this class are:
-evaluationComments: free text that can be used to comment the evaluation criteria; -evaluationDate: date when the evaluation was made; -evaluationValue: value assigned to evaluation criteria. EvaluationCriteria Class: this class represents the evaluation criteria used to evaluate the ontology. In our implemented model, this class has an evaluation criteria set as discussed in [16]. The properties of this class are:
-criteriaQuestion: question describing the evaluation criteria; -criteriaType: define whether a question accept a textual or numeric answer.
Metric Class: this class represents metrics. Such metrics can be used to score an ontology, assigning a quantitative value, in order to assist users to select ontologies. In OMV-R there are 20 implemented metrics, which will be discussed in the next subsection. The properties of this class are:
name: metric name; usages: free text to describe and guide the user how to use the metric as well how the metric results may be interpreted.
OntologyMetric Class: this class represents instances of the calculated metrics of an ontology. One or more metrics may be assigned to an ontology, they can help users to select ontologies on the basis of different points of view. The properties of this class are:
comments: free text to give more information about a metric; -creationDate: date when the metric was calculated; value: value assigned to a metric.
Project Class: this class represents research or commercial projects that use a given ontology or a set of them. The properties of this class are:
-creationDate: date of the project's beginning; description: free text to describe the project's objectives and artifacts produced by them; name: the project's name.
Relations: here we present the relations among these new classes.
-evaluatedOntology: relation between ontologies with their respective ratings; -isBasedOn: reference to another ontology which was used in its construction, this reference allows to identify the reuse level that one ontology has, based on the quantity of references; -hasEvaluator: reference to a research group or enterprise responsible by ontology evaluation; -evaluationCriteria: reference to an evaluation criteria associated to an ontology; -hasSponsor: reference to the institution sponsoring the project; -usedByProject: reference to the project that uses an ontology; -hasCreator: reference to the research group or institution that has developed the metric; -usedMetric: reference to the metric assigned to a given ontology.
From the adaptation made, the OMV-R supports now the ontology evaluation description according to the methodologies proposed by [10], [16] and [13]. Another contribution is to allow OMV-R to store new metrics whenever is needed. These metrics may be used to evaluate an ontology over different points of view. Finally, it is expected that the extended model OMV-R provides a greater flexibility in the search of ontologies, as it offers to users additional information about reuse.
In the next section we described the set of implemented metrics.
The model has been extended to accommodate a more elaborated description through metrics. In our application of the model we have implemented ways of calculating a set of metrics to describe the ontologies in the repository. The choice of metrics were based on a detailed systematic review of the literature of ontology metrics. Following the systematic review their presentation was standardized in a common representation formalism, where C is used for classes, CS is used for set classes with subclasses, CR is used for set of root classes, CL is used for set of leaf classes, H C (CR j , CL i ) for hierarchy of classes where CR j is a subclass of CL i , prop for the function that relates the classes in a non-hierarchy and att for the function that relates classes with literal values (string), I for instances and inst(C) for the function of instantiated classes [13] [12].
Besides, we use Count for count, SuperClass for superclass function, SubClass for subclass function, M ax for maximum (M axDepth and M axW idth), Depth for depth, W idth for width, Avg for average (AvgDepth, AvgM axAvg, AvgW idth, AvgAttClass, AvgP ropClass, AvgP Class, AvgCRC, AvgCRCL, AvgCIC, AvgIC) and StdDev for standard deviation (StdDevIC).
All chosen metrics presented result in numeric values and their calculation could be implemented. The metrics were separated in three groups as follows:
1. Group 1 is simply the count of the basic elements of an ontology (classes, properties, and instances), metrics 1 to 7.
Number of root classes [15] [1]
where: |CR| is the set cardinality Number of leaf classes [15] [1] [14]
where: |CL| is the set cardinality
where: |C| is the set cardinality Number of properties (attributes) [1] Count(att
where: |att| is the set cardinality Number of properties (relations) [1] Count
where: |prop| is the set cardinality Number of properties [7] [1] [14]
where: |I| is the set cardinality 2. Group 2 represents the depth and width of the structure, metrics 8 to 12.
Maximum depth [7] [1] [14] M axDepth(H C ) = (8) the major ontology depth counted from each root class (CR j )
Average depth [1] [4] [15] AvgDepth(H C ) =
the average ontology depth added from all root classes (CR) by counted from each leaf class (CL i )
Average maximum depth to average depth [14] AvgM axAvg(
Maximum width [1] M axW idth(H C ) = (11) the major ontology width counted from each class with subclass (CS k )
Average width [1] AvgW idth(H C ) = (12) the average ontology width added from all classes with subclasses (CS) by counted from each class with subclass (CS k )
3. Group 3 represents the calculation of simple averages and standard deviation from combinations of the basic elements, metrics 13 to 20.
Average number of properties (relations) to total number of classes [1] [14]
Average number of properties to total number of classes [1] AvgP Class = (Count(att) + Count(prop))/Count(C)
Average number of leaf classes to total number of classes [1] [14] [13]
Average number of leaf classes to number of classes root [14] AvgCRCL = Count(CR)/Count(CL)
Average number of classes populated to total number of classes [13] AvgCIC(
Average of the total number of instances to total number of classes
Standard deviation of the total number of instances to total number of classes [13] StdDevIC(
The AvgIC(H C ) and Count(I) were calculated and used in StdDevIC(H C ). It is not directly referenced in papers obtained in the systematic review.
Note that OMV documentation refer to four metrics only: number of class, number of property, number of individuals and number of axioms.
OntoLP Portal1 offers ontologies and ontology research related resources, with special preference to ontologies written in Portuguese language. At Home link the objectives of the portal are shown, whilst the link Resources list ontologies and related works. The link About contains information referring to the research group involved in this project. Through the Contact link others research groups in the field can send their works and suggestions. Finally, the link Links shows researches groups that have been colaborated directly or indirectly with the OntoLP Portal as well as events related with ontology research.
An ontology search was performed just to build the repository. We conducted searches in Google 2 specifying the type of query (e.g. "field filetype: owl"), in Swoogle 3 , in OntoSelect 4 and pages of projects and research groups working with the subject. Also we announce the OntoLP portal in several Brazilian research lists and organized an event with the intention of receiving these resources.
As a result we find resources in different domains such as ecology, nanotechnology, art, curriculum, emotion, privacy, network of scientific knowledge, smartphones, music and stimulus equivalence. Today the OntoLP portal has 25 ontologies. Note that the currently portal does not use any advanced structure, such as a metamodel, to store the descriptions of ontologies. Such descriptions were manually built and hard coded in HTML, which are stored in the file system on the Web server. In order to improve the current services of OntoLP portal, a new prototype application was developed that uses the OMV-R to keep the descriptions of ontologies. The next subsection will take a look deeper how this new version of OntoLP portal was developed.
A tool has been developed in order to automate the receiving process. This tool reads the ontology and includes information about ontology metrics. It has also another interface for the inclusion of other metadata, to be used in the description of ontologies in the OntoLP Portal, that corresponds with instances of OMV-R. Ontologies are thus received through a submission form. Figure 2 shows the flow of information during the ontology reception process. Besides the ability to create OMV-R's instances, the tool offers others services such as information retrieval of the ontologies, ability to create and retrieve ontology evaluation instances as well as the metrics values for each OMV-R instance that represents a description of the ontology. The recovery of information from ontologies is made through a set of methods that read data from OMV-R and then, record the information on a XML file wich will be used by presentation layer of the prototype application. Therefore, this XML file works like a communication area between OMV-R ontology and end user interface.
Finally, the tool uses the Jena's SPARQL query engine to implement the advanced search service in the OntoLP Portal. The query is performed on the OMV-R, specifically over the Ontology class to retrieve all the ontologies that match the user's filter.
Figure 3 presents the OMV-R metric instances for the stored ontologies. Notice that the OntoLP Portal prototype application can be accessed through following link: http://www.ontolp.com.br. Both versions of portals are available for readers to discover the differences between portals (take a look at previous page on footnote for the URL of the current portal).
We applied two questionnaires to evaluate the adaptations made to the portal which follows the new proposed model.
The goal of the first questionnaire was to evaluate the opinion of a users' group, researchers and engineering of ontologies about the usefulness of metrics in specific repositories, in this case in OntoLP portal.
In the first phase, the questionnaire (in print format) was applied to people in national workshop about ontologies. In the final phase the questionnaire (in digital format) was applied to visitors of the OntoLP portal. The questionnaire was available for a month (December 6, 2009 until January 6, 2010).
The final result showed that 76 % respondents have developed ontologies, 83 % have searched for ontologies written in English, Portuguese, or Spanish. The language most cited was English and Portuguese was in the second position. The sample consisted of 14 PhD, 11 MSc and 5 senior experts. At last, ontologies in many domains were mentioned.
We observed that there was no preference regarding groups 1, 2 and 3, their usefulness was better appreciation as a whole. Most people consider the use of metrics important for the portal.
The respondents commented about the lack of more specific metrics (total number of relations' types, where such relations would be " is-a", " part-of") and about the lack of the presentation in percentages the results (percentage of classes that did not have superclasses). These suggestions will be considered as future work.
The goal of the second questionnaire was to get impressions about the new services of the OntoLP Portal such as description and retrieval of ontologies based on the OMV-R metadata.
Here, the sample was formed by 19 respondents. They were instructed to visit the OntoLP Portal and to evaluate the new services such as ontology submission and search. Next, the respondents were oriented to compare the services of the current version of the portal. When asked whether these new services have eased the ontology search process, about 58% of respondents agreed, 26% remained neutral and about 15% disagreed.
Despite some disagreement about how easy is to locate ontologies in the new OntoLP Portal, about 95% of respondents believe that the new ontology submission and search services are better than the original one.
In addition to the two close questions about the satisfaction and improvement about the ontology search services, the respondents were conducted to comment referring to other information that could be added to the metadata model OMV-R. About 25% of respondents have contributed with suggestions; 32 % said that the new set of the model's information was sufficient or reasonable; and 36 % didn't answer.
The suggestions are related mainly with the ontology domain, and this information is the best way to locate ontologies. Other respondent claimed that the new portal should offer information about the relationship among ontologies, as evidence: "At first glance I would say that at some point we will need a hierarchy of ontologies (when there are more than 20 or 30 ontologies) and some mechanism to relate them (meta-ontology)". Notice tha both questionaries can by accessed through the following links: http://www.inf.pucrs.br/˜ontolp/questionario.php (for metric's questions) and http://www.ontolp.com.br/questionarioOMVR.php (for metamodel and prototype application's questions)
This paper presented an extended model based on OMV, which is called OMV-R. The new model includes key ontology metrics identified through literature review. The model was applied to OntoLP portal which aims to maintain and distribute Portuguese resources. The model is used for the description of the resources, and was also useful as a basis for new search and retrieval services. Finally, this paper discussed the surveys performed to evaluate the model, the new portal and the use of metrics, where the objective was to assess with the community the benefits and contributions of the model for ontology description and location as well as the process of reuse of such resources.
We hope to have contributed with ideas for describing ontologies that can be included in previously proposed standards. We have also shown its usefulness for the development of software that can help researchers and engineers to find and inspect available ontologies in an organized and efficient way.
Potential future work are: (i ) adaptation of the OMV-R API to work with the OMV-R model stored in commercial database as well as free ones; (ii ) the development of Web services for applications; (iii ) improving the advanced search interface of the new OntoLP Portal, in order to allow users query any property of the metamodel through free-text. Nowadays only three properties have been added to advanced search interface; (iv ) building mechanisms for visualization of ontologies described in the metadata model OMV-R.