pCaoppeyrrsightby© 2th0e10 fpoarpethrse' inaduivthidoursa.l Copying permitted only for private and aepdcuaibtdloisershm.eidc paunrdposecos.pyTrihgihstedvolubmye itiss
10-Jun-2010: submitted by Christoph Lange 11-Jun-2010: published on CEUR-WS.org
Using extended metadata model OMV and metrics in OntoLP Portal Anderson Bestteti, Larissa A. de Freitas, and Renata Vieira
College Computer, PPGCC, Ipiranga Avenue, 6681, Partenon, Building 32, Porto Alegre, RS, Brazil {anderson.bestteti,larissa.freitas,renata.vieira}@pucrs.br http://www.inf.pucrs.br/~linatural/ 1
One of the main features of ontologies is to enable humans and machines to communicate through semantic formalization. With this purpose, ontologies are used in many areas of Computer Science, such as arti cial intelligence, database and software engineering. It has been also the key component for information systems in a increasing variety of domains, well-known examples are the areas of biology, geography and law.
The Semantic Web Project contributed for popularization of ontology as an artifact to build the knowledge representation, shared and reused. With the reuse of ontologies costs can be reduced, mainly in development phase.
As presented in [
A rst problem for reuse is ontology search. Usually, repositories o er poor
navigation interfaces [
In this paper we extend and evaluate the previously proposed OMV metadata model through its application to a portal for Portuguese ontologies. The motivation is to help users searching for ontologies and ontology research related resources.
The paper is organized as follows: in Section 2 we present our extension which we call OMV-R; in Section 3 we describe a set of ontology metrics that we use in the description of the ontologies in our portal; in Section 4 we discuss the evaluation of the model; nally, in Section 5 we show the conclusion and future works. 2
Our extension of the OMV model and its application to a portal aims to assist users in the process of nding out ontologies for reuse.
The original model proposed by [
According to [
In [
Speci cally on the evaluation of ontologies, [
As OMV does not provide such information so important to enable reuse, we extended it with new classes as shown in Figure 1. 2.1
Description of the new classes, properties and relations This section will present the classes, properties and relations such as were added to metadata model OMV.
OntologyEvaluation Class: this class represents the evaluation of an ontology. The properties of this class are: { evaluationComments: free text that can be used to comment the evaluation criteria; { evaluationDate: date when the evaluation was made; { evaluationValue: value assigned to evaluation criteria.
EvaluationCriteria Class: this class represents the evaluation criteria used
to evaluate the ontology. In our implemented model, this class has an evaluation
criteria set as discussed in [
OntologyMetric Class: this class represents instances of the calculated metrics of an ontology. One or more metrics may be assigned to an ontology, they can help users to select ontologies on the basis of di erent points of view. The properties of this class are: { comments: free text to give more information about a metric; { creationDate: date when the metric was calculated; { value: value assigned to a metric. Project Class: this class represents research or commercial projects that use a given ontology or a set of them. The properties of this class are: { creationDate: date of the project's beginning; { description: free text to describe the project's objectives and artifacts produced by them; { name: the project's name.
Relations: here we present the relations among these new classes. { evaluatedOntology: relation between ontologies with their respective ratings; { isBasedOn: reference to another ontology which was used in its construction, this reference allows to identify the reuse level that one ontology has, based on the quantity of references; { hasEvaluator: reference to a research group or enterprise responsible by ontology evaluation; { evaluationCriteria: reference to an evaluation criteria associated to an ontology; { hasSponsor: reference to the institution sponsoring the project; { usedByProject: reference to the project that uses an ontology; { hasCreator: reference to the research group or institution that has developed the metric; { usedMetric: reference to the metric assigned to a given ontology.
From the adaptation made, the OMV-R supports now the ontology
evaluation description according to the methodologies proposed by [
In the next section we described the set of implemented metrics. 3
The model has been extended to accommodate a more elaborated description
through metrics. In our application of the model we have implemented ways of
calculating a set of metrics to describe the ontologies in the repository. The choice
of metrics were based on a detailed systematic review of the literature of ontology
metrics. Following the systematic review their presentation was standardized in
a common representation formalism, where C is used for classes, CS is used for
set classes with subclasses, CR is used for set of root classes, CL is used for set
of leaf classes, HC (CRj ; CLi) for hierarchy of classes where CRj is a subclass
of CLi, prop for the function that relates the classes in a non-hierarchy and att
for the function that relates classes with literal values (string ), I for instances
and inst(C) for the function of instantiated classes [
Number of root classes [
Number of classes [
Besides, we use Count for count, SuperClass for superclass function, SubClass for subclass function, M ax for maximum (M axDepth and M axW idth), Depth for depth, W idth for width, Avg for average (AvgDepth, AvgM axAvg, AvgW idth, AvgAttClass, AvgP ropClass, AvgP Class, AvgCRC, AvgCRCL, AvgCIC, AvgIC) and StdDev for standard deviation (StdDevIC).
All chosen metrics presented result in numeric values and their calculation could be implemented. The metrics were separated in three groups as follows: 1. Group 1 is simply the count of the basic elements of an ontology (classes, properties, and instances), metrics 1 to 7.
Count(CR) = jCRj where: jCRj is the set cardinality
Count(CL) = jCLj where: jCLj is the set cardinality
Count(C) = jCj where: jCj is the set cardinality
Count(att) = jattj where: jattj is the set cardinality
Count(prop) = jpropj
where: jpropj is the set cardinality
Number of properties [
Count(att) + Count(prop)
Count(I) = jIj
where: jIj is the set cardinality
2. Group 2 represents the depth and width of the structure, metrics 8 to 12.
(1)
(2)
(3)
(4)
(5)
(6)
(7)
Maximum depth [
root class (CRj ) M axDepth(HC ) =
AvgDepth(HC ) = the average ontology depth added from all root classes (CR)
by counted from each leaf class (CLi)
Average maximum depth to average depth [
AvgM axAvg(HC ) = M axDepth(HC )=AvgDepth(HC )
Maximum width [
M axW idth(HC ) = the major ontology width counted from each class with subclass (CSk)
AvgW idth(HC ) = the average ontology width added from all classes with subclasses (CS) by counted from each class with subclass (CSk) 3. Group 3 represents the calculation of simple averages and standard deviation from combinations of the basic elements, metrics 13 to 20.
Average number of properties (attributes) to total number of classes [
AvgAttClass = Count(att)=Count(C)
Average number of properties (relations) to total number of classes [
AvgP ropClass = Count(prop)=Count(C)
Average number of properties to total number of classes [
AvgP Class = (Count(att) + Count(prop))=Count(C)
Average number of leaf classes to total number of classes [
AvgCRC = Count(CR)=Count(C)
AvgCRCL = Count(CR)=Count(CL)
Average number of classes populated to total number of classes [
AvgIC(HC ) = Count(I)=Count(C)
Standard deviation of the total number of instances to total number of
classes [
The AvgIC(HC ) and Count(I) were calculated and used in StdDevIC(HC ). It is not directly referenced in papers obtained in the systematic review.
Note that OMV documentation refer to four metrics only: number of class, number of property, number of individuals and number of axioms. 4
OntoLP Portal1 o ers ontologies and ontology research related resources, with special preference to ontologies written in Portuguese language. At Home link the objectives of the portal are shown, whilst the link Resources list ontologies and related works. The link About contains information referring to the research group involved in this project. Through the Contact link others research groups in the eld can send their works and suggestions. Finally, the link Links shows researches groups that have been colaborated directly or indirectly with the OntoLP Portal as well as events related with ontology research.
An ontology search was performed just to build the repository. We conducted searches in Google 2 specifying the type of query (e.g. \ eld letype: owl"), in Swoogle 3, in OntoSelect 4 and pages of projects and research groups working with the subject. Also we announce the OntoLP portal in several Brazilian 1 http://www.inf.pucrs.br/~ ontolp/index.php 2 http://www.google.com.br/ 3 http://swoogle . umbc.edu / 4 http://olp.dfki.de/ontoselect/ research lists and organized an event with the intention of receiving these resources.
As a result we nd resources in di erent domains such as ecology, nanotechnology, art, curriculum, emotion, privacy, network of scienti c knowledge, smartphones, music and stimulus equivalence. Today the OntoLP portal has 25 ontologies. Note that the currently portal does not use any advanced structure, such as a metamodel, to store the descriptions of ontologies. Such descriptions were manually built and hard coded in HTML, which are stored in the le system on the Web server. In order to improve the current services of OntoLP portal, a new prototype application was developed that uses the OMV-R to keep the descriptions of ontologies. The next subsection will take a look deeper how this new version of OntoLP portal was developed. 4.1
Infrastructure A tool has been developed in order to automate the receiving process. This tool reads the ontology and includes information about ontology metrics. It has also another interface for the inclusion of other metadata, to be used in the description of ontologies in the OntoLP Portal, that corresponds with instances of OMV-R. Ontologies are thus received through a submission form. Figure 2 shows the ow of information during the ontology reception process.
Besides the ability to create OMV-R's instances, the tool o ers others services such as information retrieval of the ontologies, ability to create and retrieve ontology evaluation instances as well as the metrics values for each OMV-R instance that represents a description of the ontology. The recovery of information from ontologies is made through a set of methods that read data from OMV-R and then, record the information on a XML le wich will be used by presentation layer of the prototype application. Therefore, this XML le works like a communication area between OMV-R ontology and end user interface.
Finally, the tool uses the Jena's SPARQL query engine to implement the advanced search service in the OntoLP Portal. The query is performed on the OMV-R, speci cally over the Ontology class to retrieve all the ontologies that match the user's lter. We applied two questionnaires to evaluate the adaptations made to the portal which follows the new proposed model.
The goal of the rst questionnaire was to evaluate the opinion of a users' group, researchers and engineering of ontologies about the usefulness of metrics in speci c repositories, in this case in OntoLP portal.
In the rst phase, the questionnaire (in print format) was applied to people in national workshop about ontologies. In the nal phase the questionnaire (in digital format) was applied to visitors of the OntoLP portal. The questionnaire was available for a month (December 6, 2009 until January 6, 2010).
The nal result showed that 76 % respondents have developed ontologies, 83 % have searched for ontologies written in English, Portuguese, or Spanish. The language most cited was English and Portuguese was in the second position. The sample consisted of 14 PhD, 11 MSc and 5 senior experts. At last, ontologies in many domains were mentioned.
We observed that there was no preference regarding groups 1, 2 and 3, their usefulness was better appreciation as a whole. Most people consider the use of metrics important for the portal.
The respondents commented about the lack of more speci c metrics (total number of relations' types, where such relations would be \ is-a", \ part-of") and about the lack of the presentation in percentages the results (percentage of classes that did not have superclasses). These suggestions will be considered as future work.
The goal of the second questionnaire was to get impressions about the new services of the OntoLP Portal such as description and retrieval of ontologies based on the OMV-R metadata.
Here, the sample was formed by 19 respondents. They were instructed to visit the OntoLP Portal and to evaluate the new services such as ontology submission and search. Next, the respondents were oriented to compare the services of the current version of the portal. When asked whether these new services have eased the ontology search process, about 58% of respondents agreed, 26% remained neutral and about 15% disagreed.
Despite some disagreement about how easy is to locate ontologies in the new OntoLP Portal, about 95% of respondents believe that the new ontology submission and search services are better than the original one.
In addition to the two close questions about the satisfaction and improvement about the ontology search services, the respondents were conducted to comment referring to other information that could be added to the metadata model OMVR. About 25% of respondents have contributed with suggestions; 32 % said that the new set of the model's information was su cient or reasonable; and 36 % didn't answer.
The suggestions are related mainly with the ontology domain, and this information is the best way to locate ontologies. Other respondent claimed that the new portal should o er information about the relationship among ontologies, as evidence: \At rst glance I would say that at some point we will need a hierarchy of ontologies (when there are more than 20 or 30 ontologies) and some mechanism to relate them (meta-ontology)". Notice tha both questionaries can by accessed through the following links: http://www.inf.pucrs.br/~ontolp/questionario.php (for metric's questions) and http://www.ontolp.com.br/questionarioOMVR.php (for metamodel and prototype application's questions) 5
This paper presented an extended model based on OMV, which is called OMVR. The new model includes key ontology metrics identi ed through literature review. The model was applied to OntoLP portal which aims to maintain and distribute Portuguese resources. The model is used for the description of the resources, and was also useful as a basis for new search and retrieval services. Finally, this paper discussed the surveys performed to evaluate the model, the new portal and the use of metrics, where the objective was to assess with the community the bene ts and contributions of the model for ontology description and location as well as the process of reuse of such resources.
We hope to have contributed with ideas for describing ontologies that can be included in previously proposed standards. We have also shown its usefulness for the development of software that can help researchers and engineers to nd and inspect available ontologies in an organized and e cient way.
Potential future work are: (i ) adaptation of the OMV-R API to work with the OMV-R model stored in commercial database as well as free ones; (ii ) the development of Web services for distributed applications; (iii ) improving the advanced search interface of the new OntoLP Portal, in order to allow users query any property of the metamodel through free-text. Nowadays only three properties have been added to advanced search interface; (iv ) building mechanisms for visualization of ontologies described in the metadata model OMV-R.