In this article we report on our experiences using the OWL 2 reasoners HermiT and Pellet to validate domain-speci c languages dened using UML-like metamodels. Currently there exists few tools to validate metamodels. Using ontologies and reasoners to de ne and validate metamodels is a possible solution to this problem. We evaluate the reasoners according to expressiveness, correctness, performance and problem reporting capabilities. We use metamodels from the Atlantic Zoo metamodel repository as input for our comparison, and report on our experiences with the zoo.
Model Driven Engineering (MDE) [
The study of metamodeling languages has led to a number of practical tools such as model repositories, diagram editors, model transformation tools and code generation tools that simplify enormously the creation of new development tool chains that use UML or domain-speci c modeling languages in software development projects. However, the topic of metamodel validation is seldom discussed and there is a lack of tool support for this task.
In our previous work [
We proceed as follows: in Section 2 we discuss the need for metamodel validation. In Section 3 we detail the objectives of the study. In Section 4, Section 5 and Section 6 we present the materials used in the investigation. Section 7 presents the results, and Section 8 is the conclusion.
The creation of a new metamodel is not a simple task since it requires a good knowledge of the problem domain and how to capture its properties using a small set of metamodeling concepts. A metamodel often includes many constraints on how concepts in a model can be related to each other, such as multiplicity, domain and range, composition and subset constraints. If not chosen carefully, these constraints may lead to contradictions.
For example, lets assume we are creating a metamodel for a Statechart language that includes concepts such as simple states, composite states (that can contain other states) and transitions between states. A fragment of such a metamodel is shown in the top of Figure 1. This metamodel contains a rather obvious contradiction: there is an association with the minimum multiplicity larger than the maximum multiplicity. This contradiction means that there are no valid models that can make use of this association.
Another possible contradiction is shown in the bottom of Figure 1. The depicted metamodel requires each instance of State to be in a composite relationship with an instance of CompositeState. However, as CompositeState is a subclass of State it inherits this requirement, which leads to each instance of CompositeState having to be in a composite relationship with another instance of CompositeState. Given a nite set of instances of CompositeState, it is impossible not to violate one of the composition requirements | elements having only one owner and there being no cycles in the compositon. As a consequence there is no model with a nite number of elements that conforms to the given metamodel.
As shown in the examples, we consider it necessary to validate a new
metamodel to ensure that no such problems exist before actual tools and models are
created.
To tackle the previous problem, we are working towards the integration of an
automatic metamodel validation tool in a metamodel editor. The validation is based
on the idea of translating a metamodel into an ontology and using a reasoner
to check the ontology for consistency and class satis ability. We have decided to
use the OWL 2 Web Ontology Language [
OWL 2 provides classes, properties, individuals, and data values, that we
can use to de ne the main concepts of UML-like metamodeling languages as
shown by the authors [
We have noticed that OWL 2 is not expressive enough for some restrictions in
metamodels. For instance, there is no decidable combination of axioms in OWL
2 to express UML composition and its relevant restrictions. This problem could
be solved by writing extra rules in another language. One possibility is to use
the OCL Object Constraint Language [
To validate an ontology expressed in OWL 2 and SWRL, we need a reasoner
that supports both languages. There are tools which provide reasoning services
for OWL 2 ontologies as well as support of DL-safe SWRL rules [
It is still an open question if this idea is feasible in practice. In the next section we present a study that tries to answer this question. 3
The main goal of this article is to study the feasibility of using OWL 2 and SWRL reasoners to validate metamodels for domain speci c languages. The feasibility criteria is as follows:
One of the main objectives of our research is to judge the expressiveness of OWL 2 and SWRL in terms of expressing the concepts of Domain Speci c Languages developed by using an UML-like Metamodeling language. 2. Correctness of Reasoners Some of the existing reasoners are experimental research tools under constant development. We would like to know whether the current implementation of each reasoner is mature enough to produce reliable results.
We want to know how much processing time is required to validate a metamodel. Can a modern desktop computer perform this task with ease? Can the validation be performed on the y while editing a metamodel or when we save the metamodel into a le?
Are the messages produced by an OWL2 reasoner intelligible by the metamodel creator? Can we trace back inconsistencies in a generated ontology to the original metamodel elements? 5. Problems in published metamodels Since we are also planning to validate many existing metamodels, we are also interested to know if there are problems in published metamodels. 3.1
In order to answer the previous questions we select a number of metamodels, transform them into an ontology and check them for consistency and satis ability using a reasoner. We measure execution time and observe eventual error messages.
In order to asses the maturity of the reasoner implementation, we have decided to select two independently developed reasoners and compare their output.
To ensure that the metamodels to validate are representative we have not
created the metamodels ourselves. Instead we have decided to process all the
metamodels in a public repository: the Atlantic Metamodel Zoo [
We describe the materials for our study in the next three sections. 4
The Atlantic Metamodel Zoo is a library of metamodels maintained by the AtlanMod team. At the time of writing this text, it contains 286 metamodels representing diverse domains such as LateX, Java, HTML. The metamodels are available in di erent languages, including UML 2, OWL, KM3 etc.. We choose to validate the metamodels expressed in UML 2.
Metamodels expressed in UML 2 are identi ed by their headers and the elements for describing its structure. The headers must include the namespace URL for the UML metamodel and elements pre xed with uml:. An example of such a metamodel is shown in Listing 1.1. This example shows two classes: Book and Chapter associated by a simple composition.
The metamodels in the Atlantic Zoo vary considerably in size and complexity. In any case, these metamodels are considerably smaller than the UML metamodel and they do not contain OCL constraints. We consider that the metamodels in the Atlantic Zoo provide a good sample of metamodels for domain-speci c languages.
The transformation from a metamodel expressed using UML to OWL 2 is
implemented using the Model-to-Text transformation tool MOFScript [
MOFScript consists of two parts: the MOFScript tool and the MOFScript language. The MOFScript tool is an implementation of the MOFScript language and it provides ways of editing, compiling and executing MOFScript transformation code. MOFScript transformations are MOFScript language programs that de ne a set of rules that can translate metamodel elements and relations between them to expected output through print statements. The MOFScript transformation code is written based on one or more input metamodels, then compiled and executed on one or more loaded input les which contains models conforming to the input metamodels.
The input metamodel in our implementation is UML2 2.1.0, the input le is a
.uml le which contains a UML metamodel in XML syntax. There are two output
les: one contains an OWL 2 ontology written in OWL 2 functional syntax [
As mentioned in the previous sections, there are some restrictions of metamodeling concepts that have to be written in SWRL rules. However, ontology reasoners can not process OWL 2 functional syntax with SWRL rules embedded. As a solution, we create extra SWRL rules written in RDF syntax in a separate OWL document and import this ontology in the main ontology written in OWL 2 functional syntax. For instance, the generated ontologies of the metamodel shown in Listing 1.1 are Listing 1.2 and Listing 1.3. 6
Once the ontology of a metamodel has been generated by using the transformation described before, we use a reasoner to check the ontology consistency and satis ability. In terms of an ontology representing a metamodel: 1. Metamodel consistency means that there exists at least one model that conforms to that metamodel. 2. Metaclass satis ability means there is at least one model element that can belong to the class (without making the metamodel inconsistent). In the context of a metamodel, instantiating an unsatis able metaclass leads to a contradiction and therefore the metaclass cannot be used in any model. This is also refered as concept satis ability in the services provided by the reasoners.
The selection criteria for a reasoner were complete support for OWL2, SWRL and also that the reasoner is freely available as open source. The rst two requirements are motivated by the ontologies used in our study. The last requirement ensures that the study is easily repeatable by others.
Based on these criteria, we have chosen the following two reasoners:
These reasoners are either already available as a Protege (an open source ontology editor) plugin or will be supported. They can also all be used from the command line.
In our experiment, we run HermiT 1.2.1 from the command line and use the options -k and -U to check the consistency and concept satis ability respectively. In the case of Pellet, we use version 2.0.1 from the command line and use the options consistency and unsat to check the consistency and concept satis ability respectively. 7
Since metamodel validation involves both reasoners and metamodels, the results of our study depends on two aspects: the reasoning tools an the input metamodels. First, we evaluate the reasoners in terms of maturity, performance, expressiveness, and their problem reporting mechanism. Then we investigate any reported problems in the published metamodels. 7.1
The expressiveness is evaluated in two aspects: the concepts contained in the metamodels and the Description Logic which the reasoners are based on.
On one hand, the metamodels in the Atlantic Zoo are simple in the sense that they do not cover all the concepts that are within the ability of the UML metamodeling language. For example, there are no metamodels that contain any ordered properties, ordered composition or ordered subset properties. The constraints of these concepts are expressed in SWRL rules.
On the other hand, we notice problems with UML composition: translating composition to OWL 2 requires the transitivity axiom and irre exive axiom, whereas the OWL 2 speci cation forbids the two axioms used on the same object property. This can be solved by expressing them in SWRL, although SWRL rules are only applied on individuals rather than classes. 7.2
Both HermiT and Pellet can process all the ontologies generated from the Atlantic Zoo without generating any run time error or getting into an in nite loop. Furthermore, both reasoners always produce the same report for all the metamodels. This is signi cant because they have been implemented independently by two di erent development groups. Based on this, we claim that the current implementation of both HermiT and Pellet are mature enough for the task of validating metamodels.
During our experiment, we found that Pellet 2.0.1 supports OWL 1.1
functional syntax [
We run our performance experiments using a desktop computer with an Intel Core 2 6400 processor running at 2.13GHz, 2GB of RAM, Linux Fedora Core 10 and Java 1.6.0 10 rc2. We process each metamodel three times and report the average execution time.
Figure 2 shows the time required for HermiT and Pellet to process each metamodel. The x-axis represents the number of axioms in a generated ontology. The y-axis represents the total time in seconds necessary to load an ontology representing a metamodel, and check it for consistency and satis ability. As for the counting of number of axioms, though, certain expressions are a combination of OWL 2 axioms. For instance, the minimum cardinality in the Figure 1 is expressed as follows, which is a combination of axiom SubClassOf and ObjectMinCardinality, we count this as one axiom.
SubClassOf(State ObjectMinCardinality(2 State_outgoing ))
As we can observe from the gure, HermiT is consistently faster than Pellet, but both tools can process each metamodel in less than four seconds.
Based on this, we consider that the e ency of the two reasoners is satisfactory for the given problem and an average desktop computer. 7.4
Pellet produces an error message if a given input contains a syntax error or the wrong le header. Such input cannot be further checked for consistency and unsatis ability. When checking consistency, Pellet simply shows if the input ontology is consistent or not. When checking unsatis ability, given a consistent ontology, Pellet reports the number of elements checked, time used and number of unsatis able elements. If there are unsatis able elements, the speci c class names are shown.
Pellet provides an option to print verbose information while reasoning. The printed information indicating input size, speci c numbler of classes, properties and individuals, expressivity, used time summary, etc.. Once an element that leads to an inconsistency is found, Pellet stops further processing and points out a possible reason, but the actual reason still needs be veri ed by users. 4.5
4 3.5 ) s don 3 c e s (e 2.5 m i T 1.5 2 1 0.5 0
Reasoner Performance Comparision
Pellet
HermiT 500
1000 Size (lines of axioms) 1500 2000
HermiT also provides verbose infomation printing, such as the le under processing, timing for parsing, etc.. In contrast to Pellet, when an input is inconsistent, HermiT shows no possible reasons. As far as processing procedure is concerned, HermiT di ers from Pellet in that it allows checking unsatis ablity of an inconsistent metamodel.
Regardless of how the two reasoners report the results, the information they give can be di cult to interpret when a metamodel is inconsistent or has unsatis able concepts. In the best case, they show the name of the unsatis able concepts but no further explanations. More information on what makes a concept unsatis able would be extremely helpful. 7.5
During our experiment with the 286 metamodels in the Atlantic zoo, we found
279 metamodels that are consistent and satis able and 7 with validation
problems. The 7 unsatis able metamodels are ekaw, paperdyne, pcs, micro, sigkdd,
Openconf and iasted. These are all generated from ontologies dealing with
conference organization by the OntoFarm Project [
We investigated the reason of the unsatis ablity and found problems in the metamodels in the source .uml les. The original metamodels have two classes Evaluated_Paper and Assigned_Paper, which both have a self association with a minimum cardinality of 3 and a maximum cardinality of 1.
We also examined the original ontologies [
In this article we have studied the feasibility of an approach to validate metamodels using a OWL 2 reasoners. The rst step of validation is transforming UML metamodels to OWL 2 ontologies. We use MOFScript to implement the transformation from UML to OWL 2 and SWRL. The second step is to validate the generated ontologies with a reasoner. This second step was the focus of this article.
We have processed all the metamodels in the Atlantic Zoo. This comprises 286 metamodels for domain-speci c languages created independently. To our knowledge this is the most comprehensive empirical evaluation on the use of OWL 2 in metamodeling to the date.
The size of the metamodels can be considered small when compared to the metamodel of the UML language. None of the metamodels use OCL constraints or advanced metamodeling features such as subset properties. However, we consider that the Atlantic Zoo provides a good sample of how metamodel in domainspeci c languages are used in practice.
Based on our experiences, we consider that OWL 2 combined with SWRL can be used to represent all the metamodels in the repository. Also, the Pellet and HermiT reasoners could process each metamodel in less than four seconds and always provided the same results.
Appendix 1 <?xml v e r s i o n=" 1 . 0 " e n c o d i n g="ISO 8859 1 "?> 2 <uml : Model xmi : v e r s i o n=" 2 . 1 " xmlns : xmi="http : / / schema . omg . org / s p e c /XMI/ 2 . 1 " xmlns : uml="http : / /www. e c l i p s e . org /uml2 / 3 . 0 . 0 /UML" xmi : i d=" dkpFAOiaEd6gMtZRCjS81g " name="Metamodel "> 3 <packagedElement xmi : type="uml : Package "
xmi : i d=" dkpFAeiaEd6gMtZRCjS81g " name="Book "> 4 <packagedElement xmi : type="uml : A s s o c i a t i o n " xmi : i d=" dkpFCuiaEd6gMtZRCjS81g " name="A Book Chapter " memberEnd=" dkpFCeiaEd6gMtZRCjS81g dkpFC iaEd6gMtZRCjS81g "> 5 <ownedEnd xmi : i d=" dkpFC iaEd6gMtZRCjS81g " name="book " type=" dkpFA iaEd6gMtZRCjS81g " i s U n i q u e=" f a l s e " a s s o c i a t i o n=" dkpFCuiaEd6gMtZRCjS81g "> 6 <upperValue xmi : type="uml : L i t e r a l U n l i m i t e d N a t u r a l " xmi : i d=" dkpsEuiaEd6gMtZRCjS81g " v a l u e="1 "/>
Listing 1.1. An example of metamodel presented in UML 2 1 P r e f i x ( r d f s :=<http : / /www. w3 . org /2000/01/ rdf schema#>) 2 P r e f i x ( owl2xml:=<http : / /www. w3 . org /2006/12/ owl2 xml#>) 3 P r e f i x ( owl:=<http : / /www. w3 . org /2002/07/ owl#>) 4 P r e f i x ( xsd:=<http : / /www. w3 . org /2001/XMLSchema#>) 5 P r e f i x ( r d f :=<http : / /www. w3 . org /1999/02/22 rdf syntax ns#>) 41 42 D e c l a r a t i o n ( ObjectProperty ( : Book Book book ) ) 43 ObjectPropertyDomain ( : Book Book book : Book Book ) 44 ObjectPropertyRange ( : Book Book book : Book Book ) 45 SubClassOf ( : Book Book ObjectMinCardinality (1 : Book Book book ) ) 46 SubClassOf ( : Book Book ObjectMaxCardinality (1 : Book Book book ) ) 47 )
Listing 1.2. Ontology in OWL 2 functional syntax 1 <?xml v e r s i o n=" 1 . 0 " encoding="UTF 8"?>
Listing 1.3. SWRL rules written in OWL RDF syntax