Meta-models play a pivotal role in Model-Driven Engineering (MDE), as they define the abstract syntax of domain-specific languages, and hence, the structure of models. However, while they play a crucial role for the success of MDE projects, the community still lacks tools to check meta-model quality criteria, like design errors or adherence to naming conventions and best practices. In this paper, we present a language (mmSpec) and a tool (metaBest) to specify and check properties on meta-models and visualise the problematic elements. Then, we use them to evaluate over 295 meta-models of the ATL zoo by provisioning a library of 30 meta-model quality issues. Finally, from this evaluation, we draw recommendations for both MDE practitioners and meta-model tool builders.
Model-Driven Engineering (MDE) considers models as the main assets in
software development. Frequently, such models are not built using general-purpose
modelling languages, like UML, but with Domain-Specific Languages (DSLs).
Recent surveys on the use of MDE in industry [
A meta-model is considered of quality if it serves its purpose (contains all
needed abstractions of the domain), and is technically built using sound
principles (e.g., there are no repeated attributes among all sibling classes) [
In this paper, we present the language mmSpec and its supporting tool
(metaBest), directed to the specification of desired meta-model properties, their
evaluation, and the visualization of non-conforming parts of meta-models. We
have used this tool to define a reusable library of 30 meta-model quality
properties coming from quality criteria of conceptual schemas [
This paper extends [
The remaining of this paper is organized as follows. First, Section 2 introduces the mmSpec language and tool, and overviews the library of quality properties. Section 3 presents the evaluation of the library over the meta-models of the ATL zoo. Section 4 compares with related research, and Section 5 ends with the conclusions and future work. 2
We have developed a domain-specific language, named mmSpec, to specify metamodel properties and automate their evaluation on meta-models. In the following two subsections, we first introduce the main constructs of our language, and then we describe its use to construct a generic library of quality properties which can be applied on meta-models to assess their quality in an automatic way. 2.1
mmSpec allows expressing meta-model properties in a concise, intensional,
declarative, platform-independent way. It provides high-level primitives that simplify
the definition of meta-model properties, like first-order qualifiers for the length
of navigation paths or collectors of the composed cardinality in navigation paths.
Moreover, it is integrated with WordNet [
The aim of this language and its companion tool (metaBest [
Filters and conditions consist of quali ers which can be negated, combined through and/or connectives, and point to new selectors, enabling recursive checks. The main qualifiers allow expressing conditions on the existence of elements, their name (nature, synonymy, prefix, suffix, camel-phrase), abstractness, multiplicity, type, length of navigation paths, inheritance relationships, depth and width of hierarchies and trees of containment relationships, collectors of the composed cardinality in navigation paths, reachability from/to classes, and (a)cyclicity. Altogether, mmSpec promotes first-class primitives for elements (like paths or inheritance hierarchies) that need to be checked in meta-models frequently.
To evaluate a property on a meta-model, our Java-coded evaluator gathers the meta-model elements that match the property filter (or all of them, if there is no filter), verifies which ones meet the condition, and checks whether the size of the resulting subset is consistent with the selector’s quantifier.
The bottom left of Fig. 1 shows a property using mmSpec. It states that every class (selector) that is abstract (filter) should have some concrete child class (condition), or as it is expressed, be super to some class that is not abstract. If an abstract class does not fulfil this condition, it is useless because it cannot be instantiated. The meta-model in Fig. 1 (taken from the zoo) does not satisfy the property, as class BusinessEntityPropertySet is abstract with no children.
Each property can be assigned a description of its intention, which gets shown in the Test Result view (see right of Fig. 1). While this view summarizes the results of all evaluated properties, the Property Assessment view highlights the relevant elements that did (green-coloured) or did not (red-coloured) fulfil a particular property. In this case, the class BusinessEntityPropertySet is displayed in red when we double-click on the property in the Test Result view.
The language offers abstraction mechanisms to package properties into functions with parameters. When the functions are called, it is possible to include a set of elements in place of a parameter. For example, fun(every classf!abstractg) evaluates property fun for all concrete classes in the meta-model.
Finally, metaBest supports batch evaluation of a set of properties over a set of meta-models, generating a CSV file with the results. In this way, we have managed to deliver the results of evaluating a library of meta-model quality issues for a large number of meta-models, as we explain in the following sections. Code Description D01 D02 D03 D04 D05 D06 D07 D08 D09 D10
Design An attribute is not repeated among all specific classes of a hierarchy.
There are no isolated classes (i.e., not involved in any association or hierarchy). No abstract class is super to only one class (it nullifies the usefulness of the abstract class). There are no composition cycles.
There are no irrelevant classes (i.e., abstract and subclass of a concrete class).
No binary association is composite in both member ends.
There are no overridden, inherited attributes.
Every feature has a maximum multiplicity greater than 0.
No class can be contained in two classes, when it is compulsorily in one of them. No class contains one of its superclasses, with cardinality 1 in the composition end (this is not finitely satisfiable).
Best practices BP01 There are no redundant generalization paths.
BP02 There are no uninstantiable classes (i.e., abstract without concrete children). BP03 There is a root class that contains all others (best practice in EMF).
BP04 No class can be contained in two classes (weaker version of property D09). BP05 A concrete top class with subclasses is not involved in any association (the class should be probably abstract).
BP06 Two classes do not refer to each other with non-opposite references (they are likely opposite).
Naming conventions N01 Attributes are not named after their feature class (e.g., an attribute paperID in class Paper). N02 Attributes are not potential associations. If the attribute name is equal to a class, it is likely that what the designer intends to model is an association.
N03 Every binary association is named with a verb phrase.
N04 Every class is named in pascal-case, with a singular-head noun phrase.
N05 Element names are not too complex to process (i.e., too long).
N06 Every feature is named in camel-case.
N07 Every non-boolean attribute has a noun-phrase name.
N08 Every boolean attribute has a verb-phrase (e.g., isUnique).
N09 No class is named with a synonym to another class name.
Metrics M01 No class is overloaded with attributes (10-max by default).
M02 No class refers to too many others (5-max by default) – a.k.a. efferent couplings (Ce). M03 No class is referred from too many others (5-max by default) – a.k.a. afferent couplings (Ca). M04 No hierarchy is too deep (5-level max by default) – a.k.a. depth of inheritance tree (DIT). M05 No class has too many direct children (10-max by default) - a.k.a. number of children (NOC).
In order to test the quality of meta-models, we have built an mmSpec library
covering typical mistakes (some of them from [
Best practices. Basic design quality guidelines (a warning).
Naming conventions. For example, use of verbs, nouns or pascal/camel case.
Metrics. Measurements of meta-model elements and their threshold value, like
the maximum number of attributes a class should reasonably define. Most
metrics are adapted from the area of object-oriented design [
Table 1 lists the properties from these categories. To illustrate mmSpec’s expressiveness, next we show the formulation of a property from each category: { D02: There are no isolated classes. The encoding of this property is: no class => and f sub to no class, super to no class,
reach no class, reached from no class g.
The aim is to check the absence of classes that are not involved in any association or hierarchy. Thus, we use the no class selector, and check the following conditions: the class is orphan (qualifier sub-to with selector no class), childless (qualifier super-to with selector no class), contains no reference (qualifier reach with selector no class), and is not pointed by any other (qualifier reached-from with selector no class). { BP03: There is a root class that contains all others. This is a common bestpractice in EMF, where meta-models define a class from which all other classes can be reached through composition relations. Its encoding is: strictly 1 class fcont root fabsolutegg => exists.
A class satisfying cont-root is the root of a containment tree; if the root is absolute, then it contains all classes. This illustrates how mmSpec provides primitives that simplify the definition and checking of meta-model properties. { N04: Every class is named in pascal-case, with a singular-head noun phrase.
To obtain intuitive class names, these should be composed by a sequence of
words starting with capital letters, and with a singular noun as the last word
[
every class => name = pascal phrasefendfnounfsingularggg. { M01: No class is overloaded with attributes. Even in large meta-models, classes with too many attributes often evidence a questionable design. While some entities in certain domains might carry a vast load of information, commonly, this data can be split into smaller entities that are arranged using inheritance or composition. Thus, the following property states that every class should have a maximum of 10 non-inherited (!inh) attributes.
every class => with f!inhg [
To evaluate our tool and have a measure of the quality of current meta-modelling practice, we have applied our library of quality properties to the Atlan Ecore zoo of 295 meta-models. We have chosen this repository as it is representative of the meta-models that MDE practitioners build in practice. The size of the metamodels varies from tiny ones with just one class, to meta-models of medium size, the largest one having 699 classes. This is interesting as one of our goals is to detect whether the kind of quality issues depends on the meta-model size, and whether big meta-models are faultier (even if in average) than smaller ones.
Fig. 2 shows the number of quality issues detected in the analysed metamodels. Interestingly, only 5 meta-models have no issue, while no meta-model contains more than 22. The average number of issues per meta-model is 7.26.
Regarding the distribu- ls 40 tion of issues according to edom35 their kind, Fig. 3 shows how -tae 30 many meta-models fail each frebom2250 property from Table 1. De- uNm15 sign is the most relevant cate- 10 gory of properties, as it gath- 5 ers errors that may poten- 0 tially lead to a faulty design. 0 5 10 15 Qu2a0lity issues In this sense, the results for the properties in this category Fig. 2. Number of quality issues in meta-models. are good in average, as they have low rate of failure. Indeed, there are two design properties that every meta-model fulfils: D01 and D02. D01 checks the absence of repeated attributes in a hierarchy (see D01 in Fig. 4 for a faulty example), while D02 checks that the upper bound of features is not 0. However, 110 meta-models fail property D09 (37% of analysed meta-models). This error consists in making a class to be contained in two other classes, with minimum source multiplicity 1 in one of the containment relationships, as shown in Fig. 4. This is an error because, at the instance level, an instance of A could never be contained in an instance of C, as it must be mandatorily contained in an instance of B.
Surprisingly for a set of EMF meta-models, the top unmet property is BP03, an EMF best practice that states the need for a root class whose instances may contain the whole model tree. Fig. 4 shows an example meta-model that fulfils this property, and an example that does not. In BP03 (+), A contains B and C, and hence D (as it is subclass of B), so A acts as absolute root class. On the contrary, BP03 (-) does not meet the property because A does not contain D.
The next two properties not satisfied by more meta-models are N03 and N04, which are naming conventions. N03 demands the verbalization of binary association names (e.g., reaches). N04 checks the conventions for class names, as explained in Section 2.2.
Regarding the error rate evolution of each category of issues with respect to the meta-model size (measured in number of classes), Fig. 5 shows that all categories present an upward error trend as meta-models enlarge. The vertical axis in this diagram corresponds to the percentage of issues in the category that were not met by some meta-model. The growth ratio is higher in small/medium size meta-models (up to 100 classes). Then, the error growth is steadier, particularly on the design category, which remains around 20% even at the largest metamodel size. The issues of type metrics grow as the meta-model size increases, peaking 100% (i.e., all issues of the category fail in large meta-models). This might be comprehensible, since a greater number of classes usually demands a greater number of features and relationships. However, properties such as M01, M02 or M03 might be considered independent from the meta-model size, as they take care of the class feature overpopulation, which is a bad practice despite the meta-model size. More worrisome is the evolution of the best practice category. This category reflects less severe concerns than design, but they still are bad modelling practices. It is worth noting that most meta-models present a 40 to 65% error rate, which together with the design’s - almost permanent - 20%, denote an average design quality that may be improved.
Fig. 6 shows the distribution of meta-model sizes where each issue type tends to occur. For most properties, faulty meta-models have between 1 and 200 classes. However, properties M04, BP06, D07 tend to fail in large meta-models. This is natural in some cases, e.g., D07 checks the presence of overridden attributes in inheritance hierarchies. Instead, properties D06, D09, D10, BP04, N03 tend to occur in small meta-models. The fact that 3 important design properties fail in small meta-models might mean that such meta-models were built by more unexperienced designers, compared to large ones. As a matter of fact, properties with a greater number of failure occurrences (as seen in Fig. 3: D09, BP03, BP04, N03, N04, N07, M02, M03) mainly appear in really small meta-models.
Finally, if we look at the average number of quality issues per class, we find that the most frequent categories of issues are best practices and naming conventions (0.18 issue occurrences per class in both cases). Design (0.07 issues per class) and metrics (0.05 issues per class) are less frequent; nonetheless, if they are considered together, the error rate seems worrisome at least. 3.1
From the analysis of the meta-model repository, we realize that a way to improve the quality of meta-models is the inclusion of these quality checks in the metamodelling tools, for example, to discover problems like D09. Actually, for some of these problems (like the ones related to metrics) the tool could trigger some refactoring suggestions. Regarding naming conventions, we noticed the usefulness of having integrated “smart” spell checkers (i.e., to check correctness of names in camel-case).
It is worth mentioning that mmSpec is integrated with metaBup [
We could use OCL instead of mmSpec. However, OCL expressions tend to be more complex, as OCL lacks primitives (for gathering paths or inheritance hierarchies) which are part of mmSpec and have been designed to express properties on meta-models. Moreover, mmSpec supports the visualization of problematic elements (i.e., properties do not report just true/false). A comparison of OCL and mmSpec is available at: http://www.miso.es/tools/metaBest.html.
In [
In [
Some works aim at characterizing meta-model quality. For example, in [
Few works analyse the quality of real meta-models. In [
In this paper, we have introduced mmSpec, a language directed to the specification of properties to be checked on meta-models, and metaBest, a tool to visualize and report the problematic elements. We have used both to build a catalog with 30 meta-model quality properties, which has been evaluated over 295 meta-models. The obtained results show that most meta-models contain some issue; hence, the community would benefit from integrated tool support (like metaBest) for checking quality properties during meta-model construction.
In the future, we plan to analyse correlations between meta-model flaws, provide a catalog of quick fixes and recommendations, and support the creation of user-defined categories for properties. We also plan to develop a further language for meta-model testing based on constraint solving.
Acknowledgements. This work has been funded by the Spanish Ministry of Economy and Competitivity with project “Go Lite” (TIN2011-24139).