The Unified Modelling Language (UML) is widely used for modelling software systems and its integration with executable languages, such as the Action Language for Foundational UML (ALF), provides a bridge between the graphical specification techniques used by mainstream software engineers and the precise analysis and validation techniques essential for the model-driven development of information systems. As far as we know, the idea of transforming Conceptual Schemas (CS) based on UML Class Diagrams into ALF to execute systematic ALF-based test cases against these CSs and to report defects by checking logs has not been explored to date. In this paper, we use ALF to create a testing environment to validate requirements and verify some system properties at the CS level. We also report on some of the implementation details and design decisions of our proof-of-concept tool, as well as its limitations and possible use scenarios.
In previous work we proposed an approach for testing-based validation of Conceptual
Schemas (CS) in a Model-driven environment [
The paper describes the automatic transformation of a UML class diagram (CD) into ALF language in the context of the CoSTest tool 4 for the systematic testing of conceptual schemas. The resulting translated model is semantically equivalent to the original, meaning that the contract semantics (i.e. pre and post conditions), derived features, operation bodies, and association class are implemented as elements of the ALF units. The paper’s contributions are: (1) Translating UML into ALF; (2) Using ALF as a language for writing/executing test cases. We also evaluated these transformations by using our freely available CoSTest validation tool with eight CS.
The paper is structured as follows: Section 2 introduces a simple but representative example of the UML CD-based CS used. Section 3 summarizes the background. Section 4 reviews related work. Section 5 describes the mapping rules between UML and ALF notation by describing its application to the example. Section 6 gives an overview of the last phase of CoSTest to generate ALF based-test cases, as well as execution details of our validation approach. Section 7 demonstrates the application of the transformations to ALF in eight CS. Section 8 discusses the design decisions, limitations and alternative applications of the approach. The conclusions and future work are outlined in Section 9. 2
To show how ALF supports the validation in the CoSTest tool, we will use a simple model of an order from the domain of e-commerce. The first thing to decide is the 1 https://eclipse.org/papyrus/ 2 http://wiki.eclipse.org/Papyrus/UserGuide/ModelExecution 3 http://modeldriven.github.io/Alf-Reference-Implementation/ 4 https://staq.dsic.upv.es/webstaq/costest.html information that needs to be kept on an order and how this is related to information on the customer placing the order. This can be clearly represented by using a UML class diagram, such as the one shown in Figure 1 with part of an Order CS using an UML class diagram. This diagram was entered graphically using the Papyrus tool and it models an order as recording the date it was received, prepayment and dispatch status, and has a set of order lines, each of which specifies the quantity of a certain product included in the order. It also shows that an order is placed by a single customer (i.e. corporate or personal customer), who may make many orders over time. Each of these orders has several order lines, each of which refers to a single product, with net price and quantity available. Each customer has a name and address. A personal customer also has a credit card number and a corporate customer has a contact name, credit rating and credit limit. The products are identified by a name in a language, a description and a url.
Since the executable CS of a system should describe its structure and behaviour. A class
diagram is the UML’s main building block and shows structural elements of the system
at an abstract level (e.g. class, association class), their properties (owned attribute),
relationships (e.g. association and generalization) and operations [
In this context, an executable model is at the next higher layer of abstraction,
abstracting away both specific programming languages and decisions about the
organization of the software (e.g. data structure and partitioning) so that a specification
built in Executable UML can be deployed in various software environments without
change [
In this paper we propose to use a UML-based Class diagram and derive part of the CS behaviour from the constructor operations, constraints, invariants, pre and post conditions represented in the class diagram by translating them into ALF code as part of the specification of a method (i.e. constraints and invariants) or a conditional inside of the method specification (i.e. pre and post conditions). 3.2
We developed an early testing technique supported by the CoSTest tool to validate
requirements at model level [
Although there are a number of studies addressing the verification of UML models that
include actions [
The Papyrus tool [
This paper describes the use of ALF for generating/writing executable test cases as
well as for transforming a UML CD-based CS in an executable model. These
ALFbased artefacts are then used within the CoSTest process for validation of UML-based
Conceptual Schemas by executing the test cases against the executable CS in an
ALFbased testing environment. We also report on the usefulness of our UML-to-ALF
transformation and its parsing to validate a set of mutation operators [
We use the ALF language as a notation for representing UML CD-based CS and for reasoning about this model. To obtain the result outlined in the previous section we defined a model-to-text transformation of UML to ALF, which we describe in this section. The mapping is specified as an Acceleo5 transformation included in CoSTest and we outline here its points of interest. Table 1 provides information about the main transformations according to the ALF standard.
For derived associations, we add an attribute to the class (e.g. sequence) and create a getter operation (e.g., association_<DerivedAssociationName>_derivation). We then attach the operation generated from the constraint expression to the getter. Figure 4 shows highlighted with red rectangles the ALF code for the pendingDispatch derived association of our example. All the examples given in this section have been translated into ALF using our ALF translator implemented in CoSTest, and executed using the fUML execution engine.
We decided to use the Reference Implementation6 as an fUML engine because (1) it is based on the reference implementation and (2) it provides an execution log. Thanks to (1) we have confidence in its conformity to the fUML specification. And (2) means
6 http://modeldriven.github.io/fUML-Reference-Implementation/ that systematic testing (i.e. reviewing hundreds of logs) is simpler than with the Moka7 implementation, which is more suitable for an interactive testing.
The transformation of UML CD-based CS into ALF is performed in two steps: 1. Model-to-text transformation translates the UML CD-based CS into ALF units.
This transformation is written in Acceleo code. It takes as inputs an UML CD-based CS, and gives as output an ALF -based CS. The resulting ALF-based CS contains the elements generated from the transformation of all CS elements given as input.
2. ALF unit parsing. Semantically, ALF maps the CS to the Foundational UML
(fUML [
We applied our ALF -based tool to check the generation of CoSTest test cases and
measure their effectiveness in several subject CS [
The last step executed in CoSTest to generate test scenarios (see Section 3.2) is the transformation into ALF code of the abstract test cases (e.g. services, triggers, assertions and links exemplifying the interaction of actors with the system) using Acceleo, which are concretized with values entered by the tester or generated from the data model. The result of this transformation is a test suite with concrete and executable ALF-based test cases.
In ALF, an executable test case is an activity that provides the specification of parameterized behaviour as the coordinated sequencing of subordinate ALF units including assertions which describe how the model should behave. A CS can demonstrate semantic correctness and completeness with its requirements specification if no assertion is violated when executing the test suite.
Correctness covers both syntactic correctness (right or well-formedness syntax) and
semantic correctness (right meaning and relations relative to the knowledge about the
domain) [
The semantic correctness of UML-CD-based CS with respect to a domain can also be checked against the evaluation of constraints. Then the elements exercised in the test cases can be used to identify other defects, such as redundant (e.g. duplicated element) and extraneous (e.g. CS elements not used by test cases) elements.
A test suite generated by CoSTest contains hundreds of test cases, so that this automatic generation could not have been produced without ALF.
CoSTest generates three kinds of test cases and focuses on instantiation semantics of UML Structures to test: 1. The occurrence of events. An event is the execution of some operation (method) of the CS, which may have several kinds of defects, of which the following can be highlighted: (a) the pre-conditions of the event may not allow the occurrence of valid events. (b) the post-conditions may not precisely define the intended effect of events. (c) the method of an operation may produce a state that does not satisfy the CS invariants.
AssertTrue (“Object created”, order_ instanceof Order);
AssertTrue("Association Created", order_.orderlines->size()>0); 2. The non-occurrence of events. For the set of constraints defined in the CS to be correct and complete, not only must the constraints be satisfied by valid CS states, but those constraints must also rule out invalid states. Testing the CS may be a practical mean of detecting missing constraints. This is done by setting up one or more test cases, with a state established in the requirements as invalid, such as values of the range, minimum cardinality violation, unique value violation for class attributes, followed by an assertion that the state does not satisfy at least one CS constraint. For example, when an attribute of an object should be unique (e.g. unique name for Customer) CoSTest generates a test case including two instances of this object (customer in our example) with the same value in the unique attribute (i.e. name) and test the required constraint with an assertion as follows: customer_=new Customer (p_atrname=”name”, p_atraddress=”Address1”); customer2_=new Customer (p_atrname=”name”, p_atraddress=”Address2”); AssertFalse (“Not Exist”, customer_ instanceof Customer); 3. The contents of CS objects. It is often useful to include an assertion on the current state of an object instance in the CS in a test case. The purpose may be to check that one or more derivation rules derive the expected results, or that a navigational expression yields the expected results or that the effect of one or more events implies an expected result in the CS object instance. For example, CoSTest generates the following two assertions to test the derivation rules corresponding to the total derived attribute (in Order class) and the pendingDispatch derived association of our example (see Figure 1).
AssertEqual(“property_total_derivation”, order_.total, order_. orderLines->collect e(e.price)->reduce Sum); AssertEqual (“association_pendingDispatch_derivation”, customer_. pendingDispatch, customer_.orders->select e(e.dispatched=false)); The oracle and test goal of each test case is derived from the type of test cases selected (i.e. positive or negative). The expected value (oracle) for the positive test cases (assertionEqual or assertionTrue) is “true” and with negative conditions the False assertion (assertionFalse) must be true, otherwise the test case fails. The test cases are then evaluated by using these oracles and goals included in the test cases. A test case returns the verdict Pass (if the assertion is satisfied), Fail (if the assertion is not satisfied) or Inconclusive (if it was not possible to execute all the statements previous to the assertion). When the verdict of the assertion is Fail or Inconclusive, the execution trace (i.e. ALF execution log) is analysed to report the defects by using the information shown in Table 2.
In order to evaluate the syntactic correctness and completeness of the transformation
rules to ALF used in CoSTest (i.e. for CSUT and executable test cases), we applied our
UML-to-ALF transformation (see steps 8 and 9 in Figure 2) and our generator of
ALFbased test cases (see step 6 in Figure 2) to eight CSs. In particular, this experiment took
as input CSs containing a variety of characteristics that can be present in UML
CDbased CS, including classes, relations (i.e. association, composite aggregation, and
generalization) and different types of constraints (i.e. pre-condition, post-condition and
body condition). These CSs were of different sizes and domains (e.g. information
systems, games). On case is taken from industrial (i.e. IM), others CSs were found in
the literature (i.e. [
This experiment let us to evaluate the transformation rules of the CoSTest by verifying the syntactic correctness and evaluating the completeness of the transformed CSUT. Then, the results obtained by parsing for these transformations to ALF (i.e. CSUT and test cases) were 100% well-formed and complete. These CSs are publicly available in the project website https://staq.dsic.upv.es/webstaq/costest.html, so that, the test cases can be again generated and the experiment can be replicated with the CoSTest tool.
In order to validate the subject CSUT by executing test cases against them, we
injected faults into subject CSs and executed the CoSTest testing process (see steps
88 https://eclipse.org/papyrus/
11 in Figure 2). Then, defects such as missing (e.g. class, attribute, constraint,
operation, association), incorrect (e.g. operation, parameter) and extraneous elements
(e.g. derived attribute, attribute) were reported by CoSTest. These data are outside the
scope of this paper, so that, more detailed information on the testing process (e.g.
injected defects, founded defects) can be found in [
Conceptual schemas are particularly useful in discussions with problem domain
stakeholders. They are straightforward to understand and a lot of detail can be presented
in a well-laid-out, compact diagram. For most people, this is far easier to understand
than large blocks of text or written descriptions. However, in order to execute a large
number of test cases (i.e. hundreds of them) in a systematic way by exercising different
test scenarios and elements of the UML CD-based CS and then to report the defects
found, we require a tool that allows us to execute a set of test cases against a CS, so we
used ALF in the context of CoSTest tool as the execution environment that provides
the ability to execute UML models and report their defects systematically. A tool such
as CoSTest, which is based on ALF for representing UML CD-based CD, has two main
usage scenarios:
1. Verifying well-formed UML models by parsing. The syntax of the language
provides the rules for how to construct well-formed statements. The semantics of the
language provides the specification of the meaning of well-formed statements. Users
(analysts/modellers/testers) could therefore use our ALF–based tool for verifying
that a model is actually well-formed. This model can be a conceptual schema under
test, as in our validation approach, or a mutant model used to evaluate mutation
operators [
ALF and fUML are relatively new standards for building further executable UML specifications and their implementations have only appeared recently. So far, however, most fUML-based execution tooling has been intended to primarily address needs for system simulation and analyse the behaviour of the model with values manipulated by a tester.
In this paper we have shown that ALF could be used to validate requirements at
model level by systematically executing a set of the ALF-based tests. To do so, in
Section 2 we described a simple UML CD-based CS built in Papyrus, which was then
translated into ALF using a transformation we had developed. The result was a model
of a system ready to be tested (i.e. executable) using the CoSTest tool and a set of the
ALF-based test cases generated by it. The tool is able to report defects (i.e. syntactically
incorrect elements) by parsing the CS, missing and incorrect elements by executing the
model, as well as redundant and extraneous elements by coverage analysis of the testing
process. We have also used our ALF-based approach in other scenarios to evaluate
mutation operators [
We have also reported some limitations of the current ALF-based tool from user feedback. We hope the issues that are related to the standards will be addressed by the Executable UML Working Group in the near future, so that technological improvements resulting from these refinements of the standards can be integrated in the future development of CoSTest. In addition, we will do a comparative analysis of the use of other languages to write the test scripts.
This work is supported by SENESCYT of the Republic of Ecuador, Spanish Ministry of Economy, Industry, Competitiveness and the Generalitat Valenciana under the projects the PGE (TIN2016-78011-C4-1-R), FEDER (TIN2013-46238-C4-3-R), TIN2016-80811-P and PROMETEO II/2014/039, and cofinanced with ERDF.