Like any software artifact, conceptual schemas of information systems can be tested. Testing conceptual schemas has some similarities with testing programs, but there are important differences. We present a list of six kinds of tests that can be applied to conceptual schemas. Some of them require complete schemas comprising both the structural and the behavioral parts, but we show that it is useful to test incomplete schema fragments. We introduce CSTL, a language for writing automated tests of executable schemas written in UML/OCL. We sketch the implementation of a test processor to automatically execute CSTL tests as many times as needed, following the style of the modern xUnit testing frameworks.
The need and the importance of software testing in information systems engineering
is undisputed. We adopt here the precise and concise definition of testing proposed by
Meyer: “To test a program is to try to make it fail”, from which the goal of
testing becomes “to uncover faults by triggering failures” [
Currently, most work in conceptual modeling assumes that conceptual schemas are
executable, and therefore they are software [
Most of the work in software testing assumes that the System Under Test (SUT)
consists of programs (objects, components) that provide only a set of operations and
testing a SUT means calling those operations with appropriate context and input
parameters, and checking that they return the expected outputs. For example, the
recent UML Testing Profile (UTP) is based on this assumption [
If a conceptual schema were like an ordinary program, then its testing would not be
very different from testing code. However, a conceptual schema is knowledge or,
more precisely, it is the general knowledge that an information system needs to know
about the domain and about the functions it has to perform [
In this work we advocate the use of testing during the elicitation of the conceptual schema as an early error detection practice to help increasing software quality.
In our approach, the conceptual schemas under test consist of a structural
(sub)schema and a behavioral (sub)schema. The structural schema consists of a
taxonomy of entity types, a set of relationship types (either attributes or associations),
the cardinality constraints of the relationship types, and a set of other static constraints
formally defined in OCL. Entity and relationship types may be base or derived. The
behavioral schema consists of a set of event types. We adopt the view that events are
similar to ordinary entities and, therefore, that events can be modeled as a special kind
of entities, which we call event entities [
Testing conceptual schemas is as important as testing programs in projects that
follow OMG's Model Driven Development (MDD) approach [
However, we have found that it makes sense to test also incomplete conceptual
schemas, as a means to increase their quality [
In this paper, we adopt UTP's terminology and consider that a test case is a
"specification of one case to test the system including what to test with, which input,
result, and under which conditions....A test case always returns a verdict." The verdict
may be Pass, Fail, Inconclusive, and Error [
We analyzed that when we test a conceptual schema, a test case includes one or more of the following test kinds: ─ Check that a given IB state is consistent. This kind of test can be used to check that (1) the whole set of constraints and derivation rules behave as expected; and (2) the set of constraints defined in the schema is strongly satisfiable (because there is at least one consistent and non-empty IB state). ─ Check that a given IB state is inconsistent. A conceptual modeler writes this kind of test to check that (1) the OCL constraints behave as expected; or (2) the whole set of constraints and derivation rules behave as expected. ─ Check the contents of a given IB state. This kind of test can be used to check that (1) the structural schema can be instantiated to represent a particular domain state; (2) one or more derivation rules derive the expected results; (3) an OCL navigational expression yields the expected results; or (4) the effect of one or more domain events implies an expected result on the IB. ─ Check that a domain event may not occur in a given IB state. Domain event types and queries may have constraints. The meaning is that the instances of those types or queries may only occur in the domain if the constraints are satisfied. A conceptual modeler writes this kind of test to check that (1) the OCL event constraints behave as expected; and (2) the whole set of constraints defined in the event or query does not allow its occurrence as expected. ─ Check that a domain event may occur in a given IB state. This kind of test checks that the effect of a domain event occurrence is as expected. A conceptual modeler writes this kind of test to check that (1) the OCL event constraints behave as expected; (2) the whole set of constraints defined in the event behave as expected; and (3) the method and the derivation rules of the derived constant attributes and associations produce the expected results (satisfaction of postconditions and static constraints). ─ Check that a predefined query produces the expected results. This kind of test checks that the effect of a query gives the expected answer. A conceptual modeler writes this kind of test to check that: (1) the query constraints behave as expected; (2) the effect of one or more previously occurred domain events has produced the expected results on the IB; and (3) the postcondition of the query gives the expected results. 3
The Conceptual Schema Testing Language (CSTL) is a textual procedural language
for writing automated tests of executable conceptual schemas written in UML/OCL.
CSTL is inspired in, and is an evolution of ASSL [
CSTL provide constructs for specifying the test kinds enumerated in Section 2. Note that all test kinds involve an IB state that must be specified by the conceptual modeler.
A CSTL program consists of a fixture (may be empty), a set (may be empty) of fixture components, and a set of one or more test cases. The fixture is a set of statements that create a state of the IB and define the values of the common program variables. It is assumed that the execution of each test case starts with an IB state and the contents of the variables as defined by the fixture. With this assumption, the test cases of a program are independent each other, and the order of their execution is irrelevant.
In CSTL, there are three kinds of test cases: concrete, abstract and abstract invocation. A concrete test case is a set of statements that builds a state of the IB, defines values of its variables, and executes one or more tests of one of the six test kinds described in the previous section.
An abstract test case is a parameterized test case intended to be invoked one or more times in the same program. The parameters of an abstract test case may include fixture components and variables. A fixture component is a named set of statements that create a fragment of the state of the IB and define the values of a set of variables.
An abstract test case invocation is the invocation of an abstract test case with the desired values of the parameters.
The execution of a concrete test case or of an abstract test case invocation always returns a verdict. The verdict is obtained from the verdicts of the test kinds executed by the test case.
The result of the invocation of a CSTL program always returns also a verdict, which is obtained from the verdicts of its test cases. 3.2
The Test Processor
We are developing a test interpreter that reads a CSTL program and executes its statements. The test interpreter coordinates the execution of the tests (setting up fixtures, computing verdicts, and so on) and invokes the services of the information processor to create, remove and change entities, attributes and associations of the IB, and also to evaluate OCL expressions over the IB. Moreover, it shows the results of the test execution. The test manager stores the CSTL programs and requests their execution to the test interpreter. The test manager also keeps track of the test results, and maintains test statistics.
Figure 2 shows the result of the execution of an example CSTL program. There are
two test cases that have failed, and therefore the global verdict is Fail. Note that the
test processor indicates the number of the lines where the tests have failed, and an
explanation of the failure in natural language.
A conceptual schema can be executable if there is a general-purpose information
processor (virtual machine) able to behave according to the structural and behavioral
rules defined in the conceptual schema [
We are implementing the information processor reusing USE [
We have seen that, like any software artifact, conceptual schemas of information
systems can be tested with the goal of "uncover faults by triggering failures" [
We have introduced CSTL, a textual procedural language for writing automated tests of executable conceptual schemas written in UML/OCL. As far as we know, this is the first proposal of a language for testing conceptual schemas designed in the style of the modern xUnit testing frameworks. We are implementing a Test Processor that manages and executes CSTL programs.
We believe that our work opens new directions for research and development in
conceptual modeling. First, it is necessary to develop a methodology for testing
conceptual schemas. In particular, it seems interesting to develop a test-driven
conceptual modeling methodology, similar to the popular Test-Driven Development
[