In the context of software migration, existing test cases represent important assets which can be reused for migration validation. Doing so, one can save time as well as costs by avoiding design of new test cases. However, migration of test cases is a quite challenging task due to the size of the test case set and often missing conformity in the structure of the test cases. Moreover, the test cases are implemented in the same or a compatible technology as the software system they are testing, so they have to follow somehow the migration of the system. To overcome this complex task, we propose a reference process model which supports systematic test case migration during software migration projects. Our reference process model for test case migration encompasses the phases reverse engineering, restructuring, and forward engineering. An instantiation of our reference process model is presented based on an industrial case study.
Software migration is a well-established method for transferring software systems into new environments without changing their functionality. For instance, legacy software systems which are technologically obsolete, but still valuable for ongoing business, can be migrated into a new environment [Bi99].
In this context, software testing plays an important role as it verifies whether the main requirement, the preservation of the functionality, is fulfilled. Reusing existing test cases can be beneficial not just from economical perspective, as the expensive and time consuming test design is avoided [Sn99], but also from practical perspective: the existing test cases contain valuable information about the functionality of the original system. However, a direct reuse of existing test case is not always possible in real world migration projects, due to system changes as well as the differences in the source and the target environments. As test cases are coupled with the system they are testing, the system changes need to be detected, understood and then reflected on the test cases to facilitate reuse. This is a quite challenging task in various migration scenarios, because of the size of the test case set which sometimes is measured in tens of thousands of test cases and the missing conformity in the test case structure [JGY16]. Hence, first of all, a new migration method should be able to deal with the structural complexity of the test cases. Then, it should also provide an easy way to restructure the test cases and reflect the relevant system changes. Last but not least, the migrated test cases should be appropriate for the target environment, i.e., the test cases have to be re-designed for the target environment [Gr16].
The Model-Driven Engineering (MDE) software development methodology has been established to deal with the increasing complexity of today's software systems by focusing on creating and exploiting domain models. The Model-Driven Architecture (MDA) initiative, proposed by Object Management Group (OMG), puts the MDE idea in action, and clearly separates the business logic from implementation by defining software models at different levels of abstraction: The computation independent model (CIM) layer, platform-independent model (PIM) layer and platform-specific model layer (PSM).
Software migration approaches that follow the MDA principles are known as model-driven software migration (MDSM) approaches [Fu12]. Generally speaking, software migration can be seen as a transformation of the system which is done by enacting a software transformation method, which is an instance of the well-known horseshoe model [KWC98]. Therefrom, software migration is some kind of software reengineering, which according to [CC90] is defined as "examination and alteration of a subject system to reconstitute it in a new form and the subsequent implementation of the new form". In general, software reengineering consists of three consecutive phases: reverse engineering, restructuring, and forward engineering. Software testing which relies on the MDA principles is known as model-driven testing [HL03,EGL06]. Similarly to software migration, test case migration can be seen as a transformation of test cases implemented for a specific source technology to test cases implemented for a specific target technology.
Following the idea of model-driven software migration, we present a novel reference process model which supports systematic test case migration during software migration projects. Our proposed reference process model encompasses the phases Reverse Engineering, Restructuring, and Forward Engineering (Figure 1). An instantiation of our reference process model is presented based on an industrial case study which illustrates the applicability of our reference process.
The structure of the rest of the paper is as follows: In the next section 2, we present our reference process model for test case migration. The applicability of our reference process model is shown in section 3 based on an industrial case study. In section 4, we briefly discuss the related work and at the end, section 5 concludes the work and gives an outlook on future work.
In this section, we present the reference process for test case reengineering as enabler for model-driven test case migration. In general, as shown in Figure 1, our reference process comprises the main reengineering phases: Reverse engineering, Restructuring, and Forward Engineering [CC90]. Each phase consists of a set of activities and corresponding artifacts in terms of textual artifacts and models on different levels of abstraction. Here, we introduce the reference process represented as a BPMN model (Figure 2) and in the next section we come up with a concrete instantiation.
Artefacts are constituting parts of each transformation process. In Figure 2, the artefacts are represented as data objects. An artefact can either be a textual artefact, e.g., a test code, or a model. Regarding to the abstraction level, each artefact belongs to one of the three layers: System Layer, Platform-Specific Layer, and Platform-Independent Layer.
On the System Layer, the textual artifacts representing test code and models of the test code (Model of the Test Code (original) and Model of the Test Code (migrated)) are placed. Test Code (original/migrated) can be either the test cases, implemented in a specific testing framework, e.g. jUnit2 or MSUnit3, test configuration scripts or a manually implemented additional test framework. If the test cases depend on these artifacts, they also have to be considered and eventually migrated along with the test cases. Model of the Test Code (original/migrated) is an AST (Abstract Syntax Tree) representation of the test code depending on the particular language of the original or the target environment.
On Platform-Specific Layer, technology-specific test concepts are used to represent the test cases for both the source and the target environment. The Model of Executable Tests (Original/Target) is considered as platform-specific as it represents the executable test cases Testing Profile (UTP) or Test Description Language (TDL) are used. The System Behavior Model comes at the highest level of abstraction and it is a compact representation of the expected system's behavior usually represented as UML activity or sequence diagrams.
Activities in the test case reengineering process model produce or consume appropriate artifacts. These activities can be distinguished by the reengineering phase they are belonging to. For this purpose, we rely on the classification defined by [CC90], who define the reengineering as a process constituted of three phases: Reverse Engineering, Restructuring, and Forward Engineering. In Figure 2, all activities are represented as BPMN tasks.
Reverse Engineering is according to [CC90] the "process of analyzing a subject system to create representations of the system in another form or on a higher level of abstraction". Similarly, seen from software testing perspective, we define reverse engineering as a process of analyzing test cases to create representations of the test cases in another form or on a higher level of abstraction, e.g., by using test models. In general, reverse engineering can be seen as a combination of Model Discovery and Model Understanding [Br10]. Restructuring, according to [CC90], is "the transformation from one representation form to another at the same relative abstraction level, while preserving the subject system's external behavior (functionality and semantics)". In our case, we define test restructuring as the transformation from one representation to another at the same relative abstraction level, while preserving the ßemanticsöf the tests. We use "semantics" to denote the functionality of the system being checked by the tests. This activity can be applied on the System Behavior Model as well as on the Model of Abstract Test Cases. In Figure 2, we indicate Restructuring as "loop" symbols on the artifacts they are applicable on. The particular actions during Restructuring are defined by the target testing environment. But, Restructuring may also be influenced by the changes that happen in the system migration. Those system changes that are relevant for the tests have to be considered and covered by the Restructuring activity.
Language Transformation is a direct mapping between the Models of the Test Code. More precisely, it is actually a mapping between the programming languages of the source and target testing environment. Framework Transformation is performed on a higher abstraction level and defines a direct mapping between two testing frameworks, i.e., a mapping between the test concepts inside the original an the target testing framework. In Enrichment, one can insert an additional information to the tests by using annotations. This activity is applicable to various models, e.g., Model of Executable Tests, Model of Abstract Tests or System Behavior Model.
The XOR-Gateways shown in Figure 2, illustrate the flexibility of the reengineering process and its applicability in different migration contexts. For example, in a concrete migration scenario, as shown in the next section, one can decide to perform Framework Transformation, and not Language Transformation. This implies that after Model Discovery, on the first decision node, a Test Case Understanding has to be chosen.
Our reference reengineering process was applied in an industrial project which dealt with migration of parts of the well-known Eclipse Modeling Framework (EMF)6 along with the To support the migration process, we developed two Eclipse plug-ins: TestCase2TestModel and TestModel2TestCase. The TestCase2TestModel plug-in supports the reverse engineering activities by using JDT for parsing and model-to-model transformation. The forward engineering and restructuring activities, are supported by the TestModel2TestCase plug-in which is a test code generator written using Xtend, resulting at the end in MSUnit test code.
In total, 92% of the 4000 existing jUnit test cases were migrated.
It is an open question if the migration of the test cases can further be automatized. 8% of the OCL test cases were not migrated automatically, because these are regression tests that have an irregular structure which complicates an automation.
Focussing on the topic of model-driven test case migration, different research areas like model-driven testing, test case reengineering, test mining and software migration have to be analyzed.
Previous research in the area of model-driven testing already presented some proved methods that address the automation of the test case generation in a quite efficient way. In the work presented in [HL03], the authors aim to benefit from the separation of PIMs and PSMs in the generation and execution of tests by redefining the classical model-based tasks. Dai [Da04] proposes a model-driven testing methodology which takes as input UML system models and transforms them to test-specific models as instances of the UML Testing Profile. Javed et al. [JSW07] propose a generic method for model-driven testing that relies on an xUnit platform independent model. Lamancha et al. [LA09] propose also a model-driven testing method that relies on UML Testing Profile. Moreover, they present concrete transformations using the QVT transformation language. All of this methods are related to our work as they address the forward engineering phase of our reference process model.
Regarding the reverse engineering side, we have identified also some existing work in the area known as test case reengineering. employing MDE techniques to automate the reverse engineering and forward engineering phases. From our point of view, it is the most interesting project as they also advocate migration of the test cases in a model-driven way, i.e., in a similar way the system has been migrated, thus reusing, above all, the developed tools. Our work differs in that way, that we propose a systematic test case migration reference process model seen from software testing perspective.
In this paper, we present a novel reference process model for test case migration during software migration projects. Our approach supports model-driven test case migration through a systematic description of relevant reengineering activities represented in BPMN. It comprises the reengineering phases Reverse Engineering, Restructuring, and Forward Engineering as well as artefacts on different levels of abstraction specific for software testing. The proposed reference process model is a generic model with high flexibility and can be used in different software migration scenarios to support the model-driven migration of test cases. This is shown by an instantiation of our reference process model based on an industrial case study where thousand of test cases were migrated from jUnit to MSUnit, most of them completely automatic. In future work, we intend to further extend our reference process model based on the idea of situational method engineering. This way, we plan to systematically cover different test case transformation methods suitable for different migration contexts.
Object Constraint Language (OCL)7 from Java to C#. The main change performed by the system migration, besides switching from Java to C#, was to change from a Just-In-Time (JIT) compilation in the old system to an Ahead-Of-Time (AOT) compilation of OCL constraints in the migrated system. As OCL is a well-tested framework, the main goal was to reuse the existing OCL test cases to validate the migrated OCL functionality in the target environment. Consequently, in total 13 different test suites had to be migrated, each of them addressing different functional aspects of OCL.
According to this specific context information, we used our test case reengineering process model shown in Figure 2 to instantiate a suitable reengineering method. At the end, the migration method shown in Figure 3