Test Case Co-Migration Method Patterns Ivan Jovanovikj, Enes Yigitbas, Stefan Sauer, Gregor Engels Software Innovation Lab Paderborn University, Paderborn, Germany Email: firstname.lastname@uni-paderborn.de Abstract—Co-migration of test cases has a twofold benefit: development of the transformation method is a very important it reduces the cost of testing the migrated system and retains task as it influences the overall success of the migration project valuable information about the expected functionality of the in terms of effectiveness (e.g., non-functional properties) and original system and thus the desired functionality of the migrated system. The migration of test cases is shaped by the co-evolution efficiency (e.g., the time required or the budget). To achieve of the test cases, as they can be affected by the changes in the this, the situational context of the migration project should system migration. Furthermore, the situational context has to be be taken into consideration. The situational context comprises considered as it influences the quality and the effort regarding different influence factors like characteristics of the original the test case migration. To address these challenges, we propose system or target environment, the goals of the stakeholders a solution that applies situational method engineering extended with co-evolution analysis. The proposed framework enables etc. Concerning test case migration, the situational context modular construction of test transformation methods which gets even more complex as beside the influence factors of consists of a method base and a method engineering process. the system migration, test-specific influence factors like char- Method fragments are the atomic building blocks of a migration acteristics of the original test cases or test target environment method, whereas method patterns encode specific migration have to be considered as well. To develop a situation-specific strategies. Beside the basic test method patterns, we introduce co-migration patterns, which encode the dependency between transformation method is an important and challenging task, the system migration and the test case migration. The method as the previously discussed co-evolution aspect should be engineering process provides the guidance on development and incorporated when identifying the situational context from enactment of migration methods. In this paper we give an both system and test perspective. overview of the method base, in particular on the co-migration In order to address the previously mentioned challenges, method patterns, as well as a detailed discussion. Index Terms—test case migration, co-migration, co-evolution, based on the Method Engineering Framework for Situation- method engineering, method-base, co-migration method pattern Specific Software Transformation Methods (MEFiSTo) [3], we provide a framework that combines techniques from Situa- I. I NTRODUCTION tional Method Engineering (SME) [4] and Software Evolu- Reusing existing test cases is a frequently used validation tion [2]. In general, a Method Base contains the building strategy in software migration [1]. It can reduce the cost of blocks needed for assembling the migration method, namely testing the migrated system and can also help to retain valuable Method Fragments and Method Patterns. A Method Fragment information about the expected functionality of the original is an atomic building block of a migration method, whereas system and thus the desired functionality of the migrated a Method Pattern represents a strategy and indicates which system. Reusing test cases comes down to the problem of co- fragments are necessary and how to assemble them together. migration, i.e., the test cases have to be migrated along with As the test method patterns cannot express directly the de- the system to the dependency on the system migration. The co- pendency between the system and the test case migration, we migration is practically defined by the co-evolution of the test propose a set of co-migration method patterns. Technically, cases and the corresponding system. In general, co-evolution a co-migration method pattern is a combination of a system refers to two or more objects evolving alongside each other, method pattern and a test method pattern visually resembling such that there is a relationship between the two that must to a double horseshoe model. A co-migration pattern encode be maintained [2]. In our case, this refers to the test cases the relation between the applied system migration pattern and evolving alongside the migrating code, such that the test cases the selected test method pattern. remain correct for testing the migrated system. This implies The structure of the rest of the paper is as follows: In Sec- that the co-evolution analysis should be incorporated in the tion II, we introduce the test method fragments. Then, in Sec- test case migration. tion III, we introduce the test method patterns. In Section IV, When performing a test migration, a transformation method we present the test co-migration method patterns. In Section V, is required which serves as a technical guideline and describes we briefly discuss the related work and at the end, Section VI the activities necessary to perform, tools to be used, and concludes the work and gives an outlook on future work. roles to be involved in order to migrate given test cases. The Copyright © 2020 for this paper by its authors.Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). II. M ETHOD F RAGMENTS Reverse Engineering, is the process of analyzing a test case and creating another representation of them on a higher level A method fragment is an atomic building block of a of abstraction, e.g., by using test models. In general, reverse migration method, i.e., an activity, artifact or tool. As we engineering can be seen as combination of Model Discovery follow the idea of model-driven software migration [5], our and Model Understanding [9]. The Model Discovery step method fragments belong to one of the following reengineering relies on syntactical analysis and by using a parser it allows processes: Reverse Engineering, Restructuring, and Forward automatic text-to-model transformation to create a model of Engineering [6]. The activities as well as the artifacts are the test case source code represented as an Abstract Syntax represented in Figure 1 as an instance of the well-known Tree (AST) [8]. Model Understanding, in general, is a model- horseshoe model [7]. to-model transformation activity, or a chain of such activi- Artifacts are constituting parts of each migration method and ties (Test Case Understanding, Test Abstraction, and System are distinguished by the level of abstraction they are belonging Behavior Recovery), which takes the initial models, applies to. On the System Layer, textual artifacts representing test code semantic mappings and generates derived models on a higher and models of the test code are placed. Regarding the textual level of abstraction. artifacts, this is either the Original Test Code or the Migrated Restructuring, is defined as the transformation from one test Test Code. Similarly, regarding the models of the code it is representation to another at the same relative abstraction level, either the Model of Original Test Code or the Model of the while preserving the functionality that is being checked by Migrated Test Code represented in a form of in a form of the tests. This activity has been foreseen on both the System an Abstract Syntax Tree [8]. The Platform-Specific Layer is a Behavior Model and the Model of Abstract Test. The Restruc- higher level of abstraction compared to the system layer. Here, turing activity is of course influenced by the target testing technology-specific concepts are used to represent the test environment, testing tool, or by requirements on improving the cases for both the source and the target environment in terms of quality of the test cases (e.g., maintainability). Furthermore, Model of Original/Target Executable Tests. On the Platform- it could also be influenced by the changes that happen in the Independent Layer the models representing the test cases are system migration. independent of any particular testing framework or testing Forward Engineering, is the process of moving of high- technology. Here, we distinguish between two types of models, level test abstractions and logical implementation-independent the Model of the Abstract Tests and the System Behavior design to the physical implementation of the test cases. The Model. On the highest level of abstraction, we foresee the Sys- test models are used as input for a chain of model-to-model tem Behavior Model that represents a compact representation transformations (Abstract Test Derivation and Test Concretiza- of the expected behavior of the system. Behavioral diagrams tion), ending with a model-to-text transformation (Test Code like the UML activity or sequence diagram, or state machines Generation), which provides the test code as output. can be used to represent the expected behavior of the system. The Reimplementation is a text-to-text transformation which Activities in the test case reengineering horseshoe model is performed manually. The Language Transformation also produce or consume appropriate artifacts. As can be seen known as AST-based transformation [7], defines a direct in Figure 1, these activities can be distinguished by the reengi- mapping between the Original Model of the Test Code and neering process they belong to, namely Reverse Engineering, the Migrated Model of the Test Code. The Test-Language Restructuring, or Forward Engineering. Transformation, on the other hand, defines a mapping directly between two testing frameworks. The Enrichment activity is Platform-Independent Enrichment System Behavior Restucturing applicable to various models, e.g., Original Model of Exe- Model cutable Tests, Model of Abstract Tests, or System Behavior Layer System Behavior Recovery Abstract Test Derivation Model. The Removal activity is used to specify which part of Activity Specification Enrichment Model of Restucturing Exogenous Model the test case code should not be transformed. Abstract Tests Transformation Test Test Enrichment Abstraction Concretiztaion Endogenous Model Transformation III. M ETHOD PATTERNS Platform-Sepcific Text-to-Model / Model of the Framework Model of the Model-to-Text The method fragments like artifacts, activities or tools are Layer Transformation Transformation Original Migrated Executable Tests Executable Tests Text-to-Text not sufficient as no guidance is provided how to assemble Transformation Direct Test Code Test Case Test Direct Test Code No Transformation X them. A method pattern, represents construction guidelines for Understanding Understanding Concretiztaion Generation Language Artifact Specification migration methods that follows a certain strategy by defining Model of the Transformation Model of the Original Migrated Model which method fragments should be customized and how to System Layer Test Code Test Code Code Test Code put them together In the following, as shown in Figure 2, Model Discovery Generation we present an excerpt of the method patterns that preserve Reimplementation Original Migrated functionality. Test Test Code X Code The Language-based Test Transformation pattern defines the Removal migration of the functionality of the test cases by defining a mapping between the language constructs in both original Figure 1. Excerpt of Method Fragments stored in the Method Base. and target environment. The mapping is applied by a direct A co-migration method pattern is a method pattern which relates a test method pattern and a system method pattern by explicitly establishing the relation between the corresponding method fragments. Language-based Test Transformation By explicitly establishing the relation between test and system method patterns, we aim to ease the process of the X selection and configuration of a test method pattern. An Test Code Removal already configured system method pattern, with selected and Framework-based Conceptual concertized method fragments, i.e., artifacts and activities, Reimplementation Test Transformation Test Transformation suggests in what way the test method fragments should be Test Model Manual Automated No Activity X selected and configured. Consequently, it suggests in what way Code Activity Activity the tools supporting the different method fragments should be developed. Figure 2. Excerpt of basic method patterns. The co-migration patterns also facilitate reuse of existing artifacts and activities from the system migration method. transformation between Model of the Original Test Code and As an explicit relation between the system and test method Model of the Migrated Test Code. Theoretically, this pattern patterns exists, it facilitates the reuse of the already existing could be applied actually in any migration scenario, but its artifacts and activities defined for the system transformation suitability mainly depends on the complexity of the model method. Furthermore, the developed and used tooling for the transformations between both models. From test perspective, system migration, e.g., a language parser or a language meta- the transformation of the test concepts have to be done model, which correspond to an activity or an artifact, could implicitly. be reused. By using the Test Language-based Test Transformation In this work, as we already mentioned in Section III, pattern the functionality of the test cases is migrated by using we focus on the functionality preserving test method pat- an intermediate test representation on platform-specific layer. terns. Our test method patterns were mainly inspired by the Model of Original Executable Tests represents explicitly the method patterns presented in [3], where four different func- testing constructs and the test data. Doing so, transformation tionality preserving method patterns were defined, namely: step is less complex compared to the Language-based Test Reimplementation, Language-based Transformation, Concep- Transformation and it enables a direct, i.e., an explicit repre- tual Transformation, and Code-Removal. As we already said sentation and manipulation of test constructs. at the beginning of this section, a co-migration comprises The Conceptual Test Transformation pattern defines to mi- a test method pattern and a system method pattern. We grate the test functionality by using an intermediate repre- created the co-migration patterns by combining each of the sentation in terms of Model of Abstract Tests on a platform- test method patterns with each of the system method patterns, independent layer. This improves the dependent framework excluding the Code-Removal pattern. The Code-Removal was transformation on the platform-specific layer by explicitly not taken into consideration as it has no influence on reuse representing some test concepts on a higher level of abstraction of method fragments. In the following, we analyze each as part of the Model of Abstract Tests. This pattern could be co-migration method pattern regarding two aspects reusable considered suitable when some test concepts are realized com- method fragments and impacted method fragments. Reusable pletely different in both environments or when a restructuring method fragments are those method fragments from the system of the test architecture or test data is necessary. By using the transformation method which could be directly reused in the Reimplementation pattern the functionality of the test cases is test transformation method. Impacted method fragments are manually transformed by software developers and it is suitable those test method fragments which are impacted from the in cases when an automatic migration is too complex to be system method fragments. implemented. Lastly, the Test Code Removal pattern defines Figure 3 depicts the co-migration patterns that combine not to migrate certain part of the test code, e.g., when some either a Test Reimplementation method pattern (CMP1 to parts of the original system are now implicitly supported in CPM3) or a Language-based Test Transformation method pat- the new environment. tern(CMP4 to CPM6) with the three possible system migration patterns Reimplementation, Language-based Transformation, IV. C O -M IGRATION M ETHOD PATTERNS and Conceptual Transformation. As the test method patterns cannot express the relation The pattern CMP1 is a combination of two reimplemen- between the system and the test case migration, we propose tation method patterns and it is a very simple pattern which a set of co-migration method patterns. Technically, a co- suggests a manual migration of the test cases. The ease of migration method pattern is a combination of a system method reimplementation of the tests cases depends on the documenta- pattern and a test method pattern, visually resembling to a tion of the system transformation method, the more structured double horseshoe model. We define a co-migration method the better. In the case of CMP2 and CMP3 patterns, where pattern as follows: Language-based Transformation and Conceptual Transforma- CMP1 CMP2 CMP3 TMP: Test Reimplementation TMP: Test Reimplementation TMP: Test Reimplementation SMP: Reimplementation SMP: Language-based Transformation SMP: Conceptual Transformation CMP4 CMP5 CMP6 TMP: Language-based TMP: Language-based TMP: Language-based Test Transformation Test Transformation Test Transformation SMP: Reimplementation SMP: Language-based Transformation SMP: Conceptual Transformation SMP System Method Pattern Test Test Manual Automated Code Model TMP Test Method Pattern Code Model Activity Activity CMP Co-Migration Method Pattern Figure 3. Co-Migration Patterns: Part I tion are applied respectively, the reimplementation of the test is a combination of Language-based Test Transformation and cases should be easier as the transformation of the system is Conceptual Transformation. In such a constellation, both the specified explicitly in terms of transformation rules. Basically, reverse engineering and forward engineering fragments, Model no system method fragments could be directly reused. Discovery and Test Code Generation respectively, can be completely reused. Due to the difference in the abstraction The pattern CMP4 is a combination of a Language-based levels, reuse of existing method fragments in the scope of Test Transformation and a Reimplementation. This pattern is the transformation step is only possible in an indirect way. suitable if the system reimplementation was well documented Namely, the transformation on the conceptual level, could be so that some transformation rules can be derived in order used as an input when the language transformation of the test to automate the transformation of the test cases. However, it cases is performed, i.e., the conceptual transformation param- suggests implementation of a parser for the source language eterizes the language-based test transformation. Similarly as as well as a code generator for the target language. Similarly with CMP5, the complexity of the transformation could be to the previous co-migration patterns, no system method higher if the source and the target frameworks differentiate a fragments could be directly reused. lot due to the implicit transformation of the test concepts. The pattern CMP5 is a combination of Language-based Test Figure 4, depicts the co-migration patterns that combine Transformation and Language-based Transformation. This either a Test Language-based Test Transformation method pattern has a symmetric constellation, as two transformations pattern (CMP7 to CPM9) or a Conceptual Test Transformation on the same abstraction level are combined. In such a constel- method pattern (CMP10 to CPM12) with the three possible lation, both the reverse engineering and forward engineering system migration patterns Reimplementation, Language-based fragments, Model Discovery and Test Code Generation respec- Transformation, and Conceptual Transformation. tively, can be completely reused. Reuse of existing method fragments is also possible in the scope of the transformation The pattern CMP7 is a combination of a Test Language- step. However, the complexity of the transformation could based Test Transformation and Reimplementation. This pattern be higher if the source and the target frameworks differ- is suitable if the system reimplementation was well docu- entiate a lot, meaning that the transformation of the test mented so that some transformation rules can be derived relevant concepts should be done implicitly. Regarding the in order to automate the transformation of the test cases. impacted test method fragments, the Language Transformation However, it suggests implementation of a parser for the activity is impacted by the corresponding method fragment source language as well as a code generator for the target from the system transformation method. The pattern CMP6 language. Furthermore, a test case understanding fragment and CMP7 CMP8 CMP9 TMP: Test Language-based TMP: Test Language based TMP: Test Language-based Test Transformation Test Transformation Test Transformation SMP: Reimplementation SMP: Language-based Transformation SMP: Conceptual Transformation CMP10 CMP11 CMP12 TMP: Conceptual Test Transformation TMP: Conceptual Test Transformation TMP: Conceptual Test Transformation SMP: Reimplementation SMP: Reimplementation SMP: Reimplementation SMP System Method Pattern Test Test Manual Automated Code Model TMP Test Method Pattern Code Model Activity Activity CMP Co-Migration Method Pattern Figure 4. Co-Migration Patterns: Part II test case concretiztaion fragment should be configured and only possible in an indirect way. Namely, the transformation implemented in terms of model-to-model transformations. on conceptual level could be used as an input when the Test The pattern CMP8 is a combination of Test Language-based Language-based Test Transformation is configured performed, Test Transformation and Language-based Transformation. In i.e., the Conceputal Transformation parameterizes the Test such a constellation, both the reverse engineering and forward Language-based Test Transformation. Similarly to CMP8, the engineering fragments, Model Discovery and Test Code Gen- complexity of the transformation is lowered as an implicit eration respectively, can be reused. Reuse of existing method mapping between the testing languages is defined. fragments is also possible in the scope of the transformation The pattern CMP10 is a combination of Conceptual Test step. But, a test case understanding fragment and test case Transformation and Reimplementation. The suitability of this concretiztaion fragment should be still selected and imple- pattern depends on the system reimplementation, whether it mented in terms of model-to-model transformations. However, was well documented so that some transformation rules can the complexity of the transformation is lower compared to be derived in order to automate the transformation of the est CMP7, as the transformation activity from the system method cases. However, it suggests implementation of a parser for pattern could be reused to higher extent as it is specified the source language as well as a code generator for the target explicitly through a model-to-model transformation. On the language. Furthermore, a test case understanding fragment and other side, the complexity of the transformation is lowered as test case concretiztaion fragment should be configured and an explicit mapping between the testing languages is defined. implemented in terms of model-to-model transformations. The pattern CMP9 is a combination of Test Language- The pattern CMP11 is a combination of Conceptual Test based Test Transformation and Conceptual Transformation. Transformation and Language-based Transformation. In such In such a constellation, both the reverse engineering and a constellation, both the reverse engineering and forward forward engineering fragments, Model Discovery and Test engineering fragments, Model Discovery and Test Code Gen- Code Generation respectively, can be completely reused. Due eration respectively, can be reused. Reuse of existing method to the difference in the abstraction levels, reuse of existing fragments is also possible in the scope of the transformation method fragments in the scope of the transformation step is step. But, a test case understanding fragment and test case concretiztaion fragment should be still selected and imple- with incremental changes and not coarse grained changes, i.e., mented in terms of model-to-model transformations. However, conceptual changes that often happen in software migration. the complexity of the transformation is lower compared to VI. C ONCLUSION AND F UTURE W ORK CMP10, as the transformation activity from the system method pattern could be reused to higher extent as it is specified In this paper, we presented a framework that enables a explicitly through a model-to-model transformation. On the modular construction of context-specific, model-driven mi- other side, the complexity of the transformation is lowered as gration methods for test cases. The framework consists of a an explicit mapping between the testing languages is defined. method base and a method engineering process. The method The pattern CMP12 is a combination of Conceptual Test base contains method fragments, as atomic building blocks Transformation and Conceptual Transformation. In such a of a migration method, and method patterns which encode constellation, both the reverse engineering and forward engi- specific migration strategies. In order to adequately address neering fragments, Model Discovery and Test Code Generation the co-evolution in test case co-migration, we propose a set of respectively, can be completely reused. Due to the difference co-migration methods that encode the information about the in the abstraction levels, reuse of existing method fragments dependency between the system and test case migration. In in the scope of the transformation step is only possible in an future work, we intend to conduct a quality analysis of the indirect way. Namely, the transformation on conceptual level constructed test migration methods regarding quality criteria could be used used as an input when a Test Language-based like completeness or correctness. Test Transformation method pattern is configured, i.e., a Con- R EFERENCES ceputal Transformation pattern parameterizes the Conceptual [1] J. Bisbal, D. Lawless, B. Bing Wu, and J. Grimson, “Legacy information Test Transformation. Similarly to CMP11, the complexity of systems: issues and directions,” IEEE Software, 1999. the transformation is lowered as an implicit mapping between [2] T. Mens and S. Demeyer, Software Evolution, 1st ed. Springer the testing languages is defined. Publishing Company, Incorporated, 2008. [3] M. Grieger, M. Fazal-Baqaie, G. Engels, and M. Klenke, “Concept-based V. R ELATED W ORK engineering of situation-specific migration methods,” in Proceedings of the 15th International Conference on Software Reuse: Bridging with Two main research areas are relevant for this work, namely Social-Awareness, 2016. [4] B. Henderson-Sellers, J. Ralyté, P. J. Ågerfalk, and M. Rossi, “Situa- method engineering and test case evolution. Regarding method tional method engineering,” in Springer Berlin Heidelberg, 2014. engineering, different categories of method engineering ap- [5] A. Fuhr, A. Winter, U. Erdmenger, T. Horn, U. Kaiser, V. Riediger, and proaches exist: fixed methods, a selection out of set of fixed W. Teppe, “Model-Driven Software Migration - Process Model, Tool Support and Application,” in Migrating Legacy Applications: Challenges methods, configuration of a method, tailoring a method or in Service Oriented Architecture and Cloud Computing Environments. a modular construction of the method. The method tailoring IGI Global, 2012. approaches enable tailoring of a provided method, which can [6] E. J. Chikofsky and J. H. Cross, “Reverse Engineering and Design Recovery: A Taxonomy,” IEEE Software, 1990. be changed arbitrarily ( [10], [11]). However, they solely focus [7] R. Kazman, S. Woods, and S. Carriere, “Requirements for integrating on system transformation and thus no transformation of test software architecture and reengineering models: CORUM II,” in Pro- cases was considered. Consequently, they do not support co- ceedings Fifth Working Conference on Reverse Engineering. IEEE Comput. Soc, 1998. migration of test cases. The approaches that support modular [8] OMG, Architecture-driven Modernization: Abstract Syntax Tree Meta- construction of transformation methods provide a higher level model (ASTM)- Version 1.0. Object Management Group, 2011. of flexibility as they rely on a set of predefined building [9] H. Bruneliere, J. Cabot, F. Jouault, and F. Madiot, “MoDisco,” in Proceedings of the IEEE/ACM international conference on Automated blocks for methods. The method engineering approach, pre- software engineering - ASE ’10. ACM Press, 2010. sented in [12], enables modular construction, but is specialized [10] C. Zillmann, A. Winter, A. Herget, W. Teppe, M. Theurer, A. Fuhr, for migration to service-oriented environments. This issue is T. Horn, V. Riediger, U. Erdmenger, U. Kaiser, D. Uhlig, and Y. Zim- mermann, “The SOAMIG Process Model in Industrial Applications,” addressed by the MEFiSTO Framework [3] by providing a in 2011 15th European Conference on Software Maintenance and general solution for modular construction of situation-specific Reengineering. IEEE, 2011. migration methods. However, similarly to the method tailoring [11] A. Menychtas and E. Al., “Software modernization and cloudification using the ARTIST migration methodology and framework,” Scalable approaches, these two approaches do not address the migration Computing: Practice and Experience, 2014. of test cases (except ARTIST [11] to some extent) as well. The [12] R. Khadka, G. Reijnders, A. Saeidi, S. Jansen, and J. Hage, “A method existing approaches in the test case evolution are predomi- engineering based legacy to soa migration method,” in 27th IEEE International Conference on Software Maintenance (ICSM), 2011. nantly focusing on the continuous co-evolution of test cases. [13] M. Mirzaaghaei, F. Pastore, and M. Pezze, “Supporting test suite In [13], a semi-automatic approach is presented that supports evolution through test case adaptation,” in 2012 IEEE Fifth International test suite evolution through test case adaptations. Existing test Conference on Software Testing, Verification and Validation, 2012, pp. 231–240. cases are repaired and new test cases are generated to react [14] E. J. Rapos, “Co-evolution of model-based tests for industrial automotive to incremental changes in the software system. In [14], a software,” in 2015 IEEE 8th International Conference on Software method is proposed which should improve the model-based Testing, Verification and Validation (ICST), 2015, pp. 1–2. test efficiency by co-evolving test models. As part of this work, software model evolution patterns as well as their effects on test models are studied in order to apply updates directly to the tests. All in all, the existing approaches deal only