In this case study for the transformation tool contest (TTC), we propose to implement object-oriented program refactorings using transformation techniques. The case study proposes two major challenges to be solved by solution candidates: (1) bi-directional synchronization between source/target program source code and abstract program representations, and (2) program transformation rules for program refactorings. We require solutions to implement at least two prominent refactorings, namely Pull Up Method and Create Superclass. Our evaluation framework consists of collections of sample programs comprising both positive and negative cases, as well as an automated before-after testing procedure.
Challenges resulting from software aging are well known but remain open. An approach to deal with software aging is refactoring. Concerning object-oriented (OO) programs in particular, most refactorings can be formulated and applied to a high-level structure and there is no need to go down to the instruction level. Nevertheless, most recent implementations usually rely on ad-hoc program transformations directly applied to the AST (Abstract Syntax Tree). A promising alternative to tackle the challenge of identifying those (possibly concealed) program parts being subject to structural improvements is graph-based refactoring.
Here, the program is transformed into an abstract and custom-tailored program graph representation that (i) only contains relevant program elements, and (ii) makes explicit static semantic cross-AST dependencies, being crucial to reason about refactorings. Nevertheless, certain language constructs of more sophisticated programming languages pose severe challenges for a correct execution of refactorings, especially for detecting refactoring possibilities and for verifying their feasibility. As a consequence, the correct speci cation and execution of refactorings for OO languages like Java have been extensively studied for a long time in the literature and, therefore, can not serve as scope for a TTC case study to their full extent. Therefore, we propose the challenge of graph-based refactorings to be considered on a restricted sub-language of Java 1.4, further limited to core OO constructs of particular interest for the respective structural patterns.
A solution should take the source code of a given Java program as input and apply a given refactoring to an appropriate representation of that program. Ideally, a program graph conforming to a prede ned type graph is created on which the refactorings are executed and, afterwards, propagated back to the source code. However, refactorings on other representations of the source code are also allowed as long as the source code is appropriately changed. To summarize, this case has two main challenges in its full extent and a subset of these in the basic case:
I Bidirectional and incremental synchronization of the Java source code and the PG. This dimension of the case study requires special attention when it comes to maintaining the correlation between di erent kinds of program representation (textual vs. graphical) and di erent abstraction levels. Additionally, the code and the graph representation di er signi cantly w.r.t. the type of information that is displayed explicitly, concerning, e.g., method calls, eld accesses, overloading, overriding etc. As the (forward) transformation of a given Java program into a corresponding PG representation necessarily comes with loss of information, the backward transformation of (re-)building behavior-preserving Java code from the refactored PG cannot be totally independent from the forward transformation { a correct solution for this case study has to provide some means of restoring those parts of the input program which are not mapped to, or re ected in the PG. II Program refactoring by PG transformation. In our case study, refactoring operations are represented as rules consisting of a left-hand side and a right-hand side as usual. The left-hand side contains the elements which have to be present in the input and whose images in the input will be replaced by a copy of the right-hand side if the rule is applied. Therefore, the actual program refactoring part of our case study involves in any case (i) the speci cation of the refactoring rules are based on refactoring operations given in a semi-formal way, (ii) pattern matching (potentially including forbidden patterns, recursive path expressions and other advanced techniques) to nd occurrences of the pattern to be refactored in the input program and (iii) a capability of transforming the PG in order to arrive at the refactored state. Note that the classical approach to program refactoring (which is used here) never goes deeper into program structure and semantics than high-level OO building blocks, namely classes, methods and eld declarations; the declarative rewriting of more ne-grained program elements such as statements and expressions within method bodies is de nitely out of scope of our case study for TTC.
Each challenge can be solved in a basic version (with an arbitrary
intermediate representation), and in an extended version (using a separate, intermediate
representation that is at least isomorphic to our proposed type graph). Two
exemplary refactoring operations should be implemented when solving this case
study. The rst one, Pull Up Method is a classical refactoring operation { our
speci cation follows that of [
In the following, we give a detailed description of the case study to be solved by specifying the constituting artifacts, (meta-)models and transformations in Section 2. The two sample refactoring operations mentioned above are elaborated (including various examples) in Section 3. The correctness of the solutions is tested concerning sample input programs together using an automated beforeafter testing framework containing executable program test cases. Some test cases are based on the examples of Section 3, while some of them are hidden from the user { these cases check if the refactorings have been carefully implemented such that they also handle more complex situations correctly. Further details about this framework, the additional evaluation criteria, and the solution ranking system can be found in Section 4.
Based on the demanded functionality to be implemented by all solutions for the case study, further interesting extensions to those core tasks are mentioned in Section 5. 2
Before diving into the details of the actual scenario to cope with, we motivate
our case study once again by recalling the aim of refactorings. For this
purpose, we use the very words of Opdyke, the godfather of refactorings, which
say that refactoring is the same as \restructuring evolving programs to improve
their maintainability without altering their (externally visible) behaviors" [
To describe the case study in a nutshell, we provide an intuitive example here, describing a program state where a natural need for restructuring arises. Example. Refactoring Scenario 1 shows a basic example for a refactoring of a simple program. The source code of this program is shown in Appendix 1a. In this case, we expect that a program transformation takes place which moves method from all child classes of the class ParentClass to this same superclass. (This is a classical refactoring which is called Pull Up Method and builds a signi cant part of our case study. Pull Up Method will be further speci ed and exempli ed in Section 3.)
ParentClass
In the following, we give a schematic overall picture of the intended transformation chain (Figure 1) and its constituting artifacts. Solid arrows denote the extended challenge, while the dashed arrow shows the basic challenge not using a PG representation. The basic challenge can include an arbitrary intermediate representation. In Section 2.1, some details regarding the input Java code and the PG meta-model (called the type graph) are given, while Section 2.2 provides information on the individual transformation steps and the arising di culties. Java Source Code All input programs considered for refactoring for TTC are fully functioning (although, abstract) Java programs built for the very purpose of checking the correct and thorough implementation of the given refactoring operations. Some test input programs are openly available and will be also described later on, while some others serve as blind tests and are not accessible to the solution developers.
The Java programs conform to the Java 1.4 major version. Moreover, the following features and language elements are explicitly out of scope for this case study: { access modi ers (all elements have to be public) { interfaces { constructors { the keywords abstract, static and final except for public static void main(String[] args) { the keyword super { exception handling { inner, local and anonymous classes { multi-threading (synchronized, volatile, ...)
On the other hand, we would like to point out that the following Java language elements and constructs should be considered: { inheritance { method calls, method overloading and method overriding { eld accesses and eld hiding { libraries
To detect external libraries, editable classes must have an identical root package which is not the root package of any used library.
Type Graph for Representing Java Programs Figure 2 shows the type graph meta-model that is part of the extended case study assets as an EMF metamodel { nevertheless, other meta-modeling technologies are allowed in solutions as well. If the solution is designed for the extended challenge, a program graph is only allowed to contain the information visualized in Figure 2. For a technical realization of the shown types, references, and attributes tool depended tuning is allowed. It is not allowed to make additional information available in the PG.
In conformance with the restrictions on the considered Java programs and with the nature of classical refactoring, the type graph does not include any modeling possibilities for access modi ers, interfaces, etc. and any code constituents lying deeper than the method level. In the following, we describe the meaning of some of the most important nodes and edges of the type graph.
The type graph represents the basic structure of a Java program. The node TypeGraph serves as a common container for each program element as the root of the containment tree. The Java package structure is modeled by the node TPackage and the corresponding self-edge for building a package tree. The node TClass stands for Java classes and contains members (the abstract class TMember), which can be method and eld de nitions (TMethodDefinition or TFieldDefinition, respectively). In addition, a TClass refers to the abstract class TSignature, which is the common ancestor of method and eld signatures.
Methods and elds are represented by a structure consisting of three elements: { The name of the method ( eld) contained in the attribute tName of TMethod (TField), which is globally visible in the PG. { The signatures of the methods ( elds) of this name, represented by the class TMethodSignature (TFieldSignature). The signature of a method consists of its name and its list of parameter types paramList, while the signature of a eld consists of its name and its type. Di erent signatures having the same name (i.e., a common container TMethod or TField) allow overloading. Signatures have a central role in the Java language, as all method calls and eld accesses are based on signatures. { TMethodDefinition (TFieldDefinition) is an abstraction layer representing the instruction level of Java. Relevant information is expressed by reference edges in the type graph. Overloading and overriding is declared by the corresponding edges between de nition instances, although the overloading/overriding structure is also implicitly given through signatures, de nitions and inheritance. The access edges between member instances represent dependencies between one member and the other members. This single edge type stands for all kinds of semantic dependencies among class members, namely read, write and call. 2.2
The transformation chain for the extended challenge consists of three consecutive steps which are detailed here. For the basic challenge, no obligatory transformation chain is demanded.
scribed in Sec. 2.1, it has to be transformed into an abstract PG representation conforming to the type graph meta-model (ibid.). Important note: the fact that some information necessarily disappears during this transformation calls for a solution where some preservation technique is employed, i.e., it is possible to rebuild those parts in the third step (see below) which are not present in the PG.
We remark that any intermediate program representations like JaMoPP1, MoDisco2, AST models etc., are allowed to facilitate the Java-to-PG and PGto-Java transformations.
consists in an endogenous (PG-to-PG) restructuring of the program graph, according to the speci cations of the refactoring operations Pull Up Method resp. Create Superclass. For those speci cations and actual refactoring examples, see Sec. 3.
Third Step: Program Graph to Java Code As already mentioned at the rst step (Java-to-PG), one of the most di cult tasks is to create a solution which provides a means to recover the program parts not included in the PG when transforming its refactored state back into Java source code. In other words, it is impossible to implement the Java-to-PG and the PG-to-Java transformations (the rst and the third step) independently of each other. Furthermore, over the challenges posed by the abstraction level of the PG, one has to pay extra attention if a newly created PG element has to appear in the refactored code.
The resulting Java code has to ful ll the requirements of (i) having those code parts unchanged which are not a ected by the refactoring and (ii) retaining the observable behavior of the input program. These properties are checked using before-after testing (as usual in the case of behavior-based test criteria) provided by the automated test framework that is part of the case study and is further described in Section 4.
After this brief overview of both the static and the dynamic ingredients of the transformation scenario to be dealt with, we proceed as follows: In Section 3, we put the second step in Sec. 2.2 under the microscope and present the two aforementioned refactoring operations with associated examples to also provide an intuition how and why they are performed. Thereupon, in Section 4, we describe our automated before-after testing framework for checking the correctness of the implementations, which also serves as a basis for the solution ranking system described in the same section including further evaluation criteria. 3
In the following, we provide an informal speci cation of the requested refactorings. 3.1
First, we provide an intuition of Pull Up Method textually. Additionally, we give some further information and examples to clarify the requirements. Situation and action. There are methods with identical signatures (name and parameters) and equivalent behaviours in direct subclasses of a single superclass. These methods are then moved to the superclass, i.e., after the refactoring, the method is a member of the superclass and it is deleted from the subclasses. Graphical representation. Figure 3 shows a schematic representation of how Pull Up Method is performed. We use the elements of the type graph introduced in Sec. 2.1 and a notation with left- and right-hand sides as usual for graph transformation. Here, the left-hand side shows which elements have to be present or absent in the PG when applying the refactoring to it; an occurrence of the left-hand side is replaced by the right-hand side by preserving or deleting the elements of it and optionally creating some new elements and gluing them to the PG. It is implicitly given through object names which parts are preserved. In addition, we explicitly show the parts to be deleted on the left-hand side in red and marked with -- and the parts to be created on the right-hand side in green and marked with ++. The left-hand side also includes a forbidden pattern or NAC, which in this case consists of a single edge and is shown crossed through and is additionally highlighted in blue. This edge has to be absent in the input graph for the refactoring to be possible. Patterns within stacked rectangles may match multiple times.
inheritance inheritance inheritance inheritance De nition. In this case study, a Pull Up Method refactoring is speci ed as pum(parent, signature) with the following components: { a superclass parent, whose direct child classes are supposed to contain at least one equivalent method implementation, and { the method signature signature of such an equivalent method implementation, which represents the method to be pull-upped to parent.
Note that a signature consists of a name and a parameter list. The return type is not part of the signature. Anyway, within a class hierarchy, all return types of the method de nitions of a signature have to be covariant.
Two equivalent implementations of a signature do not necessarily have identical implementations. Only their behavior is crucial. As proving that two implementations have identical behavior is undecidable, this decision has to be taken by a developer before initiating the refactoring.
In case the application conditions (see below) are ful lled, the method signature signature as well as a corresponding method de nition will be part of the parent. The copies of the other de nitions of signature will be deleted from all child classes. Note that a Pull Up Method instance does not necessarily represent a valid refactoring - it marks merely a part of the input program where it is looked for a possible pull-up action.
Application conditions. In addition to the conditions shown in Figure 3, the following preconditions have to be ful lled for a Pull Up Method refactoring instance pum(parent, signature): 1. Each child class of the class parent has at least one common method signature signature with the corresponding method de nitions (def initioni for the i-th child class) having equivalent functionality. 2. Each def initioni of signature in the child classes is only accessing methods and elds accessible from parent. Methods and elds de ned in the child classes are not direct accessible . 3. The parent does not belong to a library and is editable.
Important remarks. Although it is not explicitly shown in Figure 3, all access edges in the PG pointing to a method de nition deleted by the refactoring have to be redirected to point to the one which is preserved, so that subsequent refactorings are able to consider a coherent state of the PG. The actual choice of the preserved de nition is irrelevant and the de nitions can be arbitrarily matched, as the actual method implementations are out of scope for this case study. If methods have di erent return types, then a conservative behavior, such as the denial of the refactoring is allowed.
Example 1. Our rst and most basic example for Pull Up Method is the one we have already shown as a general motivation for refactoring in the introduction part of Section 2. Example 2. Given the program in Refactoring Scenario 2, the Pull Up Method refactoring pum(ParentClass, method(String, int)) seen in the previous example is not possible. In ParentClass, a method with the given signature is already present which is overridden by methods in ChildClass1 and ChildClass2. Accordingly, the NAC shown on the left-hand side of Figure 3 is violated. Refactoring Scenario 2: Refactoring pum(ParentClass, method(String, int)) not possible { method(String, int) already exists in ParentClass Example 3. Given the program in Refactoring Scenario 3, the Pull Up Method refactoring pum(parent, method(String, int)) is not possible. In this case, Precondition 1 is not ful lled as ChildClass3 does not contain the common method with the signature method(String, int).
ParentClass Refactoring Scenario 3: Refactoring pum(ParentClass, method(String, int)) not possible { one of the child classes does not have method(String, int)
All examples shown here also have a corresponding test case in our test framework which is described in Sec. 4, with the example programs being accessible to the solution developers. In addition, there are some built-in test cases that are hidden in the framework and check trickier situations. For each of these hidden test cases, a textual hint for its purpose is provided by the test framework. 3.2
The refactoring operation Create Superclass is described in a similar fashion as the Pull Up Method refactoring above. Situation and action. There is a set of classes with similar features. As a rst step towards an improved program structure, a new common superclass of these classes is created.
Graphical representation. Figure 4 shows a schematic representation of how the Create Superclass refactoring is performed with the same notation as by the Pull Up Method refactoring above. The classes either has to have the same superclass in the PG or none of them has a superclass modeled in the PG. (Note that from a technical point of view, each Java class has a superclass. Also, the distinction above refers to the representation in the PG.) Here, both cases are shown.
inheritance De nition. In this case study, a Create Superclass instance is de ned as csc(classes, new superclass) consisting of the following components: { a list of classes classes, where all classes have identical inheritance relations (i.e., each of them inherits from the same class or they do not inherit from any class in the PG), and { a superclass new superclass, which does not exist before the refactoring and has to be generated.
In case the application pre- and postconditions (see below) are ful lled, a new class new superclass will be created which becomes the superclass of the classes in classes. Note that a Create Superclass refactoring does not necessarily represent a valid refactoring - it marks merely a part of the input program where it is looked for a possible refactoring operation.
Application conditions. In addition to the conditions shown in Figure 4, the following precondition has to be ful lled for a Create Superclass instance csc(classes, new superclass): 1. The classes contained in classes are implementing the same superclass.
Note that classes with no explicit inheritance reference in Java are implementing java.lang.Object { modeling this class explicitly in the PG is a developer decision which does not in uence the conditions for Create Superclass.
Additionally, the result of csc(classes; new superclass) has to ful l the following postconditions: 1. Each class in classes has an inheritance reference to new superclass. 2. In case the classes in classes had an explicit inheritance reference to a superclass parent before the refactoring, their new superclass new superclass has an inheritance reference to parent.
Example 1: Refactoring Scenario 4 shows the most basic example on which Create Superclass is applicable. The refactoring operation csc(fChildClass1, ChildClass2g, NewSuperclass) is possible as the desired new class does not exist yet. NewSuperclass ChildClass1
As demonstrated by the previous example, the Create Superclass refactoring itself is relatively uncomplicated, however, there are additional hidden test cases in the framework for Create Superclass as well. Note that, as already stated before, the main challenge by this refactoring is not to restructure the PG, respectively the chosen intermediate representation in the basic challenge, but to propagate the new element into the Java source code. 4
In this section, we introduce our test framework ARTE (Automated Refactoring Test Environment) for checking the correctness of implementations (Sec. 4.1) and the criteria and the scoring system which will be used to evaluate and rank the submitted solutions (Sec. 4.2). 4.1
Test Framework before execute program output execute test cases compare execute program after output
Fig. 5: Schematic Process of Before-after Testing
To enable the evaluation and ranking of the solutions for our case study, we have created an automated refactoring testing environment called ARTE, whose mechanism is sketched in Figure 5. This test framework relies on the wellknown principle of before-after testing, which is often used in behavior-critical scenarios: the behavior of the input is determined by stimulating it through the test environment and it is then checked if the output of the transformation reacts identically to the same stimulation.
In our framework, before-testing consists in compiling and executing the program and recording its console output. On the other hand, after-testing consists in compiling and executing the refactored program created by the actual solution under test, and comparing its console output to the one recorded in the before-testing phase.
The testing procedure is described in test cases. A test case consists of the following: { a Java program assigned to it, on which the transformation takes place (one program can be assigned to multiple test cases) and { a sequence of commands which can be (i) actual transformation operations or (ii) assertions to check if the transformations provided the expected result (e.g., nothing has changed if there is no correct refactoring possible). Note that a transformation operation cannot be executed without a corresponding assertion check for success.
The execution of a test case comprises the following steps: { the before-testing phase as described above, { the execution of the commands in the test case and { the after-testing phase as described above.
For further details on how to use our testing framework ARTE and how to write individual test cases, please refer to the ARTE handbook. Beyond the ones mentioned above, the number of imaginable extensions regarding the supported refactorings or the framework is unlimited. The reviewers can also reward some other creative extension approaches using the extension score. 4.2
In this section, we propose a systematic way of evaluating and ranking the solutions for the case study.
There is a total of 100 points that can be achieved by a solution. These 100 points are composed as follows (with a detailed description of the various aspects thereafter): { max. 60 points: correctness and completeness (successful execution of test cases) { max. 10 points: comparison of the execution times of the solutions { max. 30 points: quality of the solution, veri ed by the reviewers (15 points per reviewer assuming 2 reviewers per solution) { ...and a maximum of 15 points beyond the total of 100 by comparing how much of the case extensions described in Section 5 has been implemented Correctness and completeness - 60 points. By the nal ranking of the solutions, there are three kinds of test cases considered: (i) the public ones, which are part of the test framework ARTE and have been also discussed in Sec. 3, (ii) the hidden ones, also being part of ARTE but being not further speci ed except for some hints within ARTE and (iii) some additional test cases which will not be announced until the nal evaluation occurs. There is a xed amount of points assigned to each test case; these numbers are not public, however, the developers may assume that the point distribution re ects the levels of di culty. The solution developer should provide: a simple summary of the test cases accomplished by the solution. As the basic challenge o ers fewer possibilities, we will give 2/3 of all points that can be achieved in this category.
Execution times - 10 points. The test framework ARTE provides an execution time measurement (per test case), whose result is then displayed on the console in the test summary. Based on the nal test set, the fastest solution gets 10 points and the slowest 1 point, while the remaining ones will be distributed homogeneously on this scale.
Reviewer opinion - 2 x 15 points. Each of the reviewers has 15 points to award to the solution according to how much they like it.
To make the reviewers get a better insight into your solution beyond its objective correctness, it is generally a good idea to name some strong and some weak spots of the solution. It is de nitely the developer itself who can contribute the most to this topic.
The soft aspects listed below serve as guidelines or hints for the solution developers to comment on their solution beyond the scope of the actual test cases in the contest. It is not mandatory, but we are excited to learn more about the way your tool works and what it can achieve! { Comprehensibility: we think that the question if a solution works with an understandable mechanism which is not exclusively accessible for the high priests of a cult is of high importance, especially in the scope of the Transformation Tool Contest where such a comprehensible solution facilitates discussion and contributes to a pro table event. { Readability: in contrast to comprehensibility, this aspect refers to the outer appearance of the tool - whether it has a nice and/or user-friendly interface, can be easily operated, maybe even with custom-tailored commands or a DSL, ... { Communication with the user: although related to readability, this aspect refers to the quality, informativeness and level of detailedness of the actual messages given to the user while implementing a solution. In other words: Am I as user informed that everything went smoothly? In case of some failure or malfunction, am I thoroughly informed what actually went wrong? { Robustness: this classical software quality aspect characterizes how a software behaves if put into an erroneous environment, getting malformed input, ... E.g., what happens if some out-of-scope keywords appear in a Java program to be refactored? { Extensibility: this one also examines the inner structure of the solution concerning its possibilities to expand in the future. E.g., would it be easily feasible to build in the support of additional Java constructs or new refactorings? { Debugging: in contrast to readability, this aspect refers solely to the debugging capabilities of the tool used to create the solution. In case a problem is uncovered through erroneous behavior, what means are provided to locate the cause of a design failure? Does the tool provide suggestions for xing errors? How precise are the debug messages? We have created a simple online form3 for the reviewers to send in their opinion regarding the aspects above. 5
While the core case described in Sections 1-4 is already a full- edged refactoring use-case on its own right, it can still be extended in various ways inspired by the theory and practice of refactorings. Here, we mention some interesting possibilities for extending a solution beyond the requirements of the core case. New ideas are of course welcome and will be taken into account. There is a bonus of 15 points, which can be achieved by providing some (maybe partial) answers to one or more extensions or at least outlining a concept with relation to the core case solution. One convincing extension is enough to achieve the full bonus. These points are awarded according to the reviewers' opinion and we only give some recommendations which may serve as scoring guidelines. The nal bonus score is calculated as the average of the reviewers' scores. 5.1
The two refactoring operations considered in the core case, namely Pull Up Method and Create Superclass, are simple actions compared to some complex operations which are still described as a single refactoring step in the literature. A classical example for such a more complex refactoring is Extract Superclass, which can be speci ed as a combination of Create Superclass and Pull Up Method (and its pendant for elds, Pull Up Field). After executing a Create Superclass for some classes, one can use Pull Up Method resp. Pull Up Field on the newly created parent class to move the common members there. Recommendation. 15 points for a full- edged implementation which can be executed on an appropriate example program; 9 points for a working implementation which misses Pull Up Field; 1-3 points for a concept sketch using the actual rule implementations.
5.2
In our core refactoring use-case, the hypothetical user already realized the need of refactoring and also identi ed the spot where it would be possible and the kind of action to be executed. Nevertheless, one can imagine a somewhat orthogonal approach where an automatic refactoring environment permanently monitors an evolving software and proactively proposes refactorings being feasible on the code base. To be more concrete, as a rst step towards such a system, one might implement a method which takes as input the whole program and returns one (or more) feasible refactoring(s).
Recommendation. 15 points for a full- edged implementation which can be executed on an appropriate example program; 5-15 points for an alternative way of implementation according to its scope and usability; 1-5 points for a plausible concept sketch according to its ambition and clarity.
From a practical point of view, it is not unlikely that two developers of the same software might want to execute refactorings independently of each other. In this case, it can happen that the refactored code states are not compatible to each other any more and a merge is not possible. Concerning only two alternative refactorings, it is equivalent with stating that the result of the refactorings is not independent of their execution sequence. As a concrete step towards a con ict detection for refactorings, one can e.g. think of extending the framework so that it checks consequent refactoring operations and noti es the user if their execution sequence is considered as critical.
Recommendation. 15 points for a full- edged implementation which can be executed on an appropriate pair of refactorings; 9-15 points for an alternative way of implementation according to its scope and usability; 1-9 points for a plausible concept sketch according to its ambition and clarity.
ARTE is a Java terminal program which executes test cases speci ed in a Domain Speci c Language (DSL) on solutions of the OO Refactoring Case Study of the Transformation Tool Contest 2015. A test case comprises a sequence of refactoring operations on a Java program as well as the expected results. The test cases are collected in a test suite in ARTE. The tests aim at checking the correct analysis of pre- and postconditions for refactorings and the execution of these refactorings.
For executing the provided test framework, Java JDK 1.7 is needed and the path variable has to be set to point to the JDK and not to a JRE. With a JRE and no JDK, the test framework will still start but the compilation of Java programs during testing will fail.
ARTE has been tested on Windows command line and in Bash. However, ARTE should be executable in every Java-capable terminal.
A.1
A solution for the case study has to implement an interface that speci es method signatures which ARTE relies on. This interface is called TestInterface and is provided in the le TTCTestInterface.jar.
Additionally, the solutions have to be exported as a simple (not executable) JAR le. This le has to contain a folder META-INF/services with a le called ttc.testsuite.interfaces.TestInterface. This latter le has to contain the fully quali ed name of that class which implements the TestInterface. In the TTCSolutionDummy project, a dummy implementation of this interface is demonstrated.
An implementation ful lling these conditions can be dynamically loaded into ARTE using the Java ServiceLoader. Further information can be found on the Oracle website 45.
The single methods which have to be implemented are: getPluginName():
Returns the name of the actually loaded solution. setPermanentStoragePath(File path):
Is called by ARTE to hand over a location at which data can be stored permanently by the solution. setTmpPath(File path):
Is called by ARTE to hand over a location at which data can be stored temporally. All contents written to this location will be automatically deleted by closing ARTE. 4 http://www.oracle.com/technetwork/articles/javase/extensible-137159.
html 5 http://docs.oracle.com/javase/7/docs/api/java/util/ServiceLoader.html setLogPath(File path):
Is called by ARTE to hand over a location at which logs can be stored permanently. ARTE will store reports at this location as well. usesProgramGraph():
Is called by ARTE to determine whether the solution uses the program graph of the extended case. setProgramLocation(String path):
Is called in the basic case by ARTE to hand over the location of the Java program which will be refactored next. createProgramGraph(String path):
Is called in the extended case by ARTE to instruct the solution to build the program graph for the Java program located at path. applyPullUpMethod(Pull Up Refactoring refactoring):
Is called by ARTE for a Pull Up Method refactoring to be performed. The structure of the type Pull Up Refactoring is explained in the following DSL part { its elds have similar names as the corresponding keywords of the DSL. Note that name elds contain names of variables inside the DSL and not method or class names. applyCreateSuperclass(Create Superclass Refactoring refactoring): Is called by ARTE for a Create Superclass refactoring to be performed. For the type Create Superclass Refactoring holds the same as for the type Pull Up Refactoring above. synchronizeChanges():
Is called by ARTE to instruct the loaded solution to synchronize the Java source code with the PG. This means that the changes made on the PG have to be propagated into the Java program.
A.2
We provide a custom DSL to make the creation of new test cases more convenient. For developing test cases, we provide an Eclipse plug-in which supports syntax highlighting and basic validation of the test les. However, test les can be written using any text editor.
In the following, we show on a Pull Up Method and on an Create Superclass example how our DSL can be used to create test cases and how to perform tests using ARTE. We explain the commands within the examples in a practical, step-by-step fashion. For further information about commands not covered by these simple examples, refer to the in-line explanations and to Appendix C where a full command list is provided.
As Pull Up Method test case example, we recapitulate our Example 1 that has been used in Section 2 of the case description to motivate refactorings. The structure of the Java program used in this example is shown in Refactoring Scenario 1 and the corresponding source code in Appendix B. On this program, we are going to execute the refactoring pum(ParentClass, method(String, int)).
Test cases are wrapped in a TestFile environment that also de nes the name of the test case. This name should to be identical with the name of the le containing this TestFile environment. If this is not the case, it is automatically renamed during import into the test framework, which can lead to failing imports with no obvious cause. The TestFile name has to be unique.
TestFile 1.1: PUM Example 1 1
T e s t F i l e p u b l i c p u m 1 f
A TestFile contains everything needed for a test execution, namely classes, methods, refactorings and test cases. All elements used in the test cases have to be de ned in the corresponding test le. Therefore, we de ne all classes we want to use in the test case.
In our example, the rst class to be de ned is ParentClass. To unambiguously identify the class in the program during test execution, the package containing the class has to be given as well. If the class is contained in the default package, the package parameter can be omitted. 2 3 4 5 6 7 8 9 10 11 12 13 14 g g g c l a s s e x i s t i n g p a r e n t f package " example01 " name " P a r e n t C l a s s " c l a s s c h i l d 1 f package " example01 " name " C h i l d C l a s s 1 " c l a s s c h i l d 2 f package " example01 " name " C h i l d C l a s s 2 "
As we have to check after the refactoring whether the pull-upped method is no longer contained in the child classes, those have to be de ned as well.
Required primitive types and classes from libraries have to be also explicitly de ned. In this example, we need these in the signature of the method to be pulled-up.
g g g g
According to Example 1, we are going to specify the Pull Up Method refactoring pum(ParentClass, method(String,int). For this purpose, we have dened the necessary elements above and now, we combine them using the keyword pullup method. (A refactoring can be used in multiple test cases within the test le.) pullup method executable pum f p a r e n t e x i s t i n g p a r e n t method c h i l d m e t h o d c l a s s i n t f
name " i n t "
Each test case has a mandatory description, which will be displayed during execution of the test case. As second argument, the name of a Java program is given. By having a look on the le structure shown in Source Code B, it can be seen that this program name refers to the folder containing the input program. c a s e pub pum1 1 paper1 f d e s c r i p t i o n "PUM POS : ( paper ex1 ) Pull up o f two . . . " program " paper example01 " t e s t f l o w f a s s e r t T r u e ( executable pum ) e x i s t i n g p a r e n t c o n t a i n s c h i l d m e t h o d c h i l d 1 ~ c o n t a i n s c h i l d m e t h o d c h i l d 2 ~ c o n t a i n s c h i l d m e t h o d
The single steps of a test case are de ned in a list starting with the keyword testflow. A testflow environment automatically induces both before- and after-testing. As the previously de ned refactoring is supposed to succeed on the given Java program, we assert a successful refactoring by using the assertTrue command. Refactorings can only be executed with an accompanying assertion.
After executing the refactoring, we check if the resulting Java program has the structure shown in Refactoring Scenario 1. Therefore we are checking if the method has been moved to the parent and if the child classes do not contain the method anymore.
A.4
In the following, we describe a test case for a Create Superclass refactoring. For this purpose, we use again our example from the case description. The example is shown in Refactoring Scenario 4. The refactoring csc(fChildClass1, ChildClass2g, Superclass) is expected to succeed.
Again, we rst de ne the necessary elements for the refactoring. The classes for which a new superclass will be created are enumerated in a list called child by using the classes keyword. One class can be added to multiple lists and lists can be used by multiple refactorings.
T e s t F i l e p u b l i c e x s 1 f c l a s s c h i l d 1 f package " example04 " name " C h i l d C l a s s 1 " g c l a s s c h i l d 2 f package " example04 " name " C h i l d C l a s s 2 " c l a s s e s c h i l d f c h i l d 1 , c h i l d 2 g
Elements de ned in a TestFile do not have to exist in the input or output program. However, accessing these elements will result in a failure if they have not been created before by, e.g., a refactoring. Here, we de ne the variable new superclass as a \placeholder" for the class Superclass which will be created by the Create Superclass refactoring.
c l a s s n e w s u p e r c l a s s f package " example04 " name " S u p e r c l a s s " 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 g
For the de nition of the Create Superclass refactoring, we are referencing the elements de ned before.
c r e a t e s u p e r c l a s s r e f a c t o r i n g f c h i l d c h i l d s t a r g e t n e w s u p e r c l a s s c a s e p u b e x s 1 1 f d e s c r i p t i o n "EXS POS : C r e a t e a s u p e r c l a s s f o r two . . . " program " example04 " t e s t f l o w f
As we are expecting the refactoring to succeed, we use the assertTrue keyword. If we expect a refactoring to fail, we can use the keyword assertFalse. The additional keywords expectTrue and expectFalse can be used in ambiguous cases; these result in success if the expectation is ful lled and in a warning instead of a failure otherwise. Additionally, these two keywords include an elseblock where static tests on the unexpected outcome can be executed. For more details, refer to Appendix C.
a s s e r t T r u e ( r e f a c t o r i n g ) The step keyword allows for grouping the di erent stages in a testflow but has no in uence on the execution.
At this point, we have to check whether the child class extend the new superclass or not.
s t e p f c h i l d 1 e x t e n d s n e w s u p e r c l a s s c h i l d 2 e x t e n d s n e w s u p e r c l a s s g g
g It is possible to execute multiple refactorings in a single test case. A.5
On Windows, ARTE can be started by double-clicking run windows.bat. On Linux, the le run linux.sh has to be executed.
If ARTE has been launched for the rst time, a solution has to be loaded. l o a d s o l u t i o n /home/ f o o /dummy . j a r The entered path has to be absolute. The referenced solution will be copied to the permanent storage path of ARTE. The same command has to be used again to load a di erent version of the solution. The previous loaded solution will be deleted.
Test cases can be loaded similarly. In contrast to the load solution command, multiple test cases can be imported.
t e s t /home/ p u b l i c p u m 1 . t t c /home/ p u b l i c e x s 1 . t t c The import of the Java programs is a bit more complex. In addition to the path where the program is located, the main class of the program has to be given as well. The Java programs have to be structured like the example shown in Source Code B. The referenced program folder has to contain a src folder. The package structure is represented by further subfolders containing classes. The referenced program folder is equivalent to an Eclipse project folder.
A Java program loaded into ARTE has to contain a class de ning a main method that is executed during testing. s r c /home/ paper example01 main example01 . C h i l d C l a s s 1
It is possible to print out each loaded test case and Java program. t e s t c a s e s
l i s t programs
l i s t There are three ways to execute test cases: 1. Execute test cases by name. This is only possible for our public test cases and for self-written test cases. In this variant, multiple test cases can be chosen. If the name of a test le is entered, each test case in this le will be executed.
In the example below, all cases contained in the le public pum 1.ttc and the test case pub exs1 1 will be executed. e x e c u t e
t e s t p u b l i c p u m 1 . t t c p u b e x s 1 1 2. Execute all hidden test cases. e x e c u t e
hidden e x e c u t e
a l l e x i t 3. Execute all public, hidden and self-written test cases.
It is indispensable to use the exit command after using ARTE.
Most of the presented commands can be executed in various ways. Feel free to nd your favourite. The help command will help you with this. If you, e.g., want to know more about test cases, try the following: h e l p t e s t c a s e s
1 package example01 ; 2 3 public c l a s s ParentClass f 4 5 6 g public ParentClass ( ) f g C h i l d C l a s s 2 c2 = new C h i l d C l a s s 2 ( ) ; c2 . method ( " c2 : H e l l o World" , 3 ) ; public void method ( S t r i n g message , int r e p e a t ) f for ( int i =0; i <r e p e a t ; i ++)f
System . out . p r i n t l n ( message ) ; 27 package example01 ; 28 29 public c l a s s C h i l d C l a s s 2 extends ParentClass f 30 31 public C h i l d C l a s s 2 ( ) f g 32 33 34 35 36 g 37 g public void method ( S t r i n g s t r i n g , int k ) f int i = 0 ; while ( i++ < k ) System . out . p r i n t l n ( s t r i n g ) ; C
Command TestFile le id f g Command class class id f g Command classes classes id f
Subcommand Description
A test le always starts with this command. The le has to be called \ le id.ttc".
The content of a test le can be a combination of le content elements class, classes, method, pullup method,
create superclass and case.
Subcommand Description
The class command is used to describe a Java class.
An optional String value like "subsubpackpackage [String] age.subpackage.package". name [String] The name of the class.
Subcommand Description
The classes command is used to de ne sets of classes for further use.
A comma separated list of classes which class id0; :::; class idn should be grouped. Command method method id f
Subcommand Description
The method command is used to describe a Java method signature. name [String] The name of the method.
Parameters are optional and are param class d0; :::; class idn an ordered list of comma separated references to classes. g Command pullup method refactoring id f Command create superclass refactoring id f f g Command case test case id f
Subcommand parent class id
Description The de nition of a Pull Up Method refactoring.
A reference to the parent class whose childs a method should be pulled up from. method method id wAhircehfesrheonucled btoe ptuhleledmuept.hod
Subcommand Description
The de nition of a Create Superclass refactoring.
A reference to the set of classes classes id classes for which a superclass should be created.
A reference to a class varitarget class id able describing the superclass
which will be created.
Subcommand Description
A test case can be identi ed by the test suite through test case id.
A textual description of the test case. description [String] This description is also shown in the test tool.
The name of the program on which the program [String] test case should operate. test ow f A container for the test commands.
An ordered list of test commands that can contain step, assertTrue(), assertFalse(), expectTrue(), expecttest step0; :::; test stepn False(), contains, contains, ex
extends, synchronize and tends, compile. g Command Subcommand step f
Description Allows for grouping, has no e ect on the execution.
An ordered list of test steps which can contain step, assertTrue(), assertFalse(), expecttest step0; :::; test stepn True(), expectFalse(), contains, contains, extends, extends, synchronize and compile.
Command assertTrue( assertFalse(
Subcommand Description
Checks whether a refactoring has been executed successful. The result is compared with the assertion. refactoring id The refactoring which will be handed to the solution for execution. g ) g g else f LHS Variable Command RHS Variable Description
Checks if the method or eld (RHS) is conclass id contains meeltdhoidd id itfaitnheedminetthhoedcolarss e(lLdHisS)n.oTthceontteasitnecdasienfathiles class.
Command
Subcommand expectTrue( expectFalse( refactoring id
Description Checks whether a refactoring has been executed successful. The result is compared with the expected result. If the expected result is not matched, the execution can still be successful.
The refactoring which will be handed to the solution for execution.
An ordered list of test steps executed if the expectation has been matched. The test steps can contain step, assertTrue(), astest step0; :::; test stepn sertFalse(), expectTrue(), expectFalse(), contains, contains, extends, extends, synchronize and compile. LHS Variable Command RHS Variable Description
Checks if the method or eld (RHS) is not class id contains meeltdhoidd id fcaoinlstaifintehde minetthheodcloarss e(lLdHisS)c.onTthaeinteedstinctahsee class.
LHS Variable Command RHS Variable Description
Checks whether the LHS class extends the class id extends class id RHS class. The test case fails if LHS does not extend RHS.
LHS Variable Command RHS Variable Description
Checks whether the LHS class does not exclass id extends class id tend the RHS class. The test case fails if LHS extends RHS.