This paper presents a solution for the Java Refactoring Case of the 2015 Transformation Tool Contest. The solution utilises Eclipse JDT for parsing the source code, and uses a visitor to build the program graph. EMF-INCQUERY, VIATRA and the Xtend programming language are used for defining and performing the model transformations. This paper describes a solution for the extended version of the TTC 2015 Java Refactoring Case. The source code of the solution is available as an open-source project.1 There is also a SHARE image available.2 The use of automated model transformations is a key factor in modern model-driven system engineering. Model transformations allow the users to query, derive and manipulate large industrial models, including models based on existing systems, e.g. source code models created with reverse engineering techniques. Since such transformations are frequently integrated to modeling environments, they need to feature both high performance and a concise programming interface to support software engineers. EMF-INCQUERY and VIATRA aim to provide an expressive query language and a carefully designed API for defining model queries and transformations. Refactoring operations are often used in software engineering to improve the readability and maintainability of existing source code without altering the behaviour of the software. The goal of the Java Refactoring Case [10] is to use model transformation tools to refactor Java source code. We decided to solve the extended version of the case. To achieve this, the solution has to tackle the following challenges: The source code is defined in a restricted sub-language of Java 1.4. The EMF metamodel of the PG is provided in the case description. The case considers two basic refactoring operations: Pull Up Method and Create Superclass. The solution is tested in an automated test framework, ARTE (Automated Refactoring Test Environment).
1. Transforming the Java source code to a program graph (PG). 2. Performing the refactoring transformation on the program graph. 3. Synchronising the source code and the program graph.
Solving the case requires the integration of a model transformation tool and a Java source code parser. In this section, we introduce the technologies used in our solution.
EMF-INCQUERY. The objective of the EMF-INCQUERY [
VIATRA. The VIATRA framework supports the development of model transformations with a
particular emphasis on event-driven, reactive transformations [
Java Development Tools. The solution requires a technology to parse the Java code into a program
graph model and serialize the modified graph model back to source code. While the case description
mentions the JaMoPP [
The solution was developed partly in IntelliJ IDEA and partly in the Eclipse IDE. The projects are not tied
to any development environment and can be compiled with the Apache Maven [
The code is written in Java 8 and Xtend [
Figure 1 shows the high-level workflow of the transformation. This consists of five steps: the source code is parsed into an ASG 1 ; a PG is produced 2 ; based on the PG, the possible transformations are JDT 5
JDT ASG
2 source code
PG
transformation rules 3
4 calculated 3 ; if possible, these transformations are executed 4 ; finally, the results of the transformations are serialized 5 . In the following, we discuss these steps in detail. 4.2
1 The solution receives the path to the directory containing the source files. JDT parses these files and returns each file parsed as an AST. These ASTs are interconnected, which means that using JDT’s binding resolution mechanism, the developer can navigate from one AST to another one. 4.3
2 Since JDT does not produce EMF models, the generated ASTs do not support complex queries and traversal operations as the ones provided by EMF and EMF-based query languages (e.g. Eclipse OCL or EMF-INCQUERY). To extract information and to build the PG, our solution applies a visitor resulting a two-pass traversal on the ASG.
1. For each object, the visitor method creates the corresponding object(s) in the PG. Since the order of these visits is non-deterministic, the visitor maintains maps to store the mapping from the objects in the ASG to the objects in the PG. These maps provide trace information between the JDT model and the partially built PG. The visitor also collects the relations between JDT nodes and caches the unique identifiers of each connected node for every relation type. 2. After every compilation unit has been parsed, the previously populated caches are used to create the cross-references between the objects in the PG (e.g. TMember.access). 4.4
The main patterns for both refactoring operations contain a condition that EMF-INCQUERY does not support out of the box, e.g. checking “every child” (a collection of classes) of a certain class. Passing collections as pattern parameter is only possible with a workaround. Also, the INCQUERY Pattern Language does not support universal quantifiers. To overcome these limitations, we extended the program graph metamodel with a trace model shown in Figure 2. The trace model defines traces for method signatures and class lists: • MethodSignatureTrace. Java methods are uniquely identifiable by their signature. The basic PG metamodel contains a TMethodSignature class, which only identifies itself with the name of the method (using a relation to the TMethod object) and the list of its parameter types. To support querying TMethodSignature objects with EMF-INCQUERY, we created a trace reference for each of them identified by their partial3 method signature. For example, a method method() expecting a String and an Integer will have the .method(Ljava/lang/String;I) trace signature. • ClassListTrace. To express the collection of classes, a ClassListTrace object will identify them with their signatures joined by the # character. For example, a list of the ChildClass1 and ChildClass2 classes in the example04 package has the Lexample04/ChildClass1;#Lexample04/ChildClass2; trace signature.
After the PG is produced, it is extended with the trace model. The traces are based on EMF-INCQUERY patterns (Listing 1) and generated with a VIATRA transformation (Listing 4).
The universal quantifier is implemented as a double negation of the existential quantifier using the well-known identity (8 x)P(x) , ¬ (9 x)¬P(x). 4.5
The refactoring operations are implemented as model transformations on the JDT ASG and the PG. Each model transformation is defined in VIATRA: the LHS is defined with an EMF-INCQUERY pattern and the RHS is defined with imperative Xtend code. As VIATRA does not support bidirectional transformations, for each transformation on the PG, we also execute the corresponding actions on the ASG to keep the two graphs in sync. 4.5.1
Pull Up Method After creating the method signature traces, the following preconditions must be satisfied before pulling up a method: • every child class has a method with the given signature, • the parent class does not have a method with this signature, • the transformation will not create an unsatisfiable method or field access.
To decide whether the refactoring can be executed, every hparent class, method signaturei pair satisfying the preconditions is collected by the main pattern (possiblePUM). The LHS is defined with six patterns in total. The execution is controlled by parameterising the main pattern listed in Listing 2. The RHS is defined in Listing 5 using one utility pattern. 4.5.2
Create Superclass To create a new superclass, the parent class and the list of selected classes (connected to a class list trace) have to be passed to the pattern. The transformation can be executed if the following preconditions are satisfied: • the target parent class does not exist, • every selected child class has the same parent.
The LHS is defined in Listing 3 with the possibleCSC pattern using five other patterns. The RHS is defined in Listing 6, also using a utility pattern.
3The complete signature would also contain the defining type (class or interface) signature and the return type signature. 5
The changes in the ASG made by the transformations are propagated to the source code. JDT is capable of incrementally maintaining each source code file (compilation unit) based on the changes in its AST. We executed the tests and used the log files to determine the execution times. The execution times of the test cases are listed in Table 1. The results show that all public and hidden test cases have been executed successfully. Hence, we consider the solution complete and correct. As the test cases only contained small examples, we cannot draw conclusions on the performance of the solution. Still, it is worth noting that all test cases executed in less than half a second.
The implementation of the solution required quite a lot of code. The patterns were formulated in about 150 lines of INCQUERY Pattern Language code. The transformations required 400 lines of Xtend code, while implementing the interface required by ARTE and the visitor for the transformation required more than 800 lines of Java code. However, the source code is well-structured and is easy to comprehend. 6
This paper presented a solution for the Java Refactoring case of the 2015 Transformation Tool Contest. The solution addresses both challenges (bidirectional synchronisation and program transformation) and both refactoring operations (Pull Up Method, Create Superclass) defined in the case. The framework is flexible enough to allow the user to define new refactoring operations, e.g. Extract Class or Pull Up Field. Acknowledgements. The authors would like to thank Ábel Hegedüs, Oszkár Semeráth and Zoltán Ujhelyi for providing valuable insights into EMF-INCQUERY and VIATRA.
1 package hu.bme.mit.ttc.refactoring.patterns 2 3 import "platform:/plugin/TypeGraphBasic/model/TypeGraphBasic.ecore" 4 5 pattern methodSignature(methodSignature) { 6 TMethodSignature(methodSignature); 7 } 8 9 pattern tClassName(tClass, className) { 10 TClass(tClass); 11 TClass.tName(tClass, className); 12 }
Listing 1: Patterns for generating the trace model. 1 package hu.bme.mit.ttc.refactoring.patterns 2 3 import "platform:/plugin/TypeGraphBasic/model/TypeGraphBasic.ecore" 4 import "platform:/plugin/TypeGraphBasic/model/TypeGraphTrace.ecore" 5 6 /* 7 * Main decision pattern. If the preconditions are statisfied (parentClass 8 * and methodSignatureTrace can be bound as parameters), the pattern returns 9 * its parameters, if: 10 * - every child class has a method with the given signature (N = M) 11 * - the parent class does not have it already 12 * - the transformation will not create unavailable access 13 */ 14 pattern possiblePUM(parentClass : TClass, methodSignatureTrace : MethodSignatureTrace) { 15 MethodSignatureTrace.tMethodSignature(methodSignatureTrace, methodSignature); 16 17 // every child class has the method signature 18 N == count find childClassesWithSignature(parentClass, _, methodSignature); 19 M == count find childClasses(parentClass, _); 20 check(N == M && N != 0); 21 22 // parent does not already have this method 23 neg find classWithSignature(parentClass, methodSignature); 24 25 // the fields and methods will still be accessible after PUM 26 neg find childrenClassMethodDefinitionsAccessingSiblingMembers(childClass, methodSignature); 27 } 28 29 pattern childClasses(parentClass : TClass, childClass : TClass) { 30 TClass.childClasses(parentClass, childClass); 31 } 32 33 pattern childClassesWithSignature(parentClass : TClass, clazz : TClass, methodSignature : TMethodSignature) { TClass(parentClass);
TClass.childClasses(parentClass, clazz); 45 } 46 47 pattern methodsAccessingSiblingMembers(methodDefinition : TMethodDefinition) { 48 TMember.access(methodDefinition, accessedMember); 49 TClass.defines(tClass, methodDefinition); 50 TClass.defines(tClass, accessedMember); 51 } or { 52 TClass.defines(tClass, methodDefinition); 53 TMember.access(methodDefinition, accessedMember); 54 TClass.defines(otherClass, accessedMember); 55 TClass.parentClass.childClasses(tClass, otherClass); 56 } 57 58 pattern childrenClassMethodDefinitionsAccessingSiblingMembers(parentClass : TClass, methodSignature :
TMethodSignature) { TClass.childClasses(parentClass, childClass); TClass.defines(childClass, methodDefinition); TMethodSignature.definitions(methodSignature, methodDefinition); find methodsAccessingSiblingMembers(methodDefinition);
Listing 2: Patterns for the Pull Up Method refactoring. 1 package hu.bme.mit.ttc.refactoring.patterns 2 3 import "platform:/plugin/TypeGraphBasic/model/TypeGraphBasic.ecore" 4 import "platform:/plugin/TypeGraphBasic/model/TypeGraphTrace.ecore" 5 6 /* 7 * Main decision pattern. If the preconditions are statisfied (the 8 * targetClass should not exist), the pattern returns its parameters, if: 9 * - every child class has the same parent 10 */ 11 pattern possibleCSC(concatSignature, methodSignature : TMethodSignature) { 12 ClassListTrace.concatSignature(classListTrace, concatSignature); 13 ClassListTrace.tClasses.signature(classListTrace, methodSignature); 14 15 neg find childClassesWithDifferentParents(classListTrace, _, _); 16 } 17 18 pattern childClassesWithDifferentParents(classListTrace : ClassListTrace, classOne : TClass, classTwo :
TClass) { ClassListTrace.tClasses(classListTrace, classOne); ClassListTrace.tClasses(classListTrace, classTwo); find differentParents(classOne, classTwo);
Listing 3: Patterns for the Create Superclass refactoring.
A.2
Transformations 1 package hu.bme.mit.ttc.refactoring.transformations 2 3 import TypeGraphBasic.TClass 4 import TypeGraphTrace.Trace 5 import TypeGraphTrace.TypeGraphTracePackage 6 import hu.bme.mit.ttc.refactoring.patterns.TraceQueries 7 import java.util.ArrayList 8 import java.util.List 9 import org.apache.log4j.Level 10 import org.eclipse.emf.ecore.resource.Resource 11 import org.eclipse.incquery.runtime.api.AdvancedIncQueryEngine 12 import org.eclipse.incquery.runtime.evm.api.RuleEngine 13 import org.eclipse.incquery.runtime.evm.specific.RuleEngines 14 import org.eclipse.incquery.runtime.evm.specific.event.IncQueryEventRealm 15 import org.eclipse.viatra.emf.runtime.modelmanipulation.IModelManipulations 16 import org.eclipse.viatra.emf.runtime.modelmanipulation.SimpleModelManipulations 17 import org.eclipse.viatra.emf.runtime.rules.batch.BatchTransformationRuleFactory extension BatchTransformationRuleFactory factory = new BatchTransformationRuleFactory extension BatchTransformation transformation extension BatchTransformationStatements statements extension IModelManipulations manipulation new(RuleEngine ruleEngine, Resource resource) { engine = (ruleEngine.eventRealm as IncQueryEventRealm).engine as AdvancedIncQueryEngine transformation = BatchTransformation.forEngine(engine) statements = new BatchTransformationStatements(transformation) manipulation = new SimpleModelManipulations(iqEngine) transformation.ruleEngine.logger.level = Level::OFF this.resource = resource this.trace = resource.contents.get(0) as Trace val methodSignatureTraceRule = createRule.precondition(methodSignature).action [ val methodSignatureTrace = typeGraphTraceFactory.createMethodSignatureTrace trace.methodSignatures += methodSignatureTrace
Listing 4: Transformation for generating the trace model. new(AdvancedIncQueryEngine engine, String parentSignature, String methodSignature, BiMap<String, CompilationUnit> compilationUnis) { this(RuleEngines.createIncQueryRuleEngine(engine), parentSignature, methodSignature, compilationUnis) compilationUnits.values.forEach[ try { it.recordModifications } catch (Exception e) {}] val PUMRule = createRule.precondition(possiblePUM).action [
val parentClassKey = parentClass.TName val childClasses = engine.getMatcher(childClasses).getAllValuesOfchildClass(parentClass) var TypeDeclaration astParentClass var List<TypeDeclaration> astChildClasses = new ArrayList var List<MethodDeclaration> astMethodDeclarations astParentClass = findCompilationUnits(parentClassKey, childClasses, astChildClasses) astMethodDeclarations = findMethodDeclarations(astChildClasses, methodSignatureTrace) updateASTAndSerialize(astParentClass, astChildClasses, astMethodDeclarations) ) // --------------- /\ JDT transformation ------------- PG transformation \/ --------------val methodDefinitionsToDelete = engine.getMatcher(methodDefinitionInClassList).getAllMatches( parentClass, methodSignatureTrace.TMethodSignature, null, null val firstMethodDefinition = methodDefinitionsToDelete.get(0) val savedSignature = firstMethodDefinition.methodSignature val savedReturnType = firstMethodDefinition.methodDefinition.returnType val savedAccess = firstMethodDefinition.methodDefinition.access methodDefinitionsToDelete.forEach[ it.clazz.signature.remove(it.methodDefinition.signature); // remove signature from class
EcoreUtil.delete(it.methodDefinition, true) // remove the method definition val tMethodDefinition = tgPackage.typeGraphBasicFactory.createTMethodDefinition tMethodDefinition.returnType = savedReturnType tMethodDefinition.signature = savedSignature tMethodDefinition.access += savedAccess parentClass.defines += tMethodDefinition println(tMethodDefinition) ].build protected def List<MethodDeclaration> findMethodDeclarations(List<TypeDeclaration> astChildClasses, MethodSignatureTrace methodSignatureTrace) { val List<MethodDeclaration> astMethodDeclarations = new ArrayList for (childCU : astChildClasses) { val methodSignature = childCU.resolveBinding.key + methodSignatureTrace.signatureString; val types = (childCU.root as CompilationUnit).getStructuralProperty(CompilationUnit.TYPES_PROPERTY) as List<TypeDeclaration> for (type : types) { for (method : (type as TypeDeclaration).methods) { if (method.resolveBinding.key.startsWith(methodSignature)) {
astMethodDeclarations += method
return result for (methodDeclaration : astMethodDeclarations) {
methodDeclaration.delete }
} def fire() { fireAllCurrent(
PUMRule, "parentClass.tName" -> parentSignature, "MethodSignatureTrace.signatureString" -> methodSignature ) def canExecutePUM() { // get the method signature by string, then get one arbitrary match with it bound val parentTClass = engine.getMatcher(classWithName).getOneArbitraryMatch(null, parentSignature) val trace = engine.getMatcher(methodWithSignature).getOneArbitraryMatch(null, methodSignature) return parentTClass != null && trace != null && engine.getMatcher(possiblePUM).getOneArbitraryMatch(parentTClass.TClass, trace.trace) != null
Listing 5: Pull Up Method transformation. 1 package hu.bme.mit.ttc.refactoring.transformations 2 3 import TypeGraphBasic.TClass 4 import TypeGraphBasic.TMethodSignature 5 import TypeGraphBasic.TPackage 6 import TypeGraphBasic.TypeGraph 7 import TypeGraphBasic.TypeGraphBasicPackage 8 import com.google.common.collect.BiMap 52 53 54 55 new(AdvancedIncQueryEngine engine, List<String> childClassSignatures, String targetPackage, String targetName, BiMap<String, CompilationUnit> compilationUnis) { this(RuleEngines.createIncQueryRuleEngine(engine), childClassSignatures, targetPackage, targetName, compilationUnis) this.concatSignature = childClassSignatures.join("#") this.targetPackage = targetPackage this.targetName = targetName this.compilationUnits = compilationUnits compilationUnits.values.forEach[ try { it.recordModifications } catch (Exception e) {}] } val CSCRule = createRule.precondition(possibleCSC).action [ // --------------- /\ JDT transformation ------------- PG transformation \/ --------------val tClasses = engine.getMatcher(classesOfClassListTrace).getAllValuesOftClass(concatSignature) val List<TypeDeclaration> astChildClasses = findCompilationUnits(tClasses) val firstChild = astChildClasses.get(0) if (targetCU == null) { targetCU = createTargetClass(firstChild, firstChild.superclassType) var currentTPackageMatch = engine.getMatcher(packageWithName).getOneArbitraryMatch(null, current) if (currentTPackageMatch != null) {
previousTPackage = currentTPackageMatch.TPackage } else { val TPackage currentTPackage = tgPackage.typeGraphBasicFactory.createTPackage currentTPackage.TName = current if (previousTPackage != null) {
currentTPackage.parent = previousTPackage } else {
typeGraph.packages += currentTPackage val targetSignature = "L" + targetPackage.replace(’.’, ’/’) + "/" + targetName + ";"; val typeGraph = engine.getMatcher(typeGraphs).oneArbitraryMatch.typeGraph val targetTClassMatch = engine.getMatcher(classWithName).getOneArbitraryMatch(null, targetSignature) var TClass targetTClass if (targetTClassMatch == null) { targetTClass = tgPackage.typeGraphBasicFactory.createTClass targetTClass.TName = targetSignature targetTClass.package = createPackagesFor(typeGraph, targetPackage) targetTClass.parentClass = oldParentTClass } else {
targetTClass = targetTClassMatch.TClass (tClasses.get(0).eContainer as TypeGraph).classes += targetTClass targetTClass.childClasses += tClasses ].build protected def createPackagesFor(TypeGraph typeGraph, String pkg) { val String[] split = pkg.split("\\."); var previous = ""; var TPackage previousTPackage for (var i = 0; i < split.length; i++) { var String current = previous if (i != 0) {
current += "." } current += split.get(i); previousTPackage return astChildClasses
previousTPackage = currentTPackage protected def List<TypeDeclaration> findCompilationUnits(Set<TClass> childClasses) { val List<TypeDeclaration> astChildClasses = new ArrayList for (cu : compilationUnits.values) { for (child : childClasses) { if (cu.findDeclaringNode(child.TName) != null) {
astChildClasses += cu.findDeclaringNode(child.TName) as TypeDeclaration protected def List<MethodDeclaration> findMethodDeclarations(List<TypeDeclaration> astChildClasses, TMethodSignature tMethodSignature) { val List<MethodDeclaration> astMethodDeclarations = new ArrayList val methodSignatureTrace = engine.getMatcher(methodSignatureAndTrace).getAllValuesOftrace( tMethodSignature).get(0) for (childCU : astChildClasses) { val methodSignature = childCU.resolveBinding.key + methodSignatureTrace.signatureString; val types = (childCU.root as CompilationUnit).getStructuralProperty(CompilationUnit.TYPES_PROPERTY) as List<TypeDeclaration> for (type : types) { for (method : (type as TypeDeclaration).methods) { // match if (method.resolveBinding.key.startsWith(methodSignature)) {
astMethodDeclarations += method } } }
} return astMethodDeclarations protected def CompilationUnit createTargetClass(TypeDeclaration childClass, Type superClassType) { val ast = childClass.AST val compilationUnit = ast.newCompilationUnit if (targetPackage != null) { val packageDeclaration = ast.newPackageDeclaration var Name packageName for (part : targetPackage.split("\\.")) { if (packageName == null) {
packageName = ast.newSimpleName(part) } else {
packageName = ast.newQualifiedName(packageName, ast.newSimpleName(part)) }
} } packageDeclaration.name = packageName compilationUnit.package = packageDeclaration compilationUnit.imports += ASTNode.copySubtrees(ast, (childClass.root as CompilationUnit).imports) val typeDeclaration = ast.newTypeDeclaration typeDeclaration.modifiers().add(ast.newModifier(ModifierKeyword.PUBLIC_KEYWORD)) typeDeclaration.name = ast.newSimpleName(targetName) if (superClassType != null) {
typeDeclaration.superclassType = ASTNode.copySubtree(ast, superClassType) as Type compilationUnit.types += typeDeclaration compilationUnit def serializeCUs() { val targetDir = StringUtils.substringBefore( compilationUnits.keySet.get(0), "/src/" ) + "/src/" + targetPackage.replace(’.’, ’/’) val targetPath = targetDir + "/" + targetName + ".java" val targetFile = new File(targetPath) targetFile.parentFile.mkdirs compilationUnits.put(targetPath, targetCU) protected def readFileToString(String path) {
new Scanner(new File(path)).useDelimiter("\\A").next protected def insertMethodDeclaration(MethodDeclaration declaration) { val typeDeclaration = targetCU.types.get(0) as TypeDeclaration typeDeclaration.bodyDeclarations.add(ASTNode.copySubtree(targetCU.AST, declaration) as
MethodDeclaration) protected def setParentClass(List<TypeDeclaration> typeDeclarations) {
val ast = targetCU.AST protected def removeChildMethodDeclarations(List<MethodDeclaration> methodDeclarations) { for (declaration : methodDeclarations) {
declaration.delete var Type fqn if (targetPackage != null) { for (part : targetPackage.split("\\.")) { if (fqn == null) {
fqn = ast.newSimpleType(ast.newSimpleName(part)) } else {
fqn = ast.newQualifiedType(fqn, ast.newSimpleName(part)) }
} fqn = ast.newQualifiedType(fqn, ast.newSimpleName(targetName)) } else {
fqn = ast.newSimpleType(ast.newSimpleName(targetName)) for (declaration : typeDeclarations) {
declaration.superclassType = ASTNode.copySubtree(declaration.AST, fqn) as Type def canExecuteCSC() { val targetSignature = "L" + targetPackage.replace(’.’, ’/’) + "/" + targetName + ";" val targetTClass = engine.getMatcher(classWithName).getOneArbitraryMatch(null, targetSignature)
Listing 6: Create Superclass transformation.
Trace
TypeGraph [0..1] programGraph tName : EString
[0..*] classes MethodSignatureTrace signatureString : EString [0..*] methodSignatures [0..1] tMethodSignature
TMethodSignature [0..*] classLists
ClassListTrace concatSignature : EString [0..*] tClasses [0..*] childClasses [0..*] paramList
TClass tName : EString [0..1] parentClass The benchmarks were conducted on a 64-bit Arch Linux virtual machine running in SHARE. The machine utilized a single core of a 2.00 GHz Xeon E5-2650 CPU and 1 GB of RAM. We used OpenJDK 8 to run the ARTE framework and the solution. 0.463 0.013 0.333 0.094 0.189 0.093 0.063 0.013 0.114 0.082 0.081 0.007 0.058 0.189 0.179