EMF-Syncer solution to TTC’20 round-trip migration case Artur Boronat1 1 School of Computing and Mathematical Sciences, University of Leicester, University Rd, LE1 7RH, Leicester, United Kingdom Abstract In this paper, we present a solution to the TTC’20 offline case Round-trip migration of object-oriented data model instances [1]. This case involves the application of maintenance tasks to web APIs that are associated with domain models so that updates are backward compatible. The solution presented in this article features the EMF-Syncer [2], a synchronization tool for bridging MDE-agnostic programs and MDE-aware programs, which may have some similarities in their object-oriented data models at run time. EMF-Syncer provides a generic synchronization strategy that exploits such similarities automatically and is, therefore, a suitable candidate for solving the proposed problems, since each problem relies on a small change and there is a large overlap between two versions of the same data model. In the paper, we used the case benchmark framework to justify that our solution exhibits a good balance between specification conciseness and performance. Keywords Model syncing, model-driven engineering, round-trip engineering 1. Introduction assumes that MDE-agnostic programs are implemented in a JVM language and that MDE-aware programs repre- The Round-trip migration of object-oriented data model sent their data model using EMF-generated code. instances case [1] for the TTC’20 exposes an evolution Given a source data model (represented in a package problem in the context of web API development, where of classes in the MDE-agnostic program), a target data different APIs work on top of a common data model. To model (represented as an Ecore model in an EMF-based accommodate new or changing requirements in the sys- program), and a collection of objects representing the tem, API designers need to ensure backward-compatible source program state, EMF-Syncer translates the MDE- changes in the underlying data model. Assuming an agile agnostic objects as an EMF model instance in the target software development environment where data models EMF-based program, synchronizing the state of both pro- may evolve rapidly, the case proposes the use of round- grams at run time. EMF-Syncer automatically infers trip migration services, relying on Model-Driven Engi- structural similarities between object-oriented data mod- neering (MDE) technology, for enabling the co-existence els by mapping object structural features by name, when of different versions of the data model at run time. In found, translating both attribute and reference values. particular, the Eclipse Modeling Framework (EMF) [3] is This translation can be performed using a push-based used to encode data models with the EMF meta-modeling model, where the entire source program state is migrated, language, namely Ecore, so that tools built atop EMF can or using a pull-based model, where only those feature be used to implement data migration services. values accessed in the target program are migrated. Once The solution presented in this article features the EMF- synchronization is established, changes that have been Syncer [2], a synchronization tool for bridging MDE- applied to target EMF model instances can be incremen- agnostic programs and MDE-aware programs, which tally back-propagated to the source MDE-agnostic coun- may have some similarities in their object-oriented data terpart. Incrementality of back-migration entails that models1 , at run time. EMF-Syncer provides a generic only changes in target model instances are propagated synchronization strategy that exploits such similarities back and merged within the source instance. automatically and is, therefore, a suitable candidate for The aforementioned generic mapping strategy that is solving the proposed problems, since each problem relies built in EMF-Syncer can be customized in order to allow on a small change and there is a large overlap between for more complex data transformations between the data two versions of the same data model. The EMF-Syncer models involved. A domain-specific mapping strategy is TTC’20: Transformation Tool Contest, Part of the Software declared with a mapping specification that maps a source Technologies: Applications and Foundations (STAF) federated feature type to a target feature type, possibly including conferences, Eds. A. Boronat, A. García-Domínguez, G. Hinkel, and F. feature value transformations, either from source to tar- Křikava, 17 July 2020, Bergen, Norway (online). get, or from target to source, or both. Two main custom " artur.boronat@leicester.ac.uk (A. Boronat) mapping strategies can be declared: ~ https://arturboronat.github.io/ (A. Boronat)  0000-0003-2024-1736 (A. Boronat) • Renaming of feature types: such renamings may © 2021 Copyright for this paper by its authors. Use permitted under Creative CEUR Workshop http://ceur-ws.org ISSN 1613-0073 Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) affect the name of the class, where the feature is declared, and the name of the feature. This Proceedings 1 We refer to these data models as domain models in [2]. mapping strategy allows for transformations that constructor. All of the solutions use the push-based modify the data model either syntactically, when model in syncForward, which is specified by indicating either a class or a feature is renamed in the tar- syncer.syncingStrategy = SyncingStrategy.EAGER, get model, or structurally, when a feature type is forcing the migration of the entire source instance. moved to an unrelated object type. In this case, In the method migrate, the statement feature values are transformed using the generic syncer.syncForward(person_v1) migrates synchronization strategy. the source Person instance obtaining the target • Transformation of feature values, with or without instance with the age feature, which is not initial- renaming of feature types: feature value trans- ized until the statement person_v2.age = -1 is formations are expressed using Xtend lambda ex- evaluated. In the method migrateBack, the state- pressions defined for a contextual object type, ment syncer.syncBack(person_v2) migrates the from the source data model, and result in a sin- instance back to the source Java program. gle target feature value. This mapping strategy The method modify is used to implement changes to allows for semantic transformations, where the target model instances that need to be propagated back. feature value can be computed using navigation This logic is hardcoded in test cases in the benchmark expressions from the contextual type. A feature framework. EMF-Syncer needs to track which changes value transformation is applied in a single direc- are performed in the target model instance in order to tion, either from source to target or from target enable incremental back migration. Therefore, this logic to source. A programmer can opt to provide fea- has been moved from test cases to task classes. ture value transformations in both directions (as 1 class Task_1_M1_M2_M1 extends AbstractTask { in Task_2 in §2.2), only in one (as in Task_3 in 2 3 val Syncer syncer §2.3), or none at all. The last case corresponds to 4 new (EPackage model1, EPackage model2) { a simple renaming of feature types as explained 5 6 super(model1, model2); syncer = new Syncer(#[’scenario1_v1’], model2) above. 7 syncer.syncingStrategy = SyncingStrategy.EAGER 8 } For the TTC 2020 case, we are relying on the obser- 9 10 override migrate(EObject instance) { vation that an EMF-based program can be regarded as a 11 val person_v1 = instance as scenario1_v1.Person plain Java program. Hence, the given Java program will 12 val person_v2 = syncer.syncForward(person_v1) as scenario1_v2 .Person refer to the source data model and the EMF-based pro- 13 person_v2.age = -1 gram will refer to the target data model. In the following 14 15 } return person_v2; sections, we discuss the solution to the case in §2, and 16 the evaluation of the solution in §3 using the evaluation 17 18 override migrateBack(EObject instance) { val person_v2 = instance as scenario1_v2.Person criteria proposed in the case, wrapping up with some 19 return syncer.syncBack(person_v2) as scenario1_v1.Person conclusions in §4. 20 21 } 22 override modify(EObject instance) { } 23 } 2. Solution The solutions to the different tasks of the It is important to note that the bidirectional transfor- case are explained below and are available at mation between both data models is completely inferred https://github.com/emf-syncer/ttc20-roundtrip. The by the EMF-Syncer automatically in this task. In the solution has been implemented using the language back migration, as the age feature does not exist in the Xtend [4]. source program, it is not propagated. The solution to the symmetric problem Task1_M2_M1_M2 is achieved by inverting the di- 2.1. Task 1: create/delete field rection in which the EMF-Syncer is applied to the data In this task, a new feature age is added to the class models, as seen in the listing below. In this case, the data Person in the modified data model 𝑀2 and when a model M2 is regarded as the Java program and the data Person instance is migrated, its feature value is set to model M1 as the EMF program. In this case, the value -1. When the instance is migrated back, this information of the age feature of a Person class is preserved on is lost. the back migration because this feature does not exist The solution Task1_M1_M2_M1, shown in the listing in M1 and, therefore, no changes can be applied to the below, illustrates the pattern used in all of the solutions. feature age. In the following subsections, solutions to The EMF-Syncer is used by instantiating the class symmetric problems are achieved by flipping, as for this EMFSyncer between a source Java package name list task, the direction in which the EMF-Syncer is applied #[’scenario1_v1’] and a target Ecore model in the and are not included in the paper. 1 class Task_1_M2_M1_M2 extends AbstractTask { 17 Integer.valueOf(Calendar.getInstance().get(Calendar.YEAR) - 2 val EMFSyncer syncer person_v2.ybirth) as Object 3 18 ] 4 new (EPackage model1, EPackage model2) { 19 ) 5 super(model1, model2); 20 6 syncer = new Syncer(#[’scenario1_v2’], model1) 21 new (EPackage model1, EPackage model2) { 7 syncer.syncingStrategy = SyncingStrategy.EAGER 22 super(model1, model2); 8 } 23 syncer = new EMFSyncer(#[’scenario2_v1’], model2, 9 newArrayList(mapping)) 10 override migrate(EObject instance) { 24 syncer.syncingStrategy = SyncingStrategy.EAGER 11 val person_v2 = instance as scenario1_v2.Person 25 } 12 return syncer.syncForward(person_v2) as scenario1_v1.Person 26 13 } 27 override migrate(EObject instance) { 14 28 val person_v1 = instance as Person 15 override migrateBack(EObject instance) { 29 return syncer.syncForward(person_v1) as scenario2_v2.Person 16 val person_v1 = instance as scenario1_v1.Person 30 } 17 return syncer.syncBack(person_v1) as scenario1_v2.Person 31 18 } 32 override migrateBack(EObject instance) { 19 33 val person_v2 = instance as scenario2_v2.Person 20 override modify(EObject instance) { } 34 return syncer.syncBack(person_v2) as Person 21 } 35 } 36 37 override modify(EObject instance) { } 38 } 2.2. Task 2: rename field In this task, the feature age of the class Person is re- 2.3. Task 3: declare field named to ybirth, and its semantics is changed by repre- senting the age and the year of birth, respectively. This optional/mandatory renaming involves domain-specific semantics that is not In this task, the multiplicity of a feature type is modified present in the Ecore model, requiring a mapping spec- so that the feature is mandatory in one data model and ification so that the EMF-Syncer can perform the data optional in the modified version. This task exposes a transformation correctly. difference between a MDE-agnostic program, where it A mapping specification consists of a collection of is not possible to know whether a feature is optional in mappings between feature types by name, ’Person’, plain Java, and a MDE-aware one, where this informa- ’age’, /* <–> */ ’Person’, ’ybirth’, and by tion is encoded in the Ecore model. As the EMF-Syncer adding optional feature value transformations as lambda treats the source program as a plain Java program, it disre- expressions. For example, in the solution, the lambda gards the information contained in the Ecore model, and expression the logic to transform null values in to empty Strings 1 val person_v1 = it as scenario2_v1.Person needs to be provided in a custom mapping. The ex- 2 Integer.valueOf(Calendar.getInstance().get(Calendar.YEAR) - person_v1.age) as Object pression person_v2.name ?: "" returns an empty String when the name of the Person instance is null, which is checked using the Elvis operator ?:. The rest of the data transformation is fully inferred by the EMF- gets a Person instance from the source data model and Syncer . returns the ybirth value. When the EMF-Syncer ap- plies the transformation syncForward, the feature value 1 class Task_3_M1_M2_M1 extends AbstractTask { 2 val EMFSyncer syncer for Person::ybirth will be obtained by applying this 3 expression. The rest of the solution follows the same 4 // custom mapping strategy: M1 <--> M2 5 val public static mapping = new EMFSyncerMapping( structure as the solution explained in §2.1. 6 ’Person’, ’name’, /* <--> */ ’Person’, ’name’, 7 null, 1 class Task_2_M1_M2_M1 extends AbstractTask { 8 // target to source feature value transformation 2 val EMFSyncer syncer 9 [ 3 10 val person_v2 = it as scenario3_v2.Person 4 // custom mapping strategy: M1 <--> M2 11 person_v2.name ?: "" 5 val public static mapping = new EMFSyncerMapping( 12 ] 6 ’Person’, ’age’, /* <--> */ ’Person’, ’ybirth’, 13 ) 7 // source to target feature value transformation 14 8 [ 15 new (EPackage model1, EPackage model2) { 9 val person_v1 = it as scenario2_v1.Person 16 super(model1, model2); 10 Integer.valueOf(Calendar.getInstance().get(Calendar.YEAR) - 17 syncer = new EMFSyncer(#[’scenario3_v1’], model2, person_v1.age) as Object 18 newArrayList(mapping)) 11 ], 19 syncer.syncingStrategy = SyncingStrategy.EAGER 12 20 } 13 21 14 // target to source feature value transformation 22 override migrate(EObject instance) { 15 [ 23 val person_v1 = instance as scenario3_v1.Person 16 val person_v2 = it as scenario2_v2.Person 24 return syncer.syncForward(person_v1) as Person 25 } framework has been adapted in order to work with code 26 27 override migrateBack(EObject instance) { generated from Ecore models via EMF, which is required 28 val person_v2 = instance as Person by EMF-Syncer . The main properties (namely, name, 29 30 } syncer.syncBack(person_v2) as scenario3_v1.Person nsPrefix and nsUri) of the EPackage in each Ecore 31 model were updated in order to generate disjoint names- 32 33 override modify(EObject instance) { val person_v2 = instance as Person paces. In test cases, loading resources for each Ecore 34 syncer.track[ person_v2.name = null ] model was modified in order to use the corresponding 35 36 } return person_v2 generated factory for each model instance. Model in- 37 } stances, used as input/output data, were modified to re- fer to the corresponding nsUri. In some cases, the test case also performs a change in the target instance, as in This solution also contains an example of how to track Task_3, that needs to be propagated back to the source changes performed in the target instance, in the modify instance. Such modifications have been encoded in the method, that need to be migrated back. This change is task itself, in the method modify, as the EMF-Syncer encoded in the corresponding test case in the original needs to track those changes. Such changes refer to im- test framework. plementation details and do not alter the correctness properties checked2 by the test suite. 2.4. Task 4: multiple edits Solutions for all of the tasks have been implemented and all of them pass the correctness check. This task combines Task_1 and Task_2 with the aim of analysing reuse mechanisms that can be employed. 3.2. Comprehensibility The solution below reuses the transformation logic of Task_1, as it is handled by the EMF-Syncer automati- Task solutions have been implemented using Xtend. cally, and it reuses the mapping specification of Task_2, However, the EMF-Syncer can be used from Java pro- which is defined as a static field. The rest of the solution is grams as well. Given that the transformation in most as in §2.1, after renaming the corresponding namespaces. of the solutions is inferred automatically and that so- 1 class Task_4_M1_M2_M1 extends AbstractTask { lutions use generated code, instead of using the EMF 2 val EMFSyncer syncer reflection API for accessing/mutating values, we argue 3 4 new (EPackage model1, EPackage model2) { that solutions are likely to be more comprehensible than 5 super(model1, model2); the reference ones. 6 syncer = new EMFSyncer(#[’scenario4_v1’], model2, 7 newArrayList(Task_2_M1_M2_M1.mapping)) When custom mappings are required, e.g. in Task_2 8 syncer.syncingStrategy = SyncingStrategy.EAGER and in Task_4, these are defined by instantiating the 9 } 10 class EMFSyncerMapping, where feature value transfor- 11 override migrate(EObject instance) { mations are defined as Xtend lambda expressions. Such 12 val container_v1 = instance as scenario4_v1.Container 13 return syncer.syncForward(container_v1) as scenario4_v2. mapping expressions could have been defined similarly Container in Java as well. 14 } 15 16 override migrateBack(EObject instance) { 17 val container_v2 = instance as scenario4_v2.Container 3.3. Bidirectionality 18 syncer.syncBack(container_v2) as scenario4_v1.Container 19 } In the solution for most of the tasks, the EMF-Syncer 20 21 override modify(EObject instance) { } infers both transformations to be applied, syncForward 22 } and syncBack, automatically. In such cases, the pro- grammer does not need to provide a transformation spec- ification and a bidirectional data transformation is being used internally so that Java instances can be migrated to an EMF program, and back again, at run time. A cus- 3. Evaluation tom mapping specification containing only feature type renamings is fully bidirectional. Feature value transfor- In this section, we provide an evaluation of the solution mation expressions are unidirectional though. according to the evaluation criteria proposed in [1]. Therefore, the EMF-Syncer provides support for bidi- rectional transformations by default for all of the solu- 3.1. Expressiveness tions and accommodates special cases with unidirectional Two test cases are provided for checking the correct- ness of each task. A test case provides the input for a 2 The test case task_3_M1_M2_M1_b had to be updated in or- round-trip migration and the expected output. The test der to check that the name was migrated back correctly. Expressiveness Comprehensibility Bidirectionality Re-usability Task 1: "Create/Delete Field" Task_1_M1_M2_M1 2 (2) 2 (2) 1 (1) n.a. Task_1_M2_M1_M2 2 (2) 2 (2) 1 (1) n.a. Task 2: "Create/Delete Field" Task_2_M1_M2_M1 2 (2) 2 (2) 1 (1) n.a. Task_2_M2_M1_M2 2 (2) 2 (2) 1 (1) n.a. Task 3: "Create/Delete Field" Task_3_M1_M2_M1 2 (2) 2 (2) 1 (1) n.a. Task_3_M2_M1_M2 2 (2) 2 (2) 1 (1) n.a. Task 4: "Create/Delete Field" Task_4_M1_M2_M1 2 (2) 2 (2) 1 (1) 4 (4) Task_4_M2_M1_M2 2 (2) 2 (2) 1 (1) 4 (4) ∑︀ ∑︀ ∑︀ ∑︀ : 16 (16) : 16 (16) : 8 (8) : 8 (8) Table 1 Summary of evaluation results. feature value transformations, which correspond to a 35000 fraction of the data migration to be performed. 30000 25000 20000 3.4. Re-usability 15000 Re-usability is internalized in the EMF-Syncer by infer- 10000 ring transformations automatically. That is, when a user 5000 does not need to provide a mapping specification, the 0 transformation logic in the EMF-Syncer is reused for 10000 90000 170000 250000 330000 410000 490000 570000 650000 730000 810000 890000 970000 1050000 1130000 1210000 1290000 1370000 1450000 1530000 1610000 1690000 1770000 1850000 1930000 any data model change. For example, reuse of the logic transformation in Task_1 falls under this category. reference (ms) EMFSyncer (ms) On the other hand, as EMF-Syncer is a JVM library, a programmer can rely on reuse mechanisms provided by the JVM language of choice. For example, reuse of the Figure 1: Performance results (ms.) along iterations transformation logic in Task_2 falls under this category. The mapping specification defined for Task_2 is reused by calling a static field and by using a common interface 4. Conclusions for the class Person, which has been implemented using inheritance in Task_4. Following from the justification of the evaluation criteria in the section above, the results are summarized in Ta- 3.5. Performance ble 1, where the score 𝑋 out of 𝑌 is expressed as 𝑋(𝑌 ). Overall, this solution shows that the EMF-Syncer helps We have run the performance tests both for the refer- in achieving a competitive trade-off between data migra- ence solution and for the EMF-Syncer solution on a tion specification and performance. On the one hand, the MacBookPro11,5 Core i7 2.5 GHz, with four cores and automatic inference of data transformations between dif- 16 GB of RAM. The runtime results (in ms.) obtained ferent object-oriented data models reduces the need for are displayed in Fig. 1. The EMF-Syncer solution ex- specifying data transformations. When these have to be hibits a linear growth with respect to the number of specified, a programmer can rely on their programming iterations, as instructed in the case benchmark. However, skills, using a JVM programming language, for reusing the EMF-Syncer solution is more efficient than the refer- transformation logic. On the other hand, EMF-Syncer ence one thanks to its support for incremental migration can be used as an efficient data migration service at run of changes. While the reference solution takes about 31 s. time. For example, the EMF-Syncer solution is faster for 2 million iterations, the EMF-Syncer solution takes than the reference solution developed in raw Java. EMF- about 9 s, with an improvement factor of 70%. The in- Syncer can be used for building more scalable solutions, cremental propagation relies on a traceability model that involving very large models, by using the pull-based caches relevant data at run time. An analysis of memory model for propagating changes in syncForward only consumption is left for future work as it was not part of when they are required in the target program. the case benchmark. References [1] L. Beurer-Kellner, J. von Pilgrim, T. Kehrer, Round- Trip Migration of Object-Oriented Data Model In- stances, in: Proceedings of the 13th Transformation Tool Contest, a part of the Software Technologies: Applications and Foundations (STAF 2018) federa- tion of conferences, CEUR Workshop Proceedings, CEUR-WS.org, 2020. [2] A. Boronat, Code-first model-driven engineering: On the agile adoption of mde tooling, in: Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering (ASE 2019), San Diego, CA, November 11-15, ACM, 2019. [3] D. Steinberg, F. Budinsky, M. Paternostro, E. Merks, EMF: Eclipse Modeling Framework 2.0, 2nd ed., Addison-Wesley Professional, 2009. [4] T. E. Foundation, Xtend (official web page), 2018. http://www.eclipse.org/xtend/.