The Fulib solution to the TTC 2020 migration case Sebastian Copei1 , Albert Zuendorf1 1 Kassel University, Germany 1. Introduction M1Editor M2Editor At Kassel University we are working on a solution for M1 event store M2 event store bidirectional transformations based on event sourcing for about a year, now. It turned out, that the TTC 2020 - c1 - c2 - c1 - c2 migration case [1] is a special case of a bidirectional trans- M1 - ... - ... M2 formation and that our approach provides a reasonable objects objects solution for it. Figure 1: Design 2. Design The idea and design for our solution stems from Domain M1Editor M2Editor Driven Design [2] and Event Sourcing [3]. Basically, we p1: M1Person h3: HavePerson h5: HavePerson p7: M2Person use two editors M1Editor and M2Editor, one for each id = “obj0” id = “obj0” id = “obj0” id = “obj0” model of the case study, cf. Figure 1. Each editor holds the name = “Alice” name = “Alice” name = “Alice” name = “Alice” age = 25 age = “25” age = “25” ybirth= 1995 current object model (based on the corresponding ecore model). In addition, each editor provides editing com- d2: M1Dog h4: HaveDog h6: HaveDog d8: M2Dog mands following the command design pattern of [4]. All id = “obj1” id = “obj1” name = “Bob” id = “obj1” name = “Bob” id = “obj1” operations on the object model are encapsulated within name = “Bob” age = 2 age = “2” ownerId = “obj0” age = “2” ownerId = “obj0” name = “Bob” editor commands. Each editor keeps track of all executed commands (and the used command parameters) within its event store, cf. Figure 1. To enable collaboration Figure 2: Objects of M1Editor and M2Editor, both editors provide the same set of commands: a HavePerson command with param- eters id, name, and age and a HaveDog command with In [5] we have developed theoretical foundations for parameters id, ownerId, name, and age, cf. Figure 3 this command (or event) sharing between multiple edi- and Figure 2. tors: basically, we require that multiple commands that each editor implements the command execution differ- address the same object (have the same id) overwrite ently according to its specific ecore model. As an example, each other, i.e.: if c1 and c2 are commands with the same Listing 1 shows the implementation of the HavePerson id than applying c1 and then c2 is similar to applying command within M2Editor. Line 22 of Listing 1 shows only c2. In addition, commands that work on different how the age parameter of the HavePerson command is objects may be executed in any order (are commutative), turned into a ybirth value for model M2. i.e. if c3 and c4 are commands with different ids, applying Generally, we consider M1Editor and M2Editor as inde- first c3 and then c4 must result in the same object model pendent programs that may run on different computers, as applying c4 first and c3 second. While these are pretty concurrently. Therefore each editor is able to serialize its strong conditions, it turned out to be easy to implement event store (in yaml format) and to send its commands the commands according to these rules. For the TTC2020 to the other editor. Correspondingly, the receiving editor migration case we will discuss this in Section 3. is able to deserialize and execute the commands, too. Provided with overwriting and commutative com- mands, our editors are able to merge commands executed TTC’20: Transformation Tool Contest, Part of the Software by themselves or received from another editor: if a new Technologies: Applications and Foundations (STAF) federated command arrives that uses an id that is already used by conferences, Eds. A. Boronat, A. García-Domínguez, G. Hinkel, and F. some old command, the new command is executed and Křikava, 17 July 2020, Bergen, Norway (online). email: sco@uni-kassel.de (S. Copei); zuendorf@uni-kassel.de then the old command is replaced by the new command (A. Zuendorf) in the event store. If a new command arrives that uses a © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). new id, the new command is executed and added to the CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) 1. migrate M1 to M2 Command a) load M1 objects b) parse M1 objects into M1 event store id: String c) send M1 commands to M2Editor d) execute commands by M2Editor creating M2 objects and filling M2 event store run () 2. modify M2 objects to M2’ 3. migrate M2’ back to M1’ a) parse (modified) M2’ objects and detect new commands and merge the new commands into the old M2 event store b) send the updated M2 commands to the HavePerson HaveDog M1Editor c) execute (modified and new) commands on M1 name: String name: String objects and update M1 event store age: String age: String ownerId: String Figure 4: The Fulib Migration Approach run () run () object but we shall lookup the already existing object and just adjust its attributes. In Line 5 of Listing 1 our HavePerson command achieves this behavior by using Figure 3: Command Classes the getOrCreatePerson method of its editor. Our edi- tors have hash tables for each model class (i.e. for Person and Dog) and the getOrCreatePerson method looks event store. The event store is treated as a set, i.e. the up this hash table. If the hash table already has a Per- order of the commands (and the order of the command son with the given id, this Person is returned. Otherwise, execution) does not matter due to our commutativity getOrCreatePerson creates a Person object, initializes condition. its id, adds it to the hash table, and then returns the new Overall this editor design enables us to implement the Person. Thus, you may call getOrCreatePerson with TTC2020 migration case as outlined in Figure 4. The mi- a certain id as often as you like, it will always return the grate step is invoked by the test or benchmark Tasks. The same Person object. loading of objects is done by loading the corresponding This getOrCreate mechanism also helps us to xml files via EMF mechanisms. The parsing steps are achieve commutativity for our commands. Commu- discussed in Section 4. The sending is done via our yaml tativity for commands requires e.g. that you can ex- serialization and deserialization. The execution of com- ecute a HaveDog command before you execute the mands is discussed in Section 3. The modification step HavePerson command for the dog owner. To allow this, is again done by the test or benchmark tasks. Similarly, our HaveDog commands uses getOrCreatePerson the test and benchmark tasks invoke the migrate back with the ownerId to retrieve the corresponding Person step. The migrate back step uses parsing and sending object. If the owner already exists, we just use it, other- and execution similar to the migrate forward step. How- wise, the owner object is created on the fly. In the latter ever, some details dealing with missing information are case, a subsequent execution of the HavePerson com- discussed in Section 4. mand with the same id will retrieve the already existing Person object and then fill the Person’s name and age (or ybirth). 3. Commands Once the targeted Person object has been retrieved, Line 6 to Line 22 of Listing 1 fill the attributes of that Our approach relies on commands that are shared be- Person. In this solution, we use EMF dynamic editing tween the two editors. In the simple TTC20 Migration features to set the name and age or ybirth attributes. case we need only two commands, cf. Figure 3 and Fig- The different migration tasks use different ecore models ure 2. To facilitate the merging of concurrent edits, we with different properties. We store the current ecore require that commands are overwriting and commuta- model within the editors and thus the commands just tive. Thus, if we e.g. execute a HavePerson command query the current ecore model e.g. whether the current with a certain id on an empty model, the first time we Person has either an age attribute or an ybirth attribute. shall create a Person object and fill its attributes. How- Thus one implementation of our M2.HavePerson com- ever the second time, we shall not create a second Person 1 public class HavePerson extends Command { 1 public class M2Editor { 2 @Override 2 ... 3 public Object run(M2Editor editor) { 3 private void parsePerson(EObject instance) { 4 EObject person = 4 EClass eClass = instance.eClass(); 5 editor.getOrCreatePerson(getId()); 5 String name = 6 EClass personClass = 6 (String) instance 7 (EClass) editor 7 .eGet(eClass 8 .getEmfModel() 8 .getEStructuralFeature("name")); 9 .getEClassifier("Person"); 9 int age = -1; 10 person.eSet( 10 EStructuralFeature ageFeature = 11 personClass.getEStructuralFeature("name"), 11 eClass.getEStructuralFeature("age"); 12 name); 12 if (ageFeature != null) { 13 EStructuralFeature ageFeature = 13 age = (Integer) instance.eGet(ageFeature); 14 personClass.getEStructuralFeature("age"); 14 } 15 if (ageFeature != null) { 15 EStructuralFeature ybirthFeature = 16 person.eSet(ageFeature, this.age); 16 eClass.getEStructuralFeature("ybirth"); 17 } 17 if (ybirthFeature != null) { 18 EStructuralFeature ybirthFeature = 18 age = 2020 - 19 personClass 19 (Integer) instance.eGet(ybirthFeature); 20 .getEStructuralFeature("ybirth"); 20 } 21 if (ybirthFeature != null) { 21 HavePerson havePerson = new HavePerson() 22 person.eSet(ybirthFeature, 22 .setName(name) 23 2020 - this.age); 23 .setAge(age); 24 } 24 String id = getPersonId(instance); 25 return person; 25 havePerson.setId(id); 26 } 26 execute(havePerson); 27 ... 27 } 28 ... Listing 1: M2.HavePerson::run() Listing 2: M2Editor::parse() mand suffices to address all different migration tasks of the TTC2020 migration case. Note, the M1.HavePerson command. Thereby, the editor adds the new command command also has only one implementation for all tasks, to the event store. however this implementation is slightly simpler as the One crucial step during parsing is the retrieval of ids one of M2.HavePerson as no variant of model M1 deals for the model objects, cf. Line 15 of Listing 2. Usually, with ybirth attributes. we require that the model objects have an id attribute. In the TTC2020 migration case this is not true. We solve this by storing the ids for model objects within the hash 4. Parsing tables that are used by our getOrCreate methods. To look up the id of a model object that is already stored The implementation of the different migration tasks pro- in such a hash table, we search e.g. the hash table for vided by the TTC2020 migration case resources [6] load Persons for the given instance. If an entry exists, we the different start models from XML files. After the first return the corresponding id. If there is no entry yet (i.e. migration the task implementation may retrieve the mi- after loading the initial object model) we just create a grated model and it may modify the migrated model new id and add the instance to the hash table under this directly via dynamic EMF means. This means, the task new id and then return the new id. Within the backward implementation does not use our editor commands to migration step the hash table will already contain the load or to modify the models. Thus, our first task is instance and the old id is retrieved. to parse the provided model and to derive the editing There is one special case when merging the commands commands that correspond to it. created by the parse methods into the editor’s event store: For this purpose, our editors provide a parse method In migration Task4 of the TTC2020 migration case [1], that basically uses a visitor to travel through the current in model M2 the Dog has no age attribute. Thus, when model. For each model object our visitor then calls special we parse the Dog object during the backward migration parsePerson or parseDog methods, respectively. step, the parseDog method will not find any age infor- Listing 2 shows the parsePerson method of our mation. However, this age information is still contained M2Editor. Basically, Line 5 to Line 20 of Listing 2 use within the original HaveDog command that has been re- dynamic EMF features to retrieve the parameters needed ceived from M1Editor and that has been executed and for the corresponding HavePerson command. Line 20 to added to the event store of M2Editor during forward Line 25 then create the desired HavePerson command transformation. To keep the age information alive, if and provide its parameters. Finally, Line 26 executes the the parseDog method cannot find the age attribute in the parsed model object, it tries to retrieve the old com- [2] E. Evans, Domain-driven design: tackling complex- mand from the M2Editor event store and copies the age ity in the heart of software, Addison-Wesley Profes- information from there into the new command. sional, 2004. Altogether, our parsing approach utilizes that our com- [3] V. Vernon, Implementing domain-driven design, mand execution is overriding and commutative. Due to Addison-Wesley, 2013. the commutativity, the parsing visitor may visit the ob- [4] E. Gamma, R. Helm, R. Johnson, J. Vlissides, Design jects of the current input model in any order and thus patterns: Abstraction and reuse of object-oriented create the editor commands in any order. Due to the design, in: European Conference on Object-Oriented overriding property, the backward migration may just Programming, Springer, 1993, pp. 406–431. (re)execute the detected commands and this will over- [5] S. Copei, A. Zündorf, Mx for microservices, in: Proc. write the old commands and blend into the event store, Dagstuhl Seminar, volume 18491, 2019. easily.1 [6] ttc2020resources, Ttc2020 case: Round-trip migra- tion of object-oriented data model instances, github resources, https://github.com/lbeurerkellner/ttc2020, 5. Results 2020. Last viewed 13.06.2020. Overall, our editor and command approach was very well suited for the TTC2020 migration case. We were able to address all migration task with the same implementation. While our design might appear a little bit over engineered we took benefit from the FulibServiceGenerator, a tool we are just building and that generates a lot of boiler plate code for editors and commands. After all we just had to implement the run methods of the 2 commands for the two editors and the parsing methods for the two editors as described above. We believe that the use of overwriting and commuta- tive commands provides a great leverage for the parsing and merging of command sets. Overall, our design is able to handle much more complicated migration cases which we address in our current work. Unfortunately, our performance is very poor, on a test run the Fulib solution took 9.6 seconds for 10000 itera- tions while the original solution used only 0.3 seconds. We had no time to go into the details of this. You find our solution on: Github: https://github.com/fujaba/ ttc2020MigrationCaseByFulib Docker: zuendorf/fulib-solution-ttc2019 References [1] ttc2020migration, Ttc2020 case: Round-trip migration of object-oriented data model instances, https://www.transformation-tool- contest.eu/2020_roundtrip.pdf, 2020. Last viewed 13.06.2020. 1 Fortunately, the TTC2020 migration case does not include any delete operations during model modification. To handle deletion of model objects one has to remove the corresponding old commands from the event store and one has to propagate this command re- moval to the other editor and the other editor needs to remove the command, too, and it needs to undo the command in order to roll back the corresponding model changes. This rollback needs some careful dealing with our getOrCreate operations.