(Ab)using incremental ATL on the TTC 2021 incremental laboratory workflow benchmark Frédéric Jouault1 , Théo Le Calvar2 1 ERIS, ESEO-TECH, Angers, France 2 IMT Atlantique, LS2N (UMR CNRS 6004), Nantes, France Abstract The TTC 2021 Incremental Laboratory Workflow benchmark presents a challenging incremental transformation problem. This paper presents our ATOL-based partial solution, and discusses the reasons why the kind of incrementality supported by ATOL does not match the kind of incrementality required to solve the problem. Keywords ATL, Incremental Model transformation, Incremental Recompilation 1. Introduction the problem to solve is given in Section 3. Section 4 presents our ATOL-based solution to this problem. Ex- Incrementality is a mechanism by which a computation perimental results are briefly presented in Section 5, and result can be updated after input changes, without hav- finally Section 6 concludes. ing to perform all the computations required to produce the initial result again. This results in a significant per- formance advantage, because much fewer computations 2. Incremental ATL with ATOL typically need to be performed after each input change than were initially necessary. Some model transforma- The declarative nature of ATL makes it easily amenable to tions can be executed incrementally, depending on the different execution modes. The original execution mode language they are specified in, and the execution engine is called batch mode, and simply creates a1 (set of) target that is used. For instance, some ATL transformations model(s) from a (set of) source model(s) . Incremental- can be quite efficiently executed incrementally with the ity is a different execution mode supported by more re- ATOL [1] engine, but other approaches exist for ATL [2], cent ATL execution engines, which are able to propagate or other languages [3, 4, 5] source model changes into corresponding target model However, not all incremental problems have the same changes. ATOL [1] is an incremental ATL execution en- properties, and it is not clear which incremental trans- gine, which leverages composable active operations [7] formation engine can be used for each incremental prob- in order to achieve model transformation incrementality. lem. The TTC 2021 Incremental Laboratory Workflow The definition of incrementality supported by ATOL benchmark [6] was specified in order to provide means is that change propagation should be equivalent to full to compare incremental engines on a very specific incre- re-execution. In this paper, we will call this commutative mental problem. This paper presents a partial solution to incrementality. This is illustrated by the commutative this problem written in ATL, and executed with ATOL. diagram represented in Figure 1, in which: Because the required kind of incrementality does not • 𝑡 is a model transformation, which can be viewed match what ATOL provides, the whole problem could as a function. not be solved without abusing ATOL. • 𝐴, 𝐴′ , 𝐵, and 𝐵 ′ are models. 𝐵 is the result of The remainder of this paper is organized as follows. applying 𝑡 to 𝐴, which can be written: 𝐵 = 𝑡(𝐴). Section 2 reminds the reader about what kind of incre- 𝐵 ′ is the result of applying 𝑡 to 𝐴′ , which can be mentality can be achieved with ATOL. An overview of written: 𝐵 ′ = 𝑡(𝐴′ ). 𝐵 and 𝐵 ′ are target models of 𝑡, whereas 𝐴 and 𝐴′ are source models of 𝑡. TTC’21: Transformation Tool Contest, Part of the Software • 𝛿𝐴 and 𝛿𝐵 are changes, viewed as functions, re- Technologies: Applications and Foundations (STAF) federated conferences, Eds. A. Boronat, A. García-Domínguez, and G. Hinkel, spectively performed on 𝐴, and 𝐵, turning them 25 June 2021, Bergen, Norway (online). into 𝐴′ , and 𝐵 ′ . This can be written: 𝐴′ = " frederic.jouault@eseo.fr (F. Jouault); 𝛿𝐴 (𝐴), which is a source change wrt. 𝑡, and theo.le-calvar@imt-atlantique.fr (T. Le Calvar) 𝐵 ′ = 𝛿𝐵 (𝐵), which is a target change wrt. 𝑡.  0000-0002-2395-9623 (F. Jouault); 0000-0003-2273-2053 1 (T. Le Calvar) Although ATL is capable of handling multiple source and tar- © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). get models, the remainder of the paper will generally refer to a sin- CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) gle source, and a single target model to simplify explanations. and it makes a difference whether a model is up- t dated in-place or not. This is the case, for instance, A B when target model elements are linked to non- model elements, such as Graphical User Interface (GUI) elements. Updating the existing model will not break its link with the GUI elements, whereas recreating it will result in a new model that does prop δA δB not have any link with the GUI elements. Now that the definition of commutative incremental- ity on which ATOL is based has been given, the role of active operations can be explained briefly. An active operation is a basic transformation operation, such as t an OCL collect (called map in other languages), or A' B' select (called filter in other languages), equipped with a propagation algorithm respecting commutative Figure 1: Commutative incrementality diagram for transfor- incrementality. Model transformations can be specified mation 𝑡 creating 𝐵 from 𝐴 by composing these operations, and change propagation is obtained by composing their change propagation algo- rithms. • 𝑝𝑟𝑜𝑝 is a change propagation higher-order func- tion, which can translate changes on 𝐴 into changes on 𝐵. 𝛿𝐵 can thus be obtained from 𝛿𝐴 3. Problem overview by applying 𝑝𝑟𝑜𝑝, which can be written: 𝛿𝐵 = The full specification of the TTC 2021 Incremental Lab- 𝑝𝑟𝑜𝑝(𝛿𝐴 ) oratory Workflow benchmark is presented in [6]. This This diagram commutes because 𝐵 ′ = 𝑡(𝛿𝐴 (𝐴)) = section gives a brief overview of the problem definition. 𝛿𝐵 (𝑡(𝐴)). This can be rewritten without referring to Solving this problem basically requires developing a com- 𝛿𝐵 in the following way: 𝐵 ′ = 𝑝𝑟𝑜𝑝(𝛿𝐴 )(𝑡(𝐴)). Trans- piler for laboratory workflow jobs given to robotic liquid formation re-execution corresponds to 𝐵 ′ = 𝑡(𝛿𝐴 (𝐴)), handlers. These robots can perform many tasks required whereas change propagation corresponds to: 𝐵 ′ = to, for instance, analyze biological samples. However, 𝑝𝑟𝑜𝑝(𝛿𝐴 )(𝑡(𝐴)). they must be given very detailed descriptions of what ATOL automatically derives 𝑝𝑟𝑜𝑝 from 𝑡, making it low-level tasks, such as transferring tube contents, in- relatively easy for transformation developers to execute cubating samples, or washing containers, to perform on classical ATL transformations incrementally. Although which samples. An abstract syntax for these low-level the same 𝐵 ′ can be theoretically obtained from 𝐴, 𝛿𝐴 , job descriptions is provided as part of the benchmark in and 𝑡 by following either path, each one has specific the form of a metamodel called JobCollection. practical properties: Because this metamodel operates at a very low level of abstraction, a second, higher-level metamodel is in- • Change propagation is generally faster. troduced: LaboratoryWorkflow. At this level, it is only Transformation re-execution requires about the necessary to specify a collection of tasks to be applied same amount of computation for most simple to every specified sample. The job of the compiler to source changes2 . Change propagation is typically develop is first to create a model conforming to JobCol- much faster for simple changes, because it does lection for every model conforming to LaboratoryWork- not need to process the whole source model again. flow it is given. This requires assigning samples to tube • Change propagation performs in-place up- runners that may only contain a maximum of 16 samples, dates. Very often, only the actual structure of and then replicating jobs as necessary to transfer all sam- a model matters, and models duplicated by re- ples onto microplates that may only contain a maximum executing a whole transformation are equivalent of 96 samples. Moreover, all transfer operations must be to in-place updated ones for all purposes. How- performed on microplate columns that may only contain ever, in some situations, object identity matters, a maximum of 8 samples. Then, the compiler must be able to support addition 2 of new samples at any time, and to incorporate feedback A precise definition of what a simple change is is beyond the scope of this paper. The intuition is that a simple change modifies from the robot in the form of information about partially a number of model elements which is small compared to the total failed jobs. When new samples are added after older sam- number of elements in the model. ples have already been partially processed, they must be processed up to the same point by adding new jobs. as in microplate column-sized chunks. The latter are then When some samples fail to be properly processed by a distributed into microplate-sized chunks (i.e., 96/8 = 12 job, they must be marked as failed so that all future jobs rows). This results in model M2 conforming to the Labo- can skip them. However, already executed jobs must not ratoryWorkflow’ metamodel, which extends the Labora- be changed. Compilers that support applying these mod- toryWorkflow metamodel to include information about ifications incrementally are expected to be more efficient chunks. This extension is basically the same as in the than those that must recompile the full process. NMF solution, and is implemented by leveraging the ca- pability of ATOL to consider metaclasses that are not specified in Ecore. It is defined in the LaboratoryWork- 4. Solution presentation flow.xtend source file. This first transformation is rela- tively simple, but required implementing a new chunking The previous section has briefly presented the problem active operation. Because appending samples is the only to solve, and this section continues by presenting our required change, we have not implemented propagation ATOL-based solution. The solution presented here was of other changes, such as removal, or insertion. More- partially inspired by the NMF [4] solution provided in the over, because this transformation targets a superset of solutions/Reference folder of the case repository3 . its source metamodel, it was implemented using ATL’s refining mode, which supports in-place model modifica- 4.1. Problem incrementality vs. ATOL tion. This means that only the new elements must be incrementality created, and the already existing elements can be kept as is. This is relatively efficient when compared to a regular Firstly, change propagation must be able to proceed after non-refining transformation, which would have to copy either source modifications (adding samples), or target all the existing elements. modifications (marking jobs as partially failed). This re- In a second step, model M2 is then transformed by quires some level of bidirectional propagation, which Lab2Job.atl, which performs the actual translation to the active operations can achieve, but for which ATL is ill- JobCollection metamodel. Most of this transformation is equipped. Its OCL-based syntax does not support speci- relatively simple standard ATL code. The tricky part is fying enough information for reverse change propagation related to noncommutative incrementality, and is detailed to work in all cases. For instance, an OCL collect oper- in the next section. ation can only take one lambda expression as argument to process elements in the forward direction, whereas bidirectionality additionally requires a lambda expression 4.3. Abusing ATOL incrementality to process elements in the reverse direction. ATOL can, Because the problem calls for change propagation that is however, be used for this purpose because it enables users not equivalent to full transformation re-execution (i.e., to specify some helper functions in xtend4 , rather than noncommutative incrementality), solving it with ATOL is in ATL, while still keeping most of the transformation as not an easy task. We have not yet been able to fully solve declarative ATL code. the problem with ATOL. Nonetheless, we have managed Secondly, the solution must be able to perform some to abuse it into performing noncommutative change prop- changes while taking into account the fact that part agations. To achieve this, the main mechanism is change of the process has already been executed by the robot. filtering, which takes place for reverse propagation of tip This means that change propagation should not result in state to sample state. This is performed using an ad hoc the same model as what would be obtained by fully re- noncommutative active operation. It is implemented in executing the transformation. The kind of incremental- method mapState of Lab2Job.xtend. ity required here is therefore noncommutative, whereas However, even with this hack, our solution is not able ATOL was designed to make sure it performs commuta- to perform the required incremental propagations cor- tive incrementality. rectly in all cases. The remaining problems with our solution could possibly be solved by similarly abusing 4.2. Transformations ATOL, but we decided not to do so until we better under- stand what having noncommutative active operations The overall process we implemented is represented on entails. Here is the list of remaining problems in our Figure 2. The source model (called M1) conforming to solution that we have identified: the LaboratoryWorkflow metamodel is first processed by the LaboratoryChunking.atl transformation, which dis- • Problem 1. Failed samples reappear in failed tributes samples into tube runner-sized chunks, as well jobs, which is a change filtering issue. This may not be an actual problem in practice because these 3 https://github.com/tecan/ttc21incrementalLabWorkflows/ jobs will not be re-executed anyway. 4 https://www.eclipse.org/xtend/ LaboratoryWorkflow LaboratoryWorkflow' JobCollection LaboratoryChunking.atl Lab2Job.atl M1 M2 M3 Figure 2: Overview of the transformation process of our ATOL-based solution • Problem 2. Newly added samples require ad- on optimization, which could still be performed. How- ditional jobs to perform jobs already completed ever, since our solution does not completely solve the on existing samples. This is again a case of non- problem, comparing its performance to other solutions commutative incrementality, which we have not may not be that meaningful. attempted to solve. 6. Conclusion 4.4. Non-ATL supporting code In addition to ATL code, and the previously described This paper has presented our partial ATOL-based solu- xtend helpers, some xtend code was also required for tion to the TTC 2021 Incremental Laboratory Workflow the following aspects (identified with comments in the benchmark. Although this benchmark requires some LaboratoryChunking.xtend and Lab2Job.xtend files): kind of incrementality, it is not exactly the same kind of incrementality supported by ATOL, unless a better • Accessing tuple properties. This is necessary way to encode the problem as model transformation has on account of a current ATOL compiler limitation, eluded our investigations. We have shown that ATOL but this code could be automatically generated. can be abused to at least partially solve the problem, but • Enumeration literal comparison helpers. we have no theory with which to reason about our so- This could be implemented in ATL, although lution. It may therefore be the case that it could fail in string comparison would have to be used because unexpected ways beyond the already identified ones. ATOL translates enumeration literals to and from This benchmark is definitely not the one to showcase strings. ATOL on, but it presents an intriguing problem. A new • Number zero-padding. This is necessary to expanded theory of incrementality would be necessary generate tube runner and microplate names that to understand it precisely wrt. the active operation ap- conform to what is used in the benchmark’s proach. However, it is not clear whether such a new change specification files (although [6] does not theory would be general enough to support other rel- specify padding). This could be implemented in evant noncommutative incrementality problems, or if ATL, but would be cumbersome because of lim- each new problem would require extending the theory. ited OCL support for string operations. Finally, the transformation driver, and the change appli- References cation code are also written in xtend. [1] T. Le Calvar, F. Jouault, C. Chhel, M. Clavreul, Ef- ficient ATL incremental transformations, J. Object 5. Experimentations Technol. 18 (2019) 2:1–17. doi:10.5381/jot.2019. 18.3.a2. Our ATOL solution to the TTC 2021 Incremental Labo- [2] S. Martínez, M. Tisi, R. Douence, Reactive model ratory Workflow benchmark is available on GitHub5 . It transformation with ATL, Science of Computer Pro- reuses driver code from previous TTC contests. gramming 136 (2017) 1–16. doi:10.1016/j.scico. Performance seems relatively good, as can be seen on 2016.08.006. Figures 3, 4, and 5, although we have not spent much time [3] D. Varró, G. Bergmann, Á. Hegedüs, Á. Horváth, 5 https://github.com/ESEO-Tech/ I. Ráth, Z. Ujhelyi, Road to a reactive and incre- ttc21incrementalLabWorkflows mental model transformation platform: three gen- erations of the VIATRA framework, Software & Systems Modeling 15 (2016) 609–629. doi:10.1007/ Tool s10270-016-0530-4. 1000 ATL_Incremental NMF [4] G. Hinkel, NMF: A multi-platform modeling frame- Reference work, in: Theory and Practice of Model Transfor- 800 mation, Springer International Publishing, 2018, pp. Time [ms] 184–194. doi:10.1007/978-3-319-93317-7_10. 600 [5] A. Boronat, Incremental execution of rule-based model transformation, International Journal on Soft- 400 ware Tools for Technology Transfer 23 (2020) 289– 200 311. doi:10.1007/s10009-020-00583-y. [6] G. Hinkel, Incremental recompilation of laboratory 0 workflows, in: Proceedings of 14th Transformation 1 2 4 8 16 32 64 128 Model Tool Contest (TTC 2021), 2021. To appear. [7] O. Beaudoux, A. Blouin, O. Barais, J. Jézéquel, Active (a) Initial Operations on Collections, in: Model Driven Engi- neering Languages and Systems - 13th International Conference, MODELS 2010, Oslo, Norway, October 230 3-8, 2010, Proceedings, Part I, volume 6394 of Lec- ture Notes in Computer Science, Springer, 2010, pp. 220 91–105. doi:10.1007/978-3-642-16145-2_7. 210 Tool Time [ms] 200 ATL_Incremental NMF 190 Reference 180 170 1 2 4 8 16 32 64 128 Model (b) Initialization 100 80 Tool Time [ms] 60 ATL_Incremental NMF Reference 40 20 1 2 4 8 16 32 64 128 Model (c) Load 8 6 Tool Time [ms] ATL_Incremental NMF Reference 4 2 1 2 4 8 16 32 64 128 Model (d) Update Figure 3: Benchmark results for new_samples scenario Tool Tool 2000 ATL_Incremental 17500 ATL_Incremental NMF NMF Reference 15000 Reference 1500 12500 Time [ms] Time [ms] 10000 1000 7500 5000 500 2500 0 0 1 2 4 8 16 32 1 2 4 8 16 32 64 128 256 512 1024 Model Model (a) Initial (a) Initial 220 230 220 210 210 200 Tool Tool Time [ms] Time [ms] ATL_Incremental 200 ATL_Incremental 190 NMF NMF Reference 190 Reference 180 180 170 170 160 160 1 2 4 8 16 32 1 2 4 8 16 32 64 128 256 512 1024 Model Model (b) Initialization (b) Initialization 120 Tool Tool ATL_Incremental 120 ATL_Incremental NMF NMF 100 Reference Reference 100 80 80 Time [ms] Time [ms] 60 60 40 40 20 20 1 2 4 8 16 32 1 2 4 8 16 32 64 128 256 512 1024 Model Model (c) Load (c) Load 3.0 Tool Tool ATL_Incremental 4.0 ATL_Incremental 2.5 NMF NMF Reference 3.5 Reference 2.0 3.0 Time [ms] Time [ms] 2.5 1.5 2.0 1.5 1.0 1.0 0.5 0.5 1 2 4 8 16 32 1 2 4 8 16 32 64 128 256 512 1024 Model Model (d) Update (d) Update Figure 4: Benchmark results for scale_assay scenario Figure 5: Benchmark results for scale_samples scenario