=Paper=
{{Paper
|id=Vol-3089/ttc21_paper_labflow_Jouault_solution
|storemode=property
|title=(Ab)using incremental ATL on the TTC 2021 incremental laboratory workflow benchmark
|pdfUrl=https://ceur-ws.org/Vol-3089/ttc21_paper10_labflow_Jouault_solution.pdf
|volume=Vol-3089
|authors=Frédéric Jouault,Théo Le Calvar
|dblpUrl=https://dblp.org/rec/conf/ttc/JouaultC21
}}
==(Ab)using incremental ATL on the TTC 2021 incremental laboratory workflow benchmark==
https://ceur-ws.org/Vol-3089/ttc21_paper10_labflow_Jouault_solution.pdf
(Ab)using incremental ATL on the TTC 2021 incremental
laboratory workflow benchmark
Frédéric Jouault1 , Théo Le Calvar2
1
ERIS, ESEO-TECH, Angers, France
2
IMT Atlantique, LS2N (UMR CNRS 6004), Nantes, France
Abstract
The TTC 2021 Incremental Laboratory Workflow benchmark presents a challenging incremental transformation problem.
This paper presents our ATOL-based partial solution, and discusses the reasons why the kind of incrementality supported
by ATOL does not match the kind of incrementality required to solve the problem.
Keywords
ATL, Incremental Model transformation, Incremental Recompilation
1. Introduction the problem to solve is given in Section 3. Section 4
presents our ATOL-based solution to this problem. Ex-
Incrementality is a mechanism by which a computation perimental results are briefly presented in Section 5, and
result can be updated after input changes, without hav- finally Section 6 concludes.
ing to perform all the computations required to produce
the initial result again. This results in a significant per-
formance advantage, because much fewer computations 2. Incremental ATL with ATOL
typically need to be performed after each input change
than were initially necessary. Some model transforma- The declarative nature of ATL makes it easily amenable to
tions can be executed incrementally, depending on the different execution modes. The original execution mode
language they are specified in, and the execution engine is called batch mode, and simply creates a1 (set of) target
that is used. For instance, some ATL transformations model(s) from a (set of) source model(s) . Incremental-
can be quite efficiently executed incrementally with the ity is a different execution mode supported by more re-
ATOL [1] engine, but other approaches exist for ATL [2], cent ATL execution engines, which are able to propagate
or other languages [3, 4, 5] source model changes into corresponding target model
However, not all incremental problems have the same changes. ATOL [1] is an incremental ATL execution en-
properties, and it is not clear which incremental trans- gine, which leverages composable active operations [7]
formation engine can be used for each incremental prob- in order to achieve model transformation incrementality.
lem. The TTC 2021 Incremental Laboratory Workflow The definition of incrementality supported by ATOL
benchmark [6] was specified in order to provide means is that change propagation should be equivalent to full
to compare incremental engines on a very specific incre- re-execution. In this paper, we will call this commutative
mental problem. This paper presents a partial solution to incrementality. This is illustrated by the commutative
this problem written in ATL, and executed with ATOL. diagram represented in Figure 1, in which:
Because the required kind of incrementality does not • 𝑡 is a model transformation, which can be viewed
match what ATOL provides, the whole problem could as a function.
not be solved without abusing ATOL. • 𝐴, 𝐴′ , 𝐵, and 𝐵 ′ are models. 𝐵 is the result of
The remainder of this paper is organized as follows. applying 𝑡 to 𝐴, which can be written: 𝐵 = 𝑡(𝐴).
Section 2 reminds the reader about what kind of incre- 𝐵 ′ is the result of applying 𝑡 to 𝐴′ , which can be
mentality can be achieved with ATOL. An overview of written: 𝐵 ′ = 𝑡(𝐴′ ). 𝐵 and 𝐵 ′ are target models
of 𝑡, whereas 𝐴 and 𝐴′ are source models of 𝑡.
TTC’21: Transformation Tool Contest, Part of the Software
• 𝛿𝐴 and 𝛿𝐵 are changes, viewed as functions, re-
Technologies: Applications and Foundations (STAF) federated
conferences, Eds. A. Boronat, A. García-Domínguez, and G. Hinkel, spectively performed on 𝐴, and 𝐵, turning them
25 June 2021, Bergen, Norway (online). into 𝐴′ , and 𝐵 ′ . This can be written: 𝐴′ =
" frederic.jouault@eseo.fr (F. Jouault); 𝛿𝐴 (𝐴), which is a source change wrt. 𝑡, and
theo.le-calvar@imt-atlantique.fr (T. Le Calvar) 𝐵 ′ = 𝛿𝐵 (𝐵), which is a target change wrt. 𝑡.
0000-0002-2395-9623 (F. Jouault); 0000-0003-2273-2053
1
(T. Le Calvar) Although ATL is capable of handling multiple source and tar-
© 2021 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0). get models, the remainder of the paper will generally refer to a sin-
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org) gle source, and a single target model to simplify explanations.
and it makes a difference whether a model is up-
t dated in-place or not. This is the case, for instance,
A B when target model elements are linked to non-
model elements, such as Graphical User Interface
(GUI) elements. Updating the existing model will
not break its link with the GUI elements, whereas
recreating it will result in a new model that does
prop
δA δB not have any link with the GUI elements.
Now that the definition of commutative incremental-
ity on which ATOL is based has been given, the role
of active operations can be explained briefly. An active
operation is a basic transformation operation, such as
t an OCL collect (called map in other languages), or
A' B'
select (called filter in other languages), equipped
with a propagation algorithm respecting commutative
Figure 1: Commutative incrementality diagram for transfor- incrementality. Model transformations can be specified
mation 𝑡 creating 𝐵 from 𝐴
by composing these operations, and change propagation
is obtained by composing their change propagation algo-
rithms.
• 𝑝𝑟𝑜𝑝 is a change propagation higher-order func-
tion, which can translate changes on 𝐴 into
changes on 𝐵. 𝛿𝐵 can thus be obtained from 𝛿𝐴 3. Problem overview
by applying 𝑝𝑟𝑜𝑝, which can be written: 𝛿𝐵 =
The full specification of the TTC 2021 Incremental Lab-
𝑝𝑟𝑜𝑝(𝛿𝐴 )
oratory Workflow benchmark is presented in [6]. This
This diagram commutes because 𝐵 ′ = 𝑡(𝛿𝐴 (𝐴)) = section gives a brief overview of the problem definition.
𝛿𝐵 (𝑡(𝐴)). This can be rewritten without referring to Solving this problem basically requires developing a com-
𝛿𝐵 in the following way: 𝐵 ′ = 𝑝𝑟𝑜𝑝(𝛿𝐴 )(𝑡(𝐴)). Trans- piler for laboratory workflow jobs given to robotic liquid
formation re-execution corresponds to 𝐵 ′ = 𝑡(𝛿𝐴 (𝐴)), handlers. These robots can perform many tasks required
whereas change propagation corresponds to: 𝐵 ′ = to, for instance, analyze biological samples. However,
𝑝𝑟𝑜𝑝(𝛿𝐴 )(𝑡(𝐴)). they must be given very detailed descriptions of what
ATOL automatically derives 𝑝𝑟𝑜𝑝 from 𝑡, making it low-level tasks, such as transferring tube contents, in-
relatively easy for transformation developers to execute cubating samples, or washing containers, to perform on
classical ATL transformations incrementally. Although which samples. An abstract syntax for these low-level
the same 𝐵 ′ can be theoretically obtained from 𝐴, 𝛿𝐴 , job descriptions is provided as part of the benchmark in
and 𝑡 by following either path, each one has specific the form of a metamodel called JobCollection.
practical properties: Because this metamodel operates at a very low level
of abstraction, a second, higher-level metamodel is in-
• Change propagation is generally faster. troduced: LaboratoryWorkflow. At this level, it is only
Transformation re-execution requires about the necessary to specify a collection of tasks to be applied
same amount of computation for most simple to every specified sample. The job of the compiler to
source changes2 . Change propagation is typically develop is first to create a model conforming to JobCol-
much faster for simple changes, because it does lection for every model conforming to LaboratoryWork-
not need to process the whole source model again. flow it is given. This requires assigning samples to tube
• Change propagation performs in-place up- runners that may only contain a maximum of 16 samples,
dates. Very often, only the actual structure of and then replicating jobs as necessary to transfer all sam-
a model matters, and models duplicated by re- ples onto microplates that may only contain a maximum
executing a whole transformation are equivalent of 96 samples. Moreover, all transfer operations must be
to in-place updated ones for all purposes. How- performed on microplate columns that may only contain
ever, in some situations, object identity matters, a maximum of 8 samples.
Then, the compiler must be able to support addition
2 of new samples at any time, and to incorporate feedback
A precise definition of what a simple change is is beyond the
scope of this paper. The intuition is that a simple change modifies from the robot in the form of information about partially
a number of model elements which is small compared to the total failed jobs. When new samples are added after older sam-
number of elements in the model. ples have already been partially processed, they must
be processed up to the same point by adding new jobs. as in microplate column-sized chunks. The latter are then
When some samples fail to be properly processed by a distributed into microplate-sized chunks (i.e., 96/8 = 12
job, they must be marked as failed so that all future jobs rows). This results in model M2 conforming to the Labo-
can skip them. However, already executed jobs must not ratoryWorkflow’ metamodel, which extends the Labora-
be changed. Compilers that support applying these mod- toryWorkflow metamodel to include information about
ifications incrementally are expected to be more efficient chunks. This extension is basically the same as in the
than those that must recompile the full process. NMF solution, and is implemented by leveraging the ca-
pability of ATOL to consider metaclasses that are not
specified in Ecore. It is defined in the LaboratoryWork-
4. Solution presentation flow.xtend source file. This first transformation is rela-
tively simple, but required implementing a new chunking
The previous section has briefly presented the problem
active operation. Because appending samples is the only
to solve, and this section continues by presenting our
required change, we have not implemented propagation
ATOL-based solution. The solution presented here was
of other changes, such as removal, or insertion. More-
partially inspired by the NMF [4] solution provided in the
over, because this transformation targets a superset of
solutions/Reference folder of the case repository3 .
its source metamodel, it was implemented using ATL’s
refining mode, which supports in-place model modifica-
4.1. Problem incrementality vs. ATOL tion. This means that only the new elements must be
incrementality created, and the already existing elements can be kept as
is. This is relatively efficient when compared to a regular
Firstly, change propagation must be able to proceed after non-refining transformation, which would have to copy
either source modifications (adding samples), or target all the existing elements.
modifications (marking jobs as partially failed). This re- In a second step, model M2 is then transformed by
quires some level of bidirectional propagation, which Lab2Job.atl, which performs the actual translation to the
active operations can achieve, but for which ATL is ill- JobCollection metamodel. Most of this transformation is
equipped. Its OCL-based syntax does not support speci- relatively simple standard ATL code. The tricky part is
fying enough information for reverse change propagation related to noncommutative incrementality, and is detailed
to work in all cases. For instance, an OCL collect oper- in the next section.
ation can only take one lambda expression as argument
to process elements in the forward direction, whereas
bidirectionality additionally requires a lambda expression 4.3. Abusing ATOL incrementality
to process elements in the reverse direction. ATOL can, Because the problem calls for change propagation that is
however, be used for this purpose because it enables users not equivalent to full transformation re-execution (i.e.,
to specify some helper functions in xtend4 , rather than noncommutative incrementality), solving it with ATOL is
in ATL, while still keeping most of the transformation as not an easy task. We have not yet been able to fully solve
declarative ATL code. the problem with ATOL. Nonetheless, we have managed
Secondly, the solution must be able to perform some to abuse it into performing noncommutative change prop-
changes while taking into account the fact that part agations. To achieve this, the main mechanism is change
of the process has already been executed by the robot. filtering, which takes place for reverse propagation of tip
This means that change propagation should not result in state to sample state. This is performed using an ad hoc
the same model as what would be obtained by fully re- noncommutative active operation. It is implemented in
executing the transformation. The kind of incremental- method mapState of Lab2Job.xtend.
ity required here is therefore noncommutative, whereas However, even with this hack, our solution is not able
ATOL was designed to make sure it performs commuta- to perform the required incremental propagations cor-
tive incrementality. rectly in all cases. The remaining problems with our
solution could possibly be solved by similarly abusing
4.2. Transformations ATOL, but we decided not to do so until we better under-
stand what having noncommutative active operations
The overall process we implemented is represented on entails. Here is the list of remaining problems in our
Figure 2. The source model (called M1) conforming to solution that we have identified:
the LaboratoryWorkflow metamodel is first processed by
the LaboratoryChunking.atl transformation, which dis- • Problem 1. Failed samples reappear in failed
tributes samples into tube runner-sized chunks, as well jobs, which is a change filtering issue. This may
not be an actual problem in practice because these
3
https://github.com/tecan/ttc21incrementalLabWorkflows/ jobs will not be re-executed anyway.
4
https://www.eclipse.org/xtend/
LaboratoryWorkflow LaboratoryWorkflow' JobCollection
LaboratoryChunking.atl Lab2Job.atl
M1 M2 M3
Figure 2: Overview of the transformation process of our ATOL-based solution
• Problem 2. Newly added samples require ad- on optimization, which could still be performed. How-
ditional jobs to perform jobs already completed ever, since our solution does not completely solve the
on existing samples. This is again a case of non- problem, comparing its performance to other solutions
commutative incrementality, which we have not may not be that meaningful.
attempted to solve.
6. Conclusion
4.4. Non-ATL supporting code
In addition to ATL code, and the previously described This paper has presented our partial ATOL-based solu-
xtend helpers, some xtend code was also required for tion to the TTC 2021 Incremental Laboratory Workflow
the following aspects (identified with comments in the benchmark. Although this benchmark requires some
LaboratoryChunking.xtend and Lab2Job.xtend files): kind of incrementality, it is not exactly the same kind
of incrementality supported by ATOL, unless a better
• Accessing tuple properties. This is necessary way to encode the problem as model transformation has
on account of a current ATOL compiler limitation, eluded our investigations. We have shown that ATOL
but this code could be automatically generated. can be abused to at least partially solve the problem, but
• Enumeration literal comparison helpers. we have no theory with which to reason about our so-
This could be implemented in ATL, although lution. It may therefore be the case that it could fail in
string comparison would have to be used because unexpected ways beyond the already identified ones.
ATOL translates enumeration literals to and from This benchmark is definitely not the one to showcase
strings. ATOL on, but it presents an intriguing problem. A new
• Number zero-padding. This is necessary to expanded theory of incrementality would be necessary
generate tube runner and microplate names that to understand it precisely wrt. the active operation ap-
conform to what is used in the benchmark’s proach. However, it is not clear whether such a new
change specification files (although [6] does not theory would be general enough to support other rel-
specify padding). This could be implemented in evant noncommutative incrementality problems, or if
ATL, but would be cumbersome because of lim- each new problem would require extending the theory.
ited OCL support for string operations.
Finally, the transformation driver, and the change appli- References
cation code are also written in xtend.
[1] T. Le Calvar, F. Jouault, C. Chhel, M. Clavreul, Ef-
ficient ATL incremental transformations, J. Object
5. Experimentations Technol. 18 (2019) 2:1–17. doi:10.5381/jot.2019.
18.3.a2.
Our ATOL solution to the TTC 2021 Incremental Labo- [2] S. Martínez, M. Tisi, R. Douence, Reactive model
ratory Workflow benchmark is available on GitHub5 . It transformation with ATL, Science of Computer Pro-
reuses driver code from previous TTC contests. gramming 136 (2017) 1–16. doi:10.1016/j.scico.
Performance seems relatively good, as can be seen on 2016.08.006.
Figures 3, 4, and 5, although we have not spent much time [3] D. Varró, G. Bergmann, Á. Hegedüs, Á. Horváth,
5
https://github.com/ESEO-Tech/
I. Ráth, Z. Ujhelyi, Road to a reactive and incre-
ttc21incrementalLabWorkflows mental model transformation platform: three gen-
erations of the VIATRA framework, Software &
Systems Modeling 15 (2016) 609–629. doi:10.1007/ Tool
s10270-016-0530-4. 1000 ATL_Incremental
NMF
[4] G. Hinkel, NMF: A multi-platform modeling frame- Reference
work, in: Theory and Practice of Model Transfor- 800
mation, Springer International Publishing, 2018, pp.
Time [ms]
184–194. doi:10.1007/978-3-319-93317-7_10. 600
[5] A. Boronat, Incremental execution of rule-based
model transformation, International Journal on Soft- 400
ware Tools for Technology Transfer 23 (2020) 289–
200
311. doi:10.1007/s10009-020-00583-y.
[6] G. Hinkel, Incremental recompilation of laboratory 0
workflows, in: Proceedings of 14th Transformation 1 2 4 8 16 32 64 128
Model
Tool Contest (TTC 2021), 2021. To appear.
[7] O. Beaudoux, A. Blouin, O. Barais, J. Jézéquel, Active (a) Initial
Operations on Collections, in: Model Driven Engi-
neering Languages and Systems - 13th International
Conference, MODELS 2010, Oslo, Norway, October
230
3-8, 2010, Proceedings, Part I, volume 6394 of Lec-
ture Notes in Computer Science, Springer, 2010, pp. 220
91–105. doi:10.1007/978-3-642-16145-2_7. 210
Tool
Time [ms]
200 ATL_Incremental
NMF
190 Reference
180
170
1 2 4 8 16 32 64 128
Model
(b) Initialization
100
80
Tool
Time [ms]
60 ATL_Incremental
NMF
Reference
40
20
1 2 4 8 16 32 64 128
Model
(c) Load
8
6 Tool
Time [ms]
ATL_Incremental
NMF
Reference
4
2
1 2 4 8 16 32 64 128
Model
(d) Update
Figure 3: Benchmark results for new_samples scenario
Tool Tool
2000 ATL_Incremental 17500 ATL_Incremental
NMF NMF
Reference 15000 Reference
1500
12500
Time [ms]
Time [ms]
10000
1000
7500
5000
500
2500
0 0
1 2 4 8 16 32 1 2 4 8 16 32 64 128 256 512 1024
Model Model
(a) Initial (a) Initial
220 230
220
210
210
200
Tool Tool
Time [ms]
Time [ms]
ATL_Incremental 200 ATL_Incremental
190 NMF NMF
Reference 190 Reference
180
180
170 170
160 160
1 2 4 8 16 32 1 2 4 8 16 32 64 128 256 512 1024
Model Model
(b) Initialization (b) Initialization
120 Tool Tool
ATL_Incremental 120 ATL_Incremental
NMF NMF
100 Reference Reference
100
80
80
Time [ms]
Time [ms]
60 60
40 40
20 20
1 2 4 8 16 32 1 2 4 8 16 32 64 128 256 512 1024
Model Model
(c) Load (c) Load
3.0
Tool Tool
ATL_Incremental 4.0 ATL_Incremental
2.5 NMF NMF
Reference 3.5 Reference
2.0 3.0
Time [ms]
Time [ms]
2.5
1.5 2.0
1.5
1.0
1.0
0.5 0.5
1 2 4 8 16 32 1 2 4 8 16 32 64 128 256 512 1024
Model Model
(d) Update (d) Update
Figure 4: Benchmark results for scale_assay scenario Figure 5: Benchmark results for scale_samples scenario