The TTC 2021 Incremental Laboratory Workflow benchmark presents a challenging incremental transformation problem. This paper presents our ATOL-based partial solution, and discusses the reasons why the kind of incrementality supported by ATOL does not match the kind of incrementality required to solve the problem.
the problem to solve is given in Section 3. Section 4
presents our ATOL-based solution to this problem.
ExIncrementality is a mechanism by which a computation perimental results are briefly presented in Section 5, and
result can be updated after input changes, without hav- finally Section 6 concludes.
ing to perform all the computations required to produce
the initial result again. This results in a significant
performance advantage, because much fewer computations 2. Incremental ATL with ATOL
typically need to be performed after each input change
than were initially necessary. Some model transforma- The declarative nature of ATL makes it easily amenable to
tions can be executed incrementally, depending on the diferent execution modes. The original execution mode
language they are specified in, and the execution engine is called batch mode, and simply creates a (set of) target
that is used. For instance, some ATL transformations model(s) from a (set of) source model(s)1.
Incrementalcan be quite eficiently executed incrementally with the ity is a diferent execution mode supported by more
reATOL [
However, not all incremental problems have the same changes. ATOL [
The remainder of this paper is organized as follows.
Section 2 reminds the reader about what kind of incrementality can be achieved with ATOL. An overview of • is a model transformation, which can be viewed
as a function. • , ′, , and ′ are models. is the result of applying to , which can be written: = (). ′ is the result of applying to ′, which can be written: ′ = (′). and ′ are target models of , whereas and ′ are source models of . • and are changes, viewed as functions, respectively performed on , and , turning them into ′, and ′. This can be written: ′ = (), which is a source change wrt. , and ′ = (), which is a target change wrt. .
get models, the remainder of the paper will generally refer to a single source, and a single target model to simplify explanations. δA
t prop and it makes a diference whether a model is updated in-place or not. This is the case, for instance, when target model elements are linked to nonmodel elements, such as Graphical User Interface (GUI) elements. Updating the existing model will not break its link with the GUI elements, whereas recreating it will result in a new model that does not have any link with the GUI elements.
• is a change propagation higher-order function, which can translate changes on into changes on . can thus be obtained from by applying , which can be written: = ( )
oratory Workflow benchmark is presented in [
ATOL automatically derives from , making it low-level tasks, such as transferring tube contents, inrelatively easy for transformation developers to execute cubating samples, or washing containers, to perform on classical ATL transformations incrementally. Although which samples. An abstract syntax for these low-level the same ′ can be theoretically obtained from , , job descriptions is provided as part of the benchmark in and by following either path, each one has specific the form of a metamodel called JobCollection. practical properties: Because this metamodel operates at a very low level of abstraction, a second, higher-level metamodel is in• Change propagation is generally faster. troduced: LaboratoryWorkflow . At this level, it is only Transformation re-execution requires about the necessary to specify a collection of tasks to be applied same amount of computation for most simple to every specified sample. The job of the compiler to source changes2. Change propagation is typically develop is first to create a model conforming to JobColmuch faster for simple changes, because it does lection for every model conforming to LaboratoryWorknot need to process the whole source model again. flow it is given. This requires assigning samples to tube • Change propagation performs in-place up- runners that may only contain a maximum of 16 samples, dates. Very often, only the actual structure of and then replicating jobs as necessary to transfer all sama model matters, and models duplicated by re- ples onto microplates that may only contain a maximum executing a whole transformation are equivalent of 96 samples. Moreover, all transfer operations must be to in-place updated ones for all purposes. How- performed on microplate columns that may only contain ever, in some situations, object identity matters, a maximum of 8 samples.
Then, the compiler must be able to support addition
of new samples at any time, and to incorporate feedback
scop2eAofptrheicsispeadpeefinr.itTiohne oinftwuihtaiotna isstihmaptlae csihmanpgleecihsainsgbeeymoonddifiethse from the robot in the form of information about partially
a number of model elements which is small compared to the total failed jobs. When new samples are added after older
samnumber of elements in the model. ples have already been partially processed, they must
be processed up to the same point by adding new jobs. as in microplate column-sized chunks. The latter are then
When some samples fail to be properly processed by a distributed into microplate-sized chunks (i.e., 96/8 = 12
job, they must be marked as failed so that all future jobs rows). This results in model M2 conforming to the
Labocan skip them. However, already executed jobs must not ratoryWorkflow’ metamodel, which extends the
Laborabe changed. Compilers that support applying these mod- toryWorkflow metamodel to include information about
ifications incrementally are expected to be more eficient chunks. This extension is basically the same as in the
than those that must recompile the full process. NMF solution, and is implemented by leveraging the
capability of ATOL to consider metaclasses that are not
specified in Ecore. It is defined in the
LaboratoryWork4. Solution presentation lfow.xtend source file. This first transformation is
relatively simple, but required implementing a new chunking
The previous section has briefly presented the problem active operation. Because appending samples is the only
to solve, and this section continues by presenting our required change, we have not implemented propagation
ATOL-based solution. The solution presented here was of other changes, such as removal, or insertion.
Morepartially inspired by the NMF [
In a second step, model M2 is then transformed by Lab2Job.atl, which performs the actual translation to the JobCollection metamodel. Most of this transformation is relatively simple standard ATL code. The tricky part is related to noncommutative incrementality, and is detailed in the next section.
Firstly, change propagation must be able to proceed after either source modifications (adding samples), or target modifications (marking jobs as partially failed). This requires some level of bidirectional propagation, which active operations can achieve, but for which ATL is illequipped. Its OCL-based syntax does not support specifying enough information for reverse change propagation to work in all cases. For instance, an OCL collect operation can only take one lambda expression as argument to process elements in the forward direction, whereas bidirectionality additionally requires a lambda expression 4.3. Abusing ATOL incrementality to process elements in the reverse direction. ATOL can, Because the problem calls for change propagation that is however, be used for this purpose because it enables users not equivalent to full transformation re-execution (i.e., to specify some helper functions in xtend4, rather than noncommutative incrementality), solving it with ATOL is in ATL, while still keeping most of the transformation as not an easy task. We have not yet been able to fully solve declarative ATL code. the problem with ATOL. Nonetheless, we have managed
Secondly, the solution must be able to perform some to abuse it into performing noncommutative change propchanges while taking into account the fact that part agations. To achieve this, the main mechanism is change of the process has already been executed by the robot. filtering, which takes place for reverse propagation of tip This means that change propagation should not result in state to sample state. This is performed using an ad hoc the same model as what would be obtained by fully re- noncommutative active operation. It is implemented in executing the transformation. The kind of incremental- method mapState of Lab2Job.xtend. ity required here is therefore noncommutative, whereas However, even with this hack, our solution is not able ATOL was designed to make sure it performs commuta- to perform the required incremental propagations cortive incrementality. rectly in all cases. The remaining problems with our solution could possibly be solved by similarly abusing 4.2. Transformations ATOL, but we decided not to do so until we better understand what having noncommutative active operations entails. Here is the list of remaining problems in our solution that we have identified:
Figure 2. The source model (called M1) conforming to the LaboratoryWorkflow metamodel is first processed by the LaboratoryChunking.atl transformation, which distributes samples into tube runner-sized chunks, as well
• Problem 1. Failed samples reappear in failed jobs, which is a change filtering issue. This may not be an actual problem in practice because these jobs will not be re-executed anyway.
M2
M3 • Problem 2. Newly added samples require ad- on optimization, which could still be performed. Howditional jobs to perform jobs already completed ever, since our solution does not completely solve the on existing samples. This is again a case of non- problem, comparing its performance to other solutions commutative incrementality, which we have not may not be that meaningful. attempted to solve.
4.4. Non-ATL supporting code
This paper has presented our partial ATOL-based
solution to the TTC 2021 Incremental Laboratory Workflow
benchmark. Although this benchmark requires some
kind of incrementality, it is not exactly the same kind
of incrementality supported by ATOL, unless a better
• Accessing tuple properties. This is necessary way to encode the problem as model transformation has
on account of a current ATOL compiler limitation, eluded our investigations. We have shown that ATOL
but this code could be automatically generated. can be abused to at least partially solve the problem, but
• Enumeration literal comparison helpers. we have no theory with which to reason about our
soThis could be implemented in ATL, although lution. It may therefore be the case that it could fail in
string comparison would have to be used because unexpected ways beyond the already identified ones.
ATOL translates enumeration literals to and from This benchmark is definitely not the one to showcase
strings. ATOL on, but it presents an intriguing problem. A new
• Number zero-padding. This is necessary to expanded theory of incrementality would be necessary
generate tube runner and microplate names that to understand it precisely wrt. the active operation
apconform to what is used in the benchmark’s proach. However, it is not clear whether such a new
change specification files (although [
ttc21incrementalLabWorkflows 1 2 4 8 16 32 64 128
Model 2.5 2.0 1.0 0.5 4.0 3.5 3.0 1.5 1.0 0.5
Tool ATL_Incremental NMF Reference
Tool ATL_Incremental NMF Reference 32
Model
Model
Tool ATL_Incremental NMF Reference
Tool ATL_Incremental NMF Reference
Model 1