In this paper, we present the YAMTL solution to the Laboratory Workflows case of TTC 2021. This solution illustrates how to specify a consistency relation between two metamodels that may map one object of the input model to several objects of the output model using a declarative style. In addition, the solution makes use of generated boilerplate code and rule inheritance for the sake of conciseness. The initial experiments show that YAMTL introduces little overhead over the reference solution, implemented in plain code on the .NET Framework, and yet it addresses its main problems: change propagation is encoded using declarative rules and traceability is handled implicitly by YAMTL.
to previous and next steps.
Low-level models capture the execution of the The TTC 2021 case on incremental recompilation of lab- workflow on a RLH. A process is represented with oratory workflows [ 1] aims at investigating solutions a JobCollection, where each Job could refer to for a key logistic issue at the start of the Covid-19 pan- LiquidTransfer, Wash or Incubate. A LiquidTransfer demic - availability of test capacity - from a model-driven manages a microplate, either by distributing either engineering perspective. The problem consists in pro- samples from tubes or reagents from troughs into mividing an automated approach to manage the evolution croplate cavities. Each job LiquidTransfer has up to of laboratory workflows involving robotic liquid han- eight LiquidTransferTips in order to transfer samples dlers (RLHs), e.g. to repurpose existing laboratory work- or reagents at the same time. lfows in order to cope with a sudden demand of Covid- Specific logic on the behaviour of a job LiquidTransfer 19 testing. To facilitate this task declarative high-level and the mapping is documented in the case [1]. Solutions processess, specifying the configuration of a laboratory for specifying and managing such mapping are required: workflow, are to be mapped to low-level jobs that are to be highly performant during change propagation, to executable by the laboratory RLHs. Whenever there is be understandable in order to enhance maintenability, an error in a RLH low-level job (e.g. due a hardware to keep some form of memory of the state of low-level problem), the corresponding high-level process change jobs, and to have low or no specification overhead for needs to be identified and this change needs to be prop- making the mapping from high-level process to low-level agated to the low-level job. The challenge is that such jobs incremental with respect to changes in high-level low-level jobs may not be able to start from scratch to fix processes. the error, and the current state of the low-level job (e.g. The structure of the paper is as follows: section 2 current distribution of samples processed) needs to be provides a brief introduction to the YAMTL language; maintained. section 3 describes an outline of the YAMTL solution;
High-level models are defined in terms of JobRequests section 4 presents the main challenges faced; section 5 that use several Samples and that have Assays, each of presents the transformation rules used in the YAMTL which represents a protocol where there are four diferent solution; and section 6 discusses the evaluation of the ProtocolSteps: distributing sample, adding a reagent, solution with the benchmark criteria. washing or incubating. Protocol steps have dependencies
models, with support for incremental execution [3], designed as an internal DSL of Xtend.
A YAMTL model transformation is defined as a module, a class specializing the class YAMTLModule, containing the declaration of transformation rules. Each rule has an input pattern for matching variables and an output pattern languages (available in the case description [1]), and by for creating objects. An input pattern consists of in ele- defining mappings as rules. When applying a rule, perments together with a global rule filter condition, which forming a transformation step, YAMTL implicitly manis true if not specified. Each of the in elements is declared tains a traceability model that remembers which rules with a variable name, a type and a local filter condition, were applied to which parts of source models. When a which is true if not specified. An output pattern consists change is applied to the high-level model, YAMTL autoof out elements, each of which is declared with a vari- matically re-evaluates the afected transformation steps, able name, a type and an action block. Filter conditions if needed. and action blocks are specified as non-pure lambda ex- From an outline perspective, the model transformapressions in Xtend1. In such expressions, the YAMTL tion is performed as follows: initially labware is initiallanguage uses the expression 'variable'.fetch as Type ized by setting tube runners with samples, troughs with to fetch the value of a 'variable' from the execution en- reagens and microplates with cavities, and samples are vironment and for obtaining referenes to objects created allocated from tube cavities to microplate cavities. In by other rules. This is normally used to access matched a second phase, for each ProtocolStep and Sample, a objects in a filter expressions and in output expressions. LiquidTransferTip is initialized in order to perform the
When applying a YAMTL transformation to an input transfer of a sample or of a reagent. In addition, jobs are model, the pattern matcher finds all rule matches that initialized according to the type of ProtocolSteps in the satisfy local lfiter conditions. When a total match is found, high-level model. the satisfaction of that match is finally asserted by the A strength of our solution is that it is fully declararule filter condition. Once all matches are found, the tive, without involving explicit invocation of rules or transformation engine computes an output match for imposing control flow. It is also non-intrusive, as it does each input match using the expressions in the out action not require the modification of the original metamodels blocks of the corresponding rule. or adding external glueing code to execute parts of the
YAMTL also supports multiple rule inheritance. When transformation. a rule specializes another one, its execution semantics is altered as follows: in matched input elements, filter expressions are inherited using a leftmost top-down eval- 4. Challenges and YAMTL uation strategy w.r.t. the inheritance hierarchy; in output extensions elements, action expressions are also inherited following a leftmost topdown evaluation strategy w.r.t. the In the YAMTL solution, the main challenge was a conseinheritance hierarchy by default, and they can be can be quence of the additional information that low-level jobs overriden in descendant rule. need to deal with: specific source and target labware for
The YAMTL engine has been extended with an incre- each job. Such additional information means that for mental execution mode, which consists of two phases: each ProtocolStep a transformation rule has to create the initial phase, the transformation is executed in batch several Jobs, depending on the nature of the job. That is, mode but, additionally, tracks feature calls in objects of the mapping between high-level and low-level models the source model involved in transformation steps as is one-to-many for some elements. Originally, as many dependencies; and the propagation phase, the transforma- other model transformation languages with support for tion is executed incrementally for a given source update functional model transformations, YAMTL did not supand only those transformation steps afected by the up- port such type of mappings. This case has prompted the date are (re-)executed. This means that components of development of additional YAMTL extensions in order YAMTL’s execution model have been extended but the to enable their specification. syntax used to define model transformations is preserved. The new features that have been added to YAMTL are Hence, a YAMTL batch model transformation can be ex- as follows: ecuted in incremental mode, without any additional user specification overhead.
EMF metamodels for the high-level and low-level process 1For a more detailed description of the YAMTL language, the reader is referred to [2], including programmable execution strategies and multiple inheritance. • Matched rules toMany that enable repeated rule application for the same input object subject to a valid termination condition toManyCap based on the match count. The count of the present input match, to which the rule is being applied, can be retrieved with the expression 'matchCount'.fetch(). Whenever a rule toMany is involved in a rule inheritance hierarchy, all rules in that hierarchy must be toMany too. All variants of the operator fetch() have been augmented with an additional parameter, the occurrence of the transformation step from which the target object must be fetched. By default, the value of this parameter is 0, corresponding to the first transformation step that is found for the input object or input match. Hence the fetch operator is equipped for working with several matches of a rule toMany. • When the global correctness check, which ensures that a model transformation is a mapping, is disabled, there may be several rules transforming the same input object. This allows YAMTL to represent relational model transformations that are more expressive than mapping model transformations. The scope of the correctness check is at rule level though. A rule cannot be applied more than once to the same input object, unless the rule is declared as toMany and has a valid termination condition for the repetition. 1 rule('jobRequest_->_jobCollection') 2 .in('in_jobRequest', LAB.jobRequest) 3 .out('out_jobCollection', JOB.jobCollection)
Reagent.
Microplatess as required for the input JobRequest, according to the expression jobRequest.samples.size / MICROPLATE_CAPACITY, in a similar way as above. 1 rule('jobRequest_->_microplate').toMany 2 .in('jobRequest', LAB.jobRequest) 3 .toManyCap[max_count(jobRequest.samples.size,
MICROPLATE_CAPACITY)] 4 .out('microplate', JOB.microplate)[ 5 val out_jobCollection = jobRequest .fetch('out_jobCollection', 'jobRequest_->_jobCollection') as
JobCollection var microplate_list = out_jobCollection.labware.filter[ it instanceof Microplate] microplate.name = String.format('''Plate%02d''',
microplate_list.size+1) out_jobCollection.labware.add(microplate)
each input sample and adds itself to the corresponding LiquidTransferJob. When the sample has failed, the TipLiquidTransfer is removed in the undo action.
Listing 4: Rule jobRequest_->_microplate.
1 rule('tip_creation') 2 .in('in_sample', LAB.sample).filter[ 3 in_sample.state != SampleState.ERROR 4 ] 5 .in('in_step', LAB.protocolStep).filter[ 6 (in_step instanceof DistributeSample || in_step instanceof AddReagent) 1 rule('sample_->_allocation').transient 2 .in('in_sample', LAB.sample).filter [ 3 in_sample.state == SampleState.WAITING 4 ] 5 .out('out_aux', JOB.jobCollection)[ 6 val in_jobRequest = 7 in_sample.eContainer as JobRequest 8 val tubeRunnerNumber = 9 getTubeRunner_number(in_jobRequest, in_sample) 10 val tubeRunner = in_jobRequest 11 .fetch('tubeRunner', 'jobRequest_->_tubeRunner',
tubeRunnerNumber) as TubeRunner 12 tubeRunner.barcodes += in_sample.sampleID 13 val microplateNumber = 14 getMicroplate_number(in_jobRequest, in_sample) 15 val microplateCavity = 16 getMicroplate_cavity(in_jobRequest, in_sample) 17 val microplate = 18 in_jobRequest.fetch('microplate', 'jobRequest_->_microplate', microplateNumber) as Microplate
Rule sample_->_allocation computes the allocation of samples to tube runner and microplate cavities. This rule is used to complete the initialization of tube runners and microplates. The rule is transient and the JobCollection 7 ] that is created is immaterial. Therefore, this rule is only 8 .out('out_tip', JOB.tipLiquidTransfer) [ used to perform some additional initialization in other 9 val step = in_step objects. In addition, the rule remembers what sample is 10 val tip = out_tip stored in each cavity in order to implement the applica- 11 val in_jobRequest = tion of backward changes. This rule has an undo action, 12 step.eContainer.eContainer as JobRequest 13 val occurrence = which updates these backward traces when the sample 14 getTipContainerIndex(in_jobRequest, in_sample) is no longer allocated. 15 val out_job = 16 step.fetch(occurrence) as LiquidTransferJob 17 18 switch(step) { 19 DistributeSample: { 20 tip.volume = step.volume 21 // SOURCE 22 out_job.source = 23 in_jobRequest.getTubeRunner(in_sample) 24 tip.sourceCavityIndex = 25 in_jobRequest.getTubeRunner_cavity(in_sample) 26 } 27 AddReagent: { 28 tip.volume = step.volume 29 // SOURCE 30 tip.sourceCavityIndex = 0 31 val reagent = step.reagent 32 val trough = reagent.fetch() as Trough 33 out_job.source = trough 34 } 35 } 36 // TARGET 37 out_job.target = 38 in_jobRequest.getMicroplate(in_sample) // to facilitate backward propagation, which is external 39 tip.targetCavityIndex =
to YAMTL 40 in_jobRequest.getMicroplate_cavity(in_sample) // track how to retrieve sample from its cavity on a 41 // SET CONTAINER
microplate 42 out_job.tips.add(tip) backward_insert(microplate.name, microplateCavity, 43 ].undo[ in_sample) 44 // SET CONTAINER 45 val in_jobRequest = in_step.eContainer.eContainer as
JobRequest 46 val occurrence = getTipContainerIndex(in_jobRequest,
in_sample) 47 val out_job = in_step.fetch(occurrence) as
LiquidTransferJob out_job.tips.remove(out_tip) 23 ].undo[ 24 val in_jobRequest = in_sample.eContainer as JobRequest 25 val microplateNumber = 26 getMicroplate_number(in_jobRequest, in_sample) 27 val microplateCavity = 28 getMicroplate_cavity(in_jobRequest, in_sample) 29 val microplate = in_jobRequest 30 .fetch('microplate', 'jobRequest_->_microplate',
microplateNumber) as Microplate microplate_cavity_to_sample.get(microplate.name) .remove(microplateCavity)
1 rule('plateJobs').isAbstract.toMany 2 .inheritsFrom(#['job']) 3 .in('in_step', LAB.protocolStep).filter[ 4 (in_step instanceof Wash || 5 in_step instanceof Incubate) 6 ]
All in all, YAMTL extensions allowed a declarative specification of the mapping from high-level protocols to low-level jobs, involving one-to-many mappings. The implementation of the solution was not intrusive, as the metamodels provided did not need to be modified. In the following we will consider the evaluation criteria of the article: Understandability. The new features of YAMTL helped in specifying the transformation in a declarative way, managing traceability implicitly. Such type of specification can be challenging for model transformation languages with strong correctness criteria, where matches need to be unique: e.g. for ATL. This approach for defining the transformation should help in defining more complex behaviour in the liquid transfer robot, where additional cases are defined with additional rules.
Furthermore, such additional cases can be treated using specialized rules that reuse behaviour from existing ones.
Conciseness. The transformation rules for generating jobs reuse logic, both for matching and for initializing the resulting objects, via rule inheritance. In this case, rule inheritance helped reduce the amount of code needed Execution time. After two runs on a MacBookPro11,5 for each type of job. On the other hand, boilerplate code, "Core i7" 2.5 GHz with 16 GB RAM, the median run times generated by YAMTL, has helped reduce the amount (in milliseconds) both of the initial model transformation of code required for fetching values from the execution and of the updates are shown in Figure 12. The solutions environment from within filter conditions and output available in the main repository were used to compare our initialization actions. solution. In the preliminary results discussed at the workshop, YAMTL adds a little overhead over the reference Overhead specification. The incremental execution solution and it shows a very reasonably performance for of a YAMTL model transformation does not require ad- both for the initial transformation and for propagating ditional code. However, incrementality is only achieved updates forward. However, a more thorough inspection when executing the mapping from the high-level model on how changes are considered for the diferent tools is to the low-level model. Backward propagation requires required in order to be able to infer reliable conclusions. explicit coding, as in the rule sample_->_allocation, which remembers the allocation in an auxiliary map, external to YAMTL, and the explicit undo action is required to update this external data structure. Number of elements in the low-level model. The solution remembers the state of the samples in spe- [1] G. Hinkel, Incremental recompilation of laboratory cific cavities thanks to the internal traceability model workflows, in: Proceedings of the 14th Transformain YAMTL. When applying changes, only changes cor- tion Tool Contest (TTC@STAF), CEUR Workshop responding to samples that failed (that is, whose state Proceedings, 2021. (To Appear). are error) and new samples were applied. However, the [2] A. Boronat, Expressive and eficient model transforconsistency relation between high-level and low-level mation with an internal dsl of xtend, in: Proceedings models is a bit brittle as it relates specific positions within of the 21th ACM/IEEE International Conference on labware, implemented as list indexes. The EMF null con- MoDELS, ACM, 2018, pp. 78–88. straint does not allow to unset a specific list position, [3] A. Boronat, Incremental execution of rule-based hampering the reuse of microplate cavities. model transformation, International Journal on Software Tools for Technology Transfer 1433-2787 (2020). trans2fTohrmeaetviaolnusa,teionngihnaesinf oitciualsiszeadtioonn atnhde mstaogdeesl lionavdoilnvginhgavmeondoetl URL: https://doi.org/10.1007/s10009-020-00583-y. been considered as there were no relevant challenges in those stages. doi:10.1007/s10009-020-00583-y. Figure 1: Preliminary results in milliseconds: initial transformation (left) and all updates per model (right).