This paper outlines the Fulib [1, 2] solution to the Laboratory Workflow Case of the Transformation Tool Contest 2021 [3]. Our analysis of the use case showed that it provides quite a number of diferent model elements that require individual treatment but the diferent cases are relatively simple. However, some parts of the predefined Figure 1: Design EMF metamodels do not work very well with the Fulib modeling approach. For example, the predefined metamodel uses index numbers to identify the tips of a liquid very light weight implementation of our model in Java transfer job and these index numbers need to be mapped code and Fulib generates a number of dedicated Table to the barcodes of the samples that are transported by a classes that enable eficient OCL [ 4] like queries. The tip. Similarly, samples need to be mapped to cavities on actual model transformations are coded in Java against micro-plates. Thus, we took the liberty to adapt the given the generated model API. Thus, the Fulib solution has no metamodels by adding explicit associations between sam- initialization phase. ples and some labware elements, cf. Fig. 1. Note, these The various input models that describe a JobRequest, adaptions connect elements from the source and from the its Assay and the target Samples are given as EMF/XMI target metamodel of our model to model transformation. files (*/initial.xmi). We load the initial model with a For these adaptions, we loaded and combined the two generic XML parser and a DOM tree visitor, that builds given Ecore metamodels of the use case into the Fulib the model based on our light weight model implementacode generator and then did some manual modifications tion. using Fulibs metamodeling API. Due to a misinterpretation, we also changed the cardinality of the previous-next association for Jobs from many-to-many to one-to-one. 3. Creating the initial We felt this meets the semantics of the use case, too, JobCollection and it resulted in a somewhat simpler model that can be processed easier and faster. The rest of the paper outlines the implementation of the diferent model processing steps and we conclude with some measurements.
Once the JobRequest and Assay are loaded, we use an AssayToJobs visitor [5] to generate the initial JobCollection, cf. Listing 1.Using the visitor pattern allows for a nice separation of model queries that look up elements and of transformation rules that do the actual 2. Initialization and Loading operations.
The initial method of our AssayToJobs visitor The initialization phase allows to load the metamodels first creates the target JobCollection (cf. line 6 of and transformations. In our approach, Fulib generates a Listing 1). Then it iterates through the samples, reagents, and assay steps and calls appropriate assign rules (cf. lines 7 to 14).
The assignToTube rule checks, whether we have a TubeRunner that still has place for the new sample (cf. line 20). If not, a new TubeRunner is created (cf. line 22 to 26) and added to the JobCollection (cf. line 25).
Then, the sample’s barcode is added to the TubeRunner 1 public class AssayToJobs { 2 private JobCollection jc; 3 private JobRequest jr; 4 public JobCollection initial(JobRequest jr) { 5 this.jr = jr; 6 jc = new JobCollection(); 7 jr.getSamples() 8 .forEach(this::assignToTube); 9 jr.getSamples() 10 .forEach(this::assignToPlate); 11 jr.getAssay().getReagents() 12 .forEach(this::assignToTrough); 13 jr.getAssay().getSteps() 14 .forEach(this::assignJob); 15 return jc; 16 } 17 TubeRunner tube = null; 18 int tn = 1; 19 private void assignToTube(Sample sample) { 20 if (tube == null || 21 tube.getBarcodes().size() == 16) { 22 tube = new TubeRunner(); 23 tube 24 .setName(String.format("Tube%02d", tn)) 25 .setJobCollection(jc); 26 tn++; 27 } 28 tube.withBarcodes(sample.getSampleID()); 29 tube.withSamples(sample); 30 } 31 ... 54). Then, lines 56 to 66 create the corresponding 62 TipLiquidTransfer and initialize the corre- 6634 sponding attributes.
TipLiquidTransfer to its sample for easy reference. 66 67 The remaining stepAssignRuleys work similar. 68 {
DistributeSample distributeSample =
(DistributeSample) protocolStep; if (liquidTransferJob == null || liquidTransferJob.getTips().size() == 8) { liquidTransferJob =
new LiquidTransferJob(); liquidTransferJob .setProtocolStepName(
protocolStep.getId()) .setState("Planned") .setJobCollection(jobCollection) .setPrevious(lastJob); lastJob = liquidTransferJob; liquidTransferJob .setSource(sample.getTube()) .setTarget(sample.getPlate()); } TipLiquidTransfer tip =
new TipLiquidTransfer(); tip.setSourceCavityIndex (sample.getTube()
.getSamples().indexOf(sample)) .setVolume(distributeSample.getVolume()) .setTargetCavityIndex(sample.getPlate()
.getSamples().indexOf(sample)) .setStatus("Planned") .setJob(liquidTransferJob) .setSample(sample); 4. Reading Changes to Job Updates are described by text lines in predefined files. 35 Then, line 30 expands our table to Jobs attached to the 36 Microplates, i.e. we get rows for all possible triples 3387 of JobCollection, Microplate, and attached Jobs. 4390
Jobs with the right stepName. For 41 42 each resulting row, line 32 assigns the new state to the 43 44 corresponding Job. 45 46 Note, our JobCollectionTable query could also be 47 48 expressed e.g. using the Java streams API. While using 49 50 the Java stream API is quite comparable, the Java stream 51 API requires some more steps and some extra operations 5532 like flatMap and probably some extra type casts. Thus, 54 55 we prefer our FulibTables as we consider FulibTables 56 57 queries to be more concise. 58 59 Updates with dedicated new states for each sample 60 61 are handled by rule updateSamplesAndTip (cf. line 62 63 21 and 34 to 42). Rule updateSamplesAndTip uses a 64 65 FulibTables query to look up all samples attached to some 66 Microplate attached to our JobCollection. For each 6687 sample we call rule updateOneSampleAndTip (cf. line 6790 40 and line 43 to 65). Rule updateOneSampleAndTip 71 72 ifrst retrieves the result state for the current sample (cf. 73 74 lines 45 to 47) and updates the sample on failure (cf. line 75 76 50). Then the FulibTables query of lines 52 to 56 retrieves 77 78 the tip that handles the current sample within the current 79 80 stepName. Lines 57 to 65 then update the state of the 81 tip and its job.
Once the updates are propagated, the FulibTa- 8845 bles query of lines 9 to 12 of Listing 3 iter- 86 87 ates through all jobs that are still Planned and 88 89 applies rule removeOsoleteJob to them. Rule 90 91 removeObsoleteJob calls isObsolete to check, 92 93 whether the job can be removed (cf. line 70 and lines 94 95 79 to 96) and and in that case it does a classical removal 96 from a doubly linked list.
To be honest, our removal of obsolete jobs iterates 99 } through all jobs and thus it is not really incremental. 6. Conclusions This could be improved by collecting afected jobs during state changes and by investigating only afected jobs.
However, due to the low number of jobs in the example cases, we do not believe that such a caching mechanism pulls it weight and thus we did go for conciseness.
In TTC 2020 the Fulib solution used transformation code working directly, with EMF based models [6]. That solution was very slow. This, year we use the Fulib generated model implementation. As Table 1 shows, the Fulib implementation uses an average of 16 megabytes of memory to handle a case while e.g. the reference solution requires an average of 46 megabytes. We believe that this reduction of memory consumption is a result of the more space eficient model implementation provided by Fulib.
Similarly, the Fulib solution seems to be quite fast: to run all phases of the test minimal case and of all scale_samples cases and of all scale_assay cases on a laptop with Intel Core i7 CPU 3.10GHz and 16 GB RAM we use a total time of about 2 seconds, the Reference solution uses about 5.2 seconds and the NMF solution coming from the central GitHub repository uses about 76 seconds. To us it seems that EMF is a performance bottleneck.
total time (millisec)
avg. memory (mb)
Overall, the TTC 2021 Laboratory Workflow Case has reasonably simple queries and rules but it also has quite a number of diferent cases like diferent kinds of Jobs and diferent kinds of Labware that all need special treatment. The Fulib solution addresses these diferent cases using maps of rules where appropriate rules are retrieved e.g. by the types of current objects. This allows to iterate through all tasks, very conveniently. For queries, our solution uses FulibTables, which are quite similar to Java Streams or to OCL expressions. For the actual transformations, we use plain Java code working directly on the Java implementation of our model(s). Altogether, we consider our solution as easy to read and as quite concise, the whole update transformation needs roughly 90 lines of Java code.
The TTC 2021 Laboratory workflow Case is using a lot of EMF. EMF is the de-facto standard for model exchange these days. However, as we have discussed compared to our implementation, EMF has some serious performance problems. Using our own implementation (i.e. the Fulib code generator) provides us with a more eficient model implementation, however, we have to implement EMF readers and writers ourselves and we still fight some compatibility issues. These compatibility issues are another reason for our integration into the Python based test framework: we need to write EMF files that are not just EMF compatible but that are accepted by the test framework. We will improve our performance on EMF compatibility for next years TTC.