This paper presents a solution to the Incremental Class to Relational Case at the TTC 2023 using the .NET Modeling Framework (NMF), using either plain C# or NMF Synchronizations. This solution is able to derive an incremental change propagation entirely in an implicit manner.
engineering, incremental model analyses and
incremental model transformations. In particular, NMF
ExpresModels are formally defined abstractions of concepts or sions [5] is an incrementalization system able to
increphysical objects and as properties of these concepts or mentalize arbitrary function expressions and NMF
Synobjects change, so does the model. However, if other arti- chronizations [6, 7] is an incremental model
transforfacts have been derived from the model, it is often impor- mation approach. Using both tools in combination, it is
tant to keep these derived artifacts up-to-date. Because possible to solve the incremental class to relational case
recreating these artifacts from scratch takes a significant in a declarative manner such that the required change
amount of time and destroys references to individual propagations can be derived implicitly.
model elements, it is often necessary to propagate the The remainder of this paper is structured as follows:
changes. A common assumption is that implementing Section 2 gives a brief overview how NMF Expressions
this change propagation manually is cumbersome, error- and NMF Synchronizations work. Section 3 explains
prone and verbose. Model transformation languages have the actual solutions. Section 4 discusses results from
often claimed to have a superior support for change prop- the benchmark framework and Section 5 concludes the
agation [
To assess the amount of code savings possible by implicit change propagation, the Transformation Tool Contest2 2023 hosts a case for incremental transformation of class diagram models to relational database schema models. This paper presents a solution to this case using the .NET Modeling Framework (NMF) [4].
NMF is a framework built for support of model-driven
NMF Expressions [5] is an incrementalization system
integrated into the C# language. It takes expressions of
functions and automatically and implicitly derives an
incremental change propagation algorithm. This works by
setting up a dynamic dependency graph that keeps track
of the models state and adapt when necessary. The
incrementalization system is extensible and supports large
TTC’23: 15th Transformation Tool Contest, Part of the Software parts of the Standard Query Operators (SQO3).
cToenchfenroelnocgeise,sE:dAsp.pAli.cBaotiroonnsata,nAd. FGoaurncdíaa-tDioonms í(nSgTuAeFz), faendderGa.teHdinkel, 20 NMF Synchronizations is a model synchronization
apJuly 2023, Leicester, UK. proach based on the algebraic theory of synchronization
$ georg.hinkel@hs-rm.de (G. Hinkel) blocks. Synchronization blocks are a formal tool to run
https://www.hs-rm.de/de/hochschule/personen/hinkel-georg model transformations in an incremental (and
bidirec(G. Hinkel) tional) way [7]. They combine a slightly modified notion
0000-0©002022-36C4op6y2rig-h5t2fo0r 8thi(sGpa.peHrbiynitks eault)hors. Use permitted under Creative of lenses [8] with incrementalization systems. Model
CPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g CCoEmmUoRns LWiceonsrekAstthribouptionP4r.0oIncteerenadtiionnagl s(CC(CBYE4U.0)R.-WS.org) properties and methods are considered morphisms
be1For model queries, a comprehensive comparison of general- tween objects of a category that are set-theoretic products
purpose implementations and incremental query technology is
available [
A (well-behaved) in-model lens : ˓→ between types and consists of a side-efect free Get morphism ↗∈ (, ) (that does not change the global state) and a morphism ↘∈ ( × , ) called the Put function that satisfy the following conditions for all ∈ , ∈ and ∈ Ω :
↘ (, ↗ ()) = (, ) ↗ ( ↘ (, , )) = (, ˜) for some ˜ ∈ Ω .
PutGet law. Meanwhile, the second line is a bit weaker than the original GetPut because the global state may have changed. In particular, we allow the Put function to change the global state.
A (single-valued) synchronization block is an 8tuple (, , , , Φ − , Φ − , , ) that declares a synchronization action given a pair (, ) ∈ Φ − : ∼= of corresponding elements in a base isomorphism Φ − . For each such tuple in states (, ), the synchronization block specifies that the elements ( (, ), ↗ (, )) ∈ × gained by the lenses and are isomorphic with respect to Φ − . Φ− Φ−
changes to the input model should be propagated to the output model, also vice versa or not at all. • Synchronization: A client may execute the transformation in synchronization mode between a left and a right model. In that case, the engine ifnds diferences between the models and handles them according to the given strategy (only add missing elements to either side, also delete superlfuous elements on the other side or full duplex synchronization).
cation of a transformation in a broad range of diferent use cases. Furthermore, the fact that NMF Synchronizations is an internal language means that a wide range of advantages from mainstream languages, most notably modularity and tool support, can be inherited [9].
Based on this formal notion of synchronization blocks and in-model lenses, one can prove that model synchronizations built with well-behaved in-model lenses are correct and hippocratic [7], which means that updates of either model can be propagated to the other model such that the consistency relationships are restored and an update to an already consistent model does not perform any changes.
mark framework, discussed in Section 3.1. Because NMF is used in both solutions for the model serialization and deserialization, both solutions presented in this paper are afected. Afterwards, Section 3.2 explains the solution in plain C# and Section 3.3 the solution using NMF Synchronizations.
depicted in Figure 1. The usage of lenses allows these declarations to be enforced automatically and in both directions, if required. The engine computes the value that the right selector should have and enforces it using the Put operation. Similarly, a multi-valued synchronization block is a synchronization block where the lenses and are typed with collections of and , for example : ˓→ * and : ˓→ * where stars denote Kleene closures.
Synchronization blocks have been implemented in NMF Synchronizations, an internal DSL hosted by C# [6, 7]. For the incrementalization, it uses the extensible incrementalization system NMF Expressions. This DSL is able to lift the specification of a model transformation/synchronization in three orthogonal dimensions:
Transformation Zoo tend to be ignorant to the fact that the URI of a metamodel should be a URI. Unfortunately, this applies to both source and target metamodel in the case. NMF translates URI fields of Ecore metamodels to namespace URIs and is strict in that these are actually valid URIs. References that do not begin with a URI are resolved as file references and thus must fail. Therefore, additional helpers are necessary in order to help NMF resolve these changes. Furthermore, the fact that new • Direction: A client may choose between trans- classes have been added to the NMF changes metamodel formation from left to right, right to left or check- is a problem for NMF, because the (unchanged) metaonly mode. model is an integral part of NMF and once NMF loads a metamodel, it freezes it in order to prevent any changes. • Change Propagation: A client may choose whetherTherefore, it was necessary to force the code generation 3.1. Shortcomings of the benchmark framework for the changed metamodel (NMF normally does not gen- rules of the programming language, in this case C#, such erate code for metamodels for which code already exists), that the virtual machine can apply these at runtime based delete everything that is already part of NMF and make on the actual type of the object. This indirection is necsome code adjustments in the generated code. For NMF, essary because .NET in general supports many more the added classes are not necessary, because NMeta does programming languages than just C#. have an explicit class Model to represent a resource. This implementation, however, has the disadvantage that all the rules to implement the transformation of the 3.2. Plain C# with dynamic language actual model elements have to be done in methods called runtime Tifrtahnesrefoisrma mthoadtetlakelee mexeancttltyhaotnies anrogtucmoevnetr.edFubyrt htheer, existing Transform methods, this yields an exception at runtime.
Model navigation in plain C# is also very convenient since C# has a sub-language to specify queries. Using this sub-language, queries can be specified very similar to SQL but are being type-checked by the compiler and IDE.
In particular, the query used to obtain the multi-valued attributes found in the model is depicted in Listing 2.
Our first solution uses plain C#, using the dynamic language runtime, plus NMF for model serialization and deserialization. The dynamic language runtime is a language feature that allows C# programs to quit the static type system in select places. This is an advanced feature that helps to implement functionality such as trace links with few code lines at the expense of losing a lot of type system benefits that come with a static type system. It is activated by using the type dynamic. If a variable has1 the static type dynamic, the compiler does not resolve2 any method calls but emits code that will select the actua3l
4 method to invoke for a method call at runtime. This is a rather uncommon language feature used in situations where a static type system is not particularly nice to work with.
We found that one of these situations is the implementation of trace links, because we do not want to keep track of trace links in multiple hashtables just in order to have correct trace links and sometimes, the actual type of elements does not even matter. An example of the latter is that the translation of any root element should appear as a root element in the result model, regardless of what type it is. However, the translation of data types into types in the relational model typically has to be aware that the translation of a data type is a type and therefore can be used as a type of a column.
Breaking out of the static type system allows a very convenient implementation of a trace functionality in C# in just a few lines of code, as depicted in Listing 1. from cl in classModel.RootElements.OfType<IClass>() from att in cl.Attr where att.MultiValued select att
members of classes that are root elements of the model and ignore attributes found elsewhere in the hierarchy. However, the API that NMF generates for models does also include a Descendants operation to iterate over all descending model elements, for example starting from the model itself (which is also a model element in NMF).
Unfortunately, there is no equivalent trick to implement change propagation in plain C#. Of course, it would be possible to combine a manual tracing implementation with NMF Expressions for change propagation (for instance, to obtain changes for query results), but this would no longer count reasonable as plain C# and hence, we refrain from such an implementation4. For the change propagation, NMF fortunately ofers events for changes of all properties such that manual change propagation can be implemented by a simple event handler as depicted in Listing 3. 1 private Dictionary<object, IModelElement> _trace = new Dictionary< object, IModelElement>(); 2 private object TraceOrTransform(object item) 3 { 4 if (!_trace.TryGetValue(item, out var transformed)) 5 { 6 transformed = Transform((dynamic)item); 7 _trace.Add(item, transformed); 8 } 9 return transformed; 10 }
System.Object, equivalent to java.lang.Object in Java. dynamic, however, aliases to a diferent type that essentially captures both the runtime object reference and an object that encapsulates the overload resolution 1 var type = new Type 2 { 3 Name = dataType.Name 4 }; 5 dataType.NameChanged += (o, e) => type.Name = dataType.Name;
The last line of Listing 3 is an event registration, for which C# uses the += operator. Events in .NET are language implementations of the observer pattern: they 4In fact, it is already questionable whether using the dynamic language runtime already counts as not plain C# since it is a rather advanced language feature, but it ships with the default .NET SDK and it is available on all platforms. form a callback mechanism for which only registration (+=) or deregistration (-=) are allowed for callers outside the class that defines the event. Callers inside the class also have the option to raise the event which means that the registered callback methods are actually called.
The trouble starts when more dependencies are at play such as when calculating the name of the table created for a multi-valued attribute, which is calculated both from the name of the attribute and the name of the class that defined the attribute. The implementation of this change propagation is depicted in Listing 4. NMF allows to infer the change propagation rules implicitly and also has builtin support for traces. As sketched in Section 2, the idea is to structure a model transformation through isomorphisms that define pairs of model elements that correspond to each other. In the case of the classes to relational transformation, there are five such isomorphisms: • The entire class model corresponds to the entire relational model. • A class corresponds to a table. • A data type corresponds to a type. • An attribute corresponds to a column. • An attribute corresponds to a table, but only if it is multi-valued.
inherit from the generic class SynchronizationRule.
The next step is to describe these isomorphisms in terms of other isomorphisms and the identity of simple types.
For the correspondence of the entire models, this means the following: • All root elements that are data types should correspond to the root elements that are types, given the isomorphism of data types and types.
Here, we define a local method for an update routine • All root elements that are classes should correand then register and deregister this update routine dy- spond to the root elements that are tables, given namically when required. The problem here is that it is the isomorphism of classes and tables. very easy to forget to add or remove these change handlers here and thus very easy to either miss important • All multivalued attributes of classes should correupdates or to cause a memory leak. spond to the root elements that are tables, given
Manually implementing change propagation gets a lot the isomorphism of attributes and tables. worse when collections start entering the field. Because the changes that can occur on collections are more di- Note that we have two synchronization rules that tarverse, the code to handle these changes also gets a lot get all root elements that are tables. This works, because more complex and it becomes even easier to miss impor- the synchronization is only executed in one direction tant kinds of changes or cause memory leaks. The worst and we use a relaxed synchronization mode in which situation is when more complex navigation patterns are NMF does not enforce that every model element has a used, such as the query for multi-valued attributes that counterpart. needs to fetch all classes and from there return all at- From these descriptions, the last is certainly the most tributes that have the Multivalued property set to true. interesting. Its implementation is therefore depicted in
The imperative notion of the plain C# solution, how- Listing 5. ever, makes it easy to implement rather imperative as1- SynchronizeManyLeftToRightOnly(SyncRule<AttributeToTable>(), pects of the transformation. For instance, the fact that al2l m => from c in m.RootElements.OfType<IClass>() primary keys and foreign keys are to use an integer type34 wfhreormeaai.nMulct.AitVatlrued that is also the translation of the integer data type of the5 select a, input model are easy to implement. In the solution, we6 rels => rels.RootElements.OfType<IModelElement, ITable>()); statically keep a reference to the integer type in order to Listing 5: Synchronizing multi-valued attributes use it everywhere in the model transformation. 1 SynchronizeLeftToRightOnly(a => a.Owner.Name + "_" + a.Name, t =>
t.Name); 2 SynchronizeLeftToRightOnly(a => a.Owner.Name.ToCamelCase() + "Id",
t => t.Col[0].Name);
Note that the query used in Listing 5 is exactly the in a very concise manner, even if that means that certain same as the one in Listing 2, but because of a one-line type system guarantees are essentially lost. configuration at the top of the file, the compiler does not This way of implementing a trace through the dynamic resolve the syntax to the .NET builtin query operators language runtime has an important downside, though, but to the query operators in NMF. For these, NMF can and that is the lack of modular extensibility. Whereas create a dynamic dependency graph that tracks changes model transformation languages typically allow to exof the underlying models [5]. The key advantage here tend the set of model transformation rules through some is that NMF Synchronizations can infer when the query notion of extensions, this is not possible if the transformaexpression in Lines 3 to 6 in Listing 5 changes and there- tion method is resolved through the dynamic language fore, the developer does not have to specify any change runtime as in the plain C# solution. This requirement propagation implementation. is rare for toy transformations such as the transforma
Because the target isomorphism can also be the iden- tion given here, but it may be important for practical tity on any given type, it is also possible to specify syn- transformations that are typically a lot more complex. chronizations of simple attributes. For comparison, List- The trouble for the C# solution starts only when the ing 6 depicts the code necessary to synchronize the names input models are changed and these changes are to be of tables generated for multi-valued attributes, the equiv- propagated to the target model. If the use case requires alent of Listing 4. that the changes are propagated instead of rerunning the transformation, syntactic sweets of the programming language do not really help. Instead, one has to manually register and unregister to change events and handle these events appropriately. This requires dedicated supListing 6: Synchronizing the name of an attribute-table port for each type of change, which is cumbersome to and its first column implement. In its current form, the plain C# solution is not complete, meaning that by far not all changes are
Implementing more or less static references to the in- actually propagated. teger type is a bit more dificult in NMF Synchroniza- Developing the incremental version using NMF Syntions. Because NMF Synchronizations has a considerable chronizations is a diferent story. Here, the internal DSL initialization efort on the synchronization as dynamic forces the developer to think in terms of isomorphisms dependency graph templates are constructed for all of and synchronization blocks, but then the change propathe synchronization blocks and compiled for use without gation comes essentially for free, i.e., the developer does change propagation, it is not recommended to just add a not have to implement anything. ifeld to the synchronization class. Furthermore, because Consequently, whereas a large proportion of the plain unlike Java, nested classes in C# never have access to C# solution is responsible for change propagation, the an instance of the container class (in Java terms, nested NMF Synchronizations solution does not require any classes are always static), making it syntactically a lot code explicitly for change propagation, essentially bemore dificult to access these fields. However, this would cause the transformation only relies on rather simple mean to give up the thread-safety of NMF Synchroniza- model navigation queries that NMF has built-in support tions, which is also not what we want (even though not for. In particular, the query that the plain C# solution exactly required in this case). Rather, we use the syn- uses to find all the multi-valued attributes in order to chronization context data key/value container to store generate corresponding tables is the same both in the variables required during the transformation. However, plain C# solution and in the NMF Synchronizations soluthis container is unfortunately not type-safe. NMF Syn- tion. However, the diference is that whereas the plain chronizations also allows to use the dynamic language C# solution uses the .NET built-in query operators that runtime to hide the string constant, but this turns out to only execute the query in memory, the same query maps be slow. in the NMF Synchronizations solution to NMF query operators that can obtain a dynamic dependency graph that 4. Evaluation is used to attach listeners to the notification API of the models in order to update the query result as the model Creating the plain C# solution started very easy. The ini- changes. tializer syntax makes it very easy to transform elements Because a model synchronization in NMF Synchronizainto other models with a minimum of boilerplate code. tions is nothing else than a .NET class, the code required Rather, the code is a very concise notion of how to turn to set up the transformation is also rather small. Beobjects of one metamodel into objects of another. The cause in principle, NMF Synchronizations can work in ability of C# to selectively switch of the static type sys- both directions, we need to specify the direction when tem also allows support for polymorphism and tracing starting the synchronization. Therefore, it is required to have a separate variable that is then given to the syn- [3] G. Hinkel, Implicit Incremental Model Analyses chronization by reference, in C# denoted with the ref and Transformations, Ph.D. thesis, Karlsruhe Inkeyword. We may add an API in the future to have dedi- stitute of Technology, Germany, 2018. URL: https: cated support for one-way transformation to get rid of //publikationen.bibliothek.kit.edu/1000084464. this boilerplate. [4] G. Hinkel, Nmf: A multi-platform modeling framework, in: A. Rensink, J. Sánchez Cuadrado (Eds.), Theory and Practice of Model Transforma5. Conclusion tion, Springer International Publishing, Cham, 2018, pp. 184–194.
There is an ongoing debate on what claims of model trans- [5] G. Hinkel, R. Heinrich, R. Reussner, An extensible formation languages are justified and which of them can approach to implicit incremental model analyses, be backed by empirical evidence. I think that this TTC Software & Systems Modeling (2019). URL: https:// case is a good step in this direction. In my opinion, the doi.org/10.1007/s10270-019-00719-y. doi:10.1007/ plain C# solution shows that often called arguments that model transformation languages simplify model traver- [6] sG1.0H2in7k0e-l,0C19h-an0g0e7P1r9o-pya.gation in an Internal Model sal and tracing are problematic as very good support Transformation Language, in: D. Kolovos, M. Wimfor these tasks can also be found in general-purpose pro- mer (Eds.), Theory and Practice of Model Transgramming languages such as C#, which is one of the most formations: 8th International Conference, ICMT used programming languages in the world. The most 2015, Held as Part of STAF 2015, L’Aquila, Italy, important consequence of this is that because the pro- July 20-21, 2015. Proceedings, Springer Internagramming language applies to essentially any problem tional Publishing, Cham, 2015, pp. 3–17. URL: http: one could think of, developers using C# use it practically //dx.doi.org/10.1007/978-3-319-21155-8_1. doi:10. every day, whereas a model transformation language is typically limited to model transformations and hence, [7] 1G0. 0H7i/n9k7e8l,-E3.-3B1u9rg-e2r1,15C5h-a8n_g1e. propagation and developers need to switch. However, switching program- bidirectionality in internal transformation dsls, ming languages is what many developers do not like and Softw. Syst. Model. 18 (2019) 249–278. URL: https: thus, model transformation languages that only provide //doi.org/10.1007/s10270-017-0617-6. doi:10.1007/ advantages in these areas have a limited potential for adoption. [8] Js.1N0.2F7o0st-e0r,1M7.-B0.6G1r7e-en6w.ald, J. T. Moore, B. C. Pierce,
In contrast, the plain C# solution also shows that imple- A. Schmitt, Combinators for bidirectional tree transmenting change propagation manually is a diferent story, formations: A linguistic approach to the view-update as it is easy to forget changes that need to be propagated. problem, ACM Transactions on Programming The explicit implementation of the change propagation Languages and Systems (TOPLAS) 29 (2007). URL: through the standard notification API of the .NET plat- http://doi.acm.org/10.1145/1232420.1232424. doi:10. form is dificult to implement, error-prone and inhibits the readability of the transformation. In this area, model [9] 1G1. 4H5i/n1k2e3l,24T2.0G.o1l2d3sc2h4m24id.t, E. Burger, R. Reusstransformation languages that can infer change propa- ner, Using Internal Domain-Specific Languages gation implicitly, such as NMF Synchronizations, have to Inherit Tool Support and Modularity for Model a much clearer value proposition compared to general- Transformations, Software & Systems Modelpurpose programming languages. ing (2017) 1–27. URL: http://rdcu.be/oTED. doi:10. 1007/s10270-017-0578-9.