Achieving scalability in modelling and MDE involves being able to construct large models and domain-speci c languages in a systematic manner, enabling teams of modellers to construct and re ne large models in collaboration, advancing the state of the art in model querying and transformations tools so that they can cope with large models (of the scale of millions of model elements), and providing an infrastructure for e cient storage, indexing and retrieval of large models. This paper outlines how MONDO, a collaborative EC-funded project, has contributed to tackling some of these scalability-related challenges.
As MDE is increasingly applied to larger and more complex systems, the current
generation of modelling and model management technologies are being stressed
to their limits in terms of their capacity to accommodate collaborative
development, e cient management and persistence of models larger than a few hundreds
of megabytes in size. As discussed in [
This paper discusses how the MONDO project has contributed to tackling these challenges. Section 3 provides an overview of the open-source MONDO platform and then Sections 4{8 concentrate on each of the platform's major components. Section 9 outlines the ongoing evaluation process and concludes the paper. 3
The MONDO platform currently consists of the following components: 1. A framework (DSL-tao & EMF-Splitter) for automated development of scalable Eclipse-based editors for DSLs; 2. Cloud-based (CloudATL) and Reactive (ReactiveATL) versions of the widely used ATL model-to-model transformation language; 3. A new version of the VIATRA model transformation engine, built on a reactive virtual machine architecture supported by the EMF-IncQuery incremental model query framework; 4. A framework for online and o ine collaborative modelling that supports query-based access control based on IncQuery queries and VIATRA transformations; 5. A framework for indexing of heterogeneous models stored in le-based version control repositories such as Git or SVN.
These components are then integrated into ve classes of artifacts: 1. Cloud server nodes, which implement web service APIs that expose the cloud-based tools in MONDO, and may host \golden" and \front" SVN/Git repositories for collaborative modelling (further discussed in Section 8). 2. Cloud worker nodes for CloudATL-based model transformations. 3. Eclipse workbenches, which include the Eclipse-based tools from MONDO.
Some of these tools invoke the cloud API. 4. Standalone Java applications, which use the tools from MONDO as additional libraries that may invoke the cloud API. 5. Other clients, which invoke the cloud API or use the web UI of the collaboration framework.
Cloud frontends and Eclipse workbenches use OSGi to integrate the various components in MONDO in a controlled manner, ensuring they do not unintentionally interfere with each other and enabling their independent evolution beyond the lifetime of this project. The cloud API is largely implemented in Apache Thrift (thrift.apache.org), an open source library for e cient cross-language Remote Procedure Calls and serialization. Figure 1 illustrates how these components are interconnected and organised internally. In the following sections, we focus on the various ways in which the tools enable these integrations into the platform.
Hawk [
Hawk has been integrated with the rest of the platform in two ways: by integrating new components into Hawk, and by developing compatibility layers so other tools can use Hawk with minimal changes. The main components of Hawk are: { Client components: Hawk provides local and remote (Thrift-based) client components to manipulate Hawk indexes with the same UI. { Model parsers: Hawk provides components that can parse Ecore XMI, Modelio XMI/EXML, IFC2x3 STEP/XML, IFC4 STEP/XML and BPMN models. The Ecore XMI component allows Hawk to index models developed using the scalable modelling tools discussed in Section 7, produced by the transformation engine outlined in Section 5, or developed collaboratively with the framework presented in Section 8. { Version control systems: Hawk provides components for monitoring local folders, Git/SVN version control systems, workspaces and HTTP locations. The Git/SVN version control systems can be one of the \front" repositories maintained by the collaboration framework discussed in Section 8. { Backends: Hawk can use Neo4j or OrientDB for storing its graphs. Neo4j is the current market leader for graph databases, and OrientDB is another high-performance open source graph database with a more permissive license that simpli es redistribution in certain scenarios.
Hawk also provides abstractions that allow the tools developed in the other work packages to treat a local or a remote Hawk index as a standard EMFcompliant model, without requiring any changes. CloudATL can use Hawk indexes as the source of a transformation, IncQuery can query Hawk indexes and update the query results as their contents change, and DSL-tao can use a Hawk index for faster discovery and editing of cross- le references. 5
CloudATL is an extended version of the ATL transformation engine that can distribute large-scale transformations over a Hadoop cluster of worker nodes. The engine operates in a transparent manner to the user, who does not need to have any knowledge about distributed programming. The engine shares input data among the slaves, except for the the input model that is equally divided among them. The engine also relies on the MapReduce communication protocols to aggregate the intermediate results and provide the nal output model.
CloudATL can be submitted to a standard Hadoop cluster through the Hadoop APIs as well as through standalone console-based tools. The latter use a service-oriented API that turns a MONDO server into a frontend for a dedicated Hadoop cluster. The API provides services to launch, monitor and stop CloudATL transformations on the Hadoop cluster, and CloudATL jobs can read remote Hawk indexes through the EMF resource abstractions mentioned above. For validation and experimentation purposes, a virtualized Hadoop cluster (based on Docker1) has been made publicly available, and automated scripts for launching, stopping, and resizing the cluster are provided2.
Initial versions of CloudATL ran into scalability limitations of the
standard XMI model serialization format, especially the lack of support for
concurrent reads and writes. To solve this issue, a decentralized persistence backend
(NeoEMF/HBase [
NeoEMF/HBase o ers lightweight on-demand loading and e cient garbage collection. Model changes are automatically re ected in the underlying storage, making changes visible to all the clients. In the case of CloudATL, it guarantees that only the model elements needed to perform the partial transformation are loaded by each CloudATL worker. Moreover, NeoEMF/HBase has explicit support for ACID properties. This allows the use of NeoEMF/HBase as a scalable persistence backend for distributed and concurrent model transformations.
ReactiveATL3 is a reactive engine for the ATL transformation language. It is designed to perform on-demand transformation in model-driven applications, by activating only the strictly needed computation in response to updates or requests of model elements. Computation is updated when necessary, in an autonomous and optimized way by using incrementality and lazy evaluation. 6
VIATRA [
Incremental queries enable to uniformly handle elementary model changes
(e.g. change of an attribute) and aggregate model changes (e.g. disappearance of
a match in the query result set) as atomic events. To identify speci c sequences of
such events, BME introduced complex event processing to model transformations
in VIATRA [
Reactive model transformations [
MONDO partners collaborated to allow IncQuery to query Hawk indexes directly, and through the EMF resource abstractions of Hawk. An optimised
version of IncQuery was developed that can load only the part of the model that
is required. It accesses all instances of a type using direct edge traversal instead
of having to iterate through the entire model. The optimised version also passes
the Train Benchmark [
DSL-tao [
Of special relevance for scalability are infrastructure patterns related to DSL modularity. While modularity mechanisms have been developed and are well supported for programming languages, this is not so for DSLs. In order to obtain more scalable DSL modelling tools, DSL-tao includes a modularity pattern which permits customising model fragmentation strategies for DSLs. Hence, similar to how programming languages organize software projects, one can identify the meta-model elements playing the roles of project, package and unit. The generated modelling environment will then fragment and physically organize models into packages (folders) and units ( les) of smaller size, so that they can be more e ciently managed. As an example, Figure 2 shows a modelling environment that instantiates the modularity pattern for wind-turbine models, enabling the de nition of each subsystem in a di erent package, and its constituent components and state machines as les inside the package.
The modularity pattern is realised by EMFSplitter [
The user only needs to open the Eclipse preferences page produced by EMFSplitter and select Hawk to nd candidate values for non-containment references (window labelled 1 in Figure 3). The next time that the EMFSplitter dialog for editing a non-containment reference is opened, the Hawk+EMFSplitter integration will create a Hawk index (backed by OrientDB) that monitors over all the models in the workspace, and will use it to e ciently nd the candidate values. The same index will be reused on later queries, and will be kept up to date automatically by Hawk. 1 2
The dialog for editing non-containment references is the same whether Hawk is used or not (window labelled 2 in Figure 3). Depending on the scope (workspace, project, package, or unit), Hawk will be used in a di erent way. In a rst stage, Hawk will be queried to either nd all the instances of the EType of the reference across the workspace, or nd only the instances that are contained within a speci c project, package, or unit. In the second (and optional) stage, the retrieved instances can be ltered by project nature.
The MONDO Collaboration Framework provides support for both online and o ine collaborative modeling scenarios by using graph queries for de ning negrained model-level access control policies and locks, and bidirectional model transformations to derive ltered views for each collaborator and to propagate changes introduced in these views back to a server. The transformation uniformly enforces high-level ne-grained access control policies during the derivation of views and the back-propagation of changes.
In an o ine collaboration scenario (see Fig. 4), models are stored in a gold repository of an o -the-shelf version control system (VCS) that is not directly accessible by end users. Instead, the VCS server instead hosts a separate front repository dedicated to each collaborator (group) that contains a copy of the gold repository ltered according to read access privileges of that user (using the get operation of the bidirectional transformation). This front repository of the user, which contains complete version history, can also be used to commit his changes. Then the MONDO Collaboration Framework tries to propagate these changes to the gold repository (by a putback operation), but these changes may be denied based on write permissions.
In an online collaboration scenario, users can connect via their web browsers to open, view and edit models stored in the backends in live sessions. Multiple users can collaborate on the same model simultaneously along the same access control policy (used for o ine collaboration as well). The editor is provided as an Eclipse RAP-based web application, which can optionally be co-deployed with other server-side components. The main conceptual di erence with the o ine case is that the live detection of violating access control policies by incremental queries. At the end of an online collaborative session, the result is persisted back in the VCS as in the o ine case.
Con icting model changes need to be resolved by collaborators prior to a
successful commit, which is carried out by search-based automated model merge
[
In this paper we provided an outline of important scalability challenges in the context of MDE, and MONDO's technical contributions for addressing them. MONDO has already contributed novel techniques and several prototype implementations in all four identi ed key-areas. Currently, the research contributions of MONDO are under assessment in the context of four industrial case studies.
The rst case study (provided by UNINOVA4) comes from the construction domain and involves collaborative development and automated management of large computer-aided design (CAD) models of buildings. The second case study (provided by Soft-Maint5) involves exploration and automated regeneration of code from large models, which have been reverse-engineered from existing legacy codebases. The third case study (provided by IKERLAN6) involves multi-device collaborative development of models for o shore wind power generators, and the fourth case study involves managing large collections of UML models captured in a proprietary format supported by Softeam's7 Modelio8 tool. To assess the usefulness and impact of the technologies produced by MONDO, industrial partners speci ed a set of concrete requirements and measures with reference to existing state-of-the-art technologies during the rst six months of the project. We intend to present the evaluation results in a follow-up publication as soon as the evaluation reports have been produced by the respective MONDO partners.