=Paper=
{{Paper
|id=Vol-2361/short15
|storemode=property
|title=Continuous
Migration of Mass Customized
Applications
|pdfUrl=https://ceur-ws.org/Vol-2361/short15.pdf
|volume=Vol-2361
|authors=Michiel
Overeem,Slinger Jansen
|dblpUrl=https://dblp.org/rec/conf/benevol/OvereemJ18
}}
==Continuous
Migration of Mass Customized
Applications==
https://ceur-ws.org/Vol-2361/short15.pdf
Continuous Migration
of Mass Customized Applications
Michiel Overeem Slinger Jansen
dept. Product Development dept. Information and Computing Sciences
AFAS Software Utrecht University
Leusden, The Netherlands Utrecht, The Netherlands
michiel.overeem@afas.nl slinger.jansen@uu.nl
Abstract—Model-driven engineering is a known approach for by Tolvanen and Kelly [2] and Paige and Varró [3], discuss the
increasing both the quality and productivity of software develop- challenges and the effort it takes to create MDE tools. Clark
ment. It also enables business analysts to take a more active role and Muller [4] report on two startups around MDE tools and
in the development process. The model allows the application to
be tailored to the customer, who no longer needs to adjust to the lessons learned from those startups.
the software. Through the model it is possible to mass customize Our research focuses on MDE for enterprise software appli-
the developed software applications. Mass customization allows cations (ESAs). Fowler [5] states “Enterprise applications are
software producing organizations to efficiently produce and about the display, manipulation, and storage of large amounts of
maintain multiple similar software products. Through model- often complex data and the support or automation of business
driven engineering the similarities between these products can
be exploited while the variations are managed through models. processes with that data.” According to Gartner [6] examples of
The increase in quality and productivity, and the mass ESAs are CRM and ERP software. The characteristics of ESAs
customization is achieved by raising the level of abstraction are distinct from for instance games and result in different
on which software is developed. Changes done on the higher requirements for application migration.
level of abstraction should be translated into changes on the We believe that the model-centric approach of MDE is
resulting application. However, in many model-driven systems the
migration of the resulting application towards the new intended especially promising for ESAs. Brown [7] defines the model-
state is still a (partially) manual and labor-intensive step. This centric approach as “the system models have sufficient detail
manual step requires engineers to work on a lower level of to enable the generation of a full system implementation from
abstraction, and thus forgoes on the increase of quality and the models themselves". The first promise is the increase
productivity. of productivity and quality. ESAs often contain repetitive
In this research we study model-driven engineering environ-
ments in which the application is generated from a model.
patterns of functionality, such as the maintenance of data.
Whenever models or parts of these model-driven engineering MDE increases the productivity by generating the software for
environments evolve, the goal is to automatically migrate the every instance of these patterns. Not only can MDE increase
application and data as well. In order to reach this goal, we the productivity by deriving these components from a model,
formulate three solution approaches. First of all a categorization it also increases the quality of those components by doing it
of migration triggers that can occur is discussed, to increase the
understanding of the context of the migration. Migration triggers
consistently.
can be handled with different migration strategies. Secondly, we The second promise is flexibility, or mass customization
discuss the integration of the microservice architecture style to (as discussed by Krueger [8]) for multi-tenant ESAs. Through
achieve fine-grained incremental migration. Finally, we discuss the model it is possible to mass customize the developed
event sourcing as a software architecture pattern to mitigate software applications. Mass customization allows software
the complexity of data migrations. Through these solution
approaches we present our work in progress on continuous
producing organizations (SPOs) to efficiently produce and
migration for mass customized applications. maintain multiple similar software products. Through MDE
the similarities between these products can be exploited while
I. I NTRODUCTION the variations are managed through models.
The third promise is that of self-service. Not only SPOs
Model-driven engineering (MDE) is a known approach for
want to customize the software, customers (tenants) want to
increasing both the quality and productivity of software devel-
customize their own application to support their business
opment teams. According to Díaz et al. [1] these improvements
processes. Tenants want to be in control and make the
are achieved by raising the level of abstraction. MDE tools
customization themselves.
can also enable business analysts to take a more active role in
Multi-tenant ESAs can be developed with MDE through
the development process. The application of MDE, however,
Model-Driven Engineering Environments (MDEEs). As stated
is not only positive. Multiple experience reports, such as those
in our earlier research (Overeem et al. [9]), MDEEs implement
This work is a result of the AMUSE project. See amuse-project.org for the model-centric approach. One of the challenges in MDEEs
more information. is the continuous migration of mass customized applications.
1
(Overeem et al. [9]): laymen, technical business users, sql
experts, and developers.
adds requirements adds
The model is processed by the transformation engine which
implements generates an application package. The application package
takes on different formats, for example binaries when code
modeler engineer administrator user
generation is used.
interacts
develops
interacts interacts
The application package is input for the management
environment. One of the responsibilities of the management en-
modeling transformation management runtime
vironment is the deployment the application package. Through
environment environment environment environment the deployment the application instance is created. The
is processed by input for is served by
management environment is responsible for managing the
outputs generates deploys application instances. Migration of applications, caused by
different triggers, is executed by this environment. The intent
application application
model of the management environment is to automate this process,
package instance
however, the administrator can also use the management
environment to execute manual tasks.
Legend The application instance is executed by the runtime envi-
External interactor
System
Object
ronment. This runtime environment consists of infrastructure
(function)
(such as operating systems and database platforms), services,
frameworks, and other components required to execute the
application instance. The third persona, the user, interacts
Figure 1. A model-driven engineering environment enables a modeler to with this environment to execute the features offered by the
create a model in a modeling environment. The model is subsequently translated
by the model execution engine (using a model execution approach) into an
application.
application. An administrator uses the deployment environment to deploy the
application to a runtime environment. The user interacts with the runtime III. R EQUIREMENTS FOR C ONTINUOUS M IGRATION
environment. We specifically focus on those MDEEs that allow the user
to also take the role of the modeler. Without intervention of
Migration is defined as the steps that need to be executed the SPO and its engineers and administrators, the user should
to make the actual deployment of an application reflect the be able to evolve the application through model evolution. This
desired deployment. Examples of application migrations can results in more agility for the customer organization, they will
be the re-generation of the application to reflect an evolved not be dependent on the SPO for the continuous development
model, it could be the migration to a new version of the virtual of their application (at least within the capabilities of the
machine, or even the migration to a new cloud provider. model). Therefore our research focuses on four operational
requirements for the migration of applications: automation,
II. M ODEL -D RIVEN E NGINEERING E NVIRONMENTS performance, availability, and safety.
First, the migration should be automated. With automated,
Model-driven engineering environments (MDEEs) allow the we mean that the required migration steps should be derived
design and develop of software using the MDE approach. While from the migration trigger. At no point should an engineer
there are different variation points, all MDEEs share three manually analyze the required migration and develop custom
characteristics. First, they serve three key personas: the modeler, software to execute the migration. The automation will allow
the administrator, and the user. The engineer is involved as the the user/modeler to evolve the application without support of
fourth persona, but this person is not served by the MDEE, the SPO.
but builds and maintains it. Second, three key artifacts are Second, the migration should be performant. Research on
produced in the MDEE: the model, the application package, and live programming is rapidly gaining traction (see for instance
the application instance. Third, an MDEE offers four essential the work of van Rozen and van der Storm [10] and Kubelka
system functions: the modeling environment, the transformation et al. [11]). While we do not aim for a live feedback loop, we
environment, the management environment, and the runtime believe that faster feedback will improve the usability of the
environment. The MDEE concept is visualized by Fig. 1. MDEE, supporting agile development of the application.
The modeler uses the modeling environment to produce the Third, the migration should not negatively effect the
model. The modeling environment varies between different availability of the application. We aim for a fully automated
MDEEs, examples are graphical or text based modeling envi- continuously migration, meaning that scheduling should not be
ronments. Versioning, sharing, and collaboration are features necessary. Migration can thus happen on inconvenient moments
that could be offered, but are not essential for the MDEE. The (from a business perspective), and therefore should not be
essential function is the production of the model. noticed by the users. We do not expect users to asses the
The modeler persona represents different types of modelers, impact of changes on the availability, and therefore aim to let
we identified four types of modelers in earlier research the migration have no impact at all.
2
the regulations are part of the model) or to runtime environment
changes (when the regulations influence technical decisions).
Meta-model However, due to scope restrictions these are not discussed at
changes
this point. We discuss three of the identified triggers and a
possible migration strategy as examples.
The first and maybe most essential trigger is the change of
a model by the modeler. The changed model is processed to
Modeling produce a changed application package, and this application
Model changes environment
changes package needs to be redeployed in order to upgrade the
application instance. The change of the model could also be
triggered by a meta-model change or a change to the modeling
environment. In the last two cases, all existing models could
Transformation
be changed instead of a single model.
Application
enviro nment
package changes
changes
A second trigger is a change to the transformation envi-
ronment. Whenever the model transformations are changed,
existing models need to be reprocessed to produce updated
application packages. Again, those updated packages need to
Management Runtime
be redeployed in order to upgrade the instances.
Application
instance changes
enviro nment
changes
enviro nment
changes
The third trigger that we discuss as an example is a change
to the runtime environment. When a change to the runtime
environment causes changes to the application instance, the
management environment also needs to change. Therefore
the application needs to be migrated, in fact, all deployed
Application
migration
applications need to be migrated.
V. I NTEGRATING THE M ICROSERVICE A RCHITECTURE
S TYLE
Legend
Poss ibly
A second solution approach that we are focusing on is
Starting Leads to
Event
Event
leads to the integration of the microservice architecture style. We
believe that this architecture style makes it possible to achieve
fine-grained incremental migration. The integration of this
architecture style manifests itself in two ways.
First of all, the runtime environment should be able to
Figure 2. A classification of migration triggers: changes from the MDEE
leading to application migration.
host distributed microservices. The benefits of this are an
improved upgradability, scalability, resilience, and resource
sharing. The improved upgradability is the most important:
Finally, the migration should be safe. It should not be the possibility of updating a single service without affecting
possible to deploy an application that does not respond, causes the other services. The microservices make it possible to do
unexpected data-loss, or makes the application non-functional fine-grained migrations.
in some other manner. While not necessary, we believe that this style needs to
percolate all the way through to the meta-model. When
IV. A C ATEGORIZATION OF M IGRATION T RIGGERS the engineers enforce thinking in clear boundaries, smaller
A complete categorization of the migration triggers and independent model elements will arise in the meta-model.
the migration strategies is necessary to develop a MDEE These smaller model elements again help to transform the
that supports continuous migration of application. Migration model into microservices.
triggers originate from within the MDEE and require the Second, the transformation environment should consist of
migration of one or more applications. The triggers are caused independent microservices. This enables incremental transfor-
by requirements that are implemented by different personas mation of the model through the pattern Incrementality by
taking action. These requirements are added by the modeler, traceability as described by Varró [12].
administrator, or user of the MDEE (see Fig. 1).
Fig. 2 show an initial categorization of the triggers that occur VI. DATA M IGRATION IN E VENT S TORES
in the MDEE and (possibly) cause an application migration. We In earlier work (Overeem et al. [13]) we researched data
recognize that the requirements that cause these triggers (such conversion in event sourcing and proposed a framework
as changing market requirements or strategic business changes) to execute these conversions. Event sourcing is a software
are important. For instance regulatory requirements lead to architecture pattern in which not the current state of an
market requirements which can lead to model changes (when application is stored, but every change leading to the current
3
state is stored as a log of changes. These events can be used research group has published more related research: Diepen-
to derive the current state, but they can also show how that brock et al. [21], Rademacher et al. [22], Wizenty et al. [23].
state was reached. Common in event sourced systems is the They focus on the team organization and autonomy.
usage of multiple data models. The first data model is the event
store, the store with all state changes. It is the most important VIII. C ONCLUSION
data model, and recognized as the source of truth for the state.
The other data models (which can be as many as required) are This paper discusses the research that we are conducting
derived from these events. Examples are for instance: on continuous migration of applications. This challenge is
• a relational data model with the current state of the
prominent in MDEEs that are used to develop and execute
application, ESAs. The promises offered by MDEEs for ESAs (increased
• a full text search index, and
quality, productivity, and flexibility) are only delivered when
• a timeline showing the history of certain objects.
the challenge of continuous migration is solved. We discussed
three solution approaches.
Migration of applications in an MDEE do not only impact
First of all, a categorization of migration triggers including
the application, but also the application state. The data that
possible migration strategies. This categorization increases the
is stored by the application might need to be converted to
understanding of the challenge. Optimized strategies can be
conform to the new desired state. We believe that adopting event
developed when the challenges are clear.
sourcing in the runtime environment of the MDEE simplifies
Second, the integration of the microservice architecture
data migration. In event sourced systems only the events need
style in the MDEE makes fine-grained incremental migration
to be transformed, because all other data models can be derived
possible. The runtime environment benefits from loosely
from these events. We believe that in many scenarios these
coupled microservices, because it increases scalability and
events do not require migration, and thus data migration in
flexibility. Instead of re-deploying the complete application on
MDEEs is less complex when using event sourcing.
every migration trigger, the microservices allow to update only
VII. R ELATED W ORK part of the application.
Finally, the adoption of event sourcing in the runtime
Meijler et al. [14] research fine-grained evolution for gener- environment removes complexity from the data migration
ated applications. However, they take the viewpoint of a single challenge. The essential characteristic of event sourcing is
application, instead of a multi-tenant MDEE with multiple that every state change is stored as an event. Data models used
applications. The integration of the model environment and to present the state of the application are derived from these
the runtime environment is seen by Meijler et al. [14] as events. As a result of this approach, these data models are
crucial to achieve fine-grained evolution. The solution they volatile and can be rebuild when needed. These data models
descibe is focused on Model-Driven Architecture (MDA) and thus do not require migration.
the JVM environment. We propose a separate management These three solution approaches are based on an ongoing
environment instead of a tight integration of the model and case study at AFAS Software. They form the hypotheses that
runtime environment. Our approach is less coupled to specific direct our research. These three solution approaches tackle the
technology or meta-models. challenge of continuous migration from different angles.
Bruneliere et al. [15] propose Modeling as a Service (MaaS),
the synergy of cloud computing and MDE. The research agenda
R EFERENCES
they propose focuses on bringing (parts of) the MDEE to the
cloud, such as the modeling environment and the transformation [1] V. G. Díaz, E. R. N. Valdez, J. P. Espada, b. C. P. G.
engine. The (continuous) migration of resulting applications Bustelo, J. M. C. Lovelle, and C. E. M. Marín, “A
is not mentioned. Popoola et al. [16] survey different MDE brief introduction to model-driven engineering,” Tecnura,
tools and if they are capable of delivering MaaS functionality. vol. 18, no. 40, pp. 127–142, 2014.
This research too focuses on the modeling and transformation [2] J.-P. Tolvanen and S. Kelly, “Model-Driven Development
functions of the MDEE. Challenges and Solutions - Experiences with
Scalable MDE is researched by Rajbhoj and Kulkarni [17], Domain-Specific Modelling in Industry,” Proceedings
Kolovos et al. [18], Cuadrado and de Lara [19]. Rajbhoj of the 4th International Conference on Model-
and Kulkarni [17] focuses on the scalability of modeling: Driven Engineering and Software Development,
collaboration and model management. Kolovos et al. [18] no. January, pp. 711–719, 2016. [Online].
define a research agenda for scalable MDE, in which they Available: http://www.scitepress.org/DigitalLibrary/Link.
focus on language design, transformation, collaboration, and aspx?doi=10.5220/0005833207110719
persistence. Cuadrado and de Lara [19] specifically research [3] R. F. Paige and D. Varró, “Lessons learned from
model transformations, and how they can be made streaming. building model-driven development tools,” Software
Our solution proposes integration of the microservice ar- & Systems Modeling, vol. 11, no. 4, pp. 527–539,
chitecture in the MDEE. Sorgalla et al. [20] discuss how 2012. [Online]. Available: http://link.springer.com/10.
microservices can be generated from MDE platforms. Their 1007/s10270-012-0257-9
4
[4] T. Clark and P. A. Muller, “Exploiting model driven N. Matragkas, R. F. Paige, E. Guerra, J. S. Cuadrado,
technology: A tale of two startups,” Software and Systems J. De Lara, I. Ráth, and D. Varró, “A research
Modeling, vol. 11, no. 4, pp. 481–493, 2012. roadmap towards achieving scalability in model driven
[5] M. Fowler, Patterns of Enterprise Application Architec- engineering,” Proceedings of the Workshop on Scalability
ture. Addison-Wesley, 2002. in Model Driven Engineering - BigMDE ’13, pp. 1–10,
[6] Gartner, “Enterprise Application Software,” 2012. 2013. [Online]. Available: http://dl.acm.org/citation.cfm?
[Online]. Available: https://www.gartner.com/it-glossary/ doid=2487766.2487768
enterprise-application-software [19] J. S. Cuadrado and J. de Lara, “Streaming model trans-
[7] A. W. Brown, “An introduction to Model Driven formations: Scenarios, challenges and initial solutions,”
Architecture,” The Rational Edge, pp. 1–16, 2004. in International Conference on Theory and Practice of
[Online]. Available: http://www.ibm.com/developerworks/ Model Transformations. Springer, 2013, pp. 1—-16.
rational/library/3100.html [20] J. Sorgalla, F. Rademacher, S. Sachweh, and A. Zündorf,
[8] C. Krueger, “Easing the Transition to Software Mass “On Collaborative Model-driven Development of
Customization,” in International Workshop on Software Microservices,” in MSE Workshop @ STAF2018, 2018, pp.
Product-Family Engineering. Springer, 2002, pp. 282– 1–8. [Online]. Available: http://arxiv.org/abs/1805.01176
293. [Online]. Available: http://link.springer.com/10.1007/ [21] A. Diepenbrock, F. Rademacher, and S. Sachweh, “An
3-540-47833-7{_}25 Ontology-based Approach for Domain-driven Design of
[9] M. Overeem, S. Jansen, and S. Fortuin, “Generative Microservice Architectures,” INFORMATIK 2017, no.
versus interpretive model-driven development: Moving September, pp. 1–12, 2017.
past ‘It depends’,” in Communications in Computer and [22] F. Rademacher, J. Sorgalla, and S. Sachweh, “Challenges
Information Science, vol. 880, 2018, pp. 222–246. of Domain-Driven Microservice Design,” IEEE Software,
[10] R. van Rozen and T. van der Storm, “Toward live domain- p. 8, 2018.
specific languages: From text differencing to adapting [23] P. Wizenty, J. Sorgalla, F. Rademacher, and
models at run time,” Software and Systems Modeling, pp. S. Sachweh, “Magma: Build Management-based
1–18, 2017. Generation of Microservice Infrastructures,” Pro-
[11] J. Kubelka, R. Robbes, and A. Bergel, “The road ceedings of the 11th European Conference on
to live programming,” Proceedings of the 40th Software Architecture Companion Proceedings -
International Conference on Software Engineering - ECSA ’17, pp. 61–65, 2017. [Online]. Available:
ICSE ’18, pp. 1090–1101, 2018. [Online]. Available: http://dl.acm.org/citation.cfm?doid=3129790.3129821
http://dl.acm.org/citation.cfm?doid=3180155.3180200
[12] D. Varró, “Patterns and styles for incremental model
transformations,” in CEUR Workshop Proceedings, vol.
1657, 2016, pp. 41–43.
[13] M. Overeem, M. Spoor, and S. Jansen, “The Dark
Side of Event Sourcing: Managing Data Conversion,” in
IEEE 24th International Conference on Software Analysis,
Evolution and Reengineering (SANER), 2017, pp. 193–
204.
[14] T. D. Meijler, J. P. Nytun, A. Prinz, and H. Wortmann,
“Supporting fine-grained generative model-driven evolu-
tion,” Software & Systems Modeling, vol. 9, no. 3, pp.
403–424, 2010.
[15] H. Bruneliere, J. Cabot, and F. Jouault, “Combining
Model-Driven Engineering and Cloud Computing,” in
Modeling, Design, and Analysis for the Service Cloud-
MDA4ServiceCloud’10: Workshop’s 4th edition (co-
located with the 6th European Conference on Modelling
Foundations and Applications-ECMFA 2010), 2010.
[16] S. Popoola, J. Carver, and J. Gray, “Modeling as a service:
A survey of existing tools,” CEUR Workshop Proceedings,
vol. 2019, pp. 360–367, 2017.
[17] A. Rajbhoj and V. Kulkarni, “Large scale model-driven
engineering for a multi-site team-Experience report,” Pro-
ceedings - Asia-Pacific Software Engineering Conference,
APSEC, no. 2, pp. 123–128, 2013.
[18] D. S. Kolovos, M. Tisi, J. Cabot, L. M. Rose,
5