Continuous Migration of Mass Customized Applications Michiel Overeem Slinger Jansen dept. Product Development dept. Information and Computing Sciences AFAS Software Utrecht University Leusden, The Netherlands Utrecht, The Netherlands michiel.overeem@afas.nl slinger.jansen@uu.nl Abstract—Model-driven engineering is a known approach for by Tolvanen and Kelly [2] and Paige and Varró [3], discuss the increasing both the quality and productivity of software develop- challenges and the effort it takes to create MDE tools. Clark ment. It also enables business analysts to take a more active role and Muller [4] report on two startups around MDE tools and in the development process. The model allows the application to be tailored to the customer, who no longer needs to adjust to the lessons learned from those startups. the software. Through the model it is possible to mass customize Our research focuses on MDE for enterprise software appli- the developed software applications. Mass customization allows cations (ESAs). Fowler [5] states “Enterprise applications are software producing organizations to efficiently produce and about the display, manipulation, and storage of large amounts of maintain multiple similar software products. Through model- often complex data and the support or automation of business driven engineering the similarities between these products can be exploited while the variations are managed through models. processes with that data.” According to Gartner [6] examples of The increase in quality and productivity, and the mass ESAs are CRM and ERP software. The characteristics of ESAs customization is achieved by raising the level of abstraction are distinct from for instance games and result in different on which software is developed. Changes done on the higher requirements for application migration. level of abstraction should be translated into changes on the We believe that the model-centric approach of MDE is resulting application. However, in many model-driven systems the migration of the resulting application towards the new intended especially promising for ESAs. Brown [7] defines the model- state is still a (partially) manual and labor-intensive step. This centric approach as “the system models have sufficient detail manual step requires engineers to work on a lower level of to enable the generation of a full system implementation from abstraction, and thus forgoes on the increase of quality and the models themselves". The first promise is the increase productivity. of productivity and quality. ESAs often contain repetitive In this research we study model-driven engineering environ- ments in which the application is generated from a model. patterns of functionality, such as the maintenance of data. Whenever models or parts of these model-driven engineering MDE increases the productivity by generating the software for environments evolve, the goal is to automatically migrate the every instance of these patterns. Not only can MDE increase application and data as well. In order to reach this goal, we the productivity by deriving these components from a model, formulate three solution approaches. First of all a categorization it also increases the quality of those components by doing it of migration triggers that can occur is discussed, to increase the understanding of the context of the migration. Migration triggers consistently. can be handled with different migration strategies. Secondly, we The second promise is flexibility, or mass customization discuss the integration of the microservice architecture style to (as discussed by Krueger [8]) for multi-tenant ESAs. Through achieve fine-grained incremental migration. Finally, we discuss the model it is possible to mass customize the developed event sourcing as a software architecture pattern to mitigate software applications. Mass customization allows software the complexity of data migrations. Through these solution approaches we present our work in progress on continuous producing organizations (SPOs) to efficiently produce and migration for mass customized applications. maintain multiple similar software products. Through MDE the similarities between these products can be exploited while I. I NTRODUCTION the variations are managed through models. The third promise is that of self-service. Not only SPOs Model-driven engineering (MDE) is a known approach for want to customize the software, customers (tenants) want to increasing both the quality and productivity of software devel- customize their own application to support their business opment teams. According to Díaz et al. [1] these improvements processes. Tenants want to be in control and make the are achieved by raising the level of abstraction. MDE tools customization themselves. can also enable business analysts to take a more active role in Multi-tenant ESAs can be developed with MDE through the development process. The application of MDE, however, Model-Driven Engineering Environments (MDEEs). As stated is not only positive. Multiple experience reports, such as those in our earlier research (Overeem et al. [9]), MDEEs implement This work is a result of the AMUSE project. See amuse-project.org for the model-centric approach. One of the challenges in MDEEs more information. is the continuous migration of mass customized applications. 1 (Overeem et al. [9]): laymen, technical business users, sql experts, and developers. adds requirements adds The model is processed by the transformation engine which implements generates an application package. The application package takes on different formats, for example binaries when code modeler engineer administrator user generation is used. interacts develops interacts interacts The application package is input for the management environment. One of the responsibilities of the management en- modeling transformation management runtime vironment is the deployment the application package. Through environment environment environment environment the deployment the application instance is created. The is processed by input for is served by management environment is responsible for managing the outputs generates deploys application instances. Migration of applications, caused by different triggers, is executed by this environment. The intent application application model of the management environment is to automate this process, package instance however, the administrator can also use the management environment to execute manual tasks. Legend The application instance is executed by the runtime envi- External interactor System Object ronment. This runtime environment consists of infrastructure (function) (such as operating systems and database platforms), services, frameworks, and other components required to execute the application instance. The third persona, the user, interacts Figure 1. A model-driven engineering environment enables a modeler to with this environment to execute the features offered by the create a model in a modeling environment. The model is subsequently translated by the model execution engine (using a model execution approach) into an application. application. An administrator uses the deployment environment to deploy the application to a runtime environment. The user interacts with the runtime III. R EQUIREMENTS FOR C ONTINUOUS M IGRATION environment. We specifically focus on those MDEEs that allow the user to also take the role of the modeler. Without intervention of Migration is defined as the steps that need to be executed the SPO and its engineers and administrators, the user should to make the actual deployment of an application reflect the be able to evolve the application through model evolution. This desired deployment. Examples of application migrations can results in more agility for the customer organization, they will be the re-generation of the application to reflect an evolved not be dependent on the SPO for the continuous development model, it could be the migration to a new version of the virtual of their application (at least within the capabilities of the machine, or even the migration to a new cloud provider. model). Therefore our research focuses on four operational requirements for the migration of applications: automation, II. M ODEL -D RIVEN E NGINEERING E NVIRONMENTS performance, availability, and safety. First, the migration should be automated. With automated, Model-driven engineering environments (MDEEs) allow the we mean that the required migration steps should be derived design and develop of software using the MDE approach. While from the migration trigger. At no point should an engineer there are different variation points, all MDEEs share three manually analyze the required migration and develop custom characteristics. First, they serve three key personas: the modeler, software to execute the migration. The automation will allow the administrator, and the user. The engineer is involved as the the user/modeler to evolve the application without support of fourth persona, but this person is not served by the MDEE, the SPO. but builds and maintains it. Second, three key artifacts are Second, the migration should be performant. Research on produced in the MDEE: the model, the application package, and live programming is rapidly gaining traction (see for instance the application instance. Third, an MDEE offers four essential the work of van Rozen and van der Storm [10] and Kubelka system functions: the modeling environment, the transformation et al. [11]). While we do not aim for a live feedback loop, we environment, the management environment, and the runtime believe that faster feedback will improve the usability of the environment. The MDEE concept is visualized by Fig. 1. MDEE, supporting agile development of the application. The modeler uses the modeling environment to produce the Third, the migration should not negatively effect the model. The modeling environment varies between different availability of the application. We aim for a fully automated MDEEs, examples are graphical or text based modeling envi- continuously migration, meaning that scheduling should not be ronments. Versioning, sharing, and collaboration are features necessary. Migration can thus happen on inconvenient moments that could be offered, but are not essential for the MDEE. The (from a business perspective), and therefore should not be essential function is the production of the model. noticed by the users. We do not expect users to asses the The modeler persona represents different types of modelers, impact of changes on the availability, and therefore aim to let we identified four types of modelers in earlier research the migration have no impact at all. 2 the regulations are part of the model) or to runtime environment changes (when the regulations influence technical decisions). Meta-model However, due to scope restrictions these are not discussed at changes this point. We discuss three of the identified triggers and a possible migration strategy as examples. The first and maybe most essential trigger is the change of a model by the modeler. The changed model is processed to Modeling produce a changed application package, and this application Model changes environment changes package needs to be redeployed in order to upgrade the application instance. The change of the model could also be triggered by a meta-model change or a change to the modeling environment. In the last two cases, all existing models could Transformation be changed instead of a single model. Application enviro nment package changes changes A second trigger is a change to the transformation envi- ronment. Whenever the model transformations are changed, existing models need to be reprocessed to produce updated application packages. Again, those updated packages need to Management Runtime be redeployed in order to upgrade the instances. Application instance changes enviro nment changes enviro nment changes The third trigger that we discuss as an example is a change to the runtime environment. When a change to the runtime environment causes changes to the application instance, the management environment also needs to change. Therefore the application needs to be migrated, in fact, all deployed Application migration applications need to be migrated. V. I NTEGRATING THE M ICROSERVICE A RCHITECTURE S TYLE Legend Poss ibly A second solution approach that we are focusing on is Starting Leads to Event Event leads to the integration of the microservice architecture style. We believe that this architecture style makes it possible to achieve fine-grained incremental migration. The integration of this architecture style manifests itself in two ways. First of all, the runtime environment should be able to Figure 2. A classification of migration triggers: changes from the MDEE leading to application migration. host distributed microservices. The benefits of this are an improved upgradability, scalability, resilience, and resource sharing. The improved upgradability is the most important: Finally, the migration should be safe. It should not be the possibility of updating a single service without affecting possible to deploy an application that does not respond, causes the other services. The microservices make it possible to do unexpected data-loss, or makes the application non-functional fine-grained migrations. in some other manner. While not necessary, we believe that this style needs to percolate all the way through to the meta-model. When IV. A C ATEGORIZATION OF M IGRATION T RIGGERS the engineers enforce thinking in clear boundaries, smaller A complete categorization of the migration triggers and independent model elements will arise in the meta-model. the migration strategies is necessary to develop a MDEE These smaller model elements again help to transform the that supports continuous migration of application. Migration model into microservices. triggers originate from within the MDEE and require the Second, the transformation environment should consist of migration of one or more applications. The triggers are caused independent microservices. This enables incremental transfor- by requirements that are implemented by different personas mation of the model through the pattern Incrementality by taking action. These requirements are added by the modeler, traceability as described by Varró [12]. administrator, or user of the MDEE (see Fig. 1). Fig. 2 show an initial categorization of the triggers that occur VI. DATA M IGRATION IN E VENT S TORES in the MDEE and (possibly) cause an application migration. We In earlier work (Overeem et al. [13]) we researched data recognize that the requirements that cause these triggers (such conversion in event sourcing and proposed a framework as changing market requirements or strategic business changes) to execute these conversions. Event sourcing is a software are important. For instance regulatory requirements lead to architecture pattern in which not the current state of an market requirements which can lead to model changes (when application is stored, but every change leading to the current 3 state is stored as a log of changes. These events can be used research group has published more related research: Diepen- to derive the current state, but they can also show how that brock et al. [21], Rademacher et al. [22], Wizenty et al. [23]. state was reached. Common in event sourced systems is the They focus on the team organization and autonomy. usage of multiple data models. The first data model is the event store, the store with all state changes. It is the most important VIII. C ONCLUSION data model, and recognized as the source of truth for the state. The other data models (which can be as many as required) are This paper discusses the research that we are conducting derived from these events. Examples are for instance: on continuous migration of applications. This challenge is • a relational data model with the current state of the prominent in MDEEs that are used to develop and execute application, ESAs. The promises offered by MDEEs for ESAs (increased • a full text search index, and quality, productivity, and flexibility) are only delivered when • a timeline showing the history of certain objects. the challenge of continuous migration is solved. We discussed three solution approaches. Migration of applications in an MDEE do not only impact First of all, a categorization of migration triggers including the application, but also the application state. The data that possible migration strategies. This categorization increases the is stored by the application might need to be converted to understanding of the challenge. Optimized strategies can be conform to the new desired state. We believe that adopting event developed when the challenges are clear. sourcing in the runtime environment of the MDEE simplifies Second, the integration of the microservice architecture data migration. In event sourced systems only the events need style in the MDEE makes fine-grained incremental migration to be transformed, because all other data models can be derived possible. The runtime environment benefits from loosely from these events. We believe that in many scenarios these coupled microservices, because it increases scalability and events do not require migration, and thus data migration in flexibility. Instead of re-deploying the complete application on MDEEs is less complex when using event sourcing. every migration trigger, the microservices allow to update only VII. R ELATED W ORK part of the application. Finally, the adoption of event sourcing in the runtime Meijler et al. [14] research fine-grained evolution for gener- environment removes complexity from the data migration ated applications. However, they take the viewpoint of a single challenge. The essential characteristic of event sourcing is application, instead of a multi-tenant MDEE with multiple that every state change is stored as an event. Data models used applications. The integration of the model environment and to present the state of the application are derived from these the runtime environment is seen by Meijler et al. [14] as events. As a result of this approach, these data models are crucial to achieve fine-grained evolution. The solution they volatile and can be rebuild when needed. These data models descibe is focused on Model-Driven Architecture (MDA) and thus do not require migration. the JVM environment. We propose a separate management These three solution approaches are based on an ongoing environment instead of a tight integration of the model and case study at AFAS Software. They form the hypotheses that runtime environment. Our approach is less coupled to specific direct our research. These three solution approaches tackle the technology or meta-models. challenge of continuous migration from different angles. Bruneliere et al. [15] propose Modeling as a Service (MaaS), the synergy of cloud computing and MDE. The research agenda R EFERENCES they propose focuses on bringing (parts of) the MDEE to the cloud, such as the modeling environment and the transformation [1] V. G. Díaz, E. R. N. Valdez, J. P. Espada, b. C. P. G. engine. The (continuous) migration of resulting applications Bustelo, J. M. C. Lovelle, and C. E. M. Marín, “A is not mentioned. Popoola et al. [16] survey different MDE brief introduction to model-driven engineering,” Tecnura, tools and if they are capable of delivering MaaS functionality. vol. 18, no. 40, pp. 127–142, 2014. This research too focuses on the modeling and transformation [2] J.-P. Tolvanen and S. Kelly, “Model-Driven Development functions of the MDEE. Challenges and Solutions - Experiences with Scalable MDE is researched by Rajbhoj and Kulkarni [17], Domain-Specific Modelling in Industry,” Proceedings Kolovos et al. [18], Cuadrado and de Lara [19]. Rajbhoj of the 4th International Conference on Model- and Kulkarni [17] focuses on the scalability of modeling: Driven Engineering and Software Development, collaboration and model management. Kolovos et al. [18] no. January, pp. 711–719, 2016. [Online]. define a research agenda for scalable MDE, in which they Available: http://www.scitepress.org/DigitalLibrary/Link. focus on language design, transformation, collaboration, and aspx?doi=10.5220/0005833207110719 persistence. Cuadrado and de Lara [19] specifically research [3] R. F. Paige and D. Varró, “Lessons learned from model transformations, and how they can be made streaming. building model-driven development tools,” Software Our solution proposes integration of the microservice ar- & Systems Modeling, vol. 11, no. 4, pp. 527–539, chitecture in the MDEE. Sorgalla et al. [20] discuss how 2012. [Online]. Available: http://link.springer.com/10. microservices can be generated from MDE platforms. Their 1007/s10270-012-0257-9 4 [4] T. Clark and P. A. Muller, “Exploiting model driven N. Matragkas, R. F. Paige, E. Guerra, J. S. Cuadrado, technology: A tale of two startups,” Software and Systems J. De Lara, I. Ráth, and D. Varró, “A research Modeling, vol. 11, no. 4, pp. 481–493, 2012. roadmap towards achieving scalability in model driven [5] M. Fowler, Patterns of Enterprise Application Architec- engineering,” Proceedings of the Workshop on Scalability ture. Addison-Wesley, 2002. in Model Driven Engineering - BigMDE ’13, pp. 1–10, [6] Gartner, “Enterprise Application Software,” 2012. 2013. [Online]. Available: http://dl.acm.org/citation.cfm? [Online]. Available: https://www.gartner.com/it-glossary/ doid=2487766.2487768 enterprise-application-software [19] J. S. Cuadrado and J. de Lara, “Streaming model trans- [7] A. W. Brown, “An introduction to Model Driven formations: Scenarios, challenges and initial solutions,” Architecture,” The Rational Edge, pp. 1–16, 2004. in International Conference on Theory and Practice of [Online]. Available: http://www.ibm.com/developerworks/ Model Transformations. Springer, 2013, pp. 1—-16. rational/library/3100.html [20] J. Sorgalla, F. Rademacher, S. Sachweh, and A. Zündorf, [8] C. Krueger, “Easing the Transition to Software Mass “On Collaborative Model-driven Development of Customization,” in International Workshop on Software Microservices,” in MSE Workshop @ STAF2018, 2018, pp. Product-Family Engineering. Springer, 2002, pp. 282– 1–8. [Online]. Available: http://arxiv.org/abs/1805.01176 293. [Online]. Available: http://link.springer.com/10.1007/ [21] A. Diepenbrock, F. Rademacher, and S. Sachweh, “An 3-540-47833-7{_}25 Ontology-based Approach for Domain-driven Design of [9] M. Overeem, S. Jansen, and S. Fortuin, “Generative Microservice Architectures,” INFORMATIK 2017, no. versus interpretive model-driven development: Moving September, pp. 1–12, 2017. past ‘It depends’,” in Communications in Computer and [22] F. Rademacher, J. Sorgalla, and S. Sachweh, “Challenges Information Science, vol. 880, 2018, pp. 222–246. of Domain-Driven Microservice Design,” IEEE Software, [10] R. van Rozen and T. van der Storm, “Toward live domain- p. 8, 2018. specific languages: From text differencing to adapting [23] P. Wizenty, J. Sorgalla, F. Rademacher, and models at run time,” Software and Systems Modeling, pp. S. Sachweh, “Magma: Build Management-based 1–18, 2017. Generation of Microservice Infrastructures,” Pro- [11] J. Kubelka, R. Robbes, and A. Bergel, “The road ceedings of the 11th European Conference on to live programming,” Proceedings of the 40th Software Architecture Companion Proceedings - International Conference on Software Engineering - ECSA ’17, pp. 61–65, 2017. [Online]. Available: ICSE ’18, pp. 1090–1101, 2018. [Online]. Available: http://dl.acm.org/citation.cfm?doid=3129790.3129821 http://dl.acm.org/citation.cfm?doid=3180155.3180200 [12] D. Varró, “Patterns and styles for incremental model transformations,” in CEUR Workshop Proceedings, vol. 1657, 2016, pp. 41–43. [13] M. Overeem, M. Spoor, and S. Jansen, “The Dark Side of Event Sourcing: Managing Data Conversion,” in IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), 2017, pp. 193– 204. [14] T. D. Meijler, J. P. Nytun, A. Prinz, and H. Wortmann, “Supporting fine-grained generative model-driven evolu- tion,” Software & Systems Modeling, vol. 9, no. 3, pp. 403–424, 2010. [15] H. Bruneliere, J. Cabot, and F. Jouault, “Combining Model-Driven Engineering and Cloud Computing,” in Modeling, Design, and Analysis for the Service Cloud- MDA4ServiceCloud’10: Workshop’s 4th edition (co- located with the 6th European Conference on Modelling Foundations and Applications-ECMFA 2010), 2010. [16] S. Popoola, J. Carver, and J. Gray, “Modeling as a service: A survey of existing tools,” CEUR Workshop Proceedings, vol. 2019, pp. 360–367, 2017. [17] A. Rajbhoj and V. Kulkarni, “Large scale model-driven engineering for a multi-site team-Experience report,” Pro- ceedings - Asia-Pacific Software Engineering Conference, APSEC, no. 2, pp. 123–128, 2013. [18] D. S. Kolovos, M. Tisi, J. Cabot, L. M. Rose, 5