MDE-based Automated Provisioning and Management of Cloud Applications Anirban Bhattacharjee∗ ∗ Department of Electrical Engineering and Computer Science Vanderbilt University, Nashville, Tennessee, USA Email: anirban.bhattacharjee@vanderbilt.edu Abstract—Digital innovations in the modern era are driven of cloud applications, the current state of the art, and propose by cloud-native applications, cloud architectures, and cloud- solutions while providing some preliminary results. based development. In this realm, continuous architectural de- sign, framework/infrastructure (re-)configuration, deployment, II. P ROBLEM F ORMULATION migration, and resource monitoring to iteratively tune the application and underlying resources as per business demand Consider as an example a distributed, data-intensive appli- are required in the form of a self-service cloud deployment cation deployed in the cloud. It requires significant efforts for and management platform. To address these challenges while the designers - ensuring broader applicability using vendor-agnostic approaches, 1) To configure and deploy the application to use the this doctoral research envisions an intelligent and context-aware, model-driven, automated service provisioning technique, where available big data frameworks [31], manual efforts and domain expertise are alleviated to a large 2) To set up the virtual machines to host the frameworks extent. In this context, this paper describes a self-service cloud- and finally, and based application deployment and management ecosystem based 3) To establish the connection properly among application on a Model-Driven Engineering (MDE) approach. components. I. I NTRODUCTION Moreover, adding application component which requires a different framework exacerbates these problems. To address Deployment and management of cloud-based applications these problems, we envision an automated solution and outline in both homogeneous or heterogeneous cloud environments the key requirements of such an approach and their associated are often hampered by the intricacies and variabilities in the technical challenges: deployment and configuration processes stemming from the diversity in both the software stacks that control the cloud A. Requirement 1: Infrastructure Design Provisioning and platforms and the underlying platform resources. Application Configuration Automation provisioning is important since it enables the service providers Need to abstract away the architecting phase by pre-defining to deploy their service’s application components in the cloud the configuration of applications and infrastructure as intuitive provider’s resources in accordance with the service’s Quality modeling artifacts, which will speed up the deployment and of Services (QoS), availability, business policy, and cost model migration process. requirements [33], [28]. This complexity restricts the cloud ap- • Challenge 1: Capturing the Application and Cloud plication development and deployment knowledge to a narrow Specifications in the Metamodel, where all the deploy- set of experts who possess both the domain knowledge and ment and infrastructure complexities of the application application-specific knowledge to explore the power of these and cloud specification are identified and defined. techniques. Even with advances in DevOps community tools, • Challenge 2: Defining a Language for Model Trans- they demand domain expertise and hence creating full-blown formation, where the DSML will transform the abstract deployable models are error-prone and time-consuming. business model to target model (i.e. infrastructure code) To address these challenges, this doctoral dissertation en- using a knowledge base. visions an intelligent and context-aware, model -driven, au- • Challenge3: Verification of the Business Model, where tomated service provisioning platform where manual efforts all relationships and constraints of application compo- and domain expertise are minimal. To that end, in this pa- nents are checked and satisfied. per, we propose a model-driven engineering (MDE)-based ecosystem [29] to automate the process of deployment and B. Requirement 2: Support for Continuous Integration, Mi- management for cloud applications and emphasize on extensi- gration, and Delivery bility, (re-)usability and scalability. Our approach emphasizes Changes in business strategies and demands lead to evolu- vendor-neutrality in order to be broadly applicable to diverse tion of the business model, and hence the platform should be cloud platforms with their control software stacks. In the integrated with adaptation techniques. remainder of this paper, we explore the challenges faced in • Challenge 4: Extensibility and Reusability of the Appli- realizing the MDE approach for deployment and management cation Components, where the addition, upgradation or migration of application components can be done at the [10], [13]. For Requirement 3 [24], [17], dynamic resource model level, and should support collaboration and version management on enterprise application vendors is proposed by control. various frameworks based on queuing model [34], [32], or by • Challenge 5: Extensibility of the Platform, where the dynamic QoS Control [11], [16], [30]. addition of a new application type should be performed in a modularized way, and should be only a one-time IV. P ROPOSED S OLUTION effort. We propose a composable ecosystem called CloudCAMP C. Requirement 3: Dynamic Modeling and Provisioning to deploy, monitor and manage cloud application intelligently. The workflow of the proposed approach is shown in Fig. 1. Continuously monitoring the infrastructure of the system The contributions in this research include the following tasks: and applications are required, and hence the model should always reflect the current up-to-date state. • Task 1: A Deployment Framework Metamodel, to au- • Challenge 6: Context-Aware Provisioning, where opti- tomate the infrastructure design and implementation by mization of cloud resources based on cost and resource capturing variability and commonality endpoints for the availability should take place iteratively for potentially application and cloud specifications along with their SDK differing QoS needs, policies, etc. and APIs, to provide abstractions based on TOSCA- • Challenge 7: Model@Runtime, where the changes stem- specifications. This metamodel details will be stored in ming from self-adaptation of the provisioning techniques the knowledge base of the platform. • Task 2: Model-to-Infrastructure-as-code Transformation should be reflected back at the model level dynami- cally [4]. DSML, as the basis for the deployment time and runtime composition and orchestration through generative pro- III. R ELATED W ORK gramming. The DSML will also be integrated with pre- We explore existing research efforts directly aligned with defined constraints to verify the correctness of business our research goals. The script-centric DevOps community model. provides toolchains such as Chef [26], Puppet [21], or An- • Task 3: An Integrated Repository, that allows storage, sible1 to speed up deployment for developers [18]. Cloudify query, and manipulation of models, templates, metadata, and Apache Brooklyn2 enable cloud application orchestration parameters, software dependency knowledge base and of topology templates in a vendor-agnostic way according provenance in a scalable data store. to the Topology and Orchestration Specification for Cloud • Task 4: A Model@Runtime framework, that permits re- Applications (TOSCA) [27] specification. However, we need flection of the current model by capturing and monitoring declarative high-level modeling abstraction on top of these the streamlined system and application information across tools to reduce the burden of domain expertise. heterogeneous resources at runtime. To satisfy Requirement 1, multiple pattern based approaches • Task 5: Integration of Adaptive Resource Provisioning are proposed [2], [9], [8], [22] where a model-based pattern de- Engine, to provide the QoS guarantees while scaling the scribes a set of capabilities and functional and non-functional application vertically and horizontally, based on policy, properties of application service deployment in cloud infras- economic issues, etc. tructures [15], [1]. Alternatively, requirement solvers [25], [7] The approach to integrating diverse sets of deployment and are also proposed to synthesize infrastructure configuration management building blocks for a specific application and in a declarative fashion. OpenTOSCA [14] and CELAR [12] cloud providers offers significant advantages over traditional exercise the combination of MDE and TOSCA specification to methods – such as DevOps or orchestration frameworks – automate deployment of cloud applications from users’ partial by applying a blend of generative programming and micro business relevant topology. The components can be deployed service architectures. Concrete implementation our proposed and managed in the right order [5]. Later in [6] a context- generative approach will be demonstrated in a cloud-based aware provisioning and management plan along with detection MDE environment called WebGME [23]. and correction of the unintended error due to software conflicts and dependency is proposed. However, their platform is not V. C URRENT S TATUS based on any standardization, and their planner does not use Our CloudCAMP approach to integrating a diverse set of any DevOps community tools. Hence, their approach might deployment and management building blocks for a specific lack extensibility and robustness. application and cloud providers offers significant advantages In relation to Requirement 2, CHAMPS [19] focuses on over traditional methods – such as DevOps or orchestration Change Management process by which IT systems are ef- frameworks – by applying a blend of generative programming ficiently modified to minimize reconfiguration impact. Sim- and micro service architectures. ilarly, various standard modeling environments as an object- A concrete implementation of our MDE-based, generative oriented formalism for configuration problems is proposed in approach called CloudCAMP is being developed within a 1 https://www.ansible.com/ cloud-based MDE environment called WebGME [23]. A pre- 2 https://brooklyn.apache.org/ liminary version of CloudCAMP supports few application Fig. 1: The complete workflow of the proposed approach Fig. 2: THESIS Defense Timeline types in an OpenStack cloud environment. The DSML trans- Generic Modeling Environment (GME) [20] as Task 1 and 2. forms the business model to an infrastructure-as-code based Our research evaluation plans call for validating the frame- on the metamodel and pre-defined knowledge base, which is work in the context of our ongoing and upcoming industry and a MySQL database. We have integrated a monitoring tool to federally sponsored research projects. The source code is avail- check the VMs and applications health. Presently, our DSML able at https://anirban2404.github.io/DeploymentAutomation/. can realize the CloudCAMP metamodel and generate Ansible For example, one of our new projects is building a cloud-based specific infrastructure code. machine learning ecosystem which illustrates many of the The development of CloudCAMP comprises of the design needs addressed by our research. We also plan to validate our and evaluation of the expressivity of the metamodel and the framework in cloud-based, collaborative education services for DSML with all the product-line features. Figure 2 provides STEM education [3]. a timeline for this research, conducted under the supervision of Dr. Aniruddha Gokhale, Associate Professor, Vanderbilt VII. E XPECTED C ONTRIBUTIONS University. CloudCAMP will ease the deployment and management of VI. P LAN FOR E VALUATION AND VALIDATION applications based on deployment abstraction and intelligent To demonstrate the deployment and management ecosys- resource optimization. The following key contributions are tem, we are developing the CloudCAMP platform based on expected from this research: The following key contributions are expected from this [13] Guillén, J., Miranda, J., Murillo, J.M., Canal, C.: A service-oriented research: framework for developing cross cloud migratable software. Journal of Systems and Software 86(9), 2294–2308 (2013) 1) Automated generation of infrastructure code from an [14] Hirmer, P., Breitenbücher, U., Binz, T., Leymann, F., et al.: Auto- abstract business model in a standardized manner. It can matic topology completion of tosca-based cloud applications. In: GI- Jahrestagung. pp. 247–258 (2014) reduce deployment and migration time of application [15] Homer, A., Sharp, J., Brader, L., Narumoto, M., Swanson, T.: Cloud components enormously. Our CloudCAMP framework design patterns: Prescriptive architecture guidance for cloud applications can be extended, and the application components are (2014) [16] Huber, N., Brosig, F., Kounev, S.: Model-based self-adaptive resource reusable and modularized. allocation in virtualized environments. In: Proceedings of the 6th Inter- 2) Business model verification. Business model will be ver- national Symposium on Software Engineering for Adaptive and Self- ified via a set of pre-defined constraints, which collected Managing Systems. pp. 90–99. ACM (2011) [17] Huber, N., Brosig, F., Spinner, S., Kounev, S., Bahr, M.: Model-based by reverse engineering and prototyping. self-aware performance and resource management using the descartes 3) Model@Runtime framework. We will monitor and reflect modeling language. IEEE Transactions on Software Engineering (2016) the current system information in the model-level, so the [18] Humble, J., Molesky, J.: Why enterprises must adopt devops to enable continuous delivery. Cutter IT Journal 24(8), 6 (2011) business user can understand application design state and [19] Keller, A., Hellerstein, J.L., Wolf, J.L., Wu, K.L., Krishnan, F.: The can override the architecture if necessary. champs system: change management with planning and scheduling. In: 4) Integration of Resource Management Techniques. To Network Operations and Management Symposium, 2004. NOMS 2004. IEEE/IFIP. vol. 1, pp. 395–408. IEEE (2004) build the context-aware self-service framework, we will [20] Ledeczi, A., Maroti, M., Bakay, A., Karsai, G., Garrett, J., Thomason, integrate the resource management algorithm, which C., Nordstrom, G., Sprinkle, J., Volgyesi, P.: The generic modeling will trigger the addition or migration of application environment. In: Workshop on Intelligent Signal Processing, Budapest, Hungary. vol. 17, p. 1 (2001) components to scale the system based on demand. [21] Loope, J.: Managing Infrastructure with Puppet. ” O’Reilly Media, Inc.” (2011) [22] Lu, H., Shtern, M., Simmons, B., Smit, M., Litoiu, M.: Pattern-based de- R EFERENCES ployment service for next generation clouds. In: Services (SERVICES), 2013 IEEE Ninth World Congress on. pp. 464–471. IEEE (2013) [1] Ardagna, D., Di Nitto, E., Casale, G., Petcu, D., Mohagheghi, P., [23] Maróti, M., Kecskés, T., Kereskényi, R., Broll, B., Völgyesi, P., Jurácz, Mosser, S., Matthews, P., Gericke, A., Ballagny, C., D’Andria, F., et al.: L., Levendovszky, T., Lédeczi, Á.: Next generation (meta) modeling: Modaclouds: A model-driven approach for the design and execution of Web-and cloud-based collaborative tool infrastructure. MPM@ MoD- applications on multiple clouds. In: Proceedings of the 4th International ELS 1237, 41–60 (2014) Workshop on Modeling in Software Engineering. pp. 50–56. IEEE Press [24] Morin, B., Barais, O., Jezequel, J.M., Fleurey, F., Solberg, A.: Models@ (2012) run. time to support dynamic adaptation. Computer 42(10) (2009) [2] Arnold, W., Eilam, T., Kalantar, M., Konstantinou, A.V., Totok, A.A.: [25] Narain, S., Levin, G., Malik, S., Kaul, V.: Declarative infrastructure Pattern based SOA deployment. Springer (2007) configuration synthesis and debugging. Journal of Network and Systems [3] Barve, Y., Patil, P., Bhattacharjee, A., Gokhale, A.: Pads: Design and Management 16(3), 235–258 (2008) implementation of a cloud-based, immersive learning environment for [26] Nelson-Smith, S.: Test-Driven Infrastructure with Chef: Bring Behavior- distributed systems algorithms. IEEE Transactions on Emerging Topics Driven Development to Infrastructure as Code. ” O’Reilly Media, Inc.” in Computing PP(99), 1–1 (2017) (2013) [4] Blair, G., Bencomo, N., France, R.B.: Models@ run. time. Computer [27] OASIS: Topology and orchestration specification for cloud appli- 42(10) (2009) cations. http://docs.oasis-open.org/tosca/TOSCA/v1.0/TOSCA-v1.0.pdf [5] Breitenbücher, U., Binz, T., Képes, K., Kopp, O., Leymann, F., Wet- (2013), oASIS Standard tinger, J.: Combining declarative and imperative cloud application pro- [28] Peiris, C., Sharma, D., Balachandran, B.: C2tp: a service model for visioning based on tosca. In: Cloud Engineering (IC2E), 2014 IEEE cloud. International Journal of Cloud Computing 1(1), 3–22 (2011) International Conference on. pp. 87–96. IEEE (2014) [29] Schmidt, D.C.: Model-driven engineering. COMPUTER-IEEE COM- [6] Breitenbücher, U., Binz, T., Kopp, O., Leymann, F., Wieland, M.: PUTER SOCIETY- 39(2), 25 (2006) Context-aware provisioning and management of cloud applications. In: [30] Shekhar, S., Chhokra, A.D., Bhattacharjee, A., Aupy, G., Gokhale, International Conference on Cloud Computing and Services Science. pp. A.: Indices: Exploiting edge resources for performance-aware cloud- 151–168. Springer (2015) hosted services. In: Fog and Edge Computing (ICFEC), 2017 IEEE 1st [7] Di Cosmo, R., Eiche, A., Mauro, J., Zacchiroli, S., Zavattaro, G., International Conference on. pp. 75–80. IEEE (2017) Zwolakowski, J.: Automatic deployment of services in the cloud with [31] Tekiner, F., Keane, J.A.: Big data framework. In: Systems, Man, and aeolus blender. In: International Conference on Service-Oriented Com- Cybernetics (SMC), 2013 IEEE International Conference on. pp. 1494– puting. pp. 397–411. Springer (2015) 1499. IEEE (2013) [8] Eilam, T., Elder, M., Konstantinou, A.V., Snible, E.: Pattern-based [32] Urgaonkar, B., Pacifici, G., Shenoy, P., Spreitzer, M., Tantawi, A.: An composite application deployment. In: 12th IFIP/IEEE International analytical model for multi-tier internet services and its applications. In: Symposium on Integrated Network Management (IM 2011) and Work- ACM SIGMETRICS Performance Evaluation Review. vol. 33, pp. 291– shops. pp. 217–224. IEEE (2011) 302. ACM (2005) [9] Fehling, C., Leymann, F., Retter, R., Schumm, D., Schupeck, W.: An [33] Yassa, S., Chelouah, R., Kadima, H., Granado, B.: Multi-objective architectural pattern language of cloud-based applications. In: Proceed- approach for energy-aware workflow scheduling in cloud computing ings of the 18th Conference on Pattern Languages of Programs. p. 2. environments. The Scientific World Journal 2013 (2013) ACM (2011) [34] Zheng, T.: Model-based Dynamic Resource Management Multi Tier [10] Feinerer, I.: Efficient large-scale configuration via integer linear pro- Information Systems. Ph.D. thesis, Carleton University Ottawa (2007) gramming. Artificial Intelligence for Engineering Design, Analysis and Manufacturing 27(01), 37–49 (2013) [11] Gambi, A., Toffetti, G., Pautasso, C., Pezze, M.: Kriging controllers for cloud applications. IEEE Internet Computing 17(4), 40–47 (2013) [12] Giannakopoulos, I., Papailiou, N., Mantas, C., Konstantinou, I., Tsoumakos, D., Koziris, N.: Celar: automated application elasticity platform. In: Big Data (Big Data), 2014 IEEE International Conference on. pp. 23–25. IEEE (2014)