Deployment and management of cloud-based applications in both homogeneous or heterogeneous cloud environments are often hampered by the intricacies and variabilities in the deployment and configuration processes stemming from the diversity in both the software stacks that control the cloud platforms and the underlying platform resources. Application provisioning is important since it enables the service providers to deploy their service's application components in the cloud provider's resources in accordance with the service's Quality of Services (QoS), availability, business policy, and cost model requirements [33], [28]. This complexity restricts the cloud application development and deployment knowledge to a narrow set of experts who possess both the domain knowledge and application-specific knowledge to explore the power of these techniques. Even with advances in DevOps community tools, they demand domain expertise and hence creating full-blown deployable models are error-prone and time-consuming.
To address these challenges, this doctoral dissertation envisions an intelligent and context-aware, model -driven, automated service provisioning platform where manual efforts and domain expertise are minimal. To that end, in this paper, we propose a model-driven engineering (MDE)-based ecosystem [29] to automate the process of deployment and management for cloud applications and emphasize on extensibility, (re-)usability and scalability. Our approach emphasizes vendor-neutrality in order to be broadly applicable to diverse cloud platforms with their control software stacks. In the remainder of this paper, we explore the challenges faced in realizing the MDE approach for deployment and management of cloud applications, the current state of the art, and propose solutions while providing some preliminary results.
Consider as an example a distributed, data-intensive application deployed in the cloud. It requires significant efforts for the designers -1) To configure and deploy the application to use the available big data frameworks [31], 2) To set up the virtual machines to host the frameworks and finally, and 3) To establish the connection properly among application components. Moreover, adding application component which requires a different framework exacerbates these problems. To address these problems, we envision an automated solution and outline the key requirements of such an approach and their associated technical challenges:
Need to abstract away the architecting phase by pre-defining the configuration of applications and infrastructure as intuitive modeling artifacts, which will speed up the deployment and migration process.
• Challenge 1: Capturing the Application and Cloud Specifications in the Metamodel, where all the deployment and infrastructure complexities of the application and cloud specification are identified and defined. • Challenge 2: Defining a Language for Model Transformation, where the DSML will transform the abstract business model to target model (i.e. infrastructure code) using a knowledge base. • Challenge3: Verification of the Business Model, where all relationships and constraints of application components are checked and satisfied.
Changes in business strategies and demands lead to evolution of the business model, and hence the platform should be integrated with adaptation techniques. migration of application components can be done at the model level, and should support collaboration and version control.
• Challenge 5: Extensibility of the Platform, where the addition of a new application type should be performed in a modularized way, and should be only a one-time effort.
Continuously monitoring the infrastructure of the system and applications are required, and hence the model should always reflect the current up-to-date state.
• Challenge 6: Context-Aware Provisioning, where optimization of cloud resources based on cost and resource availability should take place iteratively for potentially differing QoS needs, policies, etc. • Challenge 7: Model@Runtime, where the changes stemming from self-adaptation of the provisioning techniques should be reflected back at the model level dynamically [4].
We explore existing research efforts directly aligned with our research goals. The script-centric DevOps community provides toolchains such as Chef [26], Puppet [21], or Ansible 1 to speed up deployment for developers [18]. Cloudify and Apache Brooklyn 2 enable cloud application orchestration of topology templates in a vendor-agnostic way according to the Topology and Orchestration Specification for Cloud Applications (TOSCA) [27] specification. However, we need declarative high-level modeling abstraction on top of these tools to reduce the burden of domain expertise.
To satisfy Requirement 1, multiple pattern based approaches are proposed [2], [9], [8], [22] where a model-based pattern describes a set of capabilities and functional and non-functional properties of application service deployment in cloud infrastructures [15], [1]. Alternatively, requirement solvers [25], [7] are also proposed to synthesize infrastructure configuration in a declarative fashion. OpenTOSCA [14] and CELAR [12] exercise the combination of MDE and TOSCA specification to automate deployment of cloud applications from users' partial business relevant topology. The components can be deployed and managed in the right order [5]. Later in [6] a contextaware provisioning and management plan along with detection and correction of the unintended error due to software conflicts and dependency is proposed. However, their platform is not based on any standardization, and their planner does not use any DevOps community tools. Hence, their approach might lack extensibility and robustness.
In relation to Requirement 2, CHAMPS [19] focuses on Change Management process by which IT systems are efficiently modified to minimize reconfiguration impact. Similarly, various standard modeling environments as an objectoriented formalism for configuration problems is proposed in 1 https://www.ansible.com/ 2 https://brooklyn.apache.org/ [10], [13]. For Requirement 3 [24], [17], dynamic resource management on enterprise application vendors is proposed by various frameworks based on queuing model [34], [32], or by dynamic QoS Control [11], [16], [30].
We propose a composable ecosystem called CloudCAMP to deploy, monitor and manage cloud application intelligently. The workflow of the proposed approach is shown in Fig. 1. The contributions in this research include the following tasks:
• Task 1: A Deployment Framework Metamodel, to automate the infrastructure design and implementation by capturing variability and commonality endpoints for the application and cloud specifications along with their SDK and APIs, to provide abstractions based on TOSCAspecifications. This metamodel details will be stored in the knowledge base of the platform. • Task 2: Model-to-Infrastructure-as-code Transformation DSML, as the basis for the deployment time and runtime composition and orchestration through generative programming. The DSML will also be integrated with predefined constraints to verify the correctness of business model. • Task 3: An Integrated Repository, that allows storage, query, and manipulation of models, templates, metadata, parameters, software dependency knowledge base and provenance in a scalable data store. • Task 4: A Model@Runtime framework, that permits reflection of the current model by capturing and monitoring the streamlined system and application information across heterogeneous resources at runtime. • Task 5: Integration of Adaptive Resource Provisioning Engine, to provide the QoS guarantees while scaling the application vertically and horizontally, based on policy, economic issues, etc. The approach to integrating diverse sets of deployment and management building blocks for a specific application and cloud providers offers significant advantages over traditional methods -such as DevOps or orchestration frameworksby applying a blend of generative programming and micro service architectures. Concrete implementation our proposed generative approach will be demonstrated in a cloud-based MDE environment called WebGME [23].
Our CloudCAMP approach to integrating a diverse set of deployment and management building blocks for a specific application and cloud providers offers significant advantages over traditional methods -such as DevOps or orchestration frameworks -by applying a blend of generative programming and micro service architectures.
A concrete implementation of our MDE-based, generative approach called CloudCAMP is being developed within a cloud-based MDE environment called WebGME [23]. A preliminary version of CloudCAMP supports few application The development of CloudCAMP comprises of the design and evaluation of the expressivity of the metamodel and the DSML with all the product-line features. Figure 2 provides a timeline for this research, conducted under the supervision of Dr. Aniruddha Gokhale, Associate Professor, Vanderbilt University.
To demonstrate the deployment and management ecosystem, we are developing the CloudCAMP platform based on Generic Modeling Environment (GME) [20] as Task 1 and 2.
Our research evaluation plans call for validating the framework in the context of our ongoing and upcoming industry and federally sponsored research projects. The source code is available at https://anirban2404.github.io/DeploymentAutomation/. For example, one of our new projects is building a cloud-based machine learning ecosystem which illustrates many of the needs addressed by our research. We also plan to validate our framework in cloud-based, collaborative education services for STEM education [3].
CloudCAMP will ease the deployment and management of applications based on deployment abstraction and intelligent resource optimization. The following key contributions are expected from this research:
The following key contributions are expected from this research:
1) Automated generation of infrastructure code from an abstract business model in a standardized manner. It can reduce deployment and migration time of application components enormously. Our CloudCAMP framework can be extended, and the application components are reusable and modularized. 2) Business model verification. Business model will be verified via a set of pre-defined constraints, which collected by reverse engineering and prototyping. 3) Model@Runtime framework. We will monitor and reflect the current system information in the model-level, so the business user can understand application design state and can override the architecture if necessary. 4) Integration of Resource Management Techniques. To build the context-aware self-service framework, we will integrate the resource management algorithm, which will trigger the addition or migration of application components to scale the system based on demand.