C ONSOLAS: A Model-Based Tool for Automatic Configuration and Deployment of Cloud Applications Hui Song, Franck Chauvel, Franck Fleurey, Nicolas Ferry, Arnor Solberg SINTEF ICT, Oslo, Norway. Email: {first.last}@sintef.no Abstract—This paper demonstrates C ONSOLAS, an automatic cloud resources, as well as the constraints and best patterns tool for the configuration and deployment of software applica- to configure their applications. Using the specifications as a tions in cloud. We provide Domain-Specific Modelling Languages reference, C ONSOLAS helps operators to configure and deploy for application developers to specify the components in the application and the possible cloud resources to host them, as the application iteratively: In each iteration, the operators give well as the constraints between them. Based on the specifications, simple hints on what configuration they expect or make small C ONSOLAS assists application operators in configuring and changes on the existing configuration, and C ONSOLAS returns deploying the application automatically. Operators only need to a complete and valid configuration for the next iteration. The provide simple hints on how they want to configure the applica- configurations are in the form of architectural models, which tion, and the tool generates a complete and valid configuration and deploys it. C ONSOLAS also supports operators in refining the are intuitive for operators to understand and will be deployed configuration both before and after the deployment: Operators fully automatically into the mainstream cloud platforms by make small and partial changes on an existing configuration, C ONSOLAS. The technical basis of the tool is our research on and the tool automatically completes the changes and performs multi-cloud modelling, language engineering, and constraint incremental deployment. We demo the tool with a sample case, solving on software architectures. and a video can be found at https://youtu.be/V9WWeFi1ZD8 . In the rest of this paper, we use a sample cloud applica- I. I NTRODUCTION tion (introduced in Section II) to show how developers use Many people choose to run software applications in a cloud C ONSOLAS to specify their applications (in Section IV), and environment. They obtain applications from public reposito- how operators use it to automatically configure and deploy ries such as GitHub, configure them according to their own the applications (in Section V). A demonstration video on the requirements, and deploy them on the virtual machines (VMs) same case can be found at https://youtu.be/V9WWeFi1ZD8 . that they provision from a private or a public cloud. In this way, these application operators provide their own services to II. S AMPLE C ASE customers based on the existing applications. S MART GH [1] is a route planning application developed Configuration and deployment of cloud applications is chal- by Trinity College Dublin. It searches routes on a city scale lenging. Unlike traditional desktop applications or the modern taking into consideration the sensor data such as pollution and mobile Apps that can be downloaded in a monolithic way noise. Figure 1 summarises its main components: Sensors and launched with simple set-ups, a cloud application usually collect data from city sensors or public data sources and consists of multiple components that can be hosted by different populate them into a Redis database. Hoppers link the VMs. Moreover, for performance or security purpose, the same sensor readings to OpenStreetMap maps to plan smart routes. component may have multiple instances scattered in different Webs receive routing requests and display the results from VMs. Each component may have its own parameters and Hoppers. An adaptable Load-Balancer forwards re- depend on several other components. To configure a cloud quests to the user-preferred Web. For the sake of diversity, application, operators need to decide how many instances to developers provide different Hoppers specialized for foot create for each component type, set the parameters for each routing, fast car routing (FCHopper), etc., and also different instance, link them together, and assign them to proper VMs. types of Sensors. Each component is wrapped as a docker These decision points are sophisticatedly related, because of image, which can be obtained at hub.docker.com/u/songhui/. the many constraints existing in the application or the cloud Third-party operators can configure and deploy S MART GH environment, and therefore, it is difficult and tedious to reach on their own IaaS (Infrastructure as a Service) environments, a globally valid configuration. A configuration is also hard to providing a diverse of smart routing services. Figure 2 shows refine and adjust - even a small change may require a lot of a sample configuration, where three Hoppers and their Webs subsequent revisions on different parts of the configuration to are hosted by two virtual machines. One Hopper utilizes the make it globally valid. data from a PollutionSensor, via a Redis database. The In this paper, we present the C ONSOLAS (CONstraint essential part of this configuration, as is highlighted by greyed SOLving for Architecture Setup) tool for model-based, auto- boxes in Figure 2, is to decide what types of hopper and matic configuration and deployment of cloud applications. We sensors to provide, and how many instances to create for each provide domain specific modelling languages (DSMLs) for ap- type. This part is interested to the operator, because it defines plication developers to specify the components and their target the features provided by his/her routing service. However, the Lb Deployable deploy Vm CloudML Specify applications rmem : Integer developer vmem : Integer and resources type model port: Integer Web consolas LocalRedis Sensor Specify constraints / generate constraint hp Hopper patterns / cost satisfaction problem "rmem=2" db sdb OpenStackLarge FootHopper CarHopper Redis "vmem = 4" configuration hints solve complete (partial configuration/ OpenStackHuge architecture small changes) NoiseSensor operators FastCH NormalCH "vmem = 8" CloudML "rmem=4" "rmem=2" PollutionSensor topology Fig. 1. S MART GH components model no satisfied ? w1:Web w2:Web w3:Web lb:LoadBalancer yes Automatic lb provisioning and port: 81 port: 82 port: 83 port: 80 yes mem:1 mem:1 mem:1 mem:0 deployment need hp change db LocalRedis running system h1:FastCH h2:FootHopper h3:FootHopper rmem: 4 rmem: 1 rmem: 1 rmem: 2 Fig. 3. C ONSOLAS process overview port:6397 port:8080 port:8081 port:8080 sdb deploy deploy s1:PollutionSensor v1:OpenStackHuge v2:OpenStackLarge make the changes that they cares, and the tool completes the rmem: 1 vmem: 8 vmem: 4 port:0 required co-changes automatically, and deploys all the changes Fig. 2. S MART GH components to the system incrementally. III. T OOL OVERVIEW operator has additional tedious jobs to do to reach a proper Figure 3 summarises the process of using C ONSOLAS to S MART GH configuration, such as to arrange the component configure and deploy a cloud application. The shadowed part instances into proper VMs, add auxiliary components (i.e., of the flowchart is the internal process of the tool, and the web, load balancers, database), link the components, assign rest is the activities and artefacts required by tool users. We port numbers, etc. These additional jobs are difficult because identify two different roles of tool users, i.e., the developers there are many constraints that the operator must obey. For and the operators, and divide the process into two parts, each example, a VM cannot host so many components that the for a different user role. total memory consumption of these components exceeds the Developers specify the types of components and resources. VM’s capacity; two components on the same VM should After that, they specify the constraints that a proper con- have different port numbers; every Hopper needs its own figuration should follow. The artefacts produced by the two Web; a Hopper and its database (if there is one) should activities are a type model specified in C LOUD ML (a DSML better be hosted by the same VM, etc. An ideal situation for cloud specification developed by SINTEF) and a set of would be that that operators only suggest the essential part First-Order Logic constraints (specified by a python-based of the configuration, and all the tedious part is automatically DSL, implemented for C ONSOLAS). Details and samples of generated according to the constraints and patterns. This is the specifications can be found in Section IV. the automatic configuration supported by C ONSOLAS. It also The specifications are machine-readable to C ONSOLAS, deploys the configuration automatically into the IaaS platform. which helps operators configure their own services from the Application configuration is not a once-for-all task, and application. The process starts from the operators’ providing a operators may need to adjust their S MART GH configurations partial configuration as a hint. C ONSOLAS then automatically either before they are deployed, or after they have been running generates a valid configuration, and visualize it to the operator. for a while, to optimise the service or to meet new user If the operators are not satisfied with the configuration, they requirements. For example, when there are more user requests can make changes on it and ask C ONSOLAS to complete the for walking routes, the operator may want to add a new changes. This loop continues until a satisfying configuration FootHopper into the configuration in Figure 2. For such ad- is obtained, and C ONSOLAS will automatically deploy the justment, a small change may require complicated co-changes configuration. Finally, after the application is already running, to reach a valid configuration again. For the sample scenario, operators can still change the configuration. Their changes, the co-changes will be to add a Web for the new hopper, along with the necessary subsequent changes suggested by migrate a component from v1 to v2 to save space for the new C ONSOLAS, will be deployed into the system in an in- components, linking it to the database, etc.. C ONSOLAS also cremental way. Details about how operators provide partial supports such agile and incremental reconfiguration: Operators configuration and modify it can be found in Section V. 1 deployment model SmartGHDiv 1 g_forall((x, Hopper), 2 provider openstack_nova : ... 2 g_exists((y, Web), hp(y)==x))) 3 types{ 3 g_forall([(x, Hopper)], 4 vm OpenStackLarge{ 4 soft(50, Implies(alive(db(x)), 5 provider : openstack_nova 5 deploy(x)==deploy(db(y))))) 6 ram : 4096, core : 1, 6 g_forall([(x, VM)], 7 os : "ubuntu", os64, 7 soft(mem(x)*10, Not(alive(x)))) 8 provided host ubuntuPrv 9 ...} Fig. 5. Sample constraints for S MART GH 10 internal component FootHopper{ 11 required host ubuntuReq 12 provided communication hopperPrv needs a database, then the two components should better be 13 required communication redisReq deployed on the same VM. This is a soft constraint with a 14 resource DockerImage{ priority as 50 (in the middle between 0 and 100). Finally, the 15 download: "sudo docker pull..." 16 start: "cd ˜; sudo docker run..." last constraint sets up a cost to use any VM instance, which is 17 } correlated to the memory size of the VM. This soft constraints 18 communication hp prevents C ONSOLAS from using up too much cloud resources. 19 from hopperReq to hopperPrv... 20 } V. AUTOMATIC C ONFIGURATION AND D EPLOYMENT 21 instances{\\left empty for auto config} C ONSOLAS helps operators automatically configure and 22 } deploy a cloud application in their own ways. In this section, Fig. 4. Excerpt of C LOUD ML specification for S MART GH we first use a simple scenario to illustrate the configuration and deployment process, and then briefly introduce the research approaches that enable this automatic process. IV. S PECIFICATION OF R ESOURCE AND A PPLICATION A. A configuration and deployment scenario C ONSOLAS needs application developers to specify the As a sample scenario, We show how an operator uses C ON - component and resources types, and the constraints on them. SOLAS to achieve the S MART GH configuration as described The specification is similar to the “INSTALL” manuals of in Section II and Figure 2, and deploy it on a private cloud. traditional software applications, but are machine-readable to Before configuration, the operator needs to have the applica- the C ONSOLAS tool. We provide two languages to support the tion and resource specification as described in Section IV, and specifications. an account from a cloud provider (in this case, it is SINTEF’s C ONSOLAS integrates an existing language named private cloud on OpenStack). C LOUD ML for the specification of applications and cloud In the first iteration, the operator gives no input to C ON - resources. Figure 4 shows an excerpt of the C LOUD ML SOLAS (i.e., an empty hint), and C ONSOLAS returns an initial model for S MART GH. In the types section, the S MART GH configuration, with only a BasicHopper and a Web hosted developers define the resource and component types, ı.e., by one VM. This is obviously one of the smallest valid the boxes in Figure 1. For a resource type, such as the configuration of S MART GH, and C ONSOLAS does not add VM OpenStackLarge, they specify the cloud provider, any additional component because it has a cost to do so (i.e., some common arguments across all the instances of this breaking the software constraints like Line 6 of Figure 5). type (such as 4096MB of RAM, 64bit Ubuntu OS, etc.), and The initial configuration does not satisfy the operator, who the hosting service they can provide. The developers also expects two instances of FootHoppers, one instance of define a component type FootHopper: It requires a hosting FastHopper, and a PollutionSensor. The operator service (not surprisingly the same as the one provided by inputs this requirement into C ONSOLAS as the following OpenStackLarge) and a communication service from a configuration hints. database. The component can be downloaded and started using typeof(h1)==FastCH ! the specified docker commands. Finally, a communication typeof(h2)==FootHopper ! relation connects the communication services provided by typeof(h3)==FootHopper ! Hoppers to the service required by Web. The instances And(db(h3)==sdb(s1),typeof(s1)==PollutionSensor)! section of this model is left empty, which will be filled with The hints are essentially simple constraints on the config- the automatically generated configuration. uration, and are written in the same C ONSOLAS constraint C ONSOLAS provides another domain-specific language for language as in Section IV. The first three hints require three developers to specify constraints on top of the type specifica- hoppers to be of specific types, and “!” means that the hints tions. We design and implement the language as an embedded apply to all the iterations afterwards. The last hint requires one on the basis of Python. Figure 5 shows three sample that h3 connects to a PollutionSensor via a database. constraints specified by the C ONSOLAS constraint language. The names of component instances (e.g., h2, s1) are syntax The first constraint specifies that for any Hopper, there must sugars for operators to avoid the length expressions such as be a Web for it. The second one specifies that if a Hopper ”there exists three Hoppers, they are different, and their types are...” Receiving these hints, C ONSOLAS automatically returns framework [3]. It first instantiates the required VMs, and then the configuration that has been shown in Figure 2. Only the logs-in to the VMs to execute the download, configure greyed component instances are directly required by the hints, and start commands specified with each component. but all the rest in the configuration is automatically generated by C ONSOLAS. For example, it generates two Webs for the VI. R ELATED W ORK two new Hoppers, according to the constraint at Line 1 of Chef [4] and Puppet [5] are the state-of-the-art cloud Figure 5, enlarges v1 into an OpenStackHuge VM in order management tools, and both of them support the configuration to be able to host all the Webs and two Hoppers. Then it of cloud applications based on scripting languages. Scripts instantiates a LoadBalancer because there are more than are useful when defining command sequences to configure one Webs. Finally, according to the last hints, it creates a and deploy individual components, and these tools provide PollutionSensor s1 and a LocalRedis between s1 mechanisms to reuse scripts for existing components. Yet and h1. According to the constraints in Figure 5 Line 3, h3 operators still need to manually design what components and and r1 are deployed to the same v2. resources to use and how to link them together. From this point Now the operator is satisfied with the configuration, and she of view, C ONSOLAS’s automatic architecture configuration is provides C ONSOLAS the endpoint and the account credential an ideal complement to these tools. of her target cloud platform and launches the deploy command. The automatic configuration in C ONSOLAS shares the sim- C ONSOLAS will automatically create two VMs, and instan- ilar idea with the research on auto-completion and auto cor- tiates 9 docker containers and link them together according rection of modelling tools, such as Egyed’s work for UML [6] to the configuration. After the automatic deployment process, and Xiong et al.’s work on Feature Model [7]. However, the operator can get the IP address of v1 (where the load model editors suggest completions based on general-purpose balancer is deployed) from the deployment log, and use port rules or constraints, whereas C ONSOLAS allows developers to 80 to access the routing services. define application-specific constraints. Moreover, as a tool for After the system is running, the operator can still modify automatic deployment, C ONSOLAS always looks for complete the configuration. For example, for performance purpose, she configurations rather than scattered completion suggestions. can require w3 and h3 to be hosted in the same VM: deploy(w3)==deploy(hp(w3)) VII. C ONCLUSION Due the resource limitation, C ONSOLAS will create a new VM This paper introduces the C ONSOLAS tool for automatic in OpenStackHuge, migrate all the components on v2, as configuration and deployment of cloud applications. Based well as w3 to the new VM, and finally terminate v2. on developers’ specification on the application, resource and constraints, C ONSOLAS generates configurations of the ap- B. Research behind the tool plication and deploys them automatically to the cloud en- The automatic configuration and deployment of C ONSOLAS vironment. Future work will be focused on improving the are based on our research of constraint-driven architecture usability, such as providing a web-based GUI and a set of configuration and C LOUD ML, respectively. C LOUD ML is a template for application and constraint specification. We will language for the modelling of cloud resources and topology. also investigate how to infer the user preference from their C ONSOLAS translate the C LOUD ML model together with the refinement iterations. constraints into a Satisfactory Modulo Theory (SMT) problem: Component instances are represented by enumeration items. ACKNOWLEDGEMENT Types, attributes, and connection between components are This work is supported by the EU FP7-ICT-2011-9 No. represented by uninterpreted functions. The constraints are 600654 DIVERSIFY project translated into First Order Logic assertions on these functions, and can be either hard or soft. During the configuration R EFERENCES process, C ONSOLAS first transform the operators’ hints into [1] V. Nallur, A. Elgammal, and S. Clarke, “Smart Route Planning constraints and integrate them into the SMT problem. After Using Open Data and Participatory Sensing,” in Open Source Systems: that, it launches an constraint solving process to search for Adoption and Impact. Springer, 2015, pp. 91–100. [Online]. Available: https://github.com/DIVERSIFY-project/SMART-GH a solution to the SMT problem, i.e., giving an interpretation [2] H. Song, X. Zhang, N. Ferry, F. Chauvel, A. Solberg, and G. Huang, to each of the functions. The solution satisfies all the hard “Modelling adaptation policies as domain-specific constraints,” in Model- constraints and minimize the total costs of violated soft con- Driven Engineering Languages and Systems, 2014, pp. 269–285. [3] N. Ferry, H. Song, A. Rossini, F. Chauvel, and A. Solberg, “Cloud straints. For incremental configuration, C ONSOLAS will also MF: Applying MDE to Tame the Complexity of Managing Multi-cloud generate a set of additional soft constraints according to the Applications,” in Proceedings of the 2014 IEEE/ACM 7th International previous configuration, so that the new configuration will have Conference on Utility and Cloud Computing, 2014, pp. 269–277. [4] Cloud Management Chef, https://www.chef.io. as little deviation as possible from the previous one. Due to [5] The Puppet Lab, https://puppetlabs.com/. the space limitation, we will not go into the details of the SMT [6] A. Egyed, “Fixing inconsistencies in uml design models,” in Software problem and the transformation. Interested readers can refer Engineering, 2007. ICSE 2007. 29th International Conference on. IEEE, 2007, pp. 292–301. to our previous publication [2]. C ONSOLAS transforms the [7] Y. Xiong, A. Hubaux, S. She, and K. Czarnecki, “Generating range fixes solution back into a C LOUD ML instance model for automatic for software configuration,” in Software Engineering (ICSE), 2012 34th deployment. The deployment is done by the C LOUD ML International Conference on. IEEE, 2012, pp. 58–68.