=Paper=
{{Paper
|id=Vol-1554/PD_MoDELS_2015_paper_3
|storemode=property
|title=Consolas: A Model-Based Tool for Automatic Configuration and Deployment of Cloud Applications
|pdfUrl=https://ceur-ws.org/Vol-1554/PD_MoDELS_2015_paper_3.pdf
|volume=Vol-1554
|authors=Hui Song,Franck Chauvel,Franck Fleurey,Nicolas Ferry,Arnor Solberg
|dblpUrl=https://dblp.org/rec/conf/models/SongCFFS15
}}
==Consolas: A Model-Based Tool for Automatic Configuration and Deployment of Cloud Applications==
https://ceur-ws.org/Vol-1554/PD_MoDELS_2015_paper_3.pdf
C ONSOLAS: A Model-Based Tool for Automatic
Configuration and Deployment of Cloud Applications
Hui Song, Franck Chauvel, Franck Fleurey, Nicolas Ferry, Arnor Solberg
SINTEF ICT, Oslo, Norway. Email: {first.last}@sintef.no
Abstract—This paper demonstrates C ONSOLAS, an automatic cloud resources, as well as the constraints and best patterns
tool for the configuration and deployment of software applica- to configure their applications. Using the specifications as a
tions in cloud. We provide Domain-Specific Modelling Languages reference, C ONSOLAS helps operators to configure and deploy
for application developers to specify the components in the
application and the possible cloud resources to host them, as the application iteratively: In each iteration, the operators give
well as the constraints between them. Based on the specifications, simple hints on what configuration they expect or make small
C ONSOLAS assists application operators in configuring and changes on the existing configuration, and C ONSOLAS returns
deploying the application automatically. Operators only need to a complete and valid configuration for the next iteration. The
provide simple hints on how they want to configure the applica- configurations are in the form of architectural models, which
tion, and the tool generates a complete and valid configuration
and deploys it. C ONSOLAS also supports operators in refining the are intuitive for operators to understand and will be deployed
configuration both before and after the deployment: Operators fully automatically into the mainstream cloud platforms by
make small and partial changes on an existing configuration, C ONSOLAS. The technical basis of the tool is our research on
and the tool automatically completes the changes and performs multi-cloud modelling, language engineering, and constraint
incremental deployment. We demo the tool with a sample case, solving on software architectures.
and a video can be found at https://youtu.be/V9WWeFi1ZD8 .
In the rest of this paper, we use a sample cloud applica-
I. I NTRODUCTION tion (introduced in Section II) to show how developers use
Many people choose to run software applications in a cloud C ONSOLAS to specify their applications (in Section IV), and
environment. They obtain applications from public reposito- how operators use it to automatically configure and deploy
ries such as GitHub, configure them according to their own the applications (in Section V). A demonstration video on the
requirements, and deploy them on the virtual machines (VMs) same case can be found at https://youtu.be/V9WWeFi1ZD8 .
that they provision from a private or a public cloud. In this
way, these application operators provide their own services to II. S AMPLE C ASE
customers based on the existing applications. S MART GH [1] is a route planning application developed
Configuration and deployment of cloud applications is chal- by Trinity College Dublin. It searches routes on a city scale
lenging. Unlike traditional desktop applications or the modern taking into consideration the sensor data such as pollution and
mobile Apps that can be downloaded in a monolithic way noise. Figure 1 summarises its main components: Sensors
and launched with simple set-ups, a cloud application usually collect data from city sensors or public data sources and
consists of multiple components that can be hosted by different populate them into a Redis database. Hoppers link the
VMs. Moreover, for performance or security purpose, the same sensor readings to OpenStreetMap maps to plan smart routes.
component may have multiple instances scattered in different Webs receive routing requests and display the results from
VMs. Each component may have its own parameters and Hoppers. An adaptable Load-Balancer forwards re-
depend on several other components. To configure a cloud quests to the user-preferred Web. For the sake of diversity,
application, operators need to decide how many instances to developers provide different Hoppers specialized for foot
create for each component type, set the parameters for each routing, fast car routing (FCHopper), etc., and also different
instance, link them together, and assign them to proper VMs. types of Sensors. Each component is wrapped as a docker
These decision points are sophisticatedly related, because of image, which can be obtained at hub.docker.com/u/songhui/.
the many constraints existing in the application or the cloud Third-party operators can configure and deploy S MART GH
environment, and therefore, it is difficult and tedious to reach on their own IaaS (Infrastructure as a Service) environments,
a globally valid configuration. A configuration is also hard to providing a diverse of smart routing services. Figure 2 shows
refine and adjust - even a small change may require a lot of a sample configuration, where three Hoppers and their Webs
subsequent revisions on different parts of the configuration to are hosted by two virtual machines. One Hopper utilizes the
make it globally valid. data from a PollutionSensor, via a Redis database. The
In this paper, we present the C ONSOLAS (CONstraint essential part of this configuration, as is highlighted by greyed
SOLving for Architecture Setup) tool for model-based, auto- boxes in Figure 2, is to decide what types of hopper and
matic configuration and deployment of cloud applications. We sensors to provide, and how many instances to create for each
provide domain specific modelling languages (DSMLs) for ap- type. This part is interested to the operator, because it defines
plication developers to specify the components and their target the features provided by his/her routing service. However, the
Lb Deployable
deploy Vm CloudML
Specify applications
rmem : Integer
developer
vmem : Integer and resources type model
port: Integer
Web
consolas
LocalRedis Sensor Specify constraints / generate constraint
hp
Hopper patterns / cost satisfaction problem
"rmem=2"
db sdb OpenStackLarge
FootHopper CarHopper Redis "vmem = 4" configuration hints
solve complete
(partial configuration/
OpenStackHuge architecture
small changes)
NoiseSensor
operators
FastCH NormalCH "vmem = 8"
CloudML
"rmem=4" "rmem=2" PollutionSensor
topology
Fig. 1. S MART GH components model
no satisfied
?
w1:Web w2:Web w3:Web lb:LoadBalancer yes Automatic
lb provisioning and
port: 81 port: 82 port: 83 port: 80 yes
mem:1 mem:1 mem:1 mem:0 deployment
need
hp change
db
LocalRedis running system
h1:FastCH h2:FootHopper h3:FootHopper
rmem: 4 rmem: 1 rmem: 1
rmem: 2 Fig. 3. C ONSOLAS process overview
port:6397
port:8080 port:8081 port:8080
sdb
deploy deploy
s1:PollutionSensor
v1:OpenStackHuge v2:OpenStackLarge make the changes that they cares, and the tool completes the
rmem: 1
vmem: 8 vmem: 4 port:0
required co-changes automatically, and deploys all the changes
Fig. 2. S MART GH components
to the system incrementally.
III. T OOL OVERVIEW
operator has additional tedious jobs to do to reach a proper Figure 3 summarises the process of using C ONSOLAS to
S MART GH configuration, such as to arrange the component configure and deploy a cloud application. The shadowed part
instances into proper VMs, add auxiliary components (i.e., of the flowchart is the internal process of the tool, and the
web, load balancers, database), link the components, assign rest is the activities and artefacts required by tool users. We
port numbers, etc. These additional jobs are difficult because identify two different roles of tool users, i.e., the developers
there are many constraints that the operator must obey. For and the operators, and divide the process into two parts, each
example, a VM cannot host so many components that the for a different user role.
total memory consumption of these components exceeds the Developers specify the types of components and resources.
VM’s capacity; two components on the same VM should After that, they specify the constraints that a proper con-
have different port numbers; every Hopper needs its own figuration should follow. The artefacts produced by the two
Web; a Hopper and its database (if there is one) should activities are a type model specified in C LOUD ML (a DSML
better be hosted by the same VM, etc. An ideal situation for cloud specification developed by SINTEF) and a set of
would be that that operators only suggest the essential part First-Order Logic constraints (specified by a python-based
of the configuration, and all the tedious part is automatically DSL, implemented for C ONSOLAS). Details and samples of
generated according to the constraints and patterns. This is the specifications can be found in Section IV.
the automatic configuration supported by C ONSOLAS. It also The specifications are machine-readable to C ONSOLAS,
deploys the configuration automatically into the IaaS platform. which helps operators configure their own services from the
Application configuration is not a once-for-all task, and application. The process starts from the operators’ providing a
operators may need to adjust their S MART GH configurations partial configuration as a hint. C ONSOLAS then automatically
either before they are deployed, or after they have been running generates a valid configuration, and visualize it to the operator.
for a while, to optimise the service or to meet new user If the operators are not satisfied with the configuration, they
requirements. For example, when there are more user requests can make changes on it and ask C ONSOLAS to complete the
for walking routes, the operator may want to add a new changes. This loop continues until a satisfying configuration
FootHopper into the configuration in Figure 2. For such ad- is obtained, and C ONSOLAS will automatically deploy the
justment, a small change may require complicated co-changes configuration. Finally, after the application is already running,
to reach a valid configuration again. For the sample scenario, operators can still change the configuration. Their changes,
the co-changes will be to add a Web for the new hopper, along with the necessary subsequent changes suggested by
migrate a component from v1 to v2 to save space for the new C ONSOLAS, will be deployed into the system in an in-
components, linking it to the database, etc.. C ONSOLAS also cremental way. Details about how operators provide partial
supports such agile and incremental reconfiguration: Operators configuration and modify it can be found in Section V.
1 deployment model SmartGHDiv 1 g_forall((x, Hopper),
2 provider openstack_nova : ... 2 g_exists((y, Web), hp(y)==x)))
3 types{ 3 g_forall([(x, Hopper)],
4 vm OpenStackLarge{ 4 soft(50, Implies(alive(db(x)),
5 provider : openstack_nova 5 deploy(x)==deploy(db(y)))))
6 ram : 4096, core : 1, 6 g_forall([(x, VM)],
7 os : "ubuntu", os64, 7 soft(mem(x)*10, Not(alive(x))))
8 provided host ubuntuPrv
9 ...} Fig. 5. Sample constraints for S MART GH
10 internal component FootHopper{
11 required host ubuntuReq
12 provided communication hopperPrv needs a database, then the two components should better be
13 required communication redisReq deployed on the same VM. This is a soft constraint with a
14 resource DockerImage{
priority as 50 (in the middle between 0 and 100). Finally, the
15 download: "sudo docker pull..."
16 start: "cd ˜; sudo docker run..." last constraint sets up a cost to use any VM instance, which is
17 } correlated to the memory size of the VM. This soft constraints
18 communication hp prevents C ONSOLAS from using up too much cloud resources.
19 from hopperReq to hopperPrv...
20 } V. AUTOMATIC C ONFIGURATION AND D EPLOYMENT
21 instances{\\left empty for auto config} C ONSOLAS helps operators automatically configure and
22 }
deploy a cloud application in their own ways. In this section,
Fig. 4. Excerpt of C LOUD ML specification for S MART GH we first use a simple scenario to illustrate the configuration and
deployment process, and then briefly introduce the research
approaches that enable this automatic process.
IV. S PECIFICATION OF R ESOURCE AND A PPLICATION
A. A configuration and deployment scenario
C ONSOLAS needs application developers to specify the As a sample scenario, We show how an operator uses C ON -
component and resources types, and the constraints on them. SOLAS to achieve the S MART GH configuration as described
The specification is similar to the “INSTALL” manuals of in Section II and Figure 2, and deploy it on a private cloud.
traditional software applications, but are machine-readable to Before configuration, the operator needs to have the applica-
the C ONSOLAS tool. We provide two languages to support the tion and resource specification as described in Section IV, and
specifications. an account from a cloud provider (in this case, it is SINTEF’s
C ONSOLAS integrates an existing language named private cloud on OpenStack).
C LOUD ML for the specification of applications and cloud In the first iteration, the operator gives no input to C ON -
resources. Figure 4 shows an excerpt of the C LOUD ML SOLAS (i.e., an empty hint), and C ONSOLAS returns an initial
model for S MART GH. In the types section, the S MART GH configuration, with only a BasicHopper and a Web hosted
developers define the resource and component types, ı.e., by one VM. This is obviously one of the smallest valid
the boxes in Figure 1. For a resource type, such as the configuration of S MART GH, and C ONSOLAS does not add
VM OpenStackLarge, they specify the cloud provider, any additional component because it has a cost to do so (i.e.,
some common arguments across all the instances of this breaking the software constraints like Line 6 of Figure 5).
type (such as 4096MB of RAM, 64bit Ubuntu OS, etc.), and The initial configuration does not satisfy the operator, who
the hosting service they can provide. The developers also expects two instances of FootHoppers, one instance of
define a component type FootHopper: It requires a hosting FastHopper, and a PollutionSensor. The operator
service (not surprisingly the same as the one provided by inputs this requirement into C ONSOLAS as the following
OpenStackLarge) and a communication service from a configuration hints.
database. The component can be downloaded and started using typeof(h1)==FastCH !
the specified docker commands. Finally, a communication typeof(h2)==FootHopper !
relation connects the communication services provided by typeof(h3)==FootHopper !
Hoppers to the service required by Web. The instances And(db(h3)==sdb(s1),typeof(s1)==PollutionSensor)!
section of this model is left empty, which will be filled with The hints are essentially simple constraints on the config-
the automatically generated configuration. uration, and are written in the same C ONSOLAS constraint
C ONSOLAS provides another domain-specific language for language as in Section IV. The first three hints require three
developers to specify constraints on top of the type specifica- hoppers to be of specific types, and “!” means that the hints
tions. We design and implement the language as an embedded apply to all the iterations afterwards. The last hint requires
one on the basis of Python. Figure 5 shows three sample that h3 connects to a PollutionSensor via a database.
constraints specified by the C ONSOLAS constraint language. The names of component instances (e.g., h2, s1) are syntax
The first constraint specifies that for any Hopper, there must sugars for operators to avoid the length expressions such as
be a Web for it. The second one specifies that if a Hopper ”there exists three Hoppers, they are different, and their types
are...” Receiving these hints, C ONSOLAS automatically returns framework [3]. It first instantiates the required VMs, and then
the configuration that has been shown in Figure 2. Only the logs-in to the VMs to execute the download, configure
greyed component instances are directly required by the hints, and start commands specified with each component.
but all the rest in the configuration is automatically generated
by C ONSOLAS. For example, it generates two Webs for the VI. R ELATED W ORK
two new Hoppers, according to the constraint at Line 1 of Chef [4] and Puppet [5] are the state-of-the-art cloud
Figure 5, enlarges v1 into an OpenStackHuge VM in order management tools, and both of them support the configuration
to be able to host all the Webs and two Hoppers. Then it of cloud applications based on scripting languages. Scripts
instantiates a LoadBalancer because there are more than are useful when defining command sequences to configure
one Webs. Finally, according to the last hints, it creates a and deploy individual components, and these tools provide
PollutionSensor s1 and a LocalRedis between s1 mechanisms to reuse scripts for existing components. Yet
and h1. According to the constraints in Figure 5 Line 3, h3 operators still need to manually design what components and
and r1 are deployed to the same v2. resources to use and how to link them together. From this point
Now the operator is satisfied with the configuration, and she of view, C ONSOLAS’s automatic architecture configuration is
provides C ONSOLAS the endpoint and the account credential an ideal complement to these tools.
of her target cloud platform and launches the deploy command. The automatic configuration in C ONSOLAS shares the sim-
C ONSOLAS will automatically create two VMs, and instan- ilar idea with the research on auto-completion and auto cor-
tiates 9 docker containers and link them together according rection of modelling tools, such as Egyed’s work for UML [6]
to the configuration. After the automatic deployment process, and Xiong et al.’s work on Feature Model [7]. However,
the operator can get the IP address of v1 (where the load model editors suggest completions based on general-purpose
balancer is deployed) from the deployment log, and use port rules or constraints, whereas C ONSOLAS allows developers to
80 to access the routing services. define application-specific constraints. Moreover, as a tool for
After the system is running, the operator can still modify automatic deployment, C ONSOLAS always looks for complete
the configuration. For example, for performance purpose, she configurations rather than scattered completion suggestions.
can require w3 and h3 to be hosted in the same VM:
deploy(w3)==deploy(hp(w3)) VII. C ONCLUSION
Due the resource limitation, C ONSOLAS will create a new VM This paper introduces the C ONSOLAS tool for automatic
in OpenStackHuge, migrate all the components on v2, as configuration and deployment of cloud applications. Based
well as w3 to the new VM, and finally terminate v2. on developers’ specification on the application, resource and
constraints, C ONSOLAS generates configurations of the ap-
B. Research behind the tool
plication and deploys them automatically to the cloud en-
The automatic configuration and deployment of C ONSOLAS vironment. Future work will be focused on improving the
are based on our research of constraint-driven architecture usability, such as providing a web-based GUI and a set of
configuration and C LOUD ML, respectively. C LOUD ML is a template for application and constraint specification. We will
language for the modelling of cloud resources and topology. also investigate how to infer the user preference from their
C ONSOLAS translate the C LOUD ML model together with the refinement iterations.
constraints into a Satisfactory Modulo Theory (SMT) problem:
Component instances are represented by enumeration items. ACKNOWLEDGEMENT
Types, attributes, and connection between components are This work is supported by the EU FP7-ICT-2011-9 No.
represented by uninterpreted functions. The constraints are 600654 DIVERSIFY project
translated into First Order Logic assertions on these functions,
and can be either hard or soft. During the configuration R EFERENCES
process, C ONSOLAS first transform the operators’ hints into [1] V. Nallur, A. Elgammal, and S. Clarke, “Smart Route Planning
constraints and integrate them into the SMT problem. After Using Open Data and Participatory Sensing,” in Open Source Systems:
that, it launches an constraint solving process to search for Adoption and Impact. Springer, 2015, pp. 91–100. [Online]. Available:
https://github.com/DIVERSIFY-project/SMART-GH
a solution to the SMT problem, i.e., giving an interpretation [2] H. Song, X. Zhang, N. Ferry, F. Chauvel, A. Solberg, and G. Huang,
to each of the functions. The solution satisfies all the hard “Modelling adaptation policies as domain-specific constraints,” in Model-
constraints and minimize the total costs of violated soft con- Driven Engineering Languages and Systems, 2014, pp. 269–285.
[3] N. Ferry, H. Song, A. Rossini, F. Chauvel, and A. Solberg, “Cloud
straints. For incremental configuration, C ONSOLAS will also MF: Applying MDE to Tame the Complexity of Managing Multi-cloud
generate a set of additional soft constraints according to the Applications,” in Proceedings of the 2014 IEEE/ACM 7th International
previous configuration, so that the new configuration will have Conference on Utility and Cloud Computing, 2014, pp. 269–277.
[4] Cloud Management Chef, https://www.chef.io.
as little deviation as possible from the previous one. Due to [5] The Puppet Lab, https://puppetlabs.com/.
the space limitation, we will not go into the details of the SMT [6] A. Egyed, “Fixing inconsistencies in uml design models,” in Software
problem and the transformation. Interested readers can refer Engineering, 2007. ICSE 2007. 29th International Conference on. IEEE,
2007, pp. 292–301.
to our previous publication [2]. C ONSOLAS transforms the [7] Y. Xiong, A. Hubaux, S. She, and K. Czarnecki, “Generating range fixes
solution back into a C LOUD ML instance model for automatic for software configuration,” in Software Engineering (ICSE), 2012 34th
deployment. The deployment is done by the C LOUD ML International Conference on. IEEE, 2012, pp. 58–68.