=Paper=
{{Paper
|id=Vol-1554/PD_MoDELS_2015_paper_17
|storemode=property
|title=Facilitating Agile Prototyping of Cloud Applications – A Model-based Approach
|pdfUrl=https://ceur-ws.org/Vol-1554/PD_MoDELS_2015_paper_17.pdf
|volume=Vol-1554
|authors=Ta’id Holmes
|dblpUrl=https://dblp.org/rec/conf/models/Holmes15
}}
==Facilitating Agile Prototyping of Cloud Applications – A Model-based Approach==
https://ceur-ws.org/Vol-1554/PD_MoDELS_2015_paper_17.pdf
Facilitating Agile Prototyping of
Cloud Applications — A Model-Based Approach
Ta’id Holmes
Infrastructure Cloud, Deutsche Telekom Technik GmbH
Darmstadt, Germany
t.holmes@telekom.de
Abstract—Modern cloud applications, generally, demand com- blocks (ABBs) with software building blocks (SBBs) (cf. [3]).
plex service topologies, e.g., for meeting scalability, maintenance,
The service topology, that comprises latter building blocks, may
or security requirements. Thus, often, it is desirable to increase be modeled by a team of experts. Some services are available
the complexity of service topologies in cloud applications. The re-
quired changes, however, may constitute a burden for improving off the shelf, while others need to be implemented by service
cloud applications. Changes, overall, are undertaken frequently engineers. Deployment services automate the provisioning of
in prototyping or when adopting agile development. Their costs different stages such as for testing or pre-production. Finally, a
correlate with the number of people involved. For facilitating ag- continuous integration service executes various tests that have
ile prototyping of cloud applications this demonstration presents been implemented by test developers.
a model-based approach incorporating different roles. Using,
integrating with, and building on top of open-source projects, it In case of iterative development all of these roles repeatably
comprises domain-specific language editors and showcases their need to coordinate their activities. If a cloud architect would
use and the realized automation fostering iterative development. like a particular service to be decoupled from another because
Index Terms—automation, agile, application, cloud, DSL, of scalability issues the alignment of building blocks may need
model-based, prototyping, provisioning, service topology adjustment, the model reflecting the service topology needs
to be changed, tests may have to be adapted, and service
I. I NTRODUCTION implementations have to be modified.
For various reasons, modern cloud applications are composed For lowering the burden of improving the cloud application
of increasingly complex service topologies (cf. [1]). Factors accordingly the overall turnaround time shall be kept as short
that contribute to the complexity of service topologies are the as possible in such cases. With a multitude of people and roles
ability to scale out services for meeting varying user loads, involved, agile development of complex cloud applications
security requirements that demand the separation of application is challenging: Most of all, coordination needs have to be
logic as well as of data, and maintenance considerations that addressed. Next, development of individual services needs
foster distribution of self-contained services for decoupling to be facilitated in a sense that integration into the service
their lifecycle. topology is eased. Finally, overall automation is required.
For expressing and capturing service topologies, the OASIS Tackling these issues, the demonstration proposes a model-
Topology and Orchestration Specification for Cloud Appli- based approach and presents a toolset comprising domain-
cations (TOSCA) standard 1 provides a metamodel. When specific language (DSL) editors for facilitating agile prototyping
describing a cloud application in terms of a metamodel and of cloud applications. Showcasing an overall example, it
when following a model-driven engineering (MDE) approach, combines two former contributions: Addressing coordination
provisioning can be automated (cf. [2]). Yet, development of needs, a descriptive DSL [4] permits to define roles, artifacts,
a cloud application remains difficult as its service topology and services. The alignment of ABBs and SBBs can be
needs to be modeled and the various services implemented. In undertaken by an enterprise architect using another DSL which
an agile practice, the architecture and implementation of cloud is also used for describing the provisioning of the cloud
applications usually experiences early and frequent changes application [5]. Using a running example, this demonstration
such as refactorings. For example, the topology may be changed describes the overall process, artifacts, and tools involved.
in a way that requires alignment of the model, e.g., when Abstracting from infrastructure as a service (IaaS) providers,
services are split into distributed entities. At the same time, the approach automates various tasks and enables developers
the respective services need to be adapted as well. to incrementally deploy cloud services facilitating iterative
Multiple roles with particular responsibilities are involved in prototyping of cloud applications.
the engineering process, such as architects, test developers, and The remainder is structured as follows: Section II introduces
service engineers – each one having a different set of expertise. the running example of the demonstration by describing its
For realizing a new cloud application an enterprise architect context and by characterizing related problems as a further mo-
performs a functional analysis and aligns architectural building tivation. The approach using the running example is presented
in Section III. The work is compared to some related work in
1 http://docs.oasis-open.org/tosca/TOSCA/v1.0/TOSCA-v1.0.pdf Section IV and Section V concludes.
II. RUNNING E XAMPLE : C ONTEXT AND M OTIVATION Sensor Broker Converter
Business
Dashboard Functional
Intelligence A Architecture
Let us now consider an industrial machine-to-machine Mapping
(M2M) context in which a new business idea arose. As part of Sensor
Messaging
Analytics
Database Web Web
A Abstract SW-
Platform Server Server Application Architecture
the value proposition it envisages the correlation of sensor data
Mapping
with other data and the visualization of results in a dashboard. In
PoC MQTT PoC PostgreSQL Apache PoC Concrete SW-
this scenario M2M devices emit data from sensors to an M2M Part 1 B Broker C Part 2 B C HTTP C Part 3 B Architecture
gateway. A backend server receives and processes aggregated Mapping
1 Abstract
sensor data from these gateways. For this, the data is normalized Hosting Unit Hosting Unit Hosting Unit Hosting Unit Infrastructure
and correlated with user data and data from an external data (Provisioning DSL)
Transformation
source. Results are stored in a database and visualized in a web
application. Figure 1 depicts a rudimentary server landscape 2 Concrete
Infrastructure
for the M2M scenario. DEV (EC2 IaaS Model)
TEST
Various roles participate in the engineering: among them
are service and web engineers that implement different parts
of a minimum viable product (MVP) (see below) as well as
Figure 2. From Design to Deployment of the Cloud Application
architects that describe the functional architecture.
Publish/Subscribe Dashboard
MQTT Business via hosting units. Finally the development of the various
Intelligence
parts of the MVP – as performed by software developers
(Role B) and configuration management (CM) experts (Role C)
Broker Analytics Database Web Web – requires coordination between the roles. Finally their services
Sensors Converters Server Browser have to be deployed (realized with Transformations 1 and 2).
As indicated at the bottom, this may occur for multiple stages
Figure 1. Server Landscape of a Machine-to-Machine Scenario of the engineering lifecycle (e.g., DEV or TEST).
A lack of automation renders the process – consisting of a
multitude of steps – time-consuming and costly. As a result,
Following the idea of the lean startup philosophy [6] the iterative cycles involving the reprovisioning of (parts of) the
value proposition of the business idea is to be tested with service topology may be avoided. Decreasing agility, this may
potential customers. For this, prototyping constitutes a suitable proof problematic at an early phase of product development
approach for the (rapid) development of a MVP (cf. [7]) and such as when developing a MVP. Besides the loss of flexibility
for gaining findings such as customer requirements. (e.g., as required when performing adjustments), the complexity
In situations in which products depend on several cloud of the overall process may render effective participation of
services a platform as a service (PaaS) may provide a suitable architects problematic.
basis for the development and deployment of software as a
service (SaaS). It needs to cover the requirements of the product III. D OMAIN -S PECIFIC L ANGUAGES FOR FACILITATING
in terms of deployed technologies and available cloud services. THE E NGINEERING OF C LOUD A PPLICATIONS
For the development of a MVP the decision to deploy a (certain)
PaaS may be an early decision to take. Indeed, one of the For addressing the issues mentioned in the introduction
principles of lean management is deferring decisions for being and as further elaborated in the previous section, a model-
able to also consider new information. based approach is proposed. That is, separation of concerns
As an alternative and as exercised in the demonstration, the is realized between architecture, functionality, and technology
entire cloud stack from IaaS to SaaS – realizing the MVP – platforms while automating provisioning and deployment in a
needs to be described for cloud provisioning and deployed. role-based development process. After giving an overview, the
This is particularly interesting when customized cloud stacks use of the DSLs is illustrated by means of the running example.
are preferred over a uniform cloud stack or a PaaS. At the same The demonstrator entirely relies on open source software: The
2
time the specification and deployment of a customized cloud Eclipse Modeling Framework (EMF) was chosen with Xtext
stack, e.g., in TOSCA without further tool support, requires for defining the DSLs and for realizing respective editors
expert knowledge. This burden may prevent a project to opt and Xtend for implementing the model transformations. At
3
for a tailored cloud setup and hinder lean development. runtime and besides the resulting DSL editors, Git is used
For realizing the MVP the functional architecture of the as version control system (VCS), and Puppet 4 for the CM.
5
cloud application needs to become manifested in tangible Finally, OpenStack serves as an IaaS solution.
cloud services and resources. The mapping from ABBs to 2 http://eclipse.org/modeling/emf
SBBs – performed by architects (indicated with Role A) – is 3 http://git-scm.com
shown in the top three levels of Figure 2. Describing the cloud 4 http://puppetlabs.com
provisioning, the services are associated with server instances 5 http://openstack.org
Provisioning DSL Program Software
CM Server
W7 urEnvironment project M2M_PoC CM Modules • install PoC for the
costCenter "123456789" Version Control System M2M Scenario
platform
profile ThreeStage Workspaces • deploy SW
hostingUnit sensor stage DEV TEST CM Manifests
Coordination Tools Platform
service Publisher
Mosquitto
hostingUnit broker Cloud-Init PostgreSQL
service Broker Coordination DSL Program Code Apache HTTP
artifact PoC_artifact1
Model Transformation role "SWDeveloper" Infrastructure
produces PoC_artifact1 Code IaaS Provider Members
Abstract IaaS Model Generation Volumes
service PoC_part1
Model Transformation dependsOn PoC_artifact1 set up infrastructure Security Groups
install CM clients Server
EC2 IaaS Model run CM agents
Cloud Application
Figure 3. Overview of the Model-Based Approach for the Development and Continuous Deployment of Cloud Applications
W7 urEnvironment catalogue
A. Approach Overview namespace M2M_PoC
serviceDef Publisher realizedBy PoC_part1
In addition to Figure 2, a technical overview of the model- implies services MosquittoClient PyXB
based realization is depicted in Figure 3. The cloud provisioning serviceDef Broker
implies services Mosquitto
is described using a DSL as shown in the upper left part of serviceDef Analytics realizedBy PoC_part2
implies services MosquittoClient PyXB SQLAlchemy
the figure. Through a sequence of model transformations an serviceDef Database
IaaS consumer is generated that realizes the provisioning of implies services PostgreSQL
serviceDef Report realizedBy PoC_part3
cloud infrastructure services. Platform and software services implies services SQLAlchemy ApacheWSGI
as referenced in the DSL program are provisioned using a CM
system. CM modules are looked up for such services and a Figure 4. Mapping Architectural to Software Building Blocks
CM server is instructed accordingly while server instances are
provisioned with CM clients.
A coordination DSL program declarativly describes roles, this, services are grouped in hostingUnits. Through model
artifacts, services, and dependencies between these. Coordi- transformations and automation, the latter are mapped to
nation tools are generated that interplay with a VCS and server instances or clusters and appropriate IaaS clients are
the workspaces of stakeholders. For developed services, CM generated. Please note, that the services reference the abstract
modules are prepared in addition and built for further simpli- serviceDefs from Figure 4. This and further relationships
fying the overall engineering process while profiting from the (implies) are exploited for automating the provisioning.
functionality and automation as realized by the CM system. The DSL facilitates specification of custom cloud stacks as
required for different stages of the engineering lifecycle such
B. Business Idea and Functional Analysis
as development (DEV), testing (TEST), or production (PROD).
Starting from the business idea from Section II, archi- Choosing a profile the provisioning is performed similarly
tects conduct a functional analysis. The resulting functional for each of the defined stages. If desired this is done in
architecture comprises sensors, a broker, converters, different cloud regions (also defined in the respective profile).
a business intelligence (BI), and a dashboard as If not bound to certain stages (e.g., sensor data generators
shown in Figure 2. Resulting ABBs are sensors, a for development and test) server instances (e.g., broker) are
messaging platform, an analytics engine, a database provisioned for all stages (e.g., in all the related cloud regions).
server, a web server, and a web application. Using
a DSL editor and profiting from syntax highlighting, code
D. Formalizing Dependencies of Services, Artifacts, and Roles
completion, and scoping as shown in Figure 4, the ABBs
are associated with SBBs (i.e., Mosquitto as a MQ Telemetry A declarative DSL allows to enumerate roles participat-
Transport (MQTT) broker, a PostgreSQL server, an Apache ing in the engineering process, artifacts, services, and their
HTTP server, and three different parts of a proof of concept dependencies in a generic way. From these, coordination
(PoC) implementation). The abstract building blocks are so tools are generated (cf. [4]). Integrated into a VCS these
associated with concrete building blocks to be deployed and may send out notifications when artifacts are available and
are used (i.e., referenced) for the provisioning as shown later. synchronize workspaces accordingly. In addition to supporting
the coordination, automation tools are derived from the DSL
C. Domain-Specific Language for Cloud Provisioning programs for preparing the deployment of developed services
The DSL program in the upper left part of Figure 3 to the respective hostingUnits and for pushing and applying
shows an excerpt for defining the cloud provisioning. For incremental changes through Puppet. Services that are devel-
oped such as the PoC parts are referenced by serviceDefs provisioning permits to define rudimentary service topologies.
following the realizedBy keyword (see Figure 4). While they can be mapped to TOSCA the DSL aims particularly
at giving stakeholders access to the modeling. In fact, a
E. Integration with Configuration Management Software provisioning can be described in a couple of lines and can be
Higher-level cloud services depending on IaaS are provi- changed easily. Instead of the DSL shown the approach can use
sioned through a CM system. For this, Puppet modules are any modeling as long as it does not slow down development
looked up for services using name-based matching. Using and iteration cycles. User studies have to be carried out in this
the realizedBy keyword Puppet modules are automatically regard in order to answer the question which modeling tools
synchronized with required software artifacts using the VCS. for service topologies are best for the development of cloud
Yet, module manifests need to be supplied by Puppet experts. applications as required in prototyping.
At execution time, the approach integrates with Puppet by
V. C ONCLUSION
generating manifest files and cloud-init 6 configs (passed
as user-data when launching server instances). This way Agile development of cloud applications can be facilitated
the different engineers can work independently using their using a model-based approach, e.g., using textual DSLs as
workspace focusing on their actual task while the packaging presented. For this a high degree of automation is required
and deployment is automated. beyond provisioning. Also, as many stakeholders are involved
in the engineering, it is crucial to harmonize their collaboration
F. Iterative Development and Continuous Deployment and to provide them with means for participation. This is real-
When executing the generated IaaS consumer all the cloud ized by exploiting formalized dependencies and by providing
services (see right box of Figure 3) – starting from IaaS (i.e., tailored DSL editors while abstracting from technologies such
security groups, volumes, and instances) and PaaS (i.e., Apache as CM and IaaS solutions.
HTTP server, Mosquitto, and PostgreSQL) to SaaS (i.e., PoC For wide applicability, the approach directly builds on top
Parts 1–3) – are provisioned in an interplay with Puppet and of IaaS and does not rely on a development PaaS. Yet, in
thus the cloud application is deployed. If work is performed in case the code generator is adapted, the presented work can
the VCS, e.g., when modifying a particular service, changes build on different application programming interfaces (APIs) or
can be applied incrementally using Puppet behind the scenes other technologies both for the provisioning and the CM. The
facilitating continuous deployment. Changes to the service approach was successfully adopted for the rapid development
topology are more critical: While in many cases they are of several demonstrators and a PoC in an industrial context.
respected by the demonstrator, a reprovisioning may become R EFERENCES
necessary, e.g., when renaming occurs. Please note that while
[1] V. Andrikopoulos, T. Binz, F. Leymann, and S. Strauch, “How to Adapt
the latter takes longer until all services are provisioned it is still Applications for the Cloud Environment,” in Computing. Springer, 2013,
fully automated and may be thus acceptable for prototyping. vol. 95, pp. 493–535.
[2] N. Ferry, H. Song, A. Rossini, F. Chauvel, and A. Solberg, “CloudMF:
IV. R ELATED W ORK Applying MDE to Tame the Complexity of Managing Multi-cloud
Applications,” in IEEE/ACM 7th International Conference on Utility
Rapid application development (RAD) (cf. [8]) can help to and Cloud Computing (UCC). IEEE, Dec 2014, pp. 269–277.
obtain results in a timely and efficient manner. For instance, [3] M. Iacob, D. Jonkers, H. Quartel, H. Franken, and H. van den Berg,
Delivering Enterprise Architecture with TOGAF and ARCHIMATE.
various RAD frameworks exist (cf. [9]) that are tailored towards Enschede: BIZZdesign, 2012.
web applications and that can be combined with the approach. [4] T. Holmes, “Facilitating Development and Provisioning of Service Topolo-
For the development of cloud applications a development PaaS gies through Domain-Specific Languages,” in 18th IEEE International
Enterprise Distributed Object Computing Conference Workshops and
such as the Google App Engine 7 may provide ready to use Demonstrations, G. Grossmann, S. Hallé, D. Karastoyanova, M. Reichert,
(technology) stacks of services while easing the development and S. Rinderle-Ma, Eds. IEEE, Sep. 2014, pp. 422–425.
of SaaS. In contrast this work permits to define and work with [5] ——, “Automated Provisioning of Customized Cloud Service Stacks
using Domain-Specific Languages,” in 2nd International Workshop on
customized cloud stacks. Realizing separation of concerns it Model-Driven Engineering on and for the Cloud, vol. 1242. CEUR-
incorporates different roles in the engineering using tailored WS.org, Sep. 2014, pp. 46–55.
DSLs and coordinates workspaces. As it directly operates [6] E. Ries, The Lean Startup: How Today’s Entrepreneurs Use Continuous
Innovation to Create Radically Successful Businesses. Crown Business,
on IaaS deployments in an interplay with CM systems it is 2011.
independent of a PaaS and certain (technology) stacks. [7] M. Poppendieck and M. A. Cusumano, “Lean software development: A
As mentioned, TOSCA provides a metamodel for service tutorial,” IEEE Software, vol. 29, no. 5, pp. 26–32, 2012.
[8] P. Beynon-Davies, C. Carne, H. Mackay, and D. Tudhope, “Rapid
topologies. As a standard, it constitutes a desirable basis for application development (RAD): An empirical review,” European Journal
an implementation of the approach presented. Similar to the of Information Systems, no. 8, pp. 211–223, 1999.
presented work TOSCA can be combined with CM systems [9] I. Vosloo and D. G. Kourie, “Server-centric web frameworks: An
overview,” ACM Comput. Surv., vol. 40, no. 2, 2008.
(cf. [10]). The DSL building on top of Amazon Elastic Compute [10] J. Wettinger, M. Behrendt, T. Binz, U. Breitenbücher, G. Breiter,
Cloud (EC2) 8 concepts and used in this work for describing the F. Leymann, S. Moser, I. Schwertle, and T. Spatzier, “Integrating
Configuration Management with Model-Driven Cloud Management based
6 http://launchpad.net/cloud-init on TOSCA,” in 3rd International Conference on Cloud Computing
7 http://cloud.google.com/appengine/ and Services Science, F. Desprez, D. Ferguson, E. Hadar, F. Leymann,
8 http://awsdocs.s3.amazonaws.com/EC2/latest/ec2-api.pdf M. Jarke, and M. Helfert, Eds. SciTePress, 2013, pp. 437–446.