-Modern cloud applications, generally, demand com- blocks (ABBs) with software building blocks (SBBs) (cf. [3]). plex service topologies, e.g., for meeting scalability, maintenance, The service topology, that comprises latter building blocks, may or security requirements. Thus, often, it is desirable to increase be modeled by a team of experts. Some services are available tqhueirceodmcphlaenxigteys,ofhosewrevviceer, tmopaoylocgoinesstiitnucteloaudbuarpdpelnicafotironims.pTrhoveinreg- off the shelf, while others need to be implemented by service cloud applications. Changes, overall, are undertaken frequently engineers. Deployment services automate the provisioning of in prototyping or when adopting agile development. Their costs different stages such as for testing or pre-production. Finally, a correlate with the number of people involved. For facilitating ag- continuous integration service executes various tests that have ile prototyping of cloud applications this demonstration presents been implemented by test developers. iantmegordaetiln-bgawseidth,aapnpdrobauchildiinncgoorpnotroaptinogf odpiefnfe-rseonutrcreoplerso.jeUctssi,ngit, In case of iterative development all of these roles repeatably comprises domain-specific language editors and showcases their need to coordinate their activities. If a cloud architect would use and the realized automation fostering iterative development. like a particular service to be decoupled from another because Index Terms-automation, agile, application, cloud, DSL, of scalability issues the alignment of building blocks may need model-based, prototyping, provisioning, service topology adjustment, the model reflecting the service topology needs to be changed, tests may have to be adapted, and service I. INTRODUCTION implementations have to be modified. For various reasons, modern cloud applications are composed For lowering the burden of improving the cloud application of increasingly complex service topologies (cf. [1]). Factors accordingly the overall turnaround time shall be kept as short that contribute to the complexity of service topologies are the as possible in such cases. With a multitude of people and roles ability to scale out services for meeting varying user loads, involved, agile development of complex cloud applications security requirements that demand the separation of application is challenging: Most of all, coordination needs have to be logic as well as of data, and maintenance considerations that addressed. Next, development of individual services needs foster distribution of self-contained services for decoupling to be facilitated in a sense that integration into the service their lifecycle. topology is eased. Finally, overall automation is required. For expressing and capturing service topologies, the OASIS Tackling these issues, the demonstration proposes a modelTopology and Orchestration Specification for Cloud Appli- based approach and presents a toolset comprising domaincations (TOSCA) standard 1 provides a metamodel. When specific language (DSL) editors for facilitating agile prototyping describing a cloud application in terms of a metamodel and of cloud applications. Showcasing an overall example, it when following a model-driven engineering (MDE) approach, combines two former contributions: Addressing coordination provisioning can be automated (cf. [2]). Yet, development of needs, a descriptive DSL [4] permits to define roles, artifacts, a cloud application remains difficult as its service topology and services. The alignment of ABBs and SBBs can be needs to be modeled and the various services implemented. In undertaken by an enterprise architect using another DSL which an agile practice, the architecture and implementation of cloud is also used for describing the provisioning of the cloud applications usually experiences early and frequent changes application [5]. Using a running example, this demonstration such as refactorings. For example, the topology may be changed describes the overall process, artifacts, and tools involved. in a way that requires alignment of the model, e.g., when Abstracting from infrastructure as a service (IaaS) providers, services are split into distributed entities. At the same time, the approach automates various tasks and enables developers the respective services need to be adapted as well. to incrementally deploy cloud services facilitating iterative Multiple roles with particular responsibilities are involved in prototyping of cloud applications. the engineering process, such as architects, test developers, and The remainder is structured as follows: Section II introduces service engineers - each one having a different set of expertise. the running example of the demonstration by describing its For realizing a new cloud application an enterprise architect context and by characterizing related problems as a further moperforms a functional analysis and aligns architectural building tivation. The approach using the running example is presented in Section III. The work is compared to some related work in 1http://docs.oasis-open.org/tosca/TOSCA/v1.0/TOSCA-v1.0.pdf Section IV and Section V concludes.
Sensor
Messaging Platform
Analytics
Database
Server PoC MQTT PoC Part 1 B Broker C Part 2 B
PostgreSQL
Apache PoC
C HTTP C Part 3 B Hosting Unit
Hosting Unit
Hosting Unit
Dashboard
Functional A Architecture
Mapping Web Web Server Application A AAbrscthriateccttSuWre
Mapping Concrete SWArchitecture Mapping
Abstract Hosting Unit 1 Infrastructure
(Provisioning DSL) Transformation 2 Concrete
Infrastructure (EC2 IaaS Model) II. RUNNING EXAMPLE: CONTEXT AND MOTIVATION Let us now consider an industrial machine-to-machine (M2M) context in which a new business idea arose. As part of the value proposition it envisages the correlation of sensor data with other data and the visualization of results in a dashboard. In this scenario M2M devices emit data from sensors to an M2M gateway. A backend server receives and processes aggregated sensor data from these gateways. For this, the data is normalized and correlated with user data and data from an external data source. Results are stored in a database and visualized in a web application. Figure 1 depicts a rudimentary server landscape for the M2M scenario.
Various roles participate in the engineering: among them are service and web engineers that implement different parts of a minimum viable product (MVP) (see below) as well as architects that describe the functional architecture.
Publish/Subscribe
MQTT
Business Intelligence
Dashboard
Following the idea of the lean startup philosophy [
In situations in which products depend on several cloud services a platform as a service (PaaS) may provide a suitable basis for the development and deployment of software as a service (SaaS). It needs to cover the requirements of the product III. DOMAIN-SPECIFIC LANGUAGES FOR FACILITATING in terms of deployed technologies and available cloud services. THE ENGINEERING OF CLOUD APPLICATIONS For the development of a MVP the decision to deploy a (certain) PaaS may be an early decision to take. Indeed, one of the For addressing the issues mentioned in the introduction principles of lean management is deferring decisions for being and as further elaborated in the previous section, a modelable to also consider new information. based approach is proposed. That is, separation of concerns
As an alternative and as exercised in the demonstration, the is realized between architecture, functionality, and technology entire cloud stack from IaaS to SaaS – realizing the MVP – platforms while automating provisioning and deployment in a needs to be described for cloud provisioning and deployed. role-based development process. After giving an overview, the This is particularly interesting when customized cloud stacks use of the DSLs is illustrated by means of the running example. are preferred over a uniform cloud stack or a PaaS. At the same The demonstrator entirely relies on open source software: The time the specification and deployment of a customized cloud Eclipse Modeling Framework (EMF) 2 was chosen with Xtext stack, e.g., in TOSCA without further tool support, requires for defining the DSLs and for realizing respective editors expert knowledge. This burden may prevent a project to opt and Xtend for implementing the model transformations. At for a tailored cloud setup and hinder lean development. runtime and besides the resulting DSL editors, Git 3 is used
For realizing the MVP the functional architecture of the as version control system (VCS), and Puppet 4 for the CM. cloud application needs to become manifested in tangible Finally, OpenStack 5 serves as an IaaS solution. cloud services and resources. The mapping from ABBs to SBBs – performed by architects (indicated with Role A) – is 32hhttttpp::////egcitl-ispcsme..ocrogm/modeling/emf shown in the top three levels of Figure 2. Describing the cloud 4http://puppetlabs.com provisioning, the services are associated with server instances 5http://openstack.org via hosting units. Finally the development of the various parts of the MVP – as performed by software developers (Role B) and configuration management (CM) experts (Role C) – requires coordination between the roles. Finally their services have to be deployed (realized with Transformations 1 and 2).
As indicated at the bottom, this may occur for multiple stages of the engineering lifecycle (e.g., DEV or TEST).
A lack of automation renders the process – consisting of a multitude of steps – time-consuming and costly. As a result, iterative cycles involving the reprovisioning of (parts of) the service topology may be avoided. Decreasing agility, this may proof problematic at an early phase of product development such as when developing a MVP. Besides the loss of flexibility (e.g., as required when performing adjustments), the complexity of the overall process may render effective participation of architects problematic.
Model Transformation EC2 IaaS Model
Version Control System
Workspaces
Coordination Tools Coordination DSL Program artifact PoC_artifact1 role "SWDeveloper" produces PoC_artifact1 service PoC_part1 dependsOn PoC_artifact1
CM Modules CM Manifests
Cloud-Init
Code
Code
Generation set up infrastructure irnusntaCllMCaMgcelnietsnts
CM Server • install platform • deploy SW IaaS Provider
Software PoC for the
M2M Scenario
Platform Mosquitto PostgreSQL Apache HTTP
Infrastructure Members Volumes Security Groups Server Cloud Application
In addition to Figure 2, a technical overview of the modelbased realization is depicted in Figure 3. The cloud provisioning is described using a DSL as shown in the upper left part of the figure. Through a sequence of model transformations an IaaS consumer is generated that realizes the provisioning of cloud infrastructure services. Platform and software services as referenced in the DSL program are provisioned using a CM system. CM modules are looked up for such services and a CM server is instructed accordingly while server instances are provisioned with CM clients.
A coordination DSL program declarativly describes roles, artifacts, services, and dependencies between these. Coordination tools are generated that interplay with a VCS and the workspaces of stakeholders. For developed services, CM modules are prepared in addition and built for further simplifying the overall engineering process while profiting from the functionality and automation as realized by the CM system.
Starting from the business idea from Section II, architects conduct a functional analysis. The resulting functional architecture comprises sensors, a broker, converters, a business intelligence (BI), and a dashboard as shown in Figure 2. Resulting ABBs are sensors, a messaging platform, an analytics engine, a database server, a web server, and a web application. Using a DSL editor and profiting from syntax highlighting, code completion, and scoping as shown in Figure 4, the ABBs are associated with SBBs (i.e., Mosquitto as a MQ Telemetry Transport (MQTT) broker, a PostgreSQL server, an Apache HTTP server, and three different parts of a proof of concept (PoC) implementation). The abstract building blocks are so associated with concrete building blocks to be deployed and are used (i.e., referenced) for the provisioning as shown later. C. Domain-Specific Language for Cloud Provisioning
The DSL program in the upper left part of Figure 3 shows an excerpt for defining the cloud provisioning. For W7 urEnvironment catalogue namespace M2M_PoC serviceDef Publisher realizedBy PoC_part1
implies services MosquittoClient PyXB serviceDef Broker
implies services Mosquitto serviceDef Analytics realizedBy PoC_part2
implies services MosquittoClient PyXB SQLAlchemy serviceDef Database
implies services PostgreSQL serviceDef Report realizedBy PoC_part3
implies services SQLAlchemy ApacheWSGI this, services are grouped in hostingUnits. Through model transformations and automation, the latter are mapped to server instances or clusters and appropriate IaaS clients are generated. Please note, that the services reference the abstract serviceDefs from Figure 4. This and further relationships (implies) are exploited for automating the provisioning. The DSL facilitates specification of custom cloud stacks as required for different stages of the engineering lifecycle such as development (DEV), testing (TEST), or production (PROD). Choosing a profile the provisioning is performed similarly for each of the defined stages. If desired this is done in different cloud regions (also defined in the respective profile). If not bound to certain stages (e.g., sensor data generators for development and test) server instances (e.g., broker) are provisioned for all stages (e.g., in all the related cloud regions). D. Formalizing Dependencies of Services, Artifacts, and Roles
A declarative DSL allows to enumerate roles
participating in the engineering process, artifacts, services, and their
dependencies in a generic way. From these, coordination
tools are generated (cf. [
Higher-level cloud services depending on IaaS are provisioned through a CM system. For this, Puppet modules are looked up for services using name-based matching. Using the realizedBy keyword Puppet modules are automatically synchronized with required software artifacts using the VCS. Yet, module manifests need to be supplied by Puppet experts. At execution time, the approach integrates with Puppet by generating manifest files and cloud-init 6 configs (passed as user-data when launching server instances). This way the different engineers can work independently using their workspace focusing on their actual task while the packaging and deployment is automated.
F. Iterative Development and Continuous Deployment
When executing the generated IaaS consumer all the cloud services (see right box of Figure 3) – starting from IaaS (i.e., security groups, volumes, and instances) and PaaS (i.e., Apache HTTP server, Mosquitto, and PostgreSQL) to SaaS (i.e., PoC Parts 1–3) – are provisioned in an interplay with Puppet and thus the cloud application is deployed. If work is performed in the VCS, e.g., when modifying a particular service, changes can be applied incrementally using Puppet behind the scenes facilitating continuous deployment. Changes to the service topology are more critical: While in many cases they are respected by the demonstrator, a reprovisioning may become necessary, e.g., when renaming occurs. Please note that while the latter takes longer until all services are provisioned it is still fully automated and may be thus acceptable for prototyping.
Rapid application development (RAD) (cf. [
As mentioned, TOSCA provides a metamodel for service
topologies. As a standard, it constitutes a desirable basis for
an implementation of the approach presented. Similar to the
presented work TOSCA can be combined with CM systems
(cf. [
Agile development of cloud applications can be facilitated using a model-based approach, e.g., using textual DSLs as presented. For this a high degree of automation is required beyond provisioning. Also, as many stakeholders are involved in the engineering, it is crucial to harmonize their collaboration and to provide them with means for participation. This is realized by exploiting formalized dependencies and by providing tailored DSL editors while abstracting from technologies such as CM and IaaS solutions.
For wide applicability, the approach directly builds on top of IaaS and does not rely on a development PaaS. Yet, in case the code generator is adapted, the presented work can build on different application programming interfaces (APIs) or other technologies both for the provisioning and the CM. The approach was successfully adopted for the rapid development of several demonstrators and a PoC in an industrial context.