Modern software systems evolve in a highly dynamic and open environment, where their supporting platforms and infrastructures can change on demand. Designing and operating holistic controllers able to leverage the adaptation capabilities of the complete software stack is a complex task, as it is no longer possible to foresee all possible environment states and system con gurations that would properly compensate for them. This paper presents our experience in using models@runtime to foster the systematic design and evaluation of self-adaptive systems, by enabling the coevolution of the reasoning engine and its environment. This research was carried out in the context of the Diversify project, which explores how bio-diversity can be used to enhanced the design of self-adaptive mechanisms.
Modern software systems are increasingly complex and distributed as they
materialize into large scale assemblies of technologies, such as databases, middleware,
business logics or graphical front-ends. Typically, cloud-based systems push this
complexity even further as their platform and infrastructure, traditionally xed
once and for all, can now be dynamically adjusted [
This complexity exceeds by far the capabilities of IT teams, who need to
rely on software solutions to automate as much as possible the related
maintenance operations [
A key issue, while designing such systems, is the uncertainty pertaining to
their open, dynamic and heterogeneous environment [
To minimize this risk of not being able to manage unforeseen environments,
our contribution is twofold. We rst propose an iterative process to proactively
explore possible evolutions of the environment, and allow designers to
incrementally adjust the reasoning engines accordingly. We then describe how the
models@runtime paradigm [
In the context of the Diversify project 1, we combined existing self-reparation techniques with a mechanism for exploring environment evolutions driven by an analogy with the concept of bio-diversity. The loose coupling between the reasoning engine and the environment provided by the model@runtime paradigm facilitates their interdependent evolution. On one hand, by replacing the running system with simulated environments, one can evaluate the e ectiveness of a given reasoning engine. On the other hand, by plugging alternative reasoning engines, one can evaluate their robustness to unforeseen environments.
The remainder of this paper is organized as follows. Section 2 uses a cloudbased multi-tenant system as an example of a self-adaptive system facing unforeseen environments. Section 3 presents our design process and its realization on a models@runtime platform. Section 4 details how this approach was applied to our example in the context of the Diversify project. Section 5 discusses related work before Section 6 concludes and discusses possible future work. 2
SensApp [
SensApp is implemented as four REST services to: i) register sensors, ii) store their data, iii) notify third-party services, and iv) to orchestrate these services from a uni ed facade service (dispatcher). As these services are loosely coupled (they only use their public REST APIs to communicate between them), SensApp can thus be distributed in various ways. Each service indeed only 1 http://diversify-project.eu/ 2 https://github.com/SINTEF-9012/sensapp requires a servlet container, such as Apache Tomcat or Jetty, and a set of open ports to communicate over HTTP.
In order to reduce the management overhead related to operating SensApp
in the cloud, we need to develop a self-adaptive version of SensApp, which
shall recover as much as possible from failures of its own components (i.e., its
four services) and from failures occurring in its environment (i.e., any failures of
the underlying software stack including software, platform and hardware
infrastructure). In a cloud setting, software failures are recovered by reinstalling the
erroneous component, whereas hardware failures require in addition the
provisioning of a new virtual machine. Our rst solution to self-adaptation relies on a
set of event-condition-action rules (ECA) binding some anticipated failures with
imperative repair procedures. These procedures are sequences of atomic actions
such as provisioning a virtual machine or deploying a service, which apply on
a CloudML model (cf. Section 4) describing the system and its environment
[
As cloud-based systems operate in an open environment, it is very di cult (and even not tractable) to get insight on the impact that unforeseen environments may have. In a real cloud setting, nothing guarantees for instance that the SensApp application will run in complete isolation: other applications or services may be added into the same war container, or simply be hosted on the same virtual machines. From the SensApp perspective, this kind of environment with multi-tenancy might alter its performances (e.g., response time) or the associated reasoning engine (e.g., decision relevance).
Existing approaches to manage such uncertainty (cf. Section 5) either let the system learn from its errors using machine learning techniques or directly leverage a model of the uncertainty itself. Machine learning techniques are robust but require to let the system's performance drop for a moment, until the system has learned. Modeling uncertainty to make better decisions implies the existence of partial knowledge, which is not always available.
Designing a self-adaptation mechanism, robust to unforeseen environments, thus remains a challenge. In the following, we explore how an alternative approach, inspired by agile practices, can be used to enhance the reasoning engine of our autonomic SensApp in terms of e ectiveness and performances. 3
We present in this section the design process, inspired by agile practices, and its underlying models@runtime architecture. 3.1
Our overall approach relies on an iterative process that successively re nes the design of the reasoning engine, and the environments in which it operates, until enough con dence is gained regarding the robustness of the system. The interdependent evolutions of the reasoning engine and its environment (see the two cycles in Fig. 1), referred to as coevolution, is organized as follows: 1. Check robustness to unforeseen environment: assess whether designers have gained enough con dence regarding the robustness of the system under unforeseen environments. Depending on the concern of interest, this process can be automated. Once this evaluation is successful, the overall design process is terminated. Otherwise, a new environment simulator is designed. 2. Design a new environment simulator: First, the designer identi es an uncertainty axis along which she suspects that the robustness could be an issue. Based on this, the simulated environments (potentially including some randomness) will evolve along this axis. 3. Evaluate the reasoning engine: Evaluates how the reasoner behaves with respect to the simulated environment. Depending on the concern of interest, this process can be automated. If the reasoning engine passes the evaluation then the design process restarts from Step 1, otherwise the reasoning engine is updated (Step 4). 4. Update the reasoning engine: Adapts the reasoning engine to the simulated environment. It may lead to a change of reasoning engine as well as in its con guration. Once updated, the new engine is evaluated again (Step 3). 1. Check robustness to unforseen [pass]
Evolution of the
Environment [fail] 2. Environment Simulator Design
[pass]
3. Reasoning
engine
evaluation
[fail]
4. Reasoning
engine update
Evolution of the
Reasoning Engine
Appying coevolution to address unforeseen environment assumes that the
Pareto Principle [
A Models@Runtime Architecture to Support the Process
The proposed architecture is depicted in Fig. 2 and is inspired by our previous
work [
Evaluator (step 3) Reasoning engines Evolutionhofhtheh ReasoninghEngine target system model explore alternative environments (step 1 & 2)
Synchronization
Engine diff. operator current system model
Evolutionhofhtheh Environment adjust monitor e c a tIfr e n
Simulated Unforeseenh Environments RealhRuntime Environment
The models@runtime idea is reused as such in Fig. 2, where a synchronization engine maintains the causal link between the system and the model. Any change in the environment is re ected on the model, and similarly, any change in the system model is enacted on the system. The interactions between the models@runtime engine and both interfaces consist in exchanging models or manipulating the models through an API. A reasoner can retrieve the current model of the running system and push a new target model to the engine, while the runtime environment can push a new model of the current running system to the engine.
In our approach, the con guration or replacement of a reasoning engine or of the environment is enacted by a designer. In order to help the designer in this task, an evaluator is used to assess whether the reasoner behaves correctly with respect to the simulated environment and its variations.
The Role of Models@Runtime: In this context, the separation of concerns o ered by models@runtime brings three main bene ts: (i) easier integration of the reasoning engine,(ii) easier integration of environment simulators, and (iii) coevolution of the reasoning engine and the simulator.
From the reasoning engine's point of view, models@runtime provides both abstraction and anticipation capabilities. The abstraction provided by the model let reasoning engine to analyze a simpli ed version of the running system, while the anticipation allows the reasoning engine to conduct what-if scenarios in a safe modeling space, with no impact on the running system. If and only if a valid model is identi ed, it will be automatically enacted via the causal link and the running system will consequently be dynamically adapted. If the reasoning engine at some point yields an invalid model, it is simply discarded, with no need to perform expensive roll-back since the system has not been adapted yet. The reasoning engine can be applied to any environment including simulators as far as it can be modelled by the models@runtime engine (i.e., it respects the interface).
From the environment point of view, the models@runtime architecture facilitates the simulation of alternative environments, by replacing the actual runtime environment by simulated ones. The ability to easily switch between environments and the loose coupling between the reasoning engine and the environment, avoid the recurrent design of ad hoc simulators tightly-coupled to a speci c reasoning engine. Moreover, once the design process is completed, integrating the resulting reasoning engine with the running system does not require any further change on the self-adaptation loop.
The overall architecture relies on models@runtime, as it provides a clear interface and a loose coupling between both the environment and the reasoning engine sides, which facilitates their coevolution. This in turn facilitates the continuous design of complex adaptive systems.
The next section reports on how we used this process and the related architecture in the context of the Diversify project. 4
The Diversify project aims at creating a synergy between ecology (a discipline of biology), and software engineering. We aim at understanding the key mechanisms underlying bio-diversity and how they enhance the robustness and resilience of biological systems, and thus porting them to software systems. We in particular focus on cloud systems, which o ers a large scale, distributed and complex software ecosystem, approaching the complexity of natural ecosystems. 4.1
Modeling Deployment and Provisioning of Cloud-based Systems We model the deployment and provisioning of cloud systems with CloudML, a DSML developed as a joint e ort between the REMICS, MODAClouds and PaaSage projects. As shown on Fig. 3, a CloudML model captures the deployment of a cloud-based system, including the platform and infrastructure level. In this example, SensApp is deployed on three virtual machines hosting: the \admin" application, the SensApp application, and the underlying database, respectively. The semantics of such a description is embedded in the models@runtime platform, which maintains a causal link between the model and the running cloud system. ap0:JAdmin (RUNNING)
WarContainer jt1:JJetty (RUNNING)
JRE java1:JJdk1.7 (RUNNING) ap1:JSensApp (RUNNING)
WarContainer tc1:JTomcat (RUNNING)
JRE java2:JJre1.6 (RUNNING) vm3:JMedium-Linux (RUNNING) NoSQLDB db1:JMongoDB (RUNNING) virtual machine software artifact mandatory local dependency mandatory remote dependency optional remote dependency optional local dependency
In order to observe how software diversity helps self-repair mechanisms in recovering failures, the notion of failure is re ected on the model by the state of the entity being switched from running to error. Failures are also propagated in the model along mandatory dependencies, as a software artifact cannot operate without them.
Our rst self-repair algorithm is speci ed as a set of ECA rules, which bind
potential failures to some prede ned repair procedures. We identi ed eleven
atomic modi cations on CloudML models, (e.g., provision of new node,
deployment of a given application, starting a given application), which are
combined into repair procedures. Fig. 4 depicts the repair procedure, which, given
an application, generates the following repair plan: if the application is in an
erroneous state, it rst resets the application, then it resolves all missing
dependencies, and nally starts the application. It is worth to note that the resolution
of a dependency is a separate procedure (called resolve), which is itself resolved
to generate the nal plan. In Fig. 4, the dotted arrows depict the re nements
of repair procedure according to the CloudML model introduced in Fig. 3,
where the vm2 failed. Four procedures are initially de ned: resolve, install
locally, provision and install, and repair. This planning technique is
referred in the literature as hierarchical tasks networks (HTN) [
The following paragraphs summarize how we executed the iterative process presented in Fig 1, on SensApp. In our experiment, the random generation of alternative environments is automated, whereas their integration in the models@runtime architecture is still done manually, as is the evaluation of key performance indicators.
Check Robustness to Unforeseen Environments The rst step of the process evaluates the con dence the designer has in the system to operate properly under unforeseen circumstances. repair ap1 reset ap1 resolve ap1.sensapp start ap1 pJ<- provisionAndInstall(Sensapp) bind ap1.sensapp to p provision vm2 install ap2:sensapp on vm2 resolve ap2.noSqlDb resolve ap2.wc start ap2 repair [a.status = stopped] 1. Having a clear de nition of the objectives of the self-adaptive system is the sine qua non condition to any assessment of its e ectiveness or its performance. Some self-adaptation techniques such as control theory, optimization methods, or planning techniques require a formal de nition of those objectives, but others, such as ECA rule sets, do not. In SensApp for instance, these objectives are not explicit, but the e ectiveness and the performance of the reparation process have implicitly driven the selection of the rules. 2. Once the objectives of the system are de ned, the designer must identify the key axes of uncertainty in the system's environment. SensApp may be subject to failures and may be deployed on a multi-tenant environment, where other applications that cannot be controlled may evolve and impact SensApp's behavior. Multi-tenancy and failures are thus the axes of uncertainty investigated hereafter. 3. Finally, designers have to prioritize the di erent axes of uncertainty according to their impact on the requirements and the system's objectives: the multi-tenancy hypothesis in the case of SensApp.
Since the multi-tenancy hypothesis is critical, the designer should then simulate failures in environments including applications evolving out of SensApp's control.
Design a \Diversi ed" Environment Simulator The multi-tenancy
hypothesis can be reproduced by breaking down the isolation in which the system
is running, which can be done by arti cially controlling the level of software
diversity in the environment. Indeed, isolation can be seen as a lack of software
diversity in the system's environment. Interested reader may consult [
vm4:JMedium-Linux (STOPPED) vm6:JLarge-Linux (RUNNING) NoSQLDB db2:JMongoDB (STOPPED) java3:JJdk1.6 (RUNNING)
Fig. 5 depicts an example of a simulated environment where diversity was injected into SensApp (see crosshatched virtual machines and applications). Three extra virtual machines are provisioned (i.e., vm4, vm5, and vm6) where additional applications are randomly deployed. In addition, applications are also randomly injected into existing virtual machines (i.e., ap3 is deployed on vm1 and is bound to the existing JRE as well as to the new remote applications). Our simulation introduces all possible failures of the SensApp system (e.g., failure of vm2).
Evaluation of the Reasoning Engine This step evaluates SensApp against randomly generated diversi ed environments, to check how well its self-adaptation mechanisms behave under multi-tenancy. This evaluation covers two dimensions: the e ectiveness of the reparation and its performance.
The e ectiveness of the self-repair strategies of SensApp are measured by its ability to restore itself into a running state. A repair is successful if and only if its execution results in a valid deployment of SensApp. Then, the manual inspection of simulation results showed that the e ectiveness is not impacted by multi-tenancy.
In addition, the performance of the repair procedure is also measured and is de ned as the cost of executing the resulting repair plan. In a cloud setting, di erent repair actions may have di erent costs: expensive actions are typically the provisioning of a new node and the deployment of applications on existing virtual machines (as it may require heavy network tra c). In this respect, simulation results revealed that the generated plans are suboptimal, as the planner provisions new virtual machines for each failure, without leveraging available opportunities in its environment.
Update of the Reasoning Engine Several alternative designs may help mitigating the performance drop under multi-tenancy, namely the modi cation of ECA rules or the replacement of the reasoning engine.
The rst update consisted in modifying the set of ECA rules and evaluating their e ectiveness and performance (Step 3). This updated reasoning engine preserved the scalability of the reasoning engine, but may still miss optimal plan.
A second update led to the use of automated planning techniques able to reach the optimal solution. Manual investigations of planning traces showed that generic automated planning techniques do not scale to large systems (about 30 entities in the models). 5
Uncertainty has been identi ed has one of the key obstacles to the design and
development of self-adaptive systems [
From a theoretical standpoint, several frameworks have emerged to capture
di erent types of uncertainty [
An alternative approach to uncertainty is to rely on online machine
learning. Approaches such as FUSION [
In contrast to the above approaches, our iterative process (cf. Fig. 1) shares
more with the ideas of agile practices [
Our iterative process is also inspired by existing risk analysis methods such
as CORAS [
This paper presented how models@runtime enables an iterative process to foster the design of reasoning engines for self-adaptive systems operating under certainty. This process is broken down into four steps: selection of uncertainty axis, exploration of unforeseen environments through simulation, evaluation of reasoning engine, followed by its potential re nement. Models@runtime appeared to be the architectural pattern which facilitates such a coevolution of the reasoning engine and its environments. We reported how this process is used in the Diversify project to develop a self-repair mechanism for large scale cloud-based systems, inspired by principles de ned in ecology about biological diversity.
This research e ort is a rst step toward an automated re nement process for self-adaptive systems. Our future work will consider the continuous design of distributed and collaborative reasoning engines.