=Paper= {{Paper |id=Vol-1723/4 |storemode=property |title=Fault-aware Pareto Frontier Exploration for Dependable System Architectures |pdfUrl=https://ceur-ws.org/Vol-1723/4.pdf |volume=Vol-1723 |authors=Lukas Märtin,Hauke Baller,Anne Koziolek,Ralf Reussner |dblpUrl=https://dblp.org/rec/conf/models/MartinBKR16 }} ==Fault-aware Pareto Frontier Exploration for Dependable System Architectures== https://ceur-ws.org/Vol-1723/4.pdf

Fault-aware Pareto Frontier Exploration for
Dependable System Architectures
Lukas Märtin∗ , Hauke Baller∗ , Anne Koziolek† , Ralf Reussner†
∗ TU Braunschweig, Germany

Email: {maertin,baller}@ips.cs.tu-bs.de
† Karlsruhe Institute of Technology, Germany

Email: {koziolek,reussner}@kit.edu

Abstract—While designing dependable systems, a large num- Design Space
ber of asset combinations (system configurations) with contrary
C1 C2 C3
quality objectives needs to be investigated. Basically, each feasible
configuration should be investigated. For fault-tolerant embedded
systems this problem is extended by anticipating hardware
faults leading to changed deployments of stressed resources in R1 R2

redundant constellations. The identification and evaluation of the Redundant System Design
best-fitting configuration remains a computationally intensive and
difficult task at all.
We propose a multi-stage approach (1) to sample Pareto- C1 C2 C3 R1 R2

optimal configurations for redundant system designs within
1 1 0 1 1 é è ê
hostile environments, (2) to check satisfiability of structural
constraints and (3) to measure and identify quality degradation 0 1 0 1 0 ê ê é
in fault scenarios. Thus, allowing developers to identify design 0 1 1 1 1 ì ì è
flaws, leading to large quality degradations in case of emerging Abstrac8on
faults. We use genetic algorithms (NSGA-II) for sampling a wide
range of system designs and demonstrate our approach by means Fig. 1. System Architecture for a Dependable System with Quality Ratings
of an exemplary fault-tolerant system.

I. I NTRODUCTION lifts the exploration to an abstract level of component-based
In fault-tolerant software design, the provision of depend- software engineering for embedded systems [2]. Here, the
able systems is charged with high expenses. In particular, deployment of components describes use-relations to the plat-
a diversity of replacement units (redundancies) needs to be form (resources for execution). Figure 1 shows an example
specified and addressed in redundancy methods and distributed for a redundant system design with an excerpt of feasible
well to successfully maintain faults [1]. Thus, developers are configurations, defined as the reconfiguration space by the
concerned with distinguishing many feasible system configura- developer. Each configuration requires a different subset of
tions, mostly equipped with redundant hardware resources. To hardware resources from the platform and result in varying
meet the functional requirements and simultaneously optimize ratings (values) for quality attributes. Figure 1 depicts the
multiple dimensions of system’s quality, expensive explo- differing uses of hardware resources (R1, R2) by software
rations of the design space are inevitable to find a best-fitting components (C1, C2, C3) during execution. Configurations
system variant for deployment. (rows) are defined by selecting (marked by 1) elements.
This challenge is getting even more complicated by further Some selections are optional, denoted with dashed lines. Each
considering further variants for reconfiguration upon hardware defined configuration is rated, leading to rising, falling, or con-
resource faults. Such potential faults of stressed resources in stant quality changes (arrow directions). Here, we consider the
hostile environments, e.g., cosmic radiation harming space quality attributes energy, performance and maintenance costs.
crafts and satellites, might be predicted by methods like Fault Each configuration is also validated against a set of structural
Tree Analysis, but the consequences on quality and functional constraints, representing the basic functional relations in the
validity are still expensive to inspect appropriately for rich architecture model. As soon as one of the hardware resources
design spaces. Here, each feasible configuration should be is marked as faulty (defined as fault scenario), some configu-
evaluated in face of all alternatives, leading to a exponential rations including fault-affected components may no longer be
complexity of comparisons. Even if the initial commit of a executable. To handle this partial loss of fault-tolerance, the
fault-tolerant system is usually more expensive than the initial developer needs to extend the former reconfiguration space by
commit of a regular system, the exploration of the design space additional configurations, not relying on the faulty hardware
has to be done in a systematic manner. resources. Each configuration has to be identified, quality-
From an architecture-oriented point of view, the separation rated and compared to the existing (valid) configurations.
and modeling of software components and hardware resources This procedure supports the developer in identifying possible
alternative configurations for an assumed fault scenario. In feasible system configurations, generated from queuing mod-
order to focus on significant degradations of quality attributes, els. Both approaches guide the developer to identify alterna-
a user-definable threshold for quality degradation is desirable tives for reconfiguring a system. However, the reconfiguration
from a developer’s point of view. space is not explicitly explored to identify quality drops upon
In previous work [3], we arranged alternative configurations faults in unstable hardware/software systems. Our approach
as nodes in a graph structure called Architecture Relation identifies such gaps and prioritizes near-by alternative configu-
Graph. Edges result from the reduction of available hardware ration for recommendation and decision support. In relation to
resources caused by faults. For edge prioritization, the qualities our distance measurement in neighborhood of faulty configura-
of each configuration are investigated. Such a strict hardware- tions, Barnes et al. describe relations between architectures as
oriented procedure is not feasible to evaluate alternative con- candidate evolution paths [10]. These paths specify a search-
figurations efficiently while considering quality attributes. In based reconfiguration process from a source to a pre-defined
this paper, we therefore investigate the measurement of quality target architecture via a sequence of transient architectures.
distances between configurations, including validity checking The goal is to shorten the paths to minimize reconfiguration
according to required hardware resources. efforts. However, we aim to retain as much system quality
as possible without defining a target architecture manually.
Foundations and Related Work J. R. Schott [11] defines a metric called spacing to measure
Our work relates to the concept of Degrees of Freedom [4] how well non-dominated individuals on the Pareto-frontier
to define and evaluate variability in architecture design. Pos- are distributed with respect to their neighbors. Following that
sible variation points are specified as explicit part of the idea, Gong et. al [12] also use a neighbor-based technique to
architecture model. A genetic algorithm explores the design inspect the crowding distances of non-dominated individuals
space to find Pareto-optimal solutions, i.e., the supremum and select minority isolated individuals. Thus, they refine
of all feasible solutions with respect to contrary objectives, recombination and mutation by determining nearest neighbors
respecting a set of quality attributes. This procedure can be of less-crowded individuals for the next optimization iteration.
applied to support design decisions and to explore potential In our approach, we also explore dominated neighbors to find
reconfiguration options [5]. To apply the approach to its design alternatives for individuals that became infeasible due
full extend, we need to create rich design models to gather to resource faults by comparing and minimizing distances in
ratings for quality attributes by simulation. Instead in this the objective space.
paper, we use simplified representations of design models and In the area of search-based approaches, Garvin et al. [13]
abstract quality measurements in order to provide a lightweight combine heuristic search with Feature Modeling. By using
implementation of the basic concepts in our approach. simulated annealing, the authors extend a test generation
The sampling of system configurations is performed by the algorithm to determine valid feature configurations. Based
genetic algorithm NSGA-II [6]. Echtle et al. [7] also apply on an array representation of a feature model, the algorithm
such algorithms to identify fault-tolerant system designs on perform pair-wise changes of feature selections. After each
a high level of abstraction. This work is focused on finding change a SAT check on the feature model is done. The fitness
critical fault combinations leading to invalid system designs. function of the optimization tries to maximize coverage of
We describe the variation space of the system explicitly to feature pairs. Similarly to that, Ensan et al. [14] apply a genetic
check validity of many sampled designs in a short period of algorithm to generate products (configurations) in accordance
time. More precisely, we use a propositional logic formula to to a feature model. In both approaches, each gene of a
describe the design space of examined systems and imitate chromosome represents a feature. The fitness of a product is
faults by disabling operation-critical hardware resources to coverage-oriented by evaluating the variability points and their
restrict feasible variations in configurations. Technically, we constraints from the feature model. In our approach, a feature
represent relevant parts of the architecture model as features model provides the structure for variation points and restricts
in a feature model. Feature models provide a comprehensible the selection of configurations by constraints. However, we do
graphical representation of a variant rich system. Relations not consider coverage measurements, but guide our approach
between functional components and hardware resources are by minimizing distances between configurations. Furthermore,
defined by constraints in the feature model. The fitness of a we assume, that the number of variation points is decreased
feasible solution, i.e., a configuration validated by the feature by faulty features, potentially leading to faulty configurations.
model, is based on quality assignments annotated as property Several other tools from literature apply genetic algorithms to
to each feature. generate products from a feature model in testing Software
Frey et al. [8] inspect reconfigurations as deployment op- Product Lines, e.g., PLEDGE [15].
tions for cloud-based systems derived by genetic algorithms.
The authors predefine rules at design time for systematically II. M ULTI -S TAGE A RCHITECTURE D ESIGN A NALYSIS
modified deployments of a system upon changed circum- Our approach searches for appropriate architectural design
stances in operation, e.g., system overloads. Similar to that, alternatives for reconfiguration under the assumption of pre-
Jung et al. [9] adapt a running system by policies derived at dictable hardware resource faults. The resulting set of configu-
design time. For that, a decision-tree learner is trained with rations needs to be ordered and prioritized by multiple quality
Design Space Stage 1 + 2 Stage 3 Stage 4 Stage 5
Pareto-optimal Fault Distance Aggregation +
Sampling Separation Measurement Analysis
C1 C2 C3

1.0
Healthy 1st

0.9
1st Run w/o Faults
Solutions

0.8
d1(Faulty,Healthy)

0.7
R1 R2 Quality Properties

0.6
Objectives + Values

1.0
0.5
1.0 1.2 1.4 1.6 1.8 2.0

0.9
Fault
Scenario NSGA-II Faulty 1st Euclidean

0.8
Comparison
Solutions

0.7
C1 C2 C3 R1 R2 d2 < d1

0.6
1.0
x x x x é è ê ?

0.5
0.9
1.0 1.2 1.4 1.6 1.8 2.0

0.8
x x ê ê é
ê

0.7
All 2nd
2nd Run with Faults d2(Faulty,All)
x x x ì ì è ?

0.6
Solutions

0.5
1.0 1.2 1.4 1.6 1.8 2.0

Fig. 2. Overview of our Multi-Stage Approach

dimensions. Thus, we explore the impact of resource faults faults. The presentation consists of statistics about the amount
with respect to the system’s quality dimensions in multiple of nearest neighbors of faulty solutions and quality differ-
stages. For (re-)sampling, comparing and representing feasible ences in distance matrices. Furthermore, large degradations
configurations in a comprehensible manner, we provide tool are highlighted to identify needs for design improvements.
support in five stages, depicted in Figure 2. Ultimately, the developer has just to set a threshold for distance
values as the upper limit for acceptable degradations in each
In the first two stages individual optimization runs are quality dimension. In the result all best-fitting alternatives for
performed with different settings. In stage one, no faults are a configuration, addressed by a fault scenario, are presented
considered and each locally optimal and valid alternative con- including quantified quality differences.
figuration is added to the Pareto-frontier. Although fault-prone
configurations of the first run might be detected for reevalu- Implementation
ation easily, another run is needed to determine previously Our approach was prototyped as an E CLIPSE plug-in1 .
dominated non-faulty configurations. A second Pareto-frontier Thereby, we combined the plug-in F EATURE IDE [16] for
without any faulty configurations results from the second variant-rich feature modeling and validity analysis and the
run performed in stage two. In stage three, the comparison J M ETAL [17] library for multi-objective optimization with
of both Pareto-frontiers is prepared by injecting the same meta-heuristics. Each problem-essential software component
fault scenarios in the results of the first run. Therefore, all and hardware resource of the system is identified with a feature
faulty configurations are separated from the healthy (still fully within a feature model. Furthermore, constraints in the model
operational) ones. Faults in resources might lead to side effects describe cross-cutting concerns between components and re-
in quality evaluation, e.g., if a faulty resource is cold-redundant sources, i.e., implications or excludes. In order to improve
to another still healthy resource of the configuration. Thus, an readability and to represent an architecture-oriented structure,
a posteriori re-calculation of qualities of each healthy configu- abstract features with no corresponding architectural elements
ration is performed. In a reconfiguration process, an alternative are used. To define objectives for the optimization, the root of
configuration for a faulty configuration seems to be optimal the feature model is annotated with a list of considered quality
if minimal quality losses is archived. As measurement for attributes. Based on this list, each concrete feature holds dis-
comparing neighbors of faulty configurations, the Euclidean crete ratings of one or more of these quality attributes. During
distances between the faulty configurations and healthy ones optimization these assignments are evaluated and summed up2
are determined in stage four. Next the distance measurement for each feature contributing to the configuration under the
between faulty configuration of the first run and the newly fitness analysis. We do not consider additional side constraints
sampled configurations from the second run is performed to to restrict the optimization objectives beyond the ones from the
find new nearest neighbors again. From both comparisons dis- feature model.
tance matrices result. To figure out the best-fitting alternative We apply the NSGA-II implementation from the J M ETAL
configurations, the matrices are merged to find in the combined framework to sample binary decision vectors. For each 1
results the nearest neighbors for each faulty configuration. occurring in that vector, a corresponding feature in the feature
This action already leads us to a basic transition structure to model is selected; without any propagation guided by the
judge reconfiguration decisions. Our approach is intended to rules in the feature model. The whole selection is validated
support design and maintenance activities. Therefore, the final by the SAT checker of the F EATURE IDE core engine. If the
stage five refers to the presentation of results. This allows
developers of fault-tolerant systems to identify design flaws 1 https://github.com/lmaertin/modcomp

leading to potentially large quality degradations in case of 2 Function for aggregation can be customized, e.g., Mean or Median
sampled configuration is valid, the solution is rated by the after that injector was cleaned by an additional resource.
quality assignments of the configuration, gathered from the Thus, each of these reproductions leads to changes in required
feature annotations before. If the SAT check fails, the solution resources and system’s quality.
is downgraded in each quality dimension. The system contains the following sensors and actuators.
For the given fault scenarios, we mark faulty resources Sensors: Buttons (still water, sparkling water, coke, return
by deselecting features that are related with deployments to money), counters (coins, notes), a money card terminal and
such resources. During the first optimization run, such faults filling-level meters (water, coke-mix, collector tray for clean-
are initially ignored. After the run is completed, faults are ing, cups), and a thermometer for chilling-control.
injected and all configurations from the first run are re- Actuators: Mixers (coke, CO2 ), flow controllers (pump, grav-
validated in F EATURE IDE. By rechecking satisfiability, some ity), valves (water, coke) and injectors (water, coke), money
alternative might be no longer valid. The resulting faulty (non- changers (coins, notes).
valid) and healthy (still valid) configurations are stored in two As a baseline for complexity analysis and satisfiability
independent sets for further processing. In addition, also the checks during configuration validations, we specified our de-
quality assignments of healthy configurations are reevaluated sign by a feature model, depicted in Fig. 3. The resources
according to the potentially changed number of addressed are represented as concrete features (dark blue boxes) in
quality attributes affecting the aggregated sums. the model and labeled with indexes from 0 to 20. In total
The presentation of results provides all data about feasible our system provides a design space of about 7,700 valid
alternative configuration to the developer in a comprehensive configurations with a variety of degradations in all quality
manner. In addition to general statistics (number of solutions dimensions. We assume that customers prefer to drink chilled
of both runs, ratios faulty vs. healthy and faulty vs. second drinks and like coke more than sparkling as well as sparkling
run), the data of the new reconfiguration space is aggregated water more than still water.
for decision support.
Quality Attributes and Fault Scenarios
Because of usually varying qualities in multi-objective opti-
mization, it is reasonable to let the user define a threshold for For optimization, we defined a set of quality attributes
qualities for alternative configurations. In this way, the identi- to be minimized. (1) pollution to observe the compliance
fication of a rich neighborhood set of configurations for each of hygienic value limits, (2) taste deviation according to
faulty configuration is promoted. Without a distance threshold, company’s standards, (3) response time representing the time
just the (one) best-fitting neighbor would be computed. After to drink delivery, and (4) energy consumption of the machine.
the data processing is completed, all distances are ordered In order to evaluate our approach, we use a fault scenario
beneath the threshold. On the one hand, the subset of results with significant impact on the design space, i.e., affecting
is shown as a distance matrix and new neighbors from the more than just optional features. By defining the resources
second run are highlighted. On the other hand, gaps between CokeInjector and Pump as faulty, one half of initially sampled
the Pareto-frontiers are investigated to identify most significant configurations is no longer valid. Thus, the developer has to
quality impacts. For that, a list of largest distances between figure out which alternative configurations are best-fitting.
faulty configurations and nearest neighbors is created. Results and Findings
The aggregated information can be used by the developer to
The evaluation is performed on an Intel i5 CPU at 2,5GHz
optimize the design, e.g., by adding additional resources, and
with 16GB of memory running Mac OS El Capitan, Java 8
to derive rules for reconfigurations during self-maintenance.
and E CLIPSE N EON (F EATURE IDE 3.0.1, J M ETAL 4.5.2). The
III. E VALUATION NSGA-II was set to a population size of 100 and 250 iterations
for demonstration purposes.
For evaluation purposes, we applied our tool-supported
After the first three stages are performed, the following
approach to a fault-tolerant vending machine. For simplicity,
statistics result from our experiment.
the system deals with a well-known application scenario en-
• #Solutions first run: 4
hanced by redundancies and a fair reconfiguration space. Thus,
the scenario addresses the domain of fault-tolerant embedded – #Faulty: 2
system design regarding redundant sensors and actuators. In – #Healthy: 2
addition, the system relies on software-intensive sensing and • #Solutions second run: 2
control, instead of pure mechanical solutions. Thus, the comparison of distances between faulties and health-
ies as well as faulties and new sampled is done both times in
Fault-tolerant Vending Machine ratios of 2:2. We pruned duplicate solutions, leading to a small
The vending machine offers still water, sparkling water and set of unique optimal solutions for the example system.
coke in cups, optionally chilled. Payments are accepted by With a threshold set to a maximal distance of 0.65, a
coins, notes and money card. Some sensors and actuators can distance matrix for each faulty solution is generated. Due
partially emulate other ones to support a high degree of fault- to lack of space, in Table I the neighbors for only one
tolerance without cost-intensive replication of resources. For faulty configuration are shown. The configurations are shown
instance, water can alternatively be served by the coke injector as binary vectors (optimization solution) representing the
VendingMachine

UserInterface PaymentMethods FillMeters Thermometer Mixing FlowControl MagnetValves Injector MoneyChanger

Still Sparkling Coke CoinPayment NotePayment CardPayment WaterMeter PostmixMeter CollectorMeter CupMeter Postmix CO2 Pump Gravity WaterValve CokeValve WaterInjector CokeInjector ChangeCoins ChangeNotes

0 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19 20
¬CO2 ⇒ ¬Sparkling ¬ChangeCoins ⇒ ¬CoinPayment ∧ ¬NotePayment Legend:
Coke ⇒ PostmixMeter ∧ Postmix ∧ CO2 (Still ∨ Sparkling) ∧ ¬WaterInjector ⇒ CollectorMeter ∧ CokeInjector Mandatory Or
Coke ∧ ¬CokeInjector ⇒ CollectorMeter ∧ WaterInjector ¬WaterValve ⇒ ¬Still ∧ ¬Sparkling Optional Alternativ
¬ChangeNotes ⇒ ¬NotePayment ¬CokeValve ⇒ ¬Coke Abstract Concrete

Fig. 3. Feature Model of Fault-Tolerant Vending Machine

TABLE I TABLE II
D ISTANCES FOR FAULTY S OLUTION (100100100100110101010) L ARGEST G APS IN VALUE A SSIGNMENTS TO O BJECTIVES

Dis. Qual. Assignments Solution Vector Origin Objective Gap Assignment 1 Assignment 2
0.58 0.3 1.4 1.1 0.0 100001100100101101000 healthy 1 0.00 0.3 1.4 1.1 0.0 0.3 1.4 1.1 0.0
0.58 0.3 1.4 1.1 0.0 100001100100101101000 second 2 0.00 0.3 1.4 1.1 0.0 0.3 1.4 1.1 0.0
0.64 0.3 0.9 1.2 0.5 100100100100101101010 healthy 3 0.09 0.3 0.9 1.2 0.5 0.3 1.4 1.1 0.0
0.64 0.3 0.9 1.2 0.5 100100100100101101010 second 4 0.54 0.3 0.9 1.2 0.54 0.3 1.4 1.1 0.0
... ... ... ...

selection of concrete features in order of depth-first search ●
●

1.8
corresponding to the indexes given in Fig. 3. Newly appearing ●
●

nearest neighbors from the second run are highlighted in bold ●

1.6
● ●

font. The quality assignments refer to the quality attributes and Response Time ●
●

their order introduced before. Further neighbors with distances 1.4
●
●
●
●

above the threshold value are hidden by “. . . ”. ●

●

By inspecting the values, the developer can figure out
1.2

●
●

the configuration (100001100100101101000) as the nearest ●

●
1.0

therefore and best-fitting neighbor of the faulty configuration.
●
According to the feature model, the resulting vending machine
0.7 0.8 0.9 1.0 1.1 1.2
sells still water via card payment, monitors water fill level
Energy Consumption
and number of remaining cups, supports CO2 mixing and
uses a gravity-based water flow control towards a water value
and a water injector. In this case, this configuration was Fig. 4. Plot for two Quality Dimensions
randomly sampled in first and second optimization run. To
assure fault-tolerance, the nearest neighbors shall also be faulty solutions and the frontier containing all alternative so-
considered in the rule set for run-time reconfiguration. Thus, lutions from the second run. Here, we suggest to recapture the
all neighbors under the given distance threshold are added to design to minimize that gap by resource changes contributing
a new set of solutions, representing the reconfiguration space. to meeting the objectives.
For optimization of the design space, the largest distances
between solutions are also investigated. Our implementation Discussion
performs pair-wise comparisons to find the largest distance During evaluation we were faced with some algorithmic
between quality dimensions in objective space, i.e., gaps in characteristics in optimizing multiple objectives. Despite of
Pareto-frontiers. In our experiments, we figured out the largest differing assignments of quality attributes and a complexity
distances in solutions for all quality dimensions as listed in of about 7.700 feasible configurations, the generation led to
Table II. Subsequent to those results a developer may use an just a few unique configurations and many duplicates. We
appropriate tool to visually explore gaps in the solution, e.g., presume, that this is caused by the small-scaled application
by hierarchical cluster analysis in GNU R. scenario and side-effects by similarities in quality assignments.
To visually present our idea, we performed an optimization Furthermore, we do not consult constraints for objectives, e.g.,
with two quality dimensions (Response Time and Energy minimal acceptance values as lower bounds. Nevertheless, our
Consumption), resulting in the 2D-plot in Figure 4. Faulty gap investigations were also applicable by considering only
solutions are colored in red, still healthy solutions are green widely spread objective values. Following the idea of Deb et
and new solutions from second sampling are shown in blue. al. [18], an -dominance might support the reduction of such
The plot shows a large gap between the Pareto-frontier of gaps by a better diversity to be maintained in a population.
IV. C ONCLUSIONS R EFERENCES
Even if existing techniques for fault-tolerant system design [1] A. Avizienis, “Toward systematic design of fault-tolerant systems,”
Computer, vol. 30, no. 4, pp. 51–58, 1997.
assist the developer in identifying necessary redundancies, [2] L. Grunske, P. Lindsay, E. Bondarev, Y. Papadopoulos, and D. Parker,
additional best-fitting configurations have to be figured-out. “An outline of an architecture-based method for optimizing dependabil-
Our approach guides the developer through the subset of the ity attributes of software-intensive systems,” in Architecting Dependable
Systems IV, ser. LNCS. Springer, 2007, vol. 4615, pp. 188–209.
remaining design options after a hardware resource fault is [3] L. Märtin, A. Koziolek, and R. H. Reussner, “Quality-oriented Decision
injected. Under the consideration of such a fault scenario, Support for maintaining Architectures of fault-tolerant Space Systems,”
Pareto-optimal solutions are sampled and decision support to in 9th Europ. Conf. on Software Architecture Workshops, September
2015, pp. 49:1–49:5.
identify nearest alternates to a faulty configuration is provided [4] A. Koziolek and R. Reussner, “Towards a generic quality optimisation
in five process stages. In addition, large gaps in a system’s framework for component-based system models,” in 14th Int. ACM
quality can be shown with our tooling to find flaws in Sigsoft Symp. on Component based Software Engineering, 2011, pp.
103–108.
redundant system design. [5] A. Aleti, B. Buhnova, L. Grunske, A. Koziolek, and I. Meedeniya,
“Software Architecture Optimization Methods: A Systematic Literature
Future Work Review,” IEEE Trans. on Software Engineering, vol. 39, no. 5, pp. 658–
683, 2013.
We provide lightweight tooling for mass-generating solution [6] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist
set and gap exploration of Pareto-frontiers. Based on our multiobjective genetic algorithm: NSGA-II,” IEEE Trans. on Evolution-
previous experience, we plan to integrate our exploration ary Computation, vol. 6, no. 2, pp. 182–197, 2002.
[7] K. Echtle and I. Eusgeld, A Genetic Algorithm for Fault-Tolerant System
concept within the Palladio toolset and its add-ons. For full- Design. Springer, 2003, pp. 197–213.
fledged architecture modeling [19] we will apply PALLADIO [8] S. Frey, F. Fittkau, and W. Hasselbring, “Search-based genetic opti-
B ENCH3 and P ERO PTERYX4 to define variability in models. mization for deployment and reconfiguration of software in the cloud,”
in 35th Intern. Conf. on Software Engineering, 2013, pp. 512–521.
Also the sampling with genetic algorithms is performed there. [9] G. Jung, K. Joshi, M. Hiltunen, R. Schlichting, and C. Pu, “Generating
Ratings of quality attributes are gathered by the simulation Adaptation Policies for Multi-tier Applications in Consolidated Server
engine S IMU C OM. We will make use of the results for distance Environments,” in 5th Int. Conf. on Autonomic Computing, 2008, pp.
23–32.
comparison and gap exploration proposed in this paper. [10] J. M. Barnes, A. Pandey, and D. Garlan, “Automated planning for soft-
In particular, the findings of this paper will contribute to the ware architecture evolution,” in 28th Int. Conf. on Automated Software
improvement of our decision structure Architecture Relation Eng., 2013, pp. 213–223.
[11] J. R. Schott, “Fault tolerant design using single and multicriteria genetic
Graph [3] and on-going work in area of hierarchical cluster algorithm optimization.” DTIC, Tech. Rep., 1995.
analysis. The final selection of which configurations from [12] M. Gong, L. Jiao, H. Du, and L. Bo, “Multiobjective immune algorithm
the Pareto-frontiers are added to the graph still relies on with nondominated neighbor-based selection,” IEEE Trans. on Evolu-
tionary Computation, vol. 16, no. 2, pp. 225–255, 2008.
trade-off settings preferred by the developer. Such trade-off [13] B. J. Garvin, M. B. Cohen, and M. B. Dwyer, “Evaluating improvements
analysis is supported by our tool AR EVA 25 . Based on the to a meta-heuristic search for constrained interaction testing,” Empirical
work of Florentz et al. [20] contrary quality properties are Software Engineering, vol. 16, no. 1, pp. 61–102, 2010.
[14] F. Ensan, E. Bagheri, and D. Gašević, “Evolutionary search-based test
normalized by conversion function and ordered hierarchically generation for software product line feature models,” in 24th Intern.
with weightings. This analysis needs to be integrated with the Conf. on Advanced Information Systems Engineering. Springer, 2012,
distance measurements from our tool prototype presented here. pp. 613–628.
[15] C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. L. Traon,
We plan to comprehensively evaluate the integrated tool- “PLEDGE: a product line editor and test generation tool,” in 17th Intern.
supported approach at whole with a case study. For that, we Software Product Line Conf. co-located Workshops. ACM, 2013, pp.
will build upon our previous findings in the domain of space 126–129.
[16] T. Thüm, C. Kästner, F. Benduhn, J. Meinicke, G. Saake, and T. Leich,
systems [21]. As a real-world case study, we have access to “FeatureIDE: An extensible framework for feature-oriented software
a system design of a micro-satellite provided by one of our development,” Sci. Comput. Program., vol. 79, pp. 70–85, 2014.
industrial partners. The system has a high degree of inherent [17] J. J. Durillo and A. J. Nebro, “jMetal: A Java Framework for Multi-
objective Optimization,” Adv. Eng. Softw., vol. 42, no. 10, pp. 760–771,
availability implemented by autonomy mechanisms and a large Oct. 2011.
number of redundant hardware resources. We will extend our [18] K. Deb, M. Mohan, and S. Mishra, “Towards a quick computation of
idea from previous work [22] in enhancing the system by well-spread pareto-optimal solutions,” in 2nd Intern. Conf. on Evolu-
tionary Multi-criterion Optimization. Springer, 2003, pp. 222–236.
replication-redundant capabilities as addressed here. [19] R. Reussner, S. Becker, E. Burger, J. Happe, M. Hauck, A. Koziolek,
H. Koziolek, K. Krogmann, and M. Kuperberg, “The Palladio Compo-
ACKNOWLEDGMENT nent Model,” KIT Department of Informatics, Tech. Rep., 2011.
[20] B. Florentz and M. Huhn, “Embedded systems architecture: Evaluation
This work was partially supported by the DFG (German and analysis,” in Quality of Software Architectures, ser. LNCS. Springer,
Research Foundation) under the Priority Programme SPP1593: 2006, vol. 4214, pp. 145–162.
Design For Future Managed Software Evolution. The authors [21] L. Märtin, M. Schatalov, M. Hagner, O. Maibaum, and U. Goltz, “A
Methodology for Model-based Development and Automated Verification
would like to give special thanks to Axel Busch for fruitful of Software for Aerospace Systems,” in 34th IEEE Intern. Aerospace
discussions on tool integration within P ERO PTERYX. Conf., 2013.
[22] L. Märtin and A. Nicolai, “Towards Self-Reconfiguration of Space
3 http://www.palladio-simulator.com Systems on Architectural Level based on Qualitative Ratings,” in 35th
4 https://sdqweb.ipd.kit.edu/wiki/PerOpteryx IEEE Intern. Aerospace Conf., Big Sky, USA, March 2014.
5 https://github.com/lmaertin/areva2