Fault-aware Pareto Frontier Exploration for Dependable System Architectures Lukas Märtin∗ , Hauke Baller∗ , Anne Koziolek† , Ralf Reussner† ∗ TU Braunschweig, Germany Email: {maertin,baller}@ips.cs.tu-bs.de † Karlsruhe Institute of Technology, Germany Email: {koziolek,reussner}@kit.edu Abstract—While designing dependable systems, a large num- Design Space ber of asset combinations (system configurations) with contrary C1 C2 C3 quality objectives needs to be investigated. Basically, each feasible configuration should be investigated. For fault-tolerant embedded systems this problem is extended by anticipating hardware faults leading to changed deployments of stressed resources in R1 R2 redundant constellations. The identification and evaluation of the Redundant System Design best-fitting configuration remains a computationally intensive and difficult task at all. We propose a multi-stage approach (1) to sample Pareto- C1 C2 C3 R1 R2 optimal configurations for redundant system designs within 1 1 0 1 1 é è ê hostile environments, (2) to check satisfiability of structural constraints and (3) to measure and identify quality degradation 0 1 0 1 0 ê ê é in fault scenarios. Thus, allowing developers to identify design 0 1 1 1 1 ì ì è flaws, leading to large quality degradations in case of emerging Abstrac8on faults. We use genetic algorithms (NSGA-II) for sampling a wide range of system designs and demonstrate our approach by means Fig. 1. System Architecture for a Dependable System with Quality Ratings of an exemplary fault-tolerant system. I. I NTRODUCTION lifts the exploration to an abstract level of component-based In fault-tolerant software design, the provision of depend- software engineering for embedded systems [2]. Here, the able systems is charged with high expenses. In particular, deployment of components describes use-relations to the plat- a diversity of replacement units (redundancies) needs to be form (resources for execution). Figure 1 shows an example specified and addressed in redundancy methods and distributed for a redundant system design with an excerpt of feasible well to successfully maintain faults [1]. Thus, developers are configurations, defined as the reconfiguration space by the concerned with distinguishing many feasible system configura- developer. Each configuration requires a different subset of tions, mostly equipped with redundant hardware resources. To hardware resources from the platform and result in varying meet the functional requirements and simultaneously optimize ratings (values) for quality attributes. Figure 1 depicts the multiple dimensions of system’s quality, expensive explo- differing uses of hardware resources (R1, R2) by software rations of the design space are inevitable to find a best-fitting components (C1, C2, C3) during execution. Configurations system variant for deployment. (rows) are defined by selecting (marked by 1) elements. This challenge is getting even more complicated by further Some selections are optional, denoted with dashed lines. Each considering further variants for reconfiguration upon hardware defined configuration is rated, leading to rising, falling, or con- resource faults. Such potential faults of stressed resources in stant quality changes (arrow directions). Here, we consider the hostile environments, e.g., cosmic radiation harming space quality attributes energy, performance and maintenance costs. crafts and satellites, might be predicted by methods like Fault Each configuration is also validated against a set of structural Tree Analysis, but the consequences on quality and functional constraints, representing the basic functional relations in the validity are still expensive to inspect appropriately for rich architecture model. As soon as one of the hardware resources design spaces. Here, each feasible configuration should be is marked as faulty (defined as fault scenario), some configu- evaluated in face of all alternatives, leading to a exponential rations including fault-affected components may no longer be complexity of comparisons. Even if the initial commit of a executable. To handle this partial loss of fault-tolerance, the fault-tolerant system is usually more expensive than the initial developer needs to extend the former reconfiguration space by commit of a regular system, the exploration of the design space additional configurations, not relying on the faulty hardware has to be done in a systematic manner. resources. Each configuration has to be identified, quality- From an architecture-oriented point of view, the separation rated and compared to the existing (valid) configurations. and modeling of software components and hardware resources This procedure supports the developer in identifying possible alternative configurations for an assumed fault scenario. In feasible system configurations, generated from queuing mod- order to focus on significant degradations of quality attributes, els. Both approaches guide the developer to identify alterna- a user-definable threshold for quality degradation is desirable tives for reconfiguring a system. However, the reconfiguration from a developer’s point of view. space is not explicitly explored to identify quality drops upon In previous work [3], we arranged alternative configurations faults in unstable hardware/software systems. Our approach as nodes in a graph structure called Architecture Relation identifies such gaps and prioritizes near-by alternative configu- Graph. Edges result from the reduction of available hardware ration for recommendation and decision support. In relation to resources caused by faults. For edge prioritization, the qualities our distance measurement in neighborhood of faulty configura- of each configuration are investigated. Such a strict hardware- tions, Barnes et al. describe relations between architectures as oriented procedure is not feasible to evaluate alternative con- candidate evolution paths [10]. These paths specify a search- figurations efficiently while considering quality attributes. In based reconfiguration process from a source to a pre-defined this paper, we therefore investigate the measurement of quality target architecture via a sequence of transient architectures. distances between configurations, including validity checking The goal is to shorten the paths to minimize reconfiguration according to required hardware resources. efforts. However, we aim to retain as much system quality as possible without defining a target architecture manually. Foundations and Related Work J. R. Schott [11] defines a metric called spacing to measure Our work relates to the concept of Degrees of Freedom [4] how well non-dominated individuals on the Pareto-frontier to define and evaluate variability in architecture design. Pos- are distributed with respect to their neighbors. Following that sible variation points are specified as explicit part of the idea, Gong et. al [12] also use a neighbor-based technique to architecture model. A genetic algorithm explores the design inspect the crowding distances of non-dominated individuals space to find Pareto-optimal solutions, i.e., the supremum and select minority isolated individuals. Thus, they refine of all feasible solutions with respect to contrary objectives, recombination and mutation by determining nearest neighbors respecting a set of quality attributes. This procedure can be of less-crowded individuals for the next optimization iteration. applied to support design decisions and to explore potential In our approach, we also explore dominated neighbors to find reconfiguration options [5]. To apply the approach to its design alternatives for individuals that became infeasible due full extend, we need to create rich design models to gather to resource faults by comparing and minimizing distances in ratings for quality attributes by simulation. Instead in this the objective space. paper, we use simplified representations of design models and In the area of search-based approaches, Garvin et al. [13] abstract quality measurements in order to provide a lightweight combine heuristic search with Feature Modeling. By using implementation of the basic concepts in our approach. simulated annealing, the authors extend a test generation The sampling of system configurations is performed by the algorithm to determine valid feature configurations. Based genetic algorithm NSGA-II [6]. Echtle et al. [7] also apply on an array representation of a feature model, the algorithm such algorithms to identify fault-tolerant system designs on perform pair-wise changes of feature selections. After each a high level of abstraction. This work is focused on finding change a SAT check on the feature model is done. The fitness critical fault combinations leading to invalid system designs. function of the optimization tries to maximize coverage of We describe the variation space of the system explicitly to feature pairs. Similarly to that, Ensan et al. [14] apply a genetic check validity of many sampled designs in a short period of algorithm to generate products (configurations) in accordance time. More precisely, we use a propositional logic formula to to a feature model. In both approaches, each gene of a describe the design space of examined systems and imitate chromosome represents a feature. The fitness of a product is faults by disabling operation-critical hardware resources to coverage-oriented by evaluating the variability points and their restrict feasible variations in configurations. Technically, we constraints from the feature model. In our approach, a feature represent relevant parts of the architecture model as features model provides the structure for variation points and restricts in a feature model. Feature models provide a comprehensible the selection of configurations by constraints. However, we do graphical representation of a variant rich system. Relations not consider coverage measurements, but guide our approach between functional components and hardware resources are by minimizing distances between configurations. Furthermore, defined by constraints in the feature model. The fitness of a we assume, that the number of variation points is decreased feasible solution, i.e., a configuration validated by the feature by faulty features, potentially leading to faulty configurations. model, is based on quality assignments annotated as property Several other tools from literature apply genetic algorithms to to each feature. generate products from a feature model in testing Software Frey et al. [8] inspect reconfigurations as deployment op- Product Lines, e.g., PLEDGE [15]. tions for cloud-based systems derived by genetic algorithms. The authors predefine rules at design time for systematically II. M ULTI -S TAGE A RCHITECTURE D ESIGN A NALYSIS modified deployments of a system upon changed circum- Our approach searches for appropriate architectural design stances in operation, e.g., system overloads. Similar to that, alternatives for reconfiguration under the assumption of pre- Jung et al. [9] adapt a running system by policies derived at dictable hardware resource faults. The resulting set of configu- design time. For that, a decision-tree learner is trained with rations needs to be ordered and prioritized by multiple quality Design Space Stage 1 + 2 Stage 3 Stage 4 Stage 5 Pareto-optimal Fault Distance Aggregation + Sampling Separation Measurement Analysis C1 C2 C3 1.0 Healthy 1st 0.9 1st Run w/o Faults Solutions 0.8 d1(Faulty,Healthy) 0.7 R1 R2 Quality Properties 0.6 Objectives + Values 1.0 0.5 1.0 1.2 1.4 1.6 1.8 2.0 0.9 Fault Scenario NSGA-II Faulty 1st Euclidean 0.8 Comparison Solutions 0.7 C1 C2 C3 R1 R2 d2 < d1 0.6 1.0 x x x x é è ê ? 0.5 0.9 1.0 1.2 1.4 1.6 1.8 2.0 é 0.8 x x ê ê é ê 0.7 All 2nd 2nd Run with Faults d2(Faulty,All) x x x ì ì è ? 0.6 Solutions 0.5 1.0 1.2 1.4 1.6 1.8 2.0 Fig. 2. Overview of our Multi-Stage Approach dimensions. Thus, we explore the impact of resource faults faults. The presentation consists of statistics about the amount with respect to the system’s quality dimensions in multiple of nearest neighbors of faulty solutions and quality differ- stages. For (re-)sampling, comparing and representing feasible ences in distance matrices. Furthermore, large degradations configurations in a comprehensible manner, we provide tool are highlighted to identify needs for design improvements. support in five stages, depicted in Figure 2. Ultimately, the developer has just to set a threshold for distance values as the upper limit for acceptable degradations in each In the first two stages individual optimization runs are quality dimension. In the result all best-fitting alternatives for performed with different settings. In stage one, no faults are a configuration, addressed by a fault scenario, are presented considered and each locally optimal and valid alternative con- including quantified quality differences. figuration is added to the Pareto-frontier. Although fault-prone configurations of the first run might be detected for reevalu- Implementation ation easily, another run is needed to determine previously Our approach was prototyped as an E CLIPSE plug-in1 . dominated non-faulty configurations. A second Pareto-frontier Thereby, we combined the plug-in F EATURE IDE [16] for without any faulty configurations results from the second variant-rich feature modeling and validity analysis and the run performed in stage two. In stage three, the comparison J M ETAL [17] library for multi-objective optimization with of both Pareto-frontiers is prepared by injecting the same meta-heuristics. Each problem-essential software component fault scenarios in the results of the first run. Therefore, all and hardware resource of the system is identified with a feature faulty configurations are separated from the healthy (still fully within a feature model. Furthermore, constraints in the model operational) ones. Faults in resources might lead to side effects describe cross-cutting concerns between components and re- in quality evaluation, e.g., if a faulty resource is cold-redundant sources, i.e., implications or excludes. In order to improve to another still healthy resource of the configuration. Thus, an readability and to represent an architecture-oriented structure, a posteriori re-calculation of qualities of each healthy configu- abstract features with no corresponding architectural elements ration is performed. In a reconfiguration process, an alternative are used. To define objectives for the optimization, the root of configuration for a faulty configuration seems to be optimal the feature model is annotated with a list of considered quality if minimal quality losses is archived. As measurement for attributes. Based on this list, each concrete feature holds dis- comparing neighbors of faulty configurations, the Euclidean crete ratings of one or more of these quality attributes. During distances between the faulty configurations and healthy ones optimization these assignments are evaluated and summed up2 are determined in stage four. Next the distance measurement for each feature contributing to the configuration under the between faulty configuration of the first run and the newly fitness analysis. We do not consider additional side constraints sampled configurations from the second run is performed to to restrict the optimization objectives beyond the ones from the find new nearest neighbors again. From both comparisons dis- feature model. tance matrices result. To figure out the best-fitting alternative We apply the NSGA-II implementation from the J M ETAL configurations, the matrices are merged to find in the combined framework to sample binary decision vectors. For each 1 results the nearest neighbors for each faulty configuration. occurring in that vector, a corresponding feature in the feature This action already leads us to a basic transition structure to model is selected; without any propagation guided by the judge reconfiguration decisions. Our approach is intended to rules in the feature model. The whole selection is validated support design and maintenance activities. Therefore, the final by the SAT checker of the F EATURE IDE core engine. If the stage five refers to the presentation of results. This allows developers of fault-tolerant systems to identify design flaws 1 https://github.com/lmaertin/modcomp leading to potentially large quality degradations in case of 2 Function for aggregation can be customized, e.g., Mean or Median sampled configuration is valid, the solution is rated by the after that injector was cleaned by an additional resource. quality assignments of the configuration, gathered from the Thus, each of these reproductions leads to changes in required feature annotations before. If the SAT check fails, the solution resources and system’s quality. is downgraded in each quality dimension. The system contains the following sensors and actuators. For the given fault scenarios, we mark faulty resources Sensors: Buttons (still water, sparkling water, coke, return by deselecting features that are related with deployments to money), counters (coins, notes), a money card terminal and such resources. During the first optimization run, such faults filling-level meters (water, coke-mix, collector tray for clean- are initially ignored. After the run is completed, faults are ing, cups), and a thermometer for chilling-control. injected and all configurations from the first run are re- Actuators: Mixers (coke, CO2 ), flow controllers (pump, grav- validated in F EATURE IDE. By rechecking satisfiability, some ity), valves (water, coke) and injectors (water, coke), money alternative might be no longer valid. The resulting faulty (non- changers (coins, notes). valid) and healthy (still valid) configurations are stored in two As a baseline for complexity analysis and satisfiability independent sets for further processing. In addition, also the checks during configuration validations, we specified our de- quality assignments of healthy configurations are reevaluated sign by a feature model, depicted in Fig. 3. The resources according to the potentially changed number of addressed are represented as concrete features (dark blue boxes) in quality attributes affecting the aggregated sums. the model and labeled with indexes from 0 to 20. In total The presentation of results provides all data about feasible our system provides a design space of about 7,700 valid alternative configuration to the developer in a comprehensive configurations with a variety of degradations in all quality manner. In addition to general statistics (number of solutions dimensions. We assume that customers prefer to drink chilled of both runs, ratios faulty vs. healthy and faulty vs. second drinks and like coke more than sparkling as well as sparkling run), the data of the new reconfiguration space is aggregated water more than still water. for decision support. Quality Attributes and Fault Scenarios Because of usually varying qualities in multi-objective opti- mization, it is reasonable to let the user define a threshold for For optimization, we defined a set of quality attributes qualities for alternative configurations. In this way, the identi- to be minimized. (1) pollution to observe the compliance fication of a rich neighborhood set of configurations for each of hygienic value limits, (2) taste deviation according to faulty configuration is promoted. Without a distance threshold, company’s standards, (3) response time representing the time just the (one) best-fitting neighbor would be computed. After to drink delivery, and (4) energy consumption of the machine. the data processing is completed, all distances are ordered In order to evaluate our approach, we use a fault scenario beneath the threshold. On the one hand, the subset of results with significant impact on the design space, i.e., affecting is shown as a distance matrix and new neighbors from the more than just optional features. By defining the resources second run are highlighted. On the other hand, gaps between CokeInjector and Pump as faulty, one half of initially sampled the Pareto-frontiers are investigated to identify most significant configurations is no longer valid. Thus, the developer has to quality impacts. For that, a list of largest distances between figure out which alternative configurations are best-fitting. faulty configurations and nearest neighbors is created. Results and Findings The aggregated information can be used by the developer to The evaluation is performed on an Intel i5 CPU at 2,5GHz optimize the design, e.g., by adding additional resources, and with 16GB of memory running Mac OS El Capitan, Java 8 to derive rules for reconfigurations during self-maintenance. and E CLIPSE N EON (F EATURE IDE 3.0.1, J M ETAL 4.5.2). The III. E VALUATION NSGA-II was set to a population size of 100 and 250 iterations for demonstration purposes. For evaluation purposes, we applied our tool-supported After the first three stages are performed, the following approach to a fault-tolerant vending machine. For simplicity, statistics result from our experiment. the system deals with a well-known application scenario en- • #Solutions first run: 4 hanced by redundancies and a fair reconfiguration space. Thus, the scenario addresses the domain of fault-tolerant embedded – #Faulty: 2 system design regarding redundant sensors and actuators. In – #Healthy: 2 addition, the system relies on software-intensive sensing and • #Solutions second run: 2 control, instead of pure mechanical solutions. Thus, the comparison of distances between faulties and health- ies as well as faulties and new sampled is done both times in Fault-tolerant Vending Machine ratios of 2:2. We pruned duplicate solutions, leading to a small The vending machine offers still water, sparkling water and set of unique optimal solutions for the example system. coke in cups, optionally chilled. Payments are accepted by With a threshold set to a maximal distance of 0.65, a coins, notes and money card. Some sensors and actuators can distance matrix for each faulty solution is generated. Due partially emulate other ones to support a high degree of fault- to lack of space, in Table I the neighbors for only one tolerance without cost-intensive replication of resources. For faulty configuration are shown. The configurations are shown instance, water can alternatively be served by the coke injector as binary vectors (optimization solution) representing the VendingMachine UserInterface PaymentMethods FillMeters Thermometer Mixing FlowControl MagnetValves Injector MoneyChanger 10 Still Sparkling Coke CoinPayment NotePayment CardPayment WaterMeter PostmixMeter CollectorMeter CupMeter Postmix CO2 Pump Gravity WaterValve CokeValve WaterInjector CokeInjector ChangeCoins ChangeNotes 0 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19 20 ¬CO2 ⇒ ¬Sparkling ¬ChangeCoins ⇒ ¬CoinPayment ∧ ¬NotePayment Legend: Coke ⇒ PostmixMeter ∧ Postmix ∧ CO2 (Still ∨ Sparkling) ∧ ¬WaterInjector ⇒ CollectorMeter ∧ CokeInjector Mandatory Or Coke ∧ ¬CokeInjector ⇒ CollectorMeter ∧ WaterInjector ¬WaterValve ⇒ ¬Still ∧ ¬Sparkling Optional Alternativ ¬ChangeNotes ⇒ ¬NotePayment ¬CokeValve ⇒ ¬Coke Abstract Concrete Fig. 3. Feature Model of Fault-Tolerant Vending Machine TABLE I TABLE II D ISTANCES FOR FAULTY S OLUTION (100100100100110101010) L ARGEST G APS IN VALUE A SSIGNMENTS TO O BJECTIVES Dis. Qual. Assignments Solution Vector Origin Objective Gap Assignment 1 Assignment 2 0.58 0.3 1.4 1.1 0.0 100001100100101101000 healthy 1 0.00 0.3 1.4 1.1 0.0 0.3 1.4 1.1 0.0 0.58 0.3 1.4 1.1 0.0 100001100100101101000 second 2 0.00 0.3 1.4 1.1 0.0 0.3 1.4 1.1 0.0 0.64 0.3 0.9 1.2 0.5 100100100100101101010 healthy 3 0.09 0.3 0.9 1.2 0.5 0.3 1.4 1.1 0.0 0.64 0.3 0.9 1.2 0.5 100100100100101101010 second 4 0.54 0.3 0.9 1.2 0.54 0.3 1.4 1.1 0.0 ... ... ... ... selection of concrete features in order of depth-first search ● ● 1.8 corresponding to the indexes given in Fig. 3. Newly appearing ● ● nearest neighbors from the second run are highlighted in bold ● 1.6 ● ● font. The quality assignments refer to the quality attributes and Response Time ● ● their order introduced before. Further neighbors with distances 1.4 ● ● ● ● above the threshold value are hidden by “. . . ”. ● ● By inspecting the values, the developer can figure out 1.2 ● ● the configuration (100001100100101101000) as the nearest ● ● 1.0 therefore and best-fitting neighbor of the faulty configuration. ● According to the feature model, the resulting vending machine 0.7 0.8 0.9 1.0 1.1 1.2 sells still water via card payment, monitors water fill level Energy Consumption and number of remaining cups, supports CO2 mixing and uses a gravity-based water flow control towards a water value and a water injector. In this case, this configuration was Fig. 4. Plot for two Quality Dimensions randomly sampled in first and second optimization run. To assure fault-tolerance, the nearest neighbors shall also be faulty solutions and the frontier containing all alternative so- considered in the rule set for run-time reconfiguration. Thus, lutions from the second run. Here, we suggest to recapture the all neighbors under the given distance threshold are added to design to minimize that gap by resource changes contributing a new set of solutions, representing the reconfiguration space. to meeting the objectives. For optimization of the design space, the largest distances between solutions are also investigated. Our implementation Discussion performs pair-wise comparisons to find the largest distance During evaluation we were faced with some algorithmic between quality dimensions in objective space, i.e., gaps in characteristics in optimizing multiple objectives. Despite of Pareto-frontiers. In our experiments, we figured out the largest differing assignments of quality attributes and a complexity distances in solutions for all quality dimensions as listed in of about 7.700 feasible configurations, the generation led to Table II. Subsequent to those results a developer may use an just a few unique configurations and many duplicates. We appropriate tool to visually explore gaps in the solution, e.g., presume, that this is caused by the small-scaled application by hierarchical cluster analysis in GNU R. scenario and side-effects by similarities in quality assignments. To visually present our idea, we performed an optimization Furthermore, we do not consult constraints for objectives, e.g., with two quality dimensions (Response Time and Energy minimal acceptance values as lower bounds. Nevertheless, our Consumption), resulting in the 2D-plot in Figure 4. Faulty gap investigations were also applicable by considering only solutions are colored in red, still healthy solutions are green widely spread objective values. Following the idea of Deb et and new solutions from second sampling are shown in blue. al. [18], an -dominance might support the reduction of such The plot shows a large gap between the Pareto-frontier of gaps by a better diversity to be maintained in a population. IV. C ONCLUSIONS R EFERENCES Even if existing techniques for fault-tolerant system design [1] A. Avizienis, “Toward systematic design of fault-tolerant systems,” Computer, vol. 30, no. 4, pp. 51–58, 1997. assist the developer in identifying necessary redundancies, [2] L. Grunske, P. Lindsay, E. Bondarev, Y. Papadopoulos, and D. Parker, additional best-fitting configurations have to be figured-out. “An outline of an architecture-based method for optimizing dependabil- Our approach guides the developer through the subset of the ity attributes of software-intensive systems,” in Architecting Dependable Systems IV, ser. LNCS. Springer, 2007, vol. 4615, pp. 188–209. remaining design options after a hardware resource fault is [3] L. Märtin, A. Koziolek, and R. H. Reussner, “Quality-oriented Decision injected. Under the consideration of such a fault scenario, Support for maintaining Architectures of fault-tolerant Space Systems,” Pareto-optimal solutions are sampled and decision support to in 9th Europ. Conf. on Software Architecture Workshops, September 2015, pp. 49:1–49:5. identify nearest alternates to a faulty configuration is provided [4] A. Koziolek and R. Reussner, “Towards a generic quality optimisation in five process stages. In addition, large gaps in a system’s framework for component-based system models,” in 14th Int. ACM quality can be shown with our tooling to find flaws in Sigsoft Symp. on Component based Software Engineering, 2011, pp. 103–108. redundant system design. [5] A. Aleti, B. Buhnova, L. Grunske, A. Koziolek, and I. Meedeniya, “Software Architecture Optimization Methods: A Systematic Literature Future Work Review,” IEEE Trans. on Software Engineering, vol. 39, no. 5, pp. 658– 683, 2013. We provide lightweight tooling for mass-generating solution [6] K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist set and gap exploration of Pareto-frontiers. Based on our multiobjective genetic algorithm: NSGA-II,” IEEE Trans. on Evolution- previous experience, we plan to integrate our exploration ary Computation, vol. 6, no. 2, pp. 182–197, 2002. [7] K. Echtle and I. Eusgeld, A Genetic Algorithm for Fault-Tolerant System concept within the Palladio toolset and its add-ons. For full- Design. Springer, 2003, pp. 197–213. fledged architecture modeling [19] we will apply PALLADIO [8] S. Frey, F. Fittkau, and W. Hasselbring, “Search-based genetic opti- B ENCH3 and P ERO PTERYX4 to define variability in models. mization for deployment and reconfiguration of software in the cloud,” in 35th Intern. Conf. on Software Engineering, 2013, pp. 512–521. Also the sampling with genetic algorithms is performed there. [9] G. Jung, K. Joshi, M. Hiltunen, R. Schlichting, and C. Pu, “Generating Ratings of quality attributes are gathered by the simulation Adaptation Policies for Multi-tier Applications in Consolidated Server engine S IMU C OM. We will make use of the results for distance Environments,” in 5th Int. Conf. on Autonomic Computing, 2008, pp. 23–32. comparison and gap exploration proposed in this paper. [10] J. M. Barnes, A. Pandey, and D. Garlan, “Automated planning for soft- In particular, the findings of this paper will contribute to the ware architecture evolution,” in 28th Int. Conf. on Automated Software improvement of our decision structure Architecture Relation Eng., 2013, pp. 213–223. [11] J. R. Schott, “Fault tolerant design using single and multicriteria genetic Graph [3] and on-going work in area of hierarchical cluster algorithm optimization.” DTIC, Tech. Rep., 1995. analysis. The final selection of which configurations from [12] M. Gong, L. Jiao, H. Du, and L. Bo, “Multiobjective immune algorithm the Pareto-frontiers are added to the graph still relies on with nondominated neighbor-based selection,” IEEE Trans. on Evolu- tionary Computation, vol. 16, no. 2, pp. 225–255, 2008. trade-off settings preferred by the developer. Such trade-off [13] B. J. Garvin, M. B. Cohen, and M. B. Dwyer, “Evaluating improvements analysis is supported by our tool AR EVA 25 . Based on the to a meta-heuristic search for constrained interaction testing,” Empirical work of Florentz et al. [20] contrary quality properties are Software Engineering, vol. 16, no. 1, pp. 61–102, 2010. [14] F. Ensan, E. Bagheri, and D. Gašević, “Evolutionary search-based test normalized by conversion function and ordered hierarchically generation for software product line feature models,” in 24th Intern. with weightings. This analysis needs to be integrated with the Conf. on Advanced Information Systems Engineering. Springer, 2012, distance measurements from our tool prototype presented here. pp. 613–628. [15] C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. L. Traon, We plan to comprehensively evaluate the integrated tool- “PLEDGE: a product line editor and test generation tool,” in 17th Intern. supported approach at whole with a case study. For that, we Software Product Line Conf. co-located Workshops. ACM, 2013, pp. will build upon our previous findings in the domain of space 126–129. [16] T. Thüm, C. Kästner, F. Benduhn, J. Meinicke, G. Saake, and T. Leich, systems [21]. As a real-world case study, we have access to “FeatureIDE: An extensible framework for feature-oriented software a system design of a micro-satellite provided by one of our development,” Sci. Comput. Program., vol. 79, pp. 70–85, 2014. industrial partners. The system has a high degree of inherent [17] J. J. Durillo and A. J. Nebro, “jMetal: A Java Framework for Multi- objective Optimization,” Adv. Eng. Softw., vol. 42, no. 10, pp. 760–771, availability implemented by autonomy mechanisms and a large Oct. 2011. number of redundant hardware resources. We will extend our [18] K. Deb, M. Mohan, and S. Mishra, “Towards a quick computation of idea from previous work [22] in enhancing the system by well-spread pareto-optimal solutions,” in 2nd Intern. Conf. on Evolu- tionary Multi-criterion Optimization. Springer, 2003, pp. 222–236. replication-redundant capabilities as addressed here. [19] R. Reussner, S. Becker, E. Burger, J. Happe, M. Hauck, A. Koziolek, H. Koziolek, K. Krogmann, and M. Kuperberg, “The Palladio Compo- ACKNOWLEDGMENT nent Model,” KIT Department of Informatics, Tech. Rep., 2011. [20] B. Florentz and M. Huhn, “Embedded systems architecture: Evaluation This work was partially supported by the DFG (German and analysis,” in Quality of Software Architectures, ser. LNCS. Springer, Research Foundation) under the Priority Programme SPP1593: 2006, vol. 4214, pp. 145–162. Design For Future Managed Software Evolution. The authors [21] L. Märtin, M. Schatalov, M. Hagner, O. Maibaum, and U. Goltz, “A Methodology for Model-based Development and Automated Verification would like to give special thanks to Axel Busch for fruitful of Software for Aerospace Systems,” in 34th IEEE Intern. Aerospace discussions on tool integration within P ERO PTERYX. Conf., 2013. [22] L. Märtin and A. Nicolai, “Towards Self-Reconfiguration of Space 3 http://www.palladio-simulator.com Systems on Architectural Level based on Qualitative Ratings,” in 35th 4 https://sdqweb.ipd.kit.edu/wiki/PerOpteryx IEEE Intern. Aerospace Conf., Big Sky, USA, March 2014. 5 https://github.com/lmaertin/areva2