Multi-Perspective Modeling and Performance Analysis of Software Product Lines Matthias Kowal TU Braunschweig, Germany Email: m.kowal@tu-braunschweig.de Abstract—Software system are typically available in a rich the complexity as low as possible to achieve a high level of set of variants nowadays to deal with differing customer or usability. environmental requirements and application contexts. Managing We propose a twofold modeling approach managing prob- such a software product line, gets even more difficult considering multiple involved engineering disciplines and long lifetimes, as lem and solution space as well as their mapping. Software typical for industrial systems of the automation domain. The product lines are commonly modeled using feature models thesis tackles this system diversity by modeling interdisciplinary in the problem space [2]. There is already a widespread system variability in both problem and solution space. Based on number of extensions to feature models available in literature, these models, we analyze the impact on performance properties e.g., dealing with different versions over time [3], view-based during design time giving early feedback about the system behavior. The solution space is based on a model-driven approach abstractions [4] or even combining multiple product lines [5]. with UML models, using notions of delta modeling to man- However, they all lack appropriate adaptations to deal with in- age system variability and evolution enriched with information terdisciplinary aspects, since they were originally designed for needed to automatically derive and analyze a performance model. software engineering and not mechanics and electronics [6]. Motivated by its widespread use in software engineering, we Nonetheless, we plan to use them as a foundation for our consider feature models for the problem space that are ultimately connected to the UML models. Our evaluation is based on a real- approach and extend them where it is necessary. world automation system representing a long-living and variant- For the solution space, we propose a design-level model- rich software system. ing approach using three levels of abstraction containing a workflow describing the path of a job through the system, an I. I NTRODUCTION architecture and the actual behavior. In addition, we provide a mapping between individual elements of each perspective. Long-living software systems, as in the automation domain, Variability and evolution is managed using delta modeling [7]. exist in many different variants at one point in time in order In delta modeling, we have a core model, which often is to satisfy varying user requirements (anticipated variability in the smallest possible variant of the system. The core can space). Additionally, they evolve over time in order to adapt be altered using the three basic operations of add, remove, to changing environmental conditions, such as functional, and modify. Deltas encapsulate these change operations and technical or legal requirements (unanticipated variability in therefore contain the required information to derive a specific time) [1]. Furthermore, several disciplines are involved during variant. However, the application of one or more deltas to their development with mechanical, electric and software engi- generate another variant of the system may lead to an ill- neering. As a result, product-line engineering for such systems formed system model. Hence, we have to validate that the is extremely complex. A product line is typically divided resulting variant is still consistent. In each space, we pursue into problem and solution space. The problem space captures the established principle of separation of concerns to minimize domain knowledge in an abstract way, e.g., using feature or the complexity. The perspectives are realized with a subset of decision models. The solution space consists of concrete im- UML in terms of diagrams with activity, architecture and state plementation artifacts such as source code fragments or UML- charts. diagrams. Hence, we need an interdisciplinary variability Predicting the system behavior of software product lines modeling concept on the problem space level spanning across during design time, in terms of performance, is a crucial yet the three mentioned disciplines. For the solution space, we complex task. An automation system representing a product need models with an appropriate level of abstraction to support line typically has several non-functional requirements. For the individual developers following the separation of concerns instance, it must produce or transport a specific number of principle, so that, e.g., software architects and engineers can items in a given amount of time. Feedback concerning these focus on their specific task and unnecessary information stays requirements would be beneficial at an early stage to prevent hidden. A fully integrated model-driven approach also calls for disadvantageous design decisions. However, the analysis of the connection of both spaces to support developers as much as each variant in isolation, which is called product-based per- possible and enable the generation of the configured variants in formance analysis [8], is inefficient due to the possible large the problem space with the respective solution space models. number of variants in product lines. We need appropriate In each part, we have to identify suitable methods to keep methods to overcome this aspect. Family-based approaches allow a more efficient analysis of a complete product line [9]. point can be freely selected across all variants providing more We propose such a family-based analysis based on our solution flexibility. space models to provide early feedback giving developers the Consistency checking for UML models with variability is chance to identify system bottlenecks during design time. As not yet largely considered. A brief survey of consistency performance properties, we consider throughput, utilization or checking techniques can be found in [16]. Some of the existing average queue length of the system. techniques have to be further investigated in terms of appli- The essence of this thesis is to provide a scalable modeling cability to product lines [17]. Other approaches already detect approach covering variability and evolution in both problem inconsistencies with variability [18], yet a mapping to our and solution space across several disciplines while providing general delta modeling concept has to be done. Additionally, efficient performance analyses techniques for product lines. we aim at fixing the detected inconsistencies. A few of the Research Questions. In detail, the thesis is supposed to available techniques in literature already include methods for answer the following research questions: fixing UML models in single software systems [19]. However, RQ1: How can we reuse and adapt existing feature model approaches for repairing product lines are still in the early extensions to deal with an interdisciplinary product line? stages. A first research result in this direction can be found RQ2: Can we sufficiently capture variability and evolution in [20]. We plan to build upon this foundational work. of a real-world automation system in our models while main- The proposed family-based performance analysis is related taining low complexity and high consistency? to a model checking approach of software product lines in RQ3: To what extent is our family-based performance which annotated UML sequence diagrams are used [21]. This analysis superior compared to existing techniques in terms of ultimately leads to a symbolic expression similar to ours. How- runtime? ever, we consider performance properties such as throughput and do not focus energy related parameters. Another approach II. R ELATED W ORK translates UML-annotated software product line designs to Most feature modeling approaches consider one large model layered queueing networks and analyzes them in a product- specifying the variability of a complete product line [10]. based way [22]. Although this network-type is more expressive This is not feasible considering multiple disciplines, since than ours, an efficient family-based analysis is not available. each domain solely needs to focus on their specific features and everything else would be unnecessary information. Addi- III. I NTEGRATED MODELING AND EFFICIENT tionally, one large model is hard to visualize, maintain, and PERFORMANCE ANALYSIS analyze taking evolution into account. Different views onto This section is divided into three parts and each part the feature model or the combination of two or more features represents a different contribution of the thesis. First, we models, one for each domain, as in multi software product explain the interdisciplinary modeling of a product line in the lines possible, may solve some of the existing problems [4], problem space. Second, we extend the approach to solution [5]. There are numerous more feature model extensions and space models. Third, a family-based performance analysis similar approaches available. As for now, it is an open question technique is introduced. The overall approach is exemplarily which methods can be applied to an interdisciplinary product shown in Fig. 1. line. Considering the solution space, we face the same prob- The first contribution: Interdisciplinary Modeling of Variabil- lems with multiple disciplines such as software architects ity in the Problem Space or engineers. Hence, it is a common approach to follow First contribution considers the interdisciplinary modeling the principle of separation of concerns during development. approach (cf. top-left corner of Fig. 1). We devise problem For instance, this is done in the SPES project in which an space variability modeling concepts for long-living software embedded system is developed with model-driven engineering product lines involving multiple disciplines, as typical in the and multiple viewpoints [11] or as in [12]. Our approach is automation domain. Classical feature models [23], [2] lack quite similar, but also includes a variability modeling concept expressiveness to deal with interdisciplinary systems. Hence, with delta modeling. Delta modeling is part of the trans- we develop a problem space variability modeling concept formational variability modeling concepts as is the Common capable to capture interdisciplinary variability dealing with Variability Language [13]. Comparing both approaches yields the following limitations of existing approaches (as recently no significant benefit for either one, e.g. both are language identified in [6]): (i) complex dependencies and relations independent. We argue that both emerged in parallel and between discipline-specific entities, (ii) several levels of granu- we decided to use delta modeling. Other concepts are of larity (from actuators to complete software components), and annotative or compositional nature. The former tend to have (iii) inclusion of multiple views for the individual domains unmanageable large models [14], while the latter can only add (mechanics, electronics and software). functionality making removals impossible [15]. The removal We apply existing feature model extensions to the automa- of functionality can be compensated by selecting an appropri- tion system product line. A possible extension is the usage ate variant as starting point which is incrementally extended of different views onto the feature model based on individual through addition afterwards. In delta modeling this starting domains. Furthermore, the principle of multi software product Software Design Automatic generation Performance Analysis Problem Space Solution Space Backannotation Root Perf. Workflow Consistency Index QoS F1 F2 F3 Space Architecture ODE F4 F5 ¬F1 ᴧ F4 F3 → F5 Behavior Time Time Fig. 1. The envisioned software contribution lines can be applied to solve the granularity limitation, which a workpiece throughout the system (e.g., a bottle on an basically means having separate feature models for each assembly line), while the architecture presumes the system domain. In addition, extended cross-tree constraints are avail- structure and the behavior provides the actual implementation. able in multi software product lines to capture the complex In addition, elements can be mapped between the different dependencies. perspectives. We equipped each perspective with the delta Additionally, we must as well address evolution extending modeling approach to capture variability and evolution by the the previously developed variability modeling concept with same means. Since deltas can be applied to each perspective feature versioning which is required for the case study. We in isolation, we provide a meta structure which we call base our approach on Hyper Feature Models (HFMs) [3] which interdelta containing information about the required deltas to is devised in prior work. HFMs capture feature versions and derive a valid system variant. In order to map features to the allow reasoning about dependencies of features in different solution space, we define higher-order deltas operating on the versions. As HFMs are also originally not designed to handle previously mentioned meta structure for connecting problem interdisciplinary models, we adapt HFMs suitably to integrate and solution space models. them with the previously mentioned interdisciplinary concepts. A system in development is under constant change where A closer look to traceability approaches across several models each modeling perspective may evolve independently, but con- may also be beneficial here [24]. sistency between different perspectives, e.g., activities mapped We can safely assume that during system development with to suitable components in the architecture, must be maintained. the described interconnected modeling approach, errors are a For instance, an inconsistency is an architectural component steady companion for the developers. Hence, we have to devise without a connection to any state chart. Hence, we need to analyses techniques for reasoning about consistency to provide develop a concept for incremental consistency checking to a correct and reliable design [6]. We analyze the feature model support the developer when evolving a system within the for anomalies such as dead features meaning features that can multi-perspective modeling approach. Again incrementality never be part a any variant due to constraints. In addition, is key here, as it minimizes the verification time as only we have to ensure that the derivable set of product variants is changed parts and parts possibly affected by the changes need valid, an essential prerequisite for the configuration of software to be considered. As a conceptual idea for the incremental systems. As we also consider the evolution of the feature consistency checking approach, we use previous work on model itself, all analyses have to work incrementally in order incrementally type checking of delta-oriented product lines to allow for efficient re-analyses after evolutionary changes. in Java and adapt it to our specific needs [26]. Considering For evaluation purposes, we strive for a close cooperation tooling the Epsilon framework, which includes the Epsilon with automation engineers of the TU Munich, since they are Validation Language, may be a promising candidate for an the domain experts for such systems and can give qualitative appropriate solution. feedback about the feasibility of our concepts. We want to provide the developer not only with the knowl- Preliminary Results: This contribution is the main task edge of existing errors (consistency checking), but devise a for the remaining time of the thesis. The developed results concept for repair operations as well. Similar to IDE func- should answer RQ1 using an explorative study. A first result tionality for repair operations in programming languages, e.g. about challenges here is given in [25]. adding the import statement for a used function, we want to provide repair operations for our models. For instance, a The second contribution: Multi-Perspective Modeling of Vari- component in the architecture has to be connected to a state ability and Performance in the Solution Space chart in the behavior perspective. If there is no such behavior, Second contribution is all about solution space models a repair operation is to create a default state chart, if the (cf. top-left corner of Fig. 1). The solution space is divided developer complies with the suggestion. The possible cases into three perspectives following separation of concerns and for inconsistencies have to be explored and fixing operations therefore minimizing the complexity for individual develop- for these cases have to be provided. ers: workflow, architecture, and behavior each represented Overall we have models for both problem and solution space by a different UML model with activity, block-based, and with feature and UML models. Each concept must ensure a state chart diagrams. The workflow describes the path of certain level of consistency in isolation as well as in between both sides resulting in three points that require consistency it symbolically in our family-based approach. As a result, concepts. we get a large symbolic expression in which we just have Preliminary Results: As a first result, we already have to plug-in the values for a specific variant to calculate the developed the design-level solution space modeling approach performance properties. The symbolic solving has to be done based on UML diagrams [27], [28]. In addition, we can once for the complete product line and not for each variant manage variability and evolution with delta modeling [27], over and over again. The gathered information about possible [28], [29]. Finally, we enriched the workflow perspective with system bottlenecks can influence modeling decisions at design non-functional performance properties such as arrival and time again and we achieve a round-trip engineering concept service rates at the nodes and probabilities at the transitions. as depicted in Fig. 1. However, the underlying Jackson-type This is a prerequisite for the third contribution and allows queueing networks are quite limiting in terms of service time us to do a performance analysis of the system and predict distribution and supported model elements which is why we measures such as utilization, throughput and average queue extended this work to Coxian-distributed networks [33]. lengths. As before, this also includes the full delta modeling Preliminary Results: Main contribution here is the pro- support to vary performance properties between variants [27], posed family-based performance analysis in [30]. Numerical [30]. The implementation is designed as an Eclipse plugin experiments demonstrated the superiority of the approach using XText and EMF for the individual domain-specific against a product-based solution, especially in the case of languages (DSLs) for the modeling perspectives. In total, large-scale networks with high degree of variability. The we have three DSLs for the perspectives and an additional experiment shows speed-ups of up to 2000%. The analysis one for the combining meta-structure. Although, the textual is also evaluated with the automation case study [27]. An editor has comfort functions such as auto-completion, syntax extension to Coxian-distributed networks is also available [33]. highlighting and basic syntactic consistency checks, we started However, the numerical results are not as good as before due to to develop concepts for graphical delta modeling editors using more complex formulae. A product-based performance analy- the Graphical Editing Framework (GEF) for each perspective. sis works for the workflow in our Eclipse plugin. The family- We were able to answer RQ2 as explored in three different based analysis is realized in Matlab. Each part of the tooling publications [27], [28], [29]. Method of evaluation was the real was evaluated and applied to the automation case study. This world automation system located at the AIS chair of the TU includes an actual code generation for an automation control Munich. However, the consistency concept including repair software [27]. As a result, RQ3 is successfully answered. operations is still an open point in this contribution. To conclude this section, we can safely assume that the solution space concept is mostly finished except for advanced The third contribution: Efficient Performance Analysis. consistency checks and repair operations. It is similar for the Third part of the contribution is a scalable performance performance analysis part. Main focus for the remaining thesis analysis of the complete product line. The analysis is based time lies on interdisciplinary concepts and the connection of on the previously described solution space models, especially problem and solution space models. the workflow. The workflow is interpreted as a continuous- time Markov chain that underlies a Jackson-type queueing IV. E VALUATION AND S TATUS network [31]. Fig. 1 implies an automatic generation of such The following section reflects present and future achieve- performance models. A product-based analysis is straightfor- ments in a condensed form. In addition, we provide more ward possible by solving the following equation for γ details about the evaluation methods. (I − P T )γ = λ, (1) A. Expected Contributions We expect the following contributions: where I is the identity matrix, P is the adjacency matrix and • Application of feature modeling to product lines contain- λ are the arrival rates. We can easily compute the throughput, ing hardware and software. utilization, average queue length or response time of the • Management of variability, evolution and consistency in system based on the solution [31]. However, for systems the solution space. with a large variant space such as product lines, it is not • Performance analysis techniques for product lines. efficient to compute the solution for each variant in isolation which is necessary since we cannot reuse any numerical B. Case Study results [32]. We propose a family-based analysis to overcome The proposed contributions are evaluated using a real-world this disadvantage and analyze the full product line at once. automation system called the Pick and Place Unit (PPU) Again, we benefit from the delta modeling principle for our which is provided by the AIS chair of the TU Munich and efficient analysis. We create a super-variant, also called 150%- exists in 15 different variants [34]. For the first contribution, model, based on the core and all deltas so that the full we conduct an explorative study based on the PPU. Second variability is concentrated in exactly one model. Each value contribution includes modeling and analyzing the complete that is changed by a delta is represented by a symbol and PPU product line with our solution space models. Since not a concrete value as in the product-based analysis. Again, the PPU is still a lab demonstrator, we have to conduct we can create the system of equations as in (1), but solve larger experiments for the third contribution concerning the performance analysis part to investigate the behavior in large- [5] T. Berger, R. Rublack, D. Nair, J. M. Atlee, M. Becker, K. Czarnecki, scale product lines. Finally, we want to empirically compare and A. Wasowski, “A Survey of Variability Modeling in Industrial Practice,” in VaMoS. New York, NY, USA: ACM, 2013, pp. 7:1–7:8. our textual DSL modeling concept against the graphical one to [6] S. Feldmann, C. Legat, and B. Vogel-Heuser, “Engineering support in validate the superiority of visual delta modeling e.g. in terms the machine and plant manufacturing domain through interdisciplinary of speed and understandability. A lab experiment with students product lines: An applicability analysis,” in INCOM’15, 2015. [7] I. Schaefer, “Variability modelling for model-driven development of is planned in the future. software product lines,” in VaMoS, 2010, pp. 85–92. C. Current Status [8] W. J. Stewart, Probability, Markov Chains, Queues, and Simulation. Princeton University Press, 2009. Currently six publications have emerged during this thesis [9] A. von Rhein, S. Apel, C. Kästner, T. Thüm, and I. Schaefer, “The PLA tackling solution space models and the performance analysis. model: on the combination of product-line analyses,” in VaMoS, 2013. Thus, answering RQ3 completely [30], [33], RQ2 in many [10] P.-Y. Schobbens, P. Heymans, and J.-C. Trigaux, “Feature diagrams: A survey and a formal semantics,” in RE’06. parts [27], [28], [29] except the advanced consistency and re- [11] K. Pohl, H. Hönninger, R. Achatz, and M. Broy, Model-Based Engi- pair operation concepts and RQ1 is still at the beginning [25]. neering of Embedded Systems. Springer, 2012. The time line for completion is as follows: [12] J. Kienzle, W. Al Abed, and J. Klein, “Aspect-oriented multi-view modeling,” in AOSD’09. 12/2015 Interdisciplinary Modeling Concept 9/2016 7/2017 [13] O. Haugen, B. Møller-Pedersen, J. Oldevik, G. K. Olsen, and A. Svend- for the Problem Space Start writing Thesis Written Thesis finished sen, “Adding standardized variability to domain specific languages,” in SPLC, 2008, pp. 139–148. [14] C. Kästner and S. Apel, “Integrating compositional and annotative approaches for product line engineering,” in McGPLE, 2008. 7/1/2015 4/2016 1/2017 7/31/2017 [15] C. Klein, C. Prehofer, and B. Rumpe, “Feature specification and re- Consistency Concept Connection of Problem finement with state transition diagrams,” in 4th IEEE WS on Feature and Solution Space Interactions in Telecommunications Networks. IOS Press, 1997. Fig. 2. Timeline for the remaining duration of the thesis [16] M. Usman, A. Nadeem, T. hoon Kim, and E. suk Cho, “A survey of consistency checking techniques for uml models,” in ASEA 2008. V. C ONCLUSION [17] A. Egyed, “Instant consistency checking for the uml,” in ICSE ’06. [18] R. E. Lopez-Herrejon and A. Egyed, “Detecting inconsistencies in multi- We have presented a twofold modeling approach for both view models with variability,” in ECMFA’10. problem and solution space in product lines. The abstract [19] A. Egyed, “Fixing inconsistencies in uml design models,” in ICSE’07. [20] R. E. Lopez-Herrejon and A. Egyed, “Towards fixing inconsistencies in domain knowledge is captured with common feature models. models with variability,” in VaMoS’12. However, the challenge of multiple involved disciplines de- [21] C. Ghezzi and A. M. Sharifloo, “Verifying non-functional properties of mands several feature model extensions for which we explore software product lines: Towards an efficient approach using parametric model checking,” in SPLC, 2011, pp. 170–174. the applicability to an automation system or provide neces- [22] R. Tawhid and D. C. Petriu, “Automatic derivation of a product perfor- sary adaptations. The concrete system is modeled with three mance model from a software product line model,” in SPLC, 2011. different perspectives. Each perspective allows variability and [23] K. Czarnecki and E. Ulrich, Generative Programming: Methods, Tools, and Applications: Methods, Techniques and Applications. Addison- evolution through delta modeling. Both spaces are equipped WesleyLongman, 2005. with analyses techniques and a consistency concept. The [24] I. Galvao and A. Goknil, “Survey of traceability approaches in model- feature modeling includes an analysis to detect anomalies and driven engineering,” in EDOC, 2007. [25] B. Vogel-Heuser, J. Folmer, M. Kowal, I. Schaefer, S. Lity, A. Fay, validate the set of derivable variants. The UML models are W. Lamersdorf, T. Kehrer, M. Tichy, and B. Beckert, “Selected chal- enriched with performance annotations to deduce a separate lenges of software evolution for automated production systems,” in performance model. This model can be analyzed with either Industrial Informatics, 2015. [26] L. Bettini, F. Damiani, and I. Schaefer, “Compositional type checking a product- or family-based performance analysis to provide of delta-oriented software product lines,” Acta Informatica, 2013. feedback about, e.g., bottlenecks. The complete approach is [27] M. Kowal, C. Prehofer, I. Schaefer, and M. Tribastone, “Model-based evaluated using the PPU as a long-living product line. development and performance analysis for evolving manufacturing sys- tems,” at - Automatisierungstechnik, vol. 62, pp. 794–802, 2014. ACKNOWLEDGMENT [28] M. Kowal, C. Legat, D. Lorefice, C. Prehofer, I. Schaefer, and B. Vogel- The PhD thesis is supervised by Ina Schaefer and partially Heuser, “Delta Modeling for Variant-rich and Evolving Manufacturing Systems,” in MoSEMInA, 2014. supported by the DFG (German Research Foundation) under [29] B. Vogel-Heuser, J. Mund, M. Kowal, C. Legat, J. Folmer, S. Teufl, the Priority Programme SPP1593: Design For Future — and I. Schaefer, “Towards interdisciplinary variability modeling for Managed Software Evolution. The author would like to thank automated production systems,” in Industrial Informatics, 2015. [30] M. Kowal, I. Schaefer, and M. Tribastone, “Family-Based Performance Thomas Thüm and Ina Schaefer for valuable feedback. Analysis of Variant-Rich Software Systems,” in FASE, 2014. [31] G. Bolch, S. Greiner, H. de Meer, and K. Trivedi, Queueing networks R EFERENCES and Markov chains: modeling and performance evaluation with com- [1] S. Braun, C. Bartelt, M. Obermeier, A. Rausch, and B. Vogel-Heuser, puter science applications. Wiley, 2005. “Requirements on evolution management of product lines in automation [32] T. Thüm, S. Apel, C. Kästner, I. Schaefer, and G. Saake, “A Classifi- engineering,” in Int. Conf. Math. Modelling, Vienna, Austria, 2012. cation and Survey of Analysis Strategies for Software Product Lines,” [2] K. C. Kang, S. G. Cohen, J. A. Hess, W. E. Novak, and S. P. A., “ ACM Computing Surveys, vol. 47, no. 1, pp. 6:1–6:45, Jun. 2014. Feature-Oriented Domain Analysis (FODA) Feasibility Study,” Carnegie [33] M. Kowal, M. Tschaikowski, M. Tribastone, and I. Schaefer, “Scaling Mellon University, Tech. Rep., 1990. Size and Parameter Spaces in Variability-aware Software Performance [3] C. Seidl, I. Schaefer, and U. Aßmann, “Integrated management of Models,” in ASE, 2015. variability in space and time in software families,” in SPLC ’14. [34] C. Legat, J. Folmer, and B. Vogel-Heuser, “Evolution in industrial plant [4] A. Hubaux, T. T. Tun, and P. Heymans, “Separation of Concerns in automation: A case study,” in IECON, 2013. Feature Diagram Languages: A Systematic Survey,” ACM Computing Surveys, vol. 45, no. 4, pp. 51:1–51:23, Aug. 2013.