‫ﯾﺎھﻮ‬ Behavior-Driven Development in Product Configuration Systems Sara Shafiee1 and Lars Hvam and Anders Haug and Yves Wautelet Abstract.1 Product Configuration Systems (PCS) are increasingly [13], degree of customization [14], or other factors such as used by companies to automate the performance of the sales and knowledge management and scoping process [8]. The challenges to engineering processes. Since the benefits from such projects have manage technical and commercial knowledge when implementing huge variations, it is crucial to make the right decisions when and maintaining a PCS may highly influence PCS costs and scoping and developing PCSs. The development of PCS is development time, as well as its lasting effectiveness. influenced by both business interests and technical insights. Developers of PCS face various challenges while working in team, The complexity in the development of PCSs comes from that it including different stakeholders such as business owners, involves highly complex technical knowledge from domain developers, project managers, and product experts. The more experts, [9], [10]. Hence, Scrum and agile methods attracted diverse the team is, the more significant are the challenges. This attention in PCS development. The main reasons that can be paper suggests that Behavior-driven Development (BDD) may mentioned as their ease of use, the constant communication with provide configuration teams with a specific structure to express the stakeholders and the team and fast development time [10], [15], scenarios (and thus constraints) on PCS in natural language. BDD [16]. In spite of the potential benefits of Scrum, many may yield benefits such as a better expression of PCS constraints, organizations are reluctant to throw their conventional methods more efficient communication of requirements and incorporation of away and jump into agile methods. This to some extent may be the expressed rules in a software transformation process. In other attributed some of the issues experienced with the use of Scrum, words, applying BDD may eliminate unnecessary tasks when gathering knowledge, developing, and testing PCS projects. In this including significant reduction of documentation, insufficient paper, we present a novel approach from an ongoing project on testing for mission/safety-critical projects, inadequate support for how to relate BDD to the development process of PCS while using highly stable projects, only successful with talented individuals Scrum-based methods. who favor large degrees of freedom, and inappropriate for large- scale projects [17]. As the discussion above indicates, it is very challenging to 1 INTRODUCTION specify a PCS at analysis stage. In Scrum-based development this Product Configuration Systems (PCS) are expert systems that can is traditionally done using user stories [18]. The latter nevertheless support and facilitate the sales and engineering processes [1] by do present some drawbacks because their narrative alone is not incorporating information about product features, product enough to express the constraints related to PCS. This is precisely structure, production processes, costs and prices [2]. Thereby, where BDD could offers complementary representation abilities configurators can support decision-making processes in the sales destined to address user stories limitations. Because of the use of and engineering phases of a product [3]. Configurators enable BDD next to user stories narrative, the PCS can be expressed companies to develop product alternatives to facilitate sales and without any other requirements representation artifact (traditionally production processes [4]. However, the companies that have the Product Variant Master is used for this [3]). Other positive managed to implement and utilize configurators also face various aspects of the use of BDD to further describe user stories are their challenges [1], [5]. expression in (structured) natural language making them easy to Even though advantages of PCS are evident, there are still some understand by non-technical stakeholders. They thus present difficulties associated with high costs [2], [6] and considerable advantages for communication. Finally their dynamic and process chances of failure [2], [7] in their implementation projects. This is oriented nature can be used in a systematic transformation process because companies must overcome various challenges to for the configurator design [19]. All of these elements bring implement and utilize configurators. For example, the development potential positive contributions with the use of BDD and needs to of a PCS often requires highly complex technical or commercial be more formally studied. This paper presents a first attempt knowledge, which domain experts often have a hard time towards such a research. communicating to configuration experts [8]. Furthermore, the knowledge base that contains this knowledge has to be adapted 2 LITERATURE STUDY continuously because of the changing components and configuration constraints [9], [10]. The difficulty of acquiring and modeling the required technical or commercial knowledge depends 2.1 Agile and Scrum on whether it is available in a clear and formal form [11] which in Scrum is an agile software development methodology. The turns may be contingent to company size [12], product complexity Agile Manifesto outlines the values and principles that should be supported by the various agile processes applied in software 1 Mechanical engineering department, Technical University of Denmark, development. Agile principles emphasize customer satisfaction, Denmark, email: sashaf@dtu.dk change and collaboration between domain experts and developers ‫ﯾﺎھﻮ‬ [20]. Rubin [21] highlighted that with an agile approach, the team In BDD, as compared to TDD, the task is to write a specification of starts by creating a product backlog, which is a prioritized list of system behavior that is precise enough for it to be executed as code the features and other capabilities that need to be developed. [29]. More specifically, the whole point of BDD is to ensure that Guided by the product backlog, team members address the most the real business objectives of stakeholders are met by the software important or highest priority items first; priority is based on various we deliver. If stakeholders are not involved, if discussions are not factors, but delivered business value is most often the first priority. taking place, BDD is not going to work. BDD yields benefits Scrum is an agile approach for developing innovative products and across many important areas such as: (1) Building the right thing, services [21]. (2) Reducing the risks, (3) Evolving the design [29]. Scrum facilitates cross-team coordination and collaboration The first Phase in BDD is to understand the business goals and [22]. Vlietland et al. (2016) determined that Scrum improves defining features [25]. Vision statement templates make it possible coordination through additional events, such as interteam sprint to have a well-defined set of business goals. For a good product planning meetings, interteam daily Scrums, interteam product vision statement, Moore at al. [30] propose the following contents refinements and interteam sprint reviews. A Scrum development of a template: life cycle normally consists of short iterations of two to four weeks, · For (Who will benefit from this product?) an approach that enables swift feedback from software users and · Who (What do they need?) related stakeholders regarding the developed solution [19], [23]. · The (What sort of thing are you proposing?) · That (What makes it so cool?) · Unlike (What are you competing against?) 2.2 Behavior-driven Development · Our product (Why customer prefer your solution?) In the movement of agile development, Test-Driven Development Secondly, the features have to be illustrated in natural language (TDD) has been around for a long time and can be traced back to to execute the specifications. Consequently, the scenario has the eXtreme Programming practices developed in the late 1990s [24]. structure [25]: In particular, TDD employs so-called acceptance tests as the · Given [context, initial conditions] starting point for the development process to address some of the · When [event occurs] challenges related to Scrum. Following [25], these TDD relies on · Then [outcome] two simple principles: There are several studies investigating how to automate all these · Don’t write any code until you’ve written a failing test that scenarios such as Cucumber or Jbehave [25]. demonstrates why you need this code. · Refactor regularly to avoid duplication and keep the code 2.3 Requirement artifacts in traditional PCS quality high. Behavior Driven Development (BDD) has been proposed as a projects: the Product Variant Master result of the problems that arose with TDD when applying agile Generic product structures can be illustrated using the so-called software practices [26]. It should be noticed that the language used “product variant master” (PVM) notation, which in many cases has for describing the tests, i.e. class names and operation names, plays functioned as a common language between different product, an important role both for writing test cases and for finding bugs in process and IT experts [3]. Different definitions of the PVM case of a failed test. Inspired by [27], for this purpose BDD uses notation have been proposed, and among them the definition of the natural language as a ubiquitous communication mean to describe PVM notation by Haug [11] is one of the most extensive and the acceptance tests by means of scenarios. formalized. A principal example of the PVM technique used to The cornerstone of TDD is the idea of writing a unit test before model a toy car solution space is shown in Figure 2. On the left writing the corresponding code. However, BDD is much more than side of a PVM model, part-of-structure is shown, and on the right ensuring that every user story has a corresponding set of unit tests; side, kind-of structure is shown. Classes (typically components and BDD is also about writing specifications, as opposed to tests. In assemblies) are represented by circles and may include attributes BDD, as an agile software development technique, acceptance tests and constraints (or rules). are written in natural language in order to ensure a common understanding between all members of the project [28]. Consequently, as the first step, the sentences are mapped to actual source code [28]. The shift from TDD to BDD is subtle but significant. Instead of thinking in terms of verification of a unit of code, the focus is on specifying how that code should behave, i.e., what it should do [25]. In order to be sure that of building code that matters, there is a need for specifications that describe what the code should do and how to relate it directly to the business requirements. Figure 1 shows an example of the BDD flow as it is employed in a specific tool [26]. Figure 1. Behavior Driven Development flow Figure 2. Product variant master (ToyCar example) ‫ﯾﺎھﻮ‬ 3 PROBLEM STATEMENT team’s satisfaction with BDD. Table 1 summarizes some of the details of the case projects. Using Scrum for PCS development introduces both potential benefits and challenges (as for general IT projects in Table 1). One Table 1. Scrum practice and defined roles and activities in Cases 3 and 4 of the challenges is the level of knowledge complexity in PCS that Projects Case 1 Case 2 Case 3 has to be communicated clearly to all Scrum team members in Time frame (months) 11 8 9 terms of all attributes, constraints and acceptance criteria. An Number of iterations 10 9 9 additional challenge from Scrum concerns the lack of visualization during the project (modelling techniques [3]) for product structure. BDD supports · Product owner: the stakeholders’ manager Scrum with vision statements [30] to be able to demonstrate the · Project manager · Scrum master product details and even user interface step by step. Another Roles · Development team, including: 1 application related challenge is the testing of the PCS as assessing PCSs manager, 1 project manager, 2 configuration implies to assess system features with respect to the many possible engineers, 2 developers, 1 tester · End users data and system outputs that might occur when a user is interacting · Backlog grooming with them. Testing PCSs is an arduous testing activity due to the · Sprint planning wide range of user tasks and the different combinations of testing Activities · Sprint backlog data. BDD, as an add-on of user stories, supports the testers with · Daily Scrum · Sprint review the detailed defined acceptance criteria. · Product goals and product backlog item (story) In short, these concerns bring us three main guidelines for using Artefacts (main · Product backlog and stakeholders’ requirements Scrum methods in PCS projects: specifications) (list of user stories) · Testing (acceptance criteria in user stories) · Formalize user requirements in such a way to provide · Daily Scrum testability in an ever-changing environment; · Sprint planning Planning approach · Sprint review · Guarantee consistency between user requirements and their · Feedback meetings representation in multiple artefacts during development · Same as the project roles, plus: phase; and Specific roles of · Product owner: 1 · Lay on a validation approach that could be reused to ensure meeting participants · Scrum master: 1 · Tester: 1 such a consistency for the artefacts along the project. 5 OUTLOOK 4 RESEARCH METHOD It is yet an open question, how much BDD can contribute in the The aim of this paper is to test the application of BDD in PCS development phase of PCS projects. As PCS projects own their projects in real case projects and gather the data regarding the BDD specific challenges, there is a need for further studies to test BDD application. Firstly, we review the literature of BDD to gain deeper influence specifically for PCS projects. The results from case understanding of its definition and steps. Secondly, we would studies and interviews will serve to verify or falsify the mentioned apply the findings from literature regarding BDD to the Scrum hypothesis when the study is accomplished. The structure of a user management in PCS. The authors’ ultimate goal is to outline what story presented to the case projects is demonstrated bellow: the contribution of BDD to PCS can be and discuss its importance in promoting the collaboration and communication of knowledge within the organization. Based on the mentioned challenges in PCS projects, BDD can be an effective solution to improve the definitions of different features and testing the codes in PCS projects next to the user stories. Hence, we posit the following three propositions: Proposition 1: (expressiveness) BDD allows expressing all of the necessary constraints required for documenting a Configurator using Scenarios. Proposition 2: (performance) BDD represents an effective approach for communicating the product specifications when implementing PCS as a replacement for PVM. Proposition 3: (acceptance) BDD represents an effective and practical approach (requirements artifacts comparison) for unit testing of the implemented features in development and testing Figure 3. BDD structure of a user story presented for PCS projects in the phases of PCS projects. workshops (Car example from Figure 1) We use a qualitative exploratory design based on multiple data sources: requirements artifacts, workshops, interviews and Concerning the adoption next to user stories narrative as a participant observation [31], [32]. The study is ongoing in one case replacement for PVM and the vocabulary proposed in the ontology, company using Scrum method for developing PCS for more than 4 an advantage is that requirements and tests in user stories are kept years. Workshops are conducted to train the team for BDD to be in a natural and high-level language. Based on Figure 3, the team implemented as part of the Scrum. Finally, feedback meetings are should be able to demonstrate the prototype for user interface. The held as semi-structured interviews to collect knowledge about the ‫ﯾﺎھﻮ‬ evaluation criteria of using BDD approach in the team can be Empirical Study of the External Environmental Factors considered as: Influencing the Degree of Product Customization,” no. December 1. To support enterprise modeling within agile (user centric) 2016, 2017. methods [15] B. Selic, “Agile Documentation, Anyone?,” IEEE software, vol. 26, no. 6, 2009. 2. To estimate different scenarios, architecture and integration [16] S. Ambler, Agile modeling: effective practices for extreme 3. Correct evaluation of available attributes and constraints for the programming and the unified process. John Wiley & Sons, 2002. product [17] J. Cho, “A Hybrid Software Development Method for Large- 4. To support the analysis of requirements communication Scale Projects: Rational Unified Process with Scrum,” Issues in 5. To analyze the compatibility with business modelling Information Systems, vol. 10, no. 2, pp. 340–348, 2009. 6. To prototype the desired user interface [18] Y. Wautelet, S. Heng, M. Kolp, and I. Mirbel, “Unifying and extending user story models,” in International Conference on Advanced Information Systems Engineering, 2014, pp. 211–225. REFERENCES [19] Y. Wautelet, S. Heng, S. Kiv, and M. Kolp, “User-story driven development of multi-agent systems: A process fragment for [1] A. Felfernig, L. Hotz, C. Bagley, and J. Tiihonen, Knowledge- agile methods,” Computer Languages, Systems and Structures, Based Configuration From Research to Business Cases. Newnes: vol. 50, pp. 159–176, 2017. Morgan Kaufman, 2014. [20] F. Paetsch, A. Eberlein, and F. Maurer, “Requirements [2] C. Forza and F. Salvador, Product information management for engineering and agile software development,” Proceedings. mass customization: connecting customer, front-office and back- Twelfth IEEE International Workshops on Enabling office for fast and efficient customization. New York: Palgrave Technologies: Infrastructure for Collaborative Enterprises, Macmillan, 2007. 2003., pp. 308–313, 2003. [3] L. Hvam, N. H. Mortensen, and J. Riis, Product customization. [21] K. S. Rubin, Essential Scrum: A practical guide to the most Berlin Heidelberg: Springer, 2008. popular Agile process. Addison-Wesley, 2012. [4] A. Felfernig, S. Reiterer, F. Reinfrank, G. Ninaus, and M. Jeran, [22] K. Vlaanderen, S. Jansen, S. Brinkkemper, and E. Jaspers, “The “Conflict Detection and Diagnosis in Configuration,” in agile requirements refinery: Applying SCRUM principles to Knowledge-Based Configuration: From Research to Business software product management,” Information and Software Cases, A. Felfernig, L. Hotz, C. Bagley, and J. Tiihonen, Eds. Technology, vol. 53, no. 1, pp. 58–70, 2011. Morgan Kaufman, 2014, pp. 73–87. [23] J. Vlietland, R. Van Solingen, and H. Van Vliet, “Aligning [5] L. L. Zhang, “Product configuration: a review of the state-of-the- codependent Scrum teams to enable fast business value delivery: art and future research,” International Journal of Production A governance framework and set of intervention actions,” Research, vol. 52, no. 21, pp. 6381–6398, Aug. 2014. Journal of Systems and Software, vol. 113, pp. 418–429, 2016. [6] A. Haug, L. Hvam, and N. H. Mortensen, “Definition and [24] K. Beck, Test-driven development: by example. Addison-Wesley evaluation of product configurator development strategies,” Professional, 2003. Computers in Industry, vol. 63, no. 5, pp. 471–481, Jun. 2012. [25] S. J. Ferguson, BDD in Action: Behavior-driven development for [7] S. Shafiee, K. Kristjansdottir, and L. Hvam, “Business cases for the whole software lifecycle. Manning, 2015. product configuration systems,” in 7th international conference [26] D. North, “Behavior Modification: The evolution of behavior- on mass customization and personalization in Central Europe, driven development,” Better Software, vol. 8, no. 3, 2006. 2016. [27] E. Evans, Domain-Driven Design: Tackling Complexity in the [8] S. Shafiee, K. Kristjansdottir, L. Hvam, and C. Forza, “How to Heart of Software. Addison-Wesley Professional, 2003. scope configuration projects and manage the knowledge they [28] M. Soeken, R. Wille, and R. Drechsler, “Assisted behavior driven require,” Journal of Knowledge Management, vol. 22, no. 5, pp. development using natural language processing,” in International 982–1014, 2018. Conference on Modelling Techniques and Tools for Computer [9] A. Felfernig, G. E. Friedrich, and D. Jannach, “UML as domain Performance Evaluation, 2012. specific language for the construction of knowledge-based [29] S. Nelson-Smith, Test-Driven Infrastructure with Chef: Bring configuration systems,” International Journal of Software Behavior-Driven Development to Infrastructure as Code. Engineering and Knowledge Engineering, vol. 10, no. 4, pp. 449– O’Reilly Media, Inc., 2013. 469, 2000. [30] G. A. Moore and R. McKenna, Crossing the Chasm: Marketing [10] S. Shafiee, L. Hvam, A. Haug, M. Dam, and K. Kristjansdottir, and Selling High-Tech Products to Mainstream Customers. “The documentation of product configuration systems: A HarperBusiness, 2014. framework and an IT solution,” Advanced Engineering [31] A. H. Van de Ven, “Nothing is quite so practical as a good Informatics, vol. 32, pp. 163–175, 2017. theory,” Academy of Management Review, vol. 14, no. 4, pp. [11] A. Haug, “The illusion of tacit knowledge as the great problem in 486–489, 1989. the development of product configurators,” Artificial Intelligence [32] D. M. McCutcheon and J. R. Meredith, “Conducting case study for Engineering Design, Analysis and Manufacturing, vol. 26, no. research in operations management,” Journal of Operations 1, pp. 25–37, Dec. 2010. Management, vol. 11, no. 3, pp. 239–256, Sep. 1993. [12] C. Forza and F. Salvador, “Product configuration and inter-firm coordination: An innovative solution from a small manufacturing enterprise,” Computers in Industry, vol. 49, no. 1, pp. 37–46, Sep. 2002. [13] A. Myrodia, K. Kristjansdottir, S. Shafiee, and L. Hvam, “Product configuration system and its impact on product’s life cycle complexity,” in 2016 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM), 2016, pp. 670–674. [14] E. Sandrin, “An empirical study of the external environmental factors influencing the degree of product customization An