=Paper=
{{Paper
|id=Vol-2220/09_CONFWS18_paper_21
|storemode=property
|title=Behavior-Driven Development in Product Configuration Systems (short paper)
|pdfUrl=https://ceur-ws.org/Vol-2220/09_CONFWS18_paper_21.pdf
|volume=Vol-2220
|authors=Sara Shafiee,Lars Hvam,Anders Haug,Yves Wautelet
|dblpUrl=https://dblp.org/rec/conf/confws/ShafieeHHW18
}}
==Behavior-Driven Development in Product Configuration Systems (short paper)==
https://ceur-ws.org/Vol-2220/09_CONFWS18_paper_21.pdf
ﯾﺎھﻮ
Behavior-Driven Development in Product Configuration Systems
Sara Shafiee1 and Lars Hvam and Anders Haug and Yves Wautelet
Abstract.1 Product Configuration Systems (PCS) are increasingly [13], degree of customization [14], or other factors such as
used by companies to automate the performance of the sales and knowledge management and scoping process [8]. The challenges to
engineering processes. Since the benefits from such projects have manage technical and commercial knowledge when implementing
huge variations, it is crucial to make the right decisions when and maintaining a PCS may highly influence PCS costs and
scoping and developing PCSs. The development of PCS is
development time, as well as its lasting effectiveness.
influenced by both business interests and technical insights.
Developers of PCS face various challenges while working in team, The complexity in the development of PCSs comes from that it
including different stakeholders such as business owners, involves highly complex technical knowledge from domain
developers, project managers, and product experts. The more experts, [9], [10]. Hence, Scrum and agile methods attracted
diverse the team is, the more significant are the challenges. This attention in PCS development. The main reasons that can be
paper suggests that Behavior-driven Development (BDD) may mentioned as their ease of use, the constant communication with
provide configuration teams with a specific structure to express the stakeholders and the team and fast development time [10], [15],
scenarios (and thus constraints) on PCS in natural language. BDD [16]. In spite of the potential benefits of Scrum, many
may yield benefits such as a better expression of PCS constraints, organizations are reluctant to throw their conventional methods
more efficient communication of requirements and incorporation of away and jump into agile methods. This to some extent may be
the expressed rules in a software transformation process. In other
attributed some of the issues experienced with the use of Scrum,
words, applying BDD may eliminate unnecessary tasks when
gathering knowledge, developing, and testing PCS projects. In this including significant reduction of documentation, insufficient
paper, we present a novel approach from an ongoing project on testing for mission/safety-critical projects, inadequate support for
how to relate BDD to the development process of PCS while using highly stable projects, only successful with talented individuals
Scrum-based methods. who favor large degrees of freedom, and inappropriate for large-
scale projects [17].
As the discussion above indicates, it is very challenging to
1 INTRODUCTION specify a PCS at analysis stage. In Scrum-based development this
Product Configuration Systems (PCS) are expert systems that can is traditionally done using user stories [18]. The latter nevertheless
support and facilitate the sales and engineering processes [1] by do present some drawbacks because their narrative alone is not
incorporating information about product features, product enough to express the constraints related to PCS. This is precisely
structure, production processes, costs and prices [2]. Thereby, where BDD could offers complementary representation abilities
configurators can support decision-making processes in the sales destined to address user stories limitations. Because of the use of
and engineering phases of a product [3]. Configurators enable BDD next to user stories narrative, the PCS can be expressed
companies to develop product alternatives to facilitate sales and without any other requirements representation artifact (traditionally
production processes [4]. However, the companies that have the Product Variant Master is used for this [3]). Other positive
managed to implement and utilize configurators also face various aspects of the use of BDD to further describe user stories are their
challenges [1], [5]. expression in (structured) natural language making them easy to
Even though advantages of PCS are evident, there are still some understand by non-technical stakeholders. They thus present
difficulties associated with high costs [2], [6] and considerable advantages for communication. Finally their dynamic and process
chances of failure [2], [7] in their implementation projects. This is oriented nature can be used in a systematic transformation process
because companies must overcome various challenges to for the configurator design [19]. All of these elements bring
implement and utilize configurators. For example, the development potential positive contributions with the use of BDD and needs to
of a PCS often requires highly complex technical or commercial be more formally studied. This paper presents a first attempt
knowledge, which domain experts often have a hard time towards such a research.
communicating to configuration experts [8]. Furthermore, the
knowledge base that contains this knowledge has to be adapted 2 LITERATURE STUDY
continuously because of the changing components and
configuration constraints [9], [10]. The difficulty of acquiring and
modeling the required technical or commercial knowledge depends
2.1 Agile and Scrum
on whether it is available in a clear and formal form [11] which in Scrum is an agile software development methodology. The
turns may be contingent to company size [12], product complexity Agile Manifesto outlines the values and principles that should be
supported by the various agile processes applied in software
1 Mechanical engineering department, Technical University of Denmark, development. Agile principles emphasize customer satisfaction,
Denmark, email: sashaf@dtu.dk change and collaboration between domain experts and developers
ﯾﺎھﻮ
[20]. Rubin [21] highlighted that with an agile approach, the team In BDD, as compared to TDD, the task is to write a specification of
starts by creating a product backlog, which is a prioritized list of system behavior that is precise enough for it to be executed as code
the features and other capabilities that need to be developed. [29]. More specifically, the whole point of BDD is to ensure that
Guided by the product backlog, team members address the most the real business objectives of stakeholders are met by the software
important or highest priority items first; priority is based on various we deliver. If stakeholders are not involved, if discussions are not
factors, but delivered business value is most often the first priority. taking place, BDD is not going to work. BDD yields benefits
Scrum is an agile approach for developing innovative products and across many important areas such as: (1) Building the right thing,
services [21]. (2) Reducing the risks, (3) Evolving the design [29].
Scrum facilitates cross-team coordination and collaboration The first Phase in BDD is to understand the business goals and
[22]. Vlietland et al. (2016) determined that Scrum improves defining features [25]. Vision statement templates make it possible
coordination through additional events, such as interteam sprint to have a well-defined set of business goals. For a good product
planning meetings, interteam daily Scrums, interteam product vision statement, Moore at al. [30] propose the following contents
refinements and interteam sprint reviews. A Scrum development of a template:
life cycle normally consists of short iterations of two to four weeks, · For (Who will benefit from this product?)
an approach that enables swift feedback from software users and · Who (What do they need?)
related stakeholders regarding the developed solution [19], [23]. · The (What sort of thing are you proposing?)
· That (What makes it so cool?)
· Unlike (What are you competing against?)
2.2 Behavior-driven Development · Our product (Why customer prefer your solution?)
In the movement of agile development, Test-Driven Development Secondly, the features have to be illustrated in natural language
(TDD) has been around for a long time and can be traced back to to execute the specifications. Consequently, the scenario has the
eXtreme Programming practices developed in the late 1990s [24]. structure [25]:
In particular, TDD employs so-called acceptance tests as the · Given [context, initial conditions]
starting point for the development process to address some of the · When [event occurs]
challenges related to Scrum. Following [25], these TDD relies on · Then [outcome]
two simple principles: There are several studies investigating how to automate all these
· Don’t write any code until you’ve written a failing test that scenarios such as Cucumber or Jbehave [25].
demonstrates why you need this code.
· Refactor regularly to avoid duplication and keep the code
2.3 Requirement artifacts in traditional PCS
quality high.
Behavior Driven Development (BDD) has been proposed as a
projects: the Product Variant Master
result of the problems that arose with TDD when applying agile Generic product structures can be illustrated using the so-called
software practices [26]. It should be noticed that the language used “product variant master” (PVM) notation, which in many cases has
for describing the tests, i.e. class names and operation names, plays functioned as a common language between different product,
an important role both for writing test cases and for finding bugs in process and IT experts [3]. Different definitions of the PVM
case of a failed test. Inspired by [27], for this purpose BDD uses notation have been proposed, and among them the definition of the
natural language as a ubiquitous communication mean to describe PVM notation by Haug [11] is one of the most extensive and
the acceptance tests by means of scenarios. formalized. A principal example of the PVM technique used to
The cornerstone of TDD is the idea of writing a unit test before model a toy car solution space is shown in Figure 2. On the left
writing the corresponding code. However, BDD is much more than side of a PVM model, part-of-structure is shown, and on the right
ensuring that every user story has a corresponding set of unit tests; side, kind-of structure is shown. Classes (typically components and
BDD is also about writing specifications, as opposed to tests. In assemblies) are represented by circles and may include attributes
BDD, as an agile software development technique, acceptance tests and constraints (or rules).
are written in natural language in order to ensure a common
understanding between all members of the project [28].
Consequently, as the first step, the sentences are mapped to actual
source code [28].
The shift from TDD to BDD is subtle but significant. Instead of
thinking in terms of verification of a unit of code, the focus is on
specifying how that code should behave, i.e., what it should do
[25]. In order to be sure that of building code that matters, there is
a need for specifications that describe what the code should do and
how to relate it directly to the business requirements.
Figure 1 shows an example of the BDD flow as it is employed
in a specific tool [26].
Figure 1. Behavior Driven Development flow
Figure 2. Product variant master (ToyCar example)
ﯾﺎھﻮ
3 PROBLEM STATEMENT team’s satisfaction with BDD. Table 1 summarizes some of the
details of the case projects.
Using Scrum for PCS development introduces both potential
benefits and challenges (as for general IT projects in Table 1). One Table 1. Scrum practice and defined roles and activities in Cases 3 and 4
of the challenges is the level of knowledge complexity in PCS that Projects Case 1 Case 2 Case 3
has to be communicated clearly to all Scrum team members in Time frame (months) 11 8 9
terms of all attributes, constraints and acceptance criteria. An Number of iterations
10 9 9
additional challenge from Scrum concerns the lack of visualization during the project
(modelling techniques [3]) for product structure. BDD supports · Product owner: the stakeholders’ manager
Scrum with vision statements [30] to be able to demonstrate the · Project manager
· Scrum master
product details and even user interface step by step. Another Roles · Development team, including: 1 application
related challenge is the testing of the PCS as assessing PCSs manager, 1 project manager, 2 configuration
implies to assess system features with respect to the many possible engineers, 2 developers, 1 tester
· End users
data and system outputs that might occur when a user is interacting · Backlog grooming
with them. Testing PCSs is an arduous testing activity due to the · Sprint planning
wide range of user tasks and the different combinations of testing Activities · Sprint backlog
data. BDD, as an add-on of user stories, supports the testers with · Daily Scrum
· Sprint review
the detailed defined acceptance criteria. · Product goals and product backlog item (story)
In short, these concerns bring us three main guidelines for using Artefacts (main · Product backlog and stakeholders’ requirements
Scrum methods in PCS projects: specifications) (list of user stories)
· Testing (acceptance criteria in user stories)
· Formalize user requirements in such a way to provide · Daily Scrum
testability in an ever-changing environment; · Sprint planning
Planning approach
· Sprint review
· Guarantee consistency between user requirements and their · Feedback meetings
representation in multiple artefacts during development · Same as the project roles, plus:
phase; and Specific roles of · Product owner: 1
· Lay on a validation approach that could be reused to ensure meeting participants · Scrum master: 1
· Tester: 1
such a consistency for the artefacts along the project.
5 OUTLOOK
4 RESEARCH METHOD
It is yet an open question, how much BDD can contribute in the
The aim of this paper is to test the application of BDD in PCS
development phase of PCS projects. As PCS projects own their
projects in real case projects and gather the data regarding the BDD
specific challenges, there is a need for further studies to test BDD
application. Firstly, we review the literature of BDD to gain deeper
influence specifically for PCS projects. The results from case
understanding of its definition and steps. Secondly, we would
studies and interviews will serve to verify or falsify the mentioned
apply the findings from literature regarding BDD to the Scrum
hypothesis when the study is accomplished. The structure of a user
management in PCS. The authors’ ultimate goal is to outline what
story presented to the case projects is demonstrated bellow:
the contribution of BDD to PCS can be and discuss its importance
in promoting the collaboration and communication of knowledge
within the organization.
Based on the mentioned challenges in PCS projects, BDD can
be an effective solution to improve the definitions of different
features and testing the codes in PCS projects next to the user
stories. Hence, we posit the following three propositions:
Proposition 1: (expressiveness) BDD allows expressing all of
the necessary constraints required for documenting a
Configurator using Scenarios.
Proposition 2: (performance) BDD represents an effective
approach for communicating the product specifications when
implementing PCS as a replacement for PVM.
Proposition 3: (acceptance) BDD represents an effective and
practical approach (requirements artifacts comparison) for unit
testing of the implemented features in development and testing Figure 3. BDD structure of a user story presented for PCS projects in the
phases of PCS projects. workshops (Car example from Figure 1)
We use a qualitative exploratory design based on multiple data
sources: requirements artifacts, workshops, interviews and Concerning the adoption next to user stories narrative as a
participant observation [31], [32]. The study is ongoing in one case replacement for PVM and the vocabulary proposed in the ontology,
company using Scrum method for developing PCS for more than 4 an advantage is that requirements and tests in user stories are kept
years. Workshops are conducted to train the team for BDD to be in a natural and high-level language. Based on Figure 3, the team
implemented as part of the Scrum. Finally, feedback meetings are should be able to demonstrate the prototype for user interface. The
held as semi-structured interviews to collect knowledge about the
ﯾﺎھﻮ
evaluation criteria of using BDD approach in the team can be Empirical Study of the External Environmental Factors
considered as: Influencing the Degree of Product Customization,” no. December
1. To support enterprise modeling within agile (user centric) 2016, 2017.
methods [15] B. Selic, “Agile Documentation, Anyone?,” IEEE software, vol.
26, no. 6, 2009.
2. To estimate different scenarios, architecture and integration
[16] S. Ambler, Agile modeling: effective practices for extreme
3. Correct evaluation of available attributes and constraints for the programming and the unified process. John Wiley & Sons, 2002.
product [17] J. Cho, “A Hybrid Software Development Method for Large-
4. To support the analysis of requirements communication Scale Projects: Rational Unified Process with Scrum,” Issues in
5. To analyze the compatibility with business modelling Information Systems, vol. 10, no. 2, pp. 340–348, 2009.
6. To prototype the desired user interface [18] Y. Wautelet, S. Heng, M. Kolp, and I. Mirbel, “Unifying and
extending user story models,” in International Conference on
Advanced Information Systems Engineering, 2014, pp. 211–225.
REFERENCES [19] Y. Wautelet, S. Heng, S. Kiv, and M. Kolp, “User-story driven
development of multi-agent systems: A process fragment for
[1] A. Felfernig, L. Hotz, C. Bagley, and J. Tiihonen, Knowledge-
agile methods,” Computer Languages, Systems and Structures,
Based Configuration From Research to Business Cases. Newnes:
vol. 50, pp. 159–176, 2017.
Morgan Kaufman, 2014.
[20] F. Paetsch, A. Eberlein, and F. Maurer, “Requirements
[2] C. Forza and F. Salvador, Product information management for
engineering and agile software development,” Proceedings.
mass customization: connecting customer, front-office and back-
Twelfth IEEE International Workshops on Enabling
office for fast and efficient customization. New York: Palgrave
Technologies: Infrastructure for Collaborative Enterprises,
Macmillan, 2007.
2003., pp. 308–313, 2003.
[3] L. Hvam, N. H. Mortensen, and J. Riis, Product customization.
[21] K. S. Rubin, Essential Scrum: A practical guide to the most
Berlin Heidelberg: Springer, 2008.
popular Agile process. Addison-Wesley, 2012.
[4] A. Felfernig, S. Reiterer, F. Reinfrank, G. Ninaus, and M. Jeran,
[22] K. Vlaanderen, S. Jansen, S. Brinkkemper, and E. Jaspers, “The
“Conflict Detection and Diagnosis in Configuration,” in
agile requirements refinery: Applying SCRUM principles to
Knowledge-Based Configuration: From Research to Business
software product management,” Information and Software
Cases, A. Felfernig, L. Hotz, C. Bagley, and J. Tiihonen, Eds.
Technology, vol. 53, no. 1, pp. 58–70, 2011.
Morgan Kaufman, 2014, pp. 73–87.
[23] J. Vlietland, R. Van Solingen, and H. Van Vliet, “Aligning
[5] L. L. Zhang, “Product configuration: a review of the state-of-the-
codependent Scrum teams to enable fast business value delivery:
art and future research,” International Journal of Production
A governance framework and set of intervention actions,”
Research, vol. 52, no. 21, pp. 6381–6398, Aug. 2014.
Journal of Systems and Software, vol. 113, pp. 418–429, 2016.
[6] A. Haug, L. Hvam, and N. H. Mortensen, “Definition and
[24] K. Beck, Test-driven development: by example. Addison-Wesley
evaluation of product configurator development strategies,”
Professional, 2003.
Computers in Industry, vol. 63, no. 5, pp. 471–481, Jun. 2012.
[25] S. J. Ferguson, BDD in Action: Behavior-driven development for
[7] S. Shafiee, K. Kristjansdottir, and L. Hvam, “Business cases for
the whole software lifecycle. Manning, 2015.
product configuration systems,” in 7th international conference
[26] D. North, “Behavior Modification: The evolution of behavior-
on mass customization and personalization in Central Europe,
driven development,” Better Software, vol. 8, no. 3, 2006.
2016.
[27] E. Evans, Domain-Driven Design: Tackling Complexity in the
[8] S. Shafiee, K. Kristjansdottir, L. Hvam, and C. Forza, “How to
Heart of Software. Addison-Wesley Professional, 2003.
scope configuration projects and manage the knowledge they
[28] M. Soeken, R. Wille, and R. Drechsler, “Assisted behavior driven
require,” Journal of Knowledge Management, vol. 22, no. 5, pp.
development using natural language processing,” in International
982–1014, 2018.
Conference on Modelling Techniques and Tools for Computer
[9] A. Felfernig, G. E. Friedrich, and D. Jannach, “UML as domain
Performance Evaluation, 2012.
specific language for the construction of knowledge-based
[29] S. Nelson-Smith, Test-Driven Infrastructure with Chef: Bring
configuration systems,” International Journal of Software
Behavior-Driven Development to Infrastructure as Code.
Engineering and Knowledge Engineering, vol. 10, no. 4, pp. 449–
O’Reilly Media, Inc., 2013.
469, 2000.
[30] G. A. Moore and R. McKenna, Crossing the Chasm: Marketing
[10] S. Shafiee, L. Hvam, A. Haug, M. Dam, and K. Kristjansdottir,
and Selling High-Tech Products to Mainstream Customers.
“The documentation of product configuration systems: A
HarperBusiness, 2014.
framework and an IT solution,” Advanced Engineering
[31] A. H. Van de Ven, “Nothing is quite so practical as a good
Informatics, vol. 32, pp. 163–175, 2017.
theory,” Academy of Management Review, vol. 14, no. 4, pp.
[11] A. Haug, “The illusion of tacit knowledge as the great problem in
486–489, 1989.
the development of product configurators,” Artificial Intelligence
[32] D. M. McCutcheon and J. R. Meredith, “Conducting case study
for Engineering Design, Analysis and Manufacturing, vol. 26, no.
research in operations management,” Journal of Operations
1, pp. 25–37, Dec. 2010.
Management, vol. 11, no. 3, pp. 239–256, Sep. 1993.
[12] C. Forza and F. Salvador, “Product configuration and inter-firm
coordination: An innovative solution from a small manufacturing
enterprise,” Computers in Industry, vol. 49, no. 1, pp. 37–46, Sep.
2002.
[13] A. Myrodia, K. Kristjansdottir, S. Shafiee, and L. Hvam,
“Product configuration system and its impact on product’s life
cycle complexity,” in 2016 IEEE International Conference on
Industrial Engineering and Engineering Management (IEEM),
2016, pp. 670–674.
[14] E. Sandrin, “An empirical study of the external environmental
factors influencing the degree of product customization An