A gile software development methods have become increasingly popular in the last years. However, agile methods don't specify explicitly how to deal with the quality requirements. M oreover there is little known about how organizations currently deal with this shortcoming. Based on several case studies this research will investigate real-world large-scale distributed agile projects to understand the challenges agile teams face regarding quality requirements and the approach they are currently using to cope with these challenges. After that a set of good practices will be introduced to exp licitly integrate quality requirements in agile processes. Other case studies will be conducted to validate the suggested good practices.
Agile software development methods have become increasingly popular in the last
years. The agile parad ig m is a reaction to the traditional plan -driven software
development methods such as Waterfall.
A characterizing aspect of the traditional approach is a significant up -front Require ments Engineering (RE) effort. Usually, this is a process in wh ich the software require ments should be fully e licited and documented prior to proceed to the ne xt phase (the software design process). Although this process might seem logical, it is difficult and nearly impossible [4] to achieve due to the following reasons: The require ments tend to change quickly and become outdated even before the project is completed [4][5]. Pre-specified require ments may become irre levant as the project progresses. New require ments may arise, and old requirements may become irrelevant with a better understanding of the mission [4]. The rapid change of the surrounding environment such as stakeholder preferences, technology, and time- to-market [4][5].
Responding to change is one of the agile a lliance values s tated in the agile manifesto [6]. Based on these values agile pract ices have been developed to increase the involvement of the customers and to deliver software products faster. The agile practices includes short iterations that end up with de livering a new incre mental release, gathering as many require ments as needed to start an iteration, simple incre mental design that will be evolved during the iterations and peer reviews [7].
Despite the fact that being agile is a success factor for software p rojects [1], the agile approach is still facing challenges that might cause the failure of a software project. The study of Ramesh et al. has identified seven challenges posed by agile pra ctices [8], inc luding the neglect of quality require ments (QRs). The neglect of QRs in agile projects may result in deliveries that don’t satisfy the user e xpectations. In sma ll co-located projects, this can be repaired re latively easily by adapting the ne xt batch of require ments and repairing the part of the product already delivered. This is however not possible in large projects where the tea m is necessarily distributed over space and there is no possible of ad-hoc coordination and co mmun ication a mong tea m me mbers and with clients. Hence, this doctoral research proposed in this paper addresses the problem of engineering QRs in la rge scale agile p rojects in a distributed organizat io nal context and how these organizations cope with it. The research will also suggest a set of good practices as possible solution practices. Our e xpectation is that solution for large-scale agile p rojects will also be useful for sma ller -scale agile projects. We will chec k this e xpectation later on afte r we identified solutions for la rge -scale projects. 2
QRs are those require ments that describe the qualities of the system [9]. While the functional require ments specify what the system should do in response to specific action fro m the environment, the QRs describe how the system should perform these actions. Exa mp les of the QRs are: Pe rformance, Security, Maintainability, Testability, Usability, Portability and Reusability. The term non -functional is used in many studies to point to those require ments. However, this term in this case is passive, unfortunate and doesn’t give the impression that these requirements are important [9][10]. Boeh m used the term “quality require ments (QRs)” instead of non-Functional require ments [11], wh ich in my opinion describes better the nature of those requirements. Therefo re, throughout this work the term “quality require ments ” is used to refer to the require ments that describe how the system should perform the desired actions.
In the RE literature, there is a consensus that the success or failure of a system is not only decided by the correct imple mentation of the right functional require ments. If the response time of the system for e xa mp le doesn’t meet the customer e xpectations, we can’t say that the system deliver quality [10]. Besides that, high quality software architecture is intimately connected to the achievement of the QRs [12]. A softwa re architecture that provides a high security and an acceptable performance is not the same as the software architecture that ma ximizes the performance and guarantees a certain level of security. However, since it is impossible to ma ximize the imple ment ation of a ll QRs [13], it is necessary to define the desired level o f each quality requirement to imple ment the most appropriate software architecture that delivers the e xpected business value.
A recent systematic literature revie w on agile require ments engine ering practices and challenges [14] reported that neglecting QRs is a proble m fo r agile require ments engineering processes. The same review reported that until 2013 there was just one study that comes up with a lite rature s olution proposal (NORMATIC) [15]. This study has addressed the problem of the QRs in agile practices and developed a Java-based simu lation tool for modelling QRs for semi-automatic ag ile processes. Although this study is directly related to our study, it differentiates form our study in the following aspects: NORMATIC’s problem investigation is a literature research while our investigation will be done empirically. NORMATIC hadn’t the focus on distributed agile environments and due to this nature; the challenges of being distributed and their effect on the QRs weren’t taken in consideration. The results of the NORMATIC research weren’t validated in rea l-world agile development projects and the effect iveness of the suggested tool in modelling QRs in agile projects weren’t measured.
In 2015, Darshan Do mah and Frank M itropoulos developed the NERV Met hodology [16]. Similarly to NORMATIC, NERV addresses the non-functional requirements in ag ile software develop ment based on literature study while our research as mentioned before is an empirical research. 3
The ma in goal of this research is to define a frame work to identify, model and test QRs in a large scale pro ject of a distributed agile environ ment. Based on this obje ctive, the central research question (RQ) of this study is: How to identify, implement and test the QRs in a large scale distributed agile project? To meet the research goal, the main RQ is elaborated in the following sub -questions: 1. What are the primary cha llenges to agile tea ms when working with QRs in a d istributed agile environment? 2. How do agile tea ms currently handle those challenges in a distributed agile environment? 3. Are there any gaps between the practices the agile tea ms use and the e xpected ou tcomes of deploying these practices? 4. What is important for agile teams to improve in their current way of handling QRs? 5. What improvements could be done? 6. Do the improvements work in practice? 4
A lack of e mpirica l research on agile software develop ment and QRs was observed in recent studies [17]. Particu larly, in agile literature, much of the work is focused on validating the applicability and usefulness of practices in ag ile develop ment methods. Beyond a fe w high v isibility case studies the current literature does not provide much insight on actual practices used in agile and their effect iveness in support ing RE activities [8]. To respond to the lack of e mpirica l evidence, this project will adopt case study research method [18] for the purpose of problem analysis, solution design and solution evaluation (or validation).
The design and engineering cycles consist of two ma inly stages, each of which has its own set of tasks (Fig.1). The first stage “the design cycle” is decomposed into three tasks, namely, p roble m investigation, treatment design, and treatment validation. These tasks are mu ltiple t imes ite ratively imp le mented to develop a co mplete and useful design. The results of this stage are transferred to the real world in the second stage “The engineering cycle” which deco mposed into two tasks, na mely, treat ment implementation, and implementation evaluation.
First, we will ca rry out observational case studies at agile client organizations and software development organizat ions which will help to carry out a proper proble m analysis. We assume the problems fro m a vendor and client perspectives would be diffe rent. Also, la rge government organizat ions that embark on agile might e xper ience a different set of problems compared to private mid-size co mpanies using agile. Furthermore, case studies will be used to understand which treatments companies are currently using to cope with the identified proble ms. Studying whether the treat ments are useful and whether they meet the e xpectations of co mpanies will be a lso part of the case studies.
The first four sub-questions of our research are corresponding with the proble m investigation tasks of the design cycle (Fig.1). Answering those questions will a llo w us to drive a proper treat ment design (good practices) to deal with the identified proble ms. W ith the design of the good practices the fifth sub-question of our research will be answered.
Once the good practices are developed, the plan is to validate and evaluate their utility and usefulness in follow-up case studies with practitioners working on agile projects [19]. We will use two methods to validate the frame work, na me ly, Expe rt opinion and Technical Action Research (TAR). The developed good practices are useful if the identified problems can be solved using them. The case studies will be part of each iteration to ensure better analytical results. Feedback loops will be pa rt of the interactions between industry partners and researchers to ensure the relevancy of the researched subjects. Refine ments and improve ments of our good practices and solution method will be done incrementally on step-by-step basis. In this approach, case studies are essential to ensure the practical applicability of the research for agile software businesses in the Netherlands. 5
The ma in e xpected outcome of this research is the good practices that can be used to identify, imp le ment and test the QRs in large-scale distributed agile projects. The contribution of this research is in four specific aspects: (1) Giv ing insight in the current challenges agile tea ms meet regarding QRs in large scale distributed projects. (2) The current way of working to cope with those challenges . (3) The evaluation of the current way o f working and 4) The good practices and the improve ment that could be applied to real-life pro jects that use those practices. Although there have been several proposals in scientific literature on how to treat QRs in agile projects, we found no proposal to be empirically evaluated and used in real-life settings. 6
The PhD research is started on May 2015 and expected to be fin ished in 2020. However as a software engineer with a solid experience of developing software using different software development processes I faced the challenge of neglecting the QRs in almost all the agile software projects I was part of. This motivated me to dive into the literatures searching for an appropriate solution to deal with QRs in agile context.
The search process resulted in nothing sufficient. Thereafter I started to visit seminars and workshops provided by different software co mpanies in the Netherland to find an answer to my question. I found out that each company I spoke to, had its own not validated practice to deal with the QRs. One of those companies introduced a nonofficial scru m term “Sprint zero”. In th is sprint they define the overall software arch itecture. This approach explicitly opposes the agile spirit wh ich emphasizes the eme rgence of the software architecture during the development cycle. Another company defined the QRs as constrains within user stories. Constrains are simp ly another type of requirements such as QRs but they are not the same. Constrains are not just a matter of opin ion. Any solution that does not meet the specified constraints is simp ly unacceptable [20]. In contrast to QRs constrains are not negotiable.
The next step in this doctoral research is to conduct a Systematic Literature Rev iew (SLR) [21] to answer the following questions: What are the existing agile practices used for engineering QRs? What are the challenges created by the use or absence of those practices? What are the existing solutions according to published empirical studies?
Answering these questions will g ive us insight in State of Art o f QRs within an agile context. 7 1. 2. 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21.
Shore, J., Shane, W.: The A rt of Ag ile Development. O’Reilly Media, Inc (2007).
Ramesh, B., Cao, L., Baskerville, R.: Agile requirements engineering practices and challenges: an empirical study. Inf. Syst. J. 20, 449–480 (2010). Lauesen, S.: Soft ware Requirements: Style and Techniques. Addison-Wesley Professional (2002).
Blaine, J.D., Cleland-Huang, J.: Software quality requirements: How to balance competing priorities. IEEE Softw. 25, 22–24 (2008).
Boehm, B., In, H.: Identifying Quality-Requirement Conflicts. 30602 (1996). Kazman, R., Bass, L.: To ward Deriving Software A rchitectures Fro m Quality Attributes. Engineering. 1–44 (1994).
Haigh, M.: So ftware quality, non-functional software requirements and ITbusiness alignment. Softw. Qual. J. 18, 361–385 (2010).
Inayat, I., Salwah, S., Marczak, S., Daneva, M., Shamshirband, S.: A systematic literature review on agile requirements engineering practices and cha llenges. Comput. Human Behav. (2014).
Farid, W.M., Mitropoulos, F.J.: NORMATIC: A visual tool for modeling Non-Functional Requirements in agile processes. 2012 Proc. IEEE Sout heastcon. 1–8 (2012).
Do mah, D., Mitropoulos, F.J.: The NERV Methodology : A Lightweight Process for Addressing Non-functional Requirements in Agile Software Development. (2015).
Daneva, M., Van Der Veen, E., A mrit, C., Ghaisas, S., Sikkel, K., Ku mar, R., Ajmeri, N., Ramteerthkar, U., W ieringa, R.: Agile requirements prioritization in large-scale outsourced system projects: An emp irical study. J. Syst. Softw. 86, 1333–1353 (2013).
Wieringa, R.: Emp irical research methods for technology validation: Scaling up to practice. J. Syst. Softw. 95, 19–31 (2014).
Wieringa, R., Daneva, M .: Six strategies for generalizing software enginee ring theories. Sci. Comput. Program. 101, 136–152 (2015).
Robertson, B.S., Robertson, J.: Mastering the Requirements Process. Addison Wesley Professional (2006).
Kitchenham, B., Charters, S.: Gu idelines for performing Systematic Literature Reviews in Software Engineering. Engineering. 2, 1051 (2007).