Requirements Volatility: An Industry Case Study

Abstract

Software requirements are exposed to many changes during their software development life-cycle. These changes namely additions, modifications or deletions are defined as requirements volatility. Prior requirement volatility prediction studies utilize diferent requirement volatility measures. In this study we predict number of changes per software requirement as requirement volatility for a large scale safety-critical avionics project in ASELSAN. We employ a comprehensive metric set to explain requirements volatility: requirement quality measures, project specific factors and requirement interdependencies. Predictive models are created through combining input metric sets with machine learners. Success of models in predicting requirement changes, the best performing input metric combinations, the best performing machine learners and success of models in predicting highly-volatile requirements are evaluated in this study. The best prediction results are obtained with the model employing quality metrics, project specific metrics, network metrics altogether with k-nearest neighbour machine learner (MMRE=0.366). Also the best model correctly identifies 63.2% of highly volatile requirements which are exposed to 80% of the total requirement changes. Our study results are encouraging in terms of creating requirement change prediction tools to prevent requirement volatility risks prior to the requirement review process.

1. Introduction

Although software engineering has experienced significant advancements in the last decades, majority of the large-scale software projects still try to cope with requirement changes during their software development life cycle due to dynamic nature of software development activities [ 1 ]. Changes for requirements namely additions, deletions or modifications are defined as requirements volatility [ 2 ]. Continual requirement changes during software development have tremendous impact on the cost, the schedule and the quality of the final product. Unfortunately, significant number of software projects cannot be completed successfully or completed partially because of requirements’ high volatility [ 2 ].

According to a survey conducted by Thakurta [ 3 ], project managers use various requirement volatility measures: number of changes to the identified use cases, number of changing requirements identified within the issued change requests, realized requirements out of total requirements, and amount of budget the project had to spent on the changing requirements. Alsalemi et al. [ 4 ] also report a literature review on requirements volatility prediction. Accordingly, ten studies have employed machine learning methods to predict requirements volatility until 2017. These studies utilize diferent requirement volatility measures such as number of requirement nEvelop-O CEUR htp:/ceur-ws.org ISN1613-073 © 2021 Copyright for this paper by its authors. Use permitted under Creative

CEUR

Workshop Proceedings (CEUR-WS.org) changes [ 5 ], requirement stability index of the project [ 6 ], requirements that will be changed in next iteration [ 7 ], requirement change impact [ 8 ], the impact of requirements changes on project distribution and cost factor [ 9 ], software schedule [ 10]. Related studies also propose requirements complexity metrics [ 6 ], requirement dependency metrics [ 8 ], requirement size metrics [ 5 ] and requirements evolution metrics [ 7 ] to predict their own definition of requirement volatility measure.

In our study we aim to predict number of changes per software requirement by using requirement quality measures, project specific factors and requirement interdependencies.

We define requirement volatility as the number of change requests reported for a software requirement. This change request could be either for adding a new requirement or modifying an existing requirement. We chose a safety-critical avionics software project in ASELSAN with more than 20,000 requirements for our study. Loconsole et al. [ 5 ] conducted a similar study to predict number of requirement changes using size measures on projects with less than 50 requirements. Our study complements the prior work by mining a larger dataset with thousands of requirements and a more comprehensive metric set considering quality and interdependency aspects of requirements as well as project specific factors. It should be noted that the change requests that we study in this work occured in any phase of software development after Software Requirements Specification (SRS) document has been reviewed and confirmed. Thus we investigate post-SRS requirements volatility for the avionics project under study.

The rest of paper is organized as follows. Section 2 presents related empirical studies carried on for requirement volatility prediction. Section 3 explains study design model in detail. Results and Threats to Validity of our work discussed in Section 4. Section 5 presents conclusion and points out possible directions for future work.

2. Related Work

According to the literature review, only one study conducted by Loconsole et al. [ 5 ] present an empirical study to predict number of changes per requirement, so this study is the most relevant to our work. Following size measures are used to predict number of requirement changes: number of actors interacting with use cases, number of words in each file, number of revisions of files, number of lines per file. In our study, size measures are also used to represent requirement quality but we also enriched our metric set with project specific metrics and network metrics. It should be noted that we do not take deletion requests and deleted requirements into consideration while defining requirements volatility, because in our industrial context we rarely encounter such requests for the safety-critical software. Finally, we applied our model on more than 20,000 requirements that help us assess the generalizability of our findings on predicting volatility on every software requirement using diferent metrics sets.

In this section we present previous studies that aim to predict volatility for requirements, and we focus on the input metrics they employed. We report details of five relevant studies [ 11, 6, 7, 8, 5 ] from the literature review conducted by Alsalemi et al. [ 4 ]. We also discuss the approaches of other recently published, related studies [12, 13] in this section.

Nakatani et al. [11] propose a method to predict requirement volatility using social relations between executives, competitors, cooperative organizations, and the natural environment. Those measures can be applied to customer requirements easily but it would take some efort 3. Study Design to associate them with software requirements. Christopher et al. [ 6 ] present requirements complexity metrics In this section we explain our empirical study design in to define volatility. Functional requirement complex- detail. In Section 3.1 research questions are explained. ity, non-functional requirement complexity, input-output The analyzed project for which a model would be procomplexity, interface and file complexity measures are posed is described in Section 3.2. In Section 3.3 selected used to calculate whole project’s stability, whereas we input metrics for requirement volatility prediction are deseek to predict requirement volatility for each software scribed. Section 3.4 describes the output measure of the requirement. Shi et al. [ 7 ] present a model to predict fu- prediction model. The used tools are explained in Section ture requirement changes by using previous requirement 3.5. Machine learning techniques employed in this study change metrics. They generated six history metrics for are presented in Section 3.6. Finally in 3.7, performance requirements that contain information about volatility of evaluation measures are defined for our model. topic, frequency of changes and time duration between changes. History metrics can be used to predict require- 3.1. Research Questions ments that will be changed in next iteration, but has little use in predicting requirements volatility for new projects. Our main goal is to predict requirements volatility at Pedrycz et al. [13] also employ the following change logs earlier stages of development lifecycle, and accordingly as input metrics: created version of requirement, last de- two research questions are defined. veloper, number of modifications, requirement lifetime Research Question(RQ) 1: To what extent do reduration. Change logs are created on later phases of soft- quirement quality metrics, project specific metrics and ware development thus they are again not very useful network metrics predict the volatility of a software reto predict requirements volatility for projects in earlier quirement? development phases. Goknil et al. [ 8 ] and Hein et al. [12] Previous studies used diferent metric sets to predict use requirements interrelations for volatility prediction. requirements volatility. In this study we aim to use a Goknil et al. [ 8 ] utilize formal semantics of requirement comprehensive set of input metrics, and observe their relations as input features, whereas Hein et al. [12] create individual efects on requirement volatility prediction. network metrics by using syntactical natural language While predicting the volatility, we use the number of data. We have combined both measures and created change (addition and modification) requests on a softnetwork metrics by using links between system and soft- ware requirement. Being inspired by the metric sets used ware requirements instead of lingual relations between in the literature, we form a group of requirement qualrequirement texts. Regarding network metrics we em- ity metrics and network metrics. Additionally, project ploy degree centrality, eigenvector centrality, closeness specific metrics for this particular safety-critical avionics centrality and betweenness centrality metrics, whereas project are defined and utilized throughout this study. Hein et al. [12] used 40 network metrics. During model assessment, the performance of each in

3.3. Input Metrics

We have employed several metrics to predict the volatil

Release Number Mean CR Median CR ity of each software requirement in AVPRJ. The metrics of REQs per REQ Per REQ represent three dimensions: requirement quality metrics, Release 1 8,640 0.7457 1 project specific metrics and requirement network metRelease 2 11,401 1.1165 1 rics. Requirement quality metrics extracted by NASA Release 3 2,730 0.5267 0 Automated Requirements Measurement(ARM) tool have Total 22,771 0.9051 1 been used to predict faulty modules previously [16]. Initially, we believed the way requirements are documented will afect requirements volatility besides fault proneness put metric set and the combination of those are reported. of modules. Some requirement quality size metrics are Detailed sub-questions related to RQ 1 are also listed already utilized in predicting requirements volatility [ 5 ], below: therefore we decided to include requirement quality met

RQ 1.1: Which metric group is a better indicator of ric set in our study. Network metrics are employed to the number of requirement changes? predict requirement change volatility in a recent study

RQ 1.2: Which machine learning algorithm is better [12]. This sparked the idea of utilizing network metrics at predicting the number of requirement changes? for requirements volatility prediction. Initial observation

RQ 2: How successful are the proposed models in of various change request notes confirmed that software predicting highly volatile software requirements? requirements that are changed within a particular change

Software requirements have a history of varying num- request have a tendency to be linked to similar system reber of changes during software development life cycle. quirements. Accordingly, we employed network metrics Some requirements do not change at all; however, some created by traceability information. In order to enrich requirements expose to multiple changes and pose risks input metric set with a new metric group we focused on to a software project. Practically, our model should pre- safety-critical avionics project characteristics under this dict highly volatile requirements, so those requirements study. Features are evaluated separately and the ones will be reviewed by experienced reviewers in detail. For would provide information on requirements volatility are this research question (RQ 2) we measure the success of selected as project specific metrics. Rationales of project our models on highly volatile requirements based on a specific metric selection are given in detail in subsection technique in [14]. 3.3.2. Detailed explanations for each group are given in the following subsections.

3.2. Analyzed Project

We chose a safety-critical avionics software project to

perform our analysis. We will refer to this project with AVPRJ in the rest of this paper. AVPRJ has many releases from which three releases are selected. Software requirements for those releases are related since they all belong to the same project; however they are partially distinct since each release consists of implementation of diferent software components developed by many software developers. AVPRJ has a total of 22,771 software requirements. Some release based descriptive statistic for AVPRJ are given in Table 1. CR is used as an abbreviation for change request, REQ is used as an abbreviation for a single requirement. Most of the employed requirements belong to second release and this release has highest mean change request per software requirement value. Third release has relatively fewer software requirements and less addition or modification is performed on requirements belong to this release. More than half of the requirements are modified at least once for this project; 9,848 out of 22,771 requirements are not changed which complies with Standish Group’s survey results over more than 8,000 software projects [15]. 3.3.1. Quality Metrics

While selecting requirement quality metrics to predict

volatility, we have inspired by two studies. The first study propose requirement metrics in the context of NASA Metrics Data Program(MDP) to predict software faults [16]. These metrics are calculated by automatically going through requirements documents to highlight vague, ambiguous, long, complex requirements. The second study also reports requirement quality metrics [17] to ifnd out which requirement quality analyze tool is more successful regarding measurement of those metrics.

Combining both studies’ list and customizing that to the requirements document templates in our industrial context, we present 20 quality metrics in Table 2. All of these metrics take numeric values, e.g. number of flow sentences in a requirement, number of directives in a requirement.

During the preprocessing, stage, we had to remove three metrics from our analysis since they gave little to no information for AVPRJ: Conditional, Rationale and Subjective. Only one requirement contains conditional expressions, three requirements contain rationale expres

The number of abbreviations in a software requirement. For AVPRJ permitted acronym list is used to extract this metric.

The number of actions to be performed if conditions of a software requirement are satisfied.

The number of ambiguous expressions in a software requirement, e.g. adequate, suficiently, optimal, slow.

Average character count between punctuation marks. Long sentences without punctuation marks decrease readability.

The number of conditions need to be satisfied to perform a software requirement.

The number of phrases that give the developers freedom to whether or not to implement a software requirement, e.g. maybe, can’t, would.

The number of connectors that are employed to link multiple sentences or group of words, e.g. and, or, as well as.

The number of directive expressions to refer a table, a note, a figure or an example.

The number of expressions that semantically bond a sentence to another one, e.g. although, but, else.

The number of phrases that command to perform particular actions in a software requirement, e.g. shall, must, will.

The number of pronouns that make the software requirement dificult to understand, e.g. this, that, it. A software requirement should be defined explicitly.

The number of expressions that indicate a software requirement is yet incomplete, e.g. and so on, tbd, etc.

The number of incoming links to a software requirement from other documents. For AVPRJ test cases are linked to software requirements, so the number of in links refer to the number of linked test cases.

The number of phrases that give negative meaning, e.g. doesn’t, none, can’t.

For AVPRJ nested level metric value is the greatest level in hierarchical nesting structure of a software requirement.

The number of out links of a software requirement. In AVPRJ software requirements are linked to system requirements. Therefore the number of out links is the total number of linked system requirements by a software requirement.

The number of expressions that give justification in a software requirement, e.g. thus, in order to.

The number of speculative phrases which lead to question necessity of a software requirement, e.g. normally, eventually, almost.

The number of subjective expressions presenting personal opinion rather than objectivity e.g. I think, in my opinion.

The total number of characters in a software requirement. sions and none of the requirements have subjective ex- cific metrics employed in this study. If the project follows pressions. Thus we ended up having 17 metrics repre- an inspection activity on requirements, it is more likely senting the quality aspect of requirements for predicting that the team would find the ambiguities and inconsistheir volatility. tencies on the requirements. Since derived requirements are not part of customer needs, they cannot be validated 3.3.2. Project Specific Metrics through user acceptance tests. If a requirement has a safety aspect, more comprehensive software tests will be Project specific metrics may difer regarding the scope performed, thus exposure of a potential change is highly of a software project, but the metrics we chose to use probable. Number of related components is a measure are not so specific to the development environment, pro- of impact of a software requirement on general prodgramming language, or domain in which the software is uct, thus more feedback will be given to requirements developed. We believe project specific metrics would pro- afecting many components by development team. Each vide information about development characteristics in an software release has diferent dynamics that afect reorganization, and hence the factors afecting the change quirements maturity e.g. release schedule, experience of proneness of requirements. Table 3 list these project spe- developers, complexity of system. For example if sched

ments regarding system requirement traceability links.

Software requirements which are derived from similar system requirements are tend to be closer in our model.

Weight assignment formula is given below (Equation 1).

W is weight between software requirements, NCLINK is the number of common system requirement links between two software requirements and NTOTLINK is the total number of system requirements linked from those two software requirements. After weight assignment, a symmetrical × matrix is created where n denotes the number of software requirements. Then the network metrics are computed over this matrix.

=

, ,

(1)

3.4. Model Output

Our proposed model output is the number of change requests per software requirement. After the SRS document is reviewed and completed for AVPRJ, change requests linked to each software requirement are reported in the issue management system, and the document is modified accordingly by the analysis team. Thus we define requirements volatility in our industrial context with respect to number of change requests that have been applied to add a new requirement or to modify an existing requirement in the associated SRS document. Please note that our model outputs decimal values, but number of change request per requirement in practice can only get integer values. Therefore we round fractional parts to the nearest integer. ule is too tight to complete SRS document, requirements could be immature and more requirements changes could be performed in the future for this release. 3.5. Tools

We wrote scripts to extract requirement quality and

3.3.3. Network Metrics project specific metrics from SRS documents. Later, UCINET tool [20] is used to create network metrics from Hein et al. [12] earlier utilized 40 network metrics to pre- the matrix that we extracted based on software and sysdict requirements change volatility. On the other hand, tem requirements. Regression models with diferent maValente et al. [19] present correlations between degree, chine learners are trained using WEKA tool [21]. Prebetweenness, closeness, eigenvector centrality measures, diction results are further post-processed in MATLAB to and indicate that those measures are distinct but notion- obtain the performance measures regarding all RQs. ally related. Thus in this work instead of employing 40 metrics, we chose the metrics suggested in [19] to pre- 3.6. Machine Learning Techniques dict requirements volatility for AVPRJ. These centrality metrics give each software requirement a value regard- We train models using linear regression, random foring their position in network. Brief explanations of the est regression, support vector regression and k-nearest employed network metrics are given in Table 4. neighbor regression methods. Linear regression was uti

Hein et al. [12] used language processing to create lized in [ 5 ], whereas classifier version of the other three network for requirements. In this study instead we used techniques were used in [12]. traceability information to create network graph for soft- For k-nearest neighbor regression, inversely proporware requirements. Traceability links from software re- tional weighting option is selected. Higher weights are quirements to system requirements are used for this pur- assigned to closer training samples which resulted in betpose. We assigned weights between software require- ter prediction results for our model. For support vector regression commonly used radial basis function kernel is selected. Increasing gamma parameter too much may result in over-fitting [ 22] and we also experienced a great computational cost with little to no prediction success gain for large gamma. Thus C and gamma parameters are assigned as 1.

In this study 10-fold cross validation technique is used to split training and test sets. Firstly, the dataset containing all software requirements is shufled randomly and split into 10 groups of approximately equal size. One group is labeled as a test set and other groups are used to train machine learning models. This procedure is repeated 10 times until each unique group is used as test set once.

There are requirements with zero change requests. Thus division by zero problem arises while calculating relative error. We made an assumption for unchanged requirements as presented in Equation 3.

4. Results and Discussion

| −

|

If = 0 = 1 (3) WwiethprreesspeencttatnodtwdoiscRuQsss itnhethpisersfeocrtmionan.cWeeoafltshoecommopdaerles Pred(k) is a measure of variance of the error distribu- the performance of the prediction models proposed in tion. This measure is based on relative error and it shows this study with the prior work [ 5 ] . the percentage of predictions whose errors are less than or equal to k. 4.1. RQ 1

For RQ2, we aim to predict highly volatile requirements, and thus, we first employ a method to identify those among the set of requirements: • Step 1: Rank requirements by their actual number of change requests in descending order and record their rank as . • Step 2: Obtain regression prediction results for

each software requirement. • Step 3: Rank requirements by their predicted number of change requests in descending order and record their rank as . • Step 4: Evaluate results according to the listing in Table 5. P denotes percentage of requirements which are perceived as highly volatile, and denotes total number of requirements in validation set.

After obtaining processed data, machine learning regression methods are applied to answer the question if requirement quality metrics, network metrics, project specific metrics can be used to predict the number of changes on each software requirement by employing machine learning methods. Model performance results are gathered for all input metric and machine learning method combinations separately.

Results for RQ 1 is given in Table 6. The following abbreviations are used: ML for machine learning, Q for requirement quality metrics, P for project specific metrics, N for network metrics, KNN for k-nearest neighbor regression, LR for linear regression, RF for random forest regression and SVR for support vector regression.

In terms of input metric combinations, the best MMRE results are achieved with Q&P&N(0.366), Q&N(0.381) and tion models give the same best result, we do not rank the best performing models with regard to MdMRE.

The best performance with respect to Pred(0.25) and Pred(0.5) are obtained with Q&P&N+KNN. Q&N+KNN and Q&P+KNN report the second and third best results.

Those results indicate that requirement quality metrics and k-nearest neighbor algorithm are also successful with respect to Pred measures.

To sum up, best performance results are achieved by using requirement quality metrics, project specific metrics and network metrics altogether. Accordingly, K-nearest neighbor algorithm gives best performance results for all measures. In all best performing models, quality metrics are utilized either as a pair with project or network metrics or as combination of all three. It seems the way requirements are documented has a high efect on the volatility rates.

We compared our findings against the study conducted by Loconsole et al. [ 5 ]. Table 7 reports the performance of linear regression model with the best metric set in our study, our best performing model and the best performing model of [ 5 ]. If we compare the findings only on LR, we observe that using number of lines predicts volatility better on their commercial setting, while in our context using quality metrics only does not give the best result.

Other algorithms like KNN in combination with all metrics significantly improve the prediction performance by reducing MMRE down to 0.36 and MdMRE down to 0, and increasing Pred(0.25) up to 57%. 4.2. RQ 2 Table 7 RQ 2 aims to measure the success of our model in predictComparison of our performance (RQ 1) against [ 5 ] ing highly volatile requirements. We present our technique to identify highly volatile requirements in Section

MMRE MdMRE Pred(0.25) Pred(0.5) 3.7. We first need to determine change request coverQ+LR 0.51 0.5 0.43 0.54 age to categorize highly-volatile requirements, and later Best model 0.36 0 0.57 0.68 calculate recall, accuracy and false alarm rates. Rates NLines+LR [ 5 ] 0.58 0.27 0.5 0.63 for various change request coverage by most volatile requirements are given in Table 8. As change request coverage grows more requirements are labeled as highly Q&P(0.402). We may interpret that requirement quality volatile. We chose 80% change request coverage since metrics (Q) are successful at predicting number of change approximately 40% percent of reviewers are considered requests per software requirement, and its combinations as well-experienced in AVPRJ. Therefore by applying with the other metrics also give good results. With re- this model we could assign review task of 38.6% of total spect to the machine learning algorithm, the three best requirements, which are possibly highly volatile, to experforming metric combinations give the highest predic- perienced developers in early phase of development. We tion performance when k-nearest neighbor algorithm is did not present other coverage results in this study due utilized. to page limitation.

MdMRE is zero for the following metric and ma- In Table 9 the best recall results are achieved chine learner combinations: Q&P&N+KNN, Q&P&N+RF, with Q&P&N+KNN(0.632), Q&N+RF(0.616) and Q&P&N+SVR, Q&P+KNN, P&N+KNN, Q&N+KNN, Q&P+KNN(0.604). Again, all best performing models Q&N+RF and Q&N+SVR. Number of change requests have requirement quality metrics in common, whereas for more than half of the software requirements are pre- the best combination consists of all metrics. The best dicted correctly with these models. Since many predic- accuracy results are obtained from Q&P&N+KNN(0.716),

Q&N+RF(0.703) and Q&P+KNN(0.694). The lowest false

alarm rate results are achieved by Q&P&N+KNN(0.232), Q&N+RF(0.242) and Q&P+KNN(0.249). K-nearest neighbor and random forest regression methods are successful in predicting highly volatile requirements for 80% change request coverage.

The most important measure for RQ 2 is recall since the purpose of this question is to measure success on predicting highly volatile requirements. We correctly identify 63.2% of highly volatile requirements which are exposed to 80% of the total requirement changes. Internal validity: In this study we present requirements volatility in a software project can be predicted to some extent utilizing requirement quality metrics, project specific factors and network metrics altogether. However, this does not imply causal relationship between input and output metrics since we did not conduct a controlled experiment.

External validity: We have conducted the case study on one project, so results have local validity. However, the dataset is quite large with more than 20,000 requirements from three distinct releases developed by many software developers. Nonetheless, applying the predictive models on diferent projects in the future would be better in terms of generalization of results.

Construct validity: Developers did not use their native language in software requirements. Thus there could be some typos which may afect textual requirement quality metrics. Also there could be some expressions used by developers in software requirements, e.g. subjective expressions that should have taken into consideration while creating requirement quality metrics but we missed. Due to the size of dataset we couldn’t manually check these kinds of typos and grammatical errors, but we know that reviewers are responsible for correcting those. We create network graphs based on traceability links between software and system requirements as indicated in the SRS. We could have use linguistic data to connect software requirements as previous study [12] and it may reflect relationship between requirements in a better way. We plan to do it as a future work.

Conclusion validity: For RQ 2 only the results of 80% change request coverage are presented due to page limitation. Regarding the results of other CR coverage, we observe higher recall and accuracy whereas false alarm rate grows undesirably as the coverage grows. Therefore RQ 2 results would difer in that way if we had chosen other CR coverage rate.

5. Conclusion and Future Work

In this paper, we have carried out an empirical study to predict number of changes per software requirement by using requirement quality measures, project specific factors and requirement interdependencies. 22,771 software requirements from a safety-critical software project in ASELSAN are utilized to build 28 prediction models and assess the best performing metric suite and algorithm. We conclude that we can predict volatility of requirements with an average MMRE of 36% by observing metrics of similar requirements through KNN. We also observe that measuring requirements from diferent aspects like quality, project and network dependencies gives a much better performance. We plan to integrate

References

such a predictor model into requirement management [10] X. Wang , C. Wu , L. Ma, Software project schedule

activity so that highly-volatile requirements could be au- 2010 IEEE International Conference on Advanced

tomatically and accurately identified . This way, software Management Science , volume 2 of ICAMS 2010 ,

development leads could take precautions beforehand to IEEE, Chengdu , China, 2010 , pp. 26 - 30 .

reduce requirements volatility related risks . Since there [11] T. Nakatani , T. Tsumaki, Predicting requirements

the best performing models . on Conceptual Modelling , volume 154 of APCCM 2014 , Auckland, New Zealand, 2014 , pp. 65 - 70 . [12] P. H. Hein , E. Kames , C. Chen , B. Morkos , Employ-

[1] N. Nurmuliani , D. Zowghi , S. Powell , Analysis of re- Design 32 ( 2021 ) 245 - 269 . quirements volatility during software development [13] W. Pedrycz , J. Iljazi , A. Sillitti , G. Succi , Prediction life cycle, in: 2004 Australian Software Engineering of the successful completion of requirements in Conference , ASWEC '04, IEEE, 2004 , pp. 28 - 37 . software development-an initial study , in: Agent

[2] G. Swathi , A. Jagan , C. Prasad , Writing software and Multi-Agent Systems: Technology and Applirequirements specification quality requirements: cations , Springer, 2016 , pp. 261 - 269 . An approach to manage requirements volatility , Int. [14] T. J. Ostrand , E. J. Weyuker , How to measure sucJ. Comp. Tech. Appl . 2 ( 2011 ) 631 - 638 . cess of fault prediction models , in: Fourth interna-

[3] R. Thakurta , A mixed mode analysis of the im- tional workshop on Software quality assurance: in pact of requirement volatility on software project conjunction with the 6th ESEC/FSE joint meeting, success , Journal of International Technology and SOQUA'07 , Dubrovnik , Croatia, 2007 , pp. 25 - 30 . Information Management 20 ( 2011 ). [15] T. Clancy , The standish group report, Chaos report

[4] A. M. Alsalemi , E.-T. Yeoh, A systematic literature ( 1995 ). review of requirements volatility prediction , in: [16] Y. Jiang , B. Cukic , T. Menzies, Fault prediction using 2017 International Conference on Current Trends early lifecycle data , in: The 18th IEEE International in Computer, Electrical, Electronics and Communi- Symposium on Software Reliability, ISSRE'07 , IEEE, cation, ICCTCEEC-2017, IEEE, 2017 , pp. 55 - 64 . 2007 , pp. 237 - 246 .

[5] A. Loconsole , J. Börstler , Construction and Valida- [17] G. Génova , J. M. Fuentes , J. Llorens , O. Hurtado, tion of Prediction Models for Number of Changes V. Moreno, A framework to measure and improve to Requirements , Umeå University Technical Re- the quality of textual requirements , Requirements port UMINF-07 .03, Umeå University Department of engineering 18 ( 2013 ) 25 - 41 . Computing Science, UMEÅ , SWEDEN, 2007 . [18] A. Faisandier , Systems architecture and design, Sin-

[6] D. F. X. Christopher , E. Chandra , Prediction of ergy'Com Belberaud, France, 2013 . software requirements stability based on complex - [19] T. W. Valente , K. Coronges , C. Lakon , E. Costenity point measurement using multi-criteria fuzzy bader, How correlated are network centrality meaapproach , International Journal of Software Engi- sures?, Connections 28 ( 2008 ) 16 - 26 . neering & Applications 3 ( 2012 ) 101 - 115 . [20] S. P. Borgatti , M. G. Everett , L. C. Freeman , Ucinet

[7] L. Shi , Q. Wang , M. Li , Learning from evolution his- for windows: Software for social network analysis, tory to predict future requirement changes , in: 2013 Harvard, MA: analytic technologies 6 ( 2002 ). 21st IEEE International Requirements Engineering [21] F. Eibe , M. A. Hall , I. H. Witten , The weka workConference, RE-2013, IEEE, 2013 , pp. 135 - 144 . bench. online appendix for data mining: practical

[8] A. Goknil , R. van Domburg, I. Kurtev , K. van den machine learning tools and techniques , in: Morgan Berg, F. Wijnhoven , Experimental evaluation of a Kaufmann , 2016 . tool for change impact prediction in requirements [22] A . Ben-Hur , J. Weston , A user's guide to support models: Design, results, and lessons learned, in: vector machines, in: Data mining techniques for 2014 IEEE 4th International Model-Driven Require- the life sciences , Springer, 2010 , pp. 223 - 239 . ments Engineering Workshop (MoDRE), MoDRE [23] D. Zhang , J. J. Tsai , Machine learning applications in 2014 , IEEE, Karlskrona, Sweden, 2014 , pp. 57 - 66 . software engineering, volume 16 , World Scientific ,

[9] B. Morkos , J. Mathieson , J. D. Summers , Compar- 2005. ative analysis of requirements change prediction models: manual, linguistic, and neural network , Research in Engineering Design 25 ( 2014 ) 139 - 156 .