This paper proposes a three-factor nonlinear regression model for Java-software size estimation. Java is one of the most widely used programming languages in the world and is actively used in the development of various software projects. The software size estimation plays a key planning role at the early stages of software project planning for the successful implementation of software development projects. The software size estimation is valuable information that is used to predict software development effort estimation using parametric models such as COCOMO, COCOMO II, COSYSMO, etc. The aim of the study is to build the three-factor nonlinear regression model for early KLOC size estimation of Java-software on the basis of multivariate Johnson SB family normalizing transformations to increase the reliability and accuracy of the software size estimation at the early stage of software project planning using UML class diagram metrics. The object of the study is the process of size estimation for open-source Java-software. The subject of the study is the nonlinear regression models to estimate the software size. To achieve this goal we collected software code metrics information from 571 open-source Java software projects. The obtained dataset is split up in training and testing samples accordingly Cross-Validation technique. The regression model building is based on an iterative method that includes bijective multivariate normalizing transformation, multivariable outliers detection, confidence and prediction interval building. The obtained nonlinear regression model is compared to the existing models by the regression models quality criteria such as the determination coefficient, mean magnitude of relative error (MMRE) and the percentage of prediction of the relative error level 0.25 (PRED(0.25)). The comparison results confirm increasing the accuracy of the software size estimation using the obtained regression model on both training and testing samples. The prospects for further research may include increasing the samples of Java-software code metrics, usage of different software code metrics, extending the set of independent factors and using different normalizing transformations for early KLOC estimation.
planning for the successful implementation of software development projects. The software size can be represented as functional points or number of code lines (KLOC - kilo lines of code). Both variants have their own advantages, drawbacks and restrictions. One of the crucial advantages of
include programming languages or software category [1] and KLOC parameter is widely used in software development effort estimation parametric models such as COCOMO, COCOMO II, SLIM [2], COSYSMO [3], etc. Software development effort estimation is one of the significant indicators of budget, resources and duration planning of any project for software development business. Reliable estimates of software development effort provide valuable information for project planning, and it helps to take into account risks, to recognize planning gaps and to increase the efficiency of the development process [4].
Java programming language is one of the most widely used in software development [5]. Java software varies from utility and micro web-service applications to huge information or automotive or operation system development. Java is object oriented programming (OOP) language and it executes in a separated environment - Java Virtual Machine that makes it a universal instrument for any purpose [6]. Java as an OOP language has compatibility with UML software projection therefore they both have a shared set of software code metrics such as quantity of classes, total quantity of methods and fields or depending on access modifiers, inheritance, association or complexity metrics.
projects have much higher failure rates in comparison with small, moderate or medium projects. Software development of most of the large projects is based on traditional project management methodologies like software development life cycle and waterfall, obviously it makes most of the parametric models more suitable for software development effort estimation than agile methodologies. Otherwise, company or team specific calibration of the parametric models that can work with uncertainty, such as COCOMO II and could provide reliable estimates for the projects that follow Agile methodol stages of development because of the complexity of integrating with highly distributed and different data and flows.
important task of the software project management life cycle and the share of failures or problems in software development is still high, lead us to confirmation the calculating requirement of reliable and accurate estimates of the size in code lines requires appropriate models for Java software to provide valuable information for project planning at the early stages.
estimation of Java-software on the basis of multivariate Johnson family normalizing transformations to increase the reliability and accuracy of the Java-software size estimation at the early stage of software project planning using UML class diagram metrics. The obtained math model should achieve higher quality criteria values in comparison with the existing math models of
equations and models for general Java-software or categories such as web-service, informationalsystems, etc on the metrics basis of the conceptual data model derived from a UML class diagram.
(CBO) and total quantity of fields per class (TFQ) and the [9] is based on CLASS, CBO and average value of TFQ per class and proposes better variant of the equation in comparison with [8].
Java-software size estimation depending on different variants of software code metrics. A good benefit of the models is a proposed prediction intervals for possibility to make expected, pessimistic and optimistic KLOC estimates. The model [10] is based on dataset from researches [8] and [9], and it can not guarantee robustness and accuracy of KLOC estimation due to Java language evolution and changing in modern software development practices that makes significant impact on software metrics previously and nowadays. The [10, 11, 12] regression models are based on restricted samples of Java applications that also have an impact on reliability and robustness of the nonlinear regression models. The latest research [13] proves the models [10, 11, 12] are restricted in estimation of code lines on different samples of code metrics from Java-software therefore the accurate result is not guaranteed. The [13] research offers one-factors and two-factor nonlinear regression models which were built using Cross-Validation technique. The regression models quality criteria such as the coefficient of determination R2, a mean magnitude of relative error MMRE and percentage of prediction for magnitude of relative error (MRE) level 0.25 PRED(0.25) reveal good quality level of the obtained two-factor model but the values of some quality criteria thresholds are not exceeded. Thus, study [14] confirms that with increasing sample sizes, the values of the quality criteria for the accuracy of predicting mathematical models for training and testing samples approach each other, which indicates the benefits of using large samples to build mathematical models in combination with Cross-Validation technique and adding more factors to nonlinear regression models allows to achieve higher accuracy accordingly quality criteria [13].
To achieve the aim of the research it is necessary to build the three-factor nonlinear regression
model on the basis of multivariate normalizing transformation using the original training and
testing non-Gaussian four-dimension samples of Java-software metrics to build and verify the
regression model. The training and testing sample includes the thousand lines of code (KLOC) Y,
the total number of classes X1, total quantity of unique methods invocations in the code (RFC
response for class) X2 and average value of public and protected methods per class (VMQ - visual
methods quantity) X3 from code metrics information of Java open-source software. The code
metrics information of projects does not intersect between training and testing samples. Suppose
there are bijective multivariate normalizing transformation of non-Gaussian random vector =
{ , 1, 2, . . . , } into Gaussian random vector = { , 1, 2, . . . , } is given by
= ( ), (
form = ( , 1, 2, 3, ), where is a Gaussian random variable that defines residuals, F is three-factor nonlinear regression models and Y is the dependent variable KLOC. 4. Materials, research methods and three-factor nonlinear regression model constructing 4.1. Three-factor nonlinear regression model constructing
platform (https://github.com) using the CK tool (https://github.com/mauricioaniche/ck). The following metrics were acquired: the actual total of Java-software size measured in number of thousand lines of code (KLOC), total quantity of classes (CLASS) X1, total number of unique method calls in classes (RFC) X2 and average value of visible methods per class of the software (aVMQ) X3.
conceptual model. The obtained dataset was randomly divided into training and testing samples with sizes of 286 and 285 rows of Java-software project data, respectively. The distribution of metrics relative to KLOC is shown in Figure 1 for training dataset and in Figure 2 for testing dataset.
level to avoid a high correlation between regression factors and sensitivity of the model to random changes in the data.
For a multiple regression model with factors Xi , = 1,2, . . . , k, the VIFs are represented by the diagonal elements of the inverse correlation × matrix. If the value of the VIF coefficient exceeds 10 (the threshold value), a high level of multicollinearity exists between the independent variables [15].
indicates the absence of multicollinearity between the factors of the regression model.
17] and it is based on detecting and discarding outliers in nonlinear regression analysis of non
detection, regression residuals distribution verification and prediction interval detection.
According to the technique it is recommended to detect and discard only one outlier in an iterative way once it is detected. In case if an outlier is detected, the technique starts from the first step using the modified sample without the detected outlier from the previous iteration. Otherwise the nonlinear regression model is successfully constructed. The first step of the technique is based on normalizing transformation.
= ̂ + ̂ (
− ̂ ̂ + ̂ − ), where respectively.
parameters of Johnson SB family normalizing transformation, < Х < + , > 0, > 0; - non-Gaussian random variable which X equals to Y, X1, X2, X3,
In the first step for normalizing the multivariate non-Gaussian data of the training sample, we
use the Johnson univariate and multivariate transformation for the SB family. It suggests a rich set
of parameters for accurate calibration of the transformation. In our case the Johnson normalizing
translation of SB family is given by [18]:
(
= ̂ + ̂ [1 + −( −̂ )/̂ ]−1, where Z is a Gaussian random variable which Z equals to ZY, ZX1, ZX2, ZX3, respectively.
Parameters of Johnson univariate and multivariate transformations (
{ 2 + (ℇ − 3)2 + ̅Z̅2̅ + ( 2 − 1)2 }, where is a vector of estimators,
= { , , , }; is skewness of a random variable, ℇ is kurtosis of a random variable; ̅= 1 ∑ =1 ; 2 = 1
∑ =1( − ̅)2; - i-th value of random variable Z of the sample size N. For the multidimensional data, we use measurement of multivariate skewness (β1, k) and kurtosis (β2, k) of the sample proposed in a Mardia [20] multivariate normality test.
Estimators for the parameters of multivariate transformation (
The second step includes outlier detection using squared Mahalanobis distance (MD) technique in the sample of normalized data. The squared Mahalanobis distance are elements on the main diagonal of the 2 matrix of size N x N. The MD formula is given by and is given by 2 = ( − Z̅) −1( − Z̅)
1 =1 =
∑( − Z̅)( − Z̅) , freedom. dimensional sample is given by
where Z is a normally distributed random variable = ( 1 , 2 , … , variance matrix and is a means vector of independent variable of the sample, Z̅ = ) , SN is a biased sample (̅Z̅1̅, ̅Z̅2̅, . . . , ̅Z̅k̅) .
The elements of the main diagonal 2, = 1,2, . . . matrix are detected as outliers if the values exceed the threshold value of the Chi-Square 2 distribution quantile for the significant level For the training sample 17 data rows were detected as outliers because their 2 values greater then threshold value 18.55 of the Chi-Square 2 for the significant level = 0.005 and 20 degrees of
= ̂ + = ̂ 0 + ̂ 1 1 + ̂ 2 2 + ̂ 3 3 + ,
where is Gaussian random variable, ∼
the linear regression model (
The estimators are calculated by the least square method: ̂ 0 = −23.784722, ̂ 1 = −0.005724, ̂ 2 = −0.005062, ̂ 3 = −0.008718 .
iterations respectively and the values are bigger than quantile 16.81 of the Chi-Square distribution
2 ); ̂ 0, ̂ 1, ̂ 2, ̂ 3 - estimators for parameters of
for 6 degrees of freedom.
iteratively.
(
= −1( ̂ + ) = −1( ̂ 0 + ̂ 1 1( 1)+ ̂ 2 2( 2)+ ̂ 3 3( 3)+ ),
(
In the sixth step we constructed prediction interval ̂
of the nonlinear regression model (
(
(
], where
For the training sample, 1 outlier is detected and discarded. For the obtained prediction interval, the values of normalized sample means Z̅1, Z̅
2 and Z̅3 are −1321.94070, −1447.061944 and
−1008.068369, respectively. The /2, = 2.594580 for significant level = 0.01 and 264
degrees of freedom; = 0.089394. The inverse matrix of (
The Gaussian distribution of the four-dimension training data set ( = 268) is verified by a multivariate normality test proposed by Mardia [20] at the latest iteration. The test is based on measurement of multivariate skewness (β1, k) and kurtosis(β2, k) of the sample.
4.2. Quality criteria of the obtained nonlinear regression model
The obtained nonlinear regression model (
=1( − ̂ ) ; + is a vector of central moments of independent where N - sample size and is the value of the magnitude of relative error for the i-th datapoint of the sample.
The calculation of prediction percentage (PRED) for the magnitude of relative error level 0.25 is given by (0.25) = 1 ∑ { =1 1 0 ≤ 0.25 ℎ .
≤ 0.25 and (0.25) ≥ 0.75 for the measurement of the regression models accuracy of prediction results. The coefficient of determination ( 2) value is acceptable if it is more or equals to 0.75 [21].
The obtained three-factor nonlinear regression model is tested with the quality criteria to assess
the predictive reliability and accuracy on the training and the testing samples and compared with
(
# 1 2 3 4 19.818 ̂ 40.095 208.967
Prediction intervals of the one-factor and three-factor nonlinear regression models are compared by given formula
∑ =1[ 1 − 1 ]
∑ =1[ 3 − 3 ]) ⋅ 100%,
(
To make visualization of the comparison of the one-factor nonlinear model and the three-factor nonlinear model accuracy we sorted all 571 data rows in ascending order by actual KLOC values of the multidimensional dataset with KLOC estimates and prediction intervals of the target models and first 400 records were chosen by x-axis and 150 KLOC max value was chosen by y-axis due to scaling. The KLOC estimates and prediction intervals were smoothed by moving averages technique on a basis of 5 values regarding the centric value of them. The obtained charts are demonstrated on Figure 3.
are closer to the actual KLOC values than the estimates of the one-factor nonlinear model.
nonlinear model on the basis of the same normalizing transformation.
testing sample confirms the model is not overfitted that confirms robustness and reliability. 2, and (0.25) values of training and testing samples are close to each other thus confirm the samples have a high level of representativity of the population of the open-source
software using appropriate techniques for constructing non-linear regression models on the basis of multivariate normalizing Johnson SB family transformation. The technique is chosen due to the non-Gaussian distribution of four-dimensional data of the code metrics according to Mardia criteria and the regression residuals according to the Chi-Square test.
Table 1 and models comparison in [13] confirms that the results of this research demonstrate
the effectiveness and accuracy of the proposed three-factor nonlinear regression model for early
KLOC estimation in comparison with existing linear and non-linear regression models by quality
criteria 2, and (0.25). The criteria for the model are exceeded expectation 2 ≥
0.75, ≤ 0.25 and (0.25) ≥ 0.75 for both training and testing samples. The values
are 2 = 0.9073, = 0.1645, (0.25)0.7692 for training sample and 2 = 0.9016,
= 0.1617, (0.25) = 0.8175 for the testing sample. Cross-Validation technique
increases guarantees of robustness of the model. Moreover, the width of prediction interval (
Java-software using three code metrics such as total quantity of classes (CLASS), total number of
unique method calls in classes (RFC) and average value of visible methods per class (aVMQ) that
could be obtained on the early stages of the project planning from UML class diagrams. The
disadvantages of the proposed model (
confirms that the KLOC estimates by the three factor model (
problem of Java-software size estimation at the early stages of software development project planning using UML class diagram metrics: total quantity of classes (CLASS), total quantity of responses for class (RFC) and average value of public and protected methods per class (aVMQ).
The scientific novelty of the obtained results is that the three-factor nonlinear regression model is improved in comparison with existing models and equation for KLOC estimation of Javasoftware; Multivariate Johnson SB family normalizing transformation is firstly used for model construction using CLASS, RFC and aVMQ multidimensional dataset; the three-factor nonlinear regression model is firstly built using large sample size ( = 286) and the robustness and reliability verified by using the testing sample with the similar size. The model, in comparison with other nonlinear regression models, has higher value of coefficient of determination 2, a lower value of the mean relative error , higher value of the percentage of prediction of the relative error level (0.25) for both training and testing samples and the prediction interval is smaller in comparison with one-factor nonlinear regression models. The obtainer quality criteria values confirm the training and the testing samples are close to each other thus verifies the samples have a high level of representativity of the population of the open-source Java-software
The practical significance of the obtained results allows us to recommend the built model for use in practice. The proposed model is implemented as a software product that can be used by project managers for Java software development effort estimation at early stages of project planning to reduce risks and cost.
The authors would like to express their sincere gratitude to the Doctor of Technical Sciences, Professor Sergiy B. Prykhodko, NUOS, Mykolaiv, Ukraine for his support in applying the mathematical apparatus for the research and to PhD. Maurício Aniche, Assistant Professor in Software Engineering at TU Delft, for providing valuable Java code static analysis tools CK.
doi:10.1109/TCSET49122.2020.235464. [18] P. M. Stanfield, J. R. Wilson, G. A. Mirka, N. F. Glasscock, J. P. Psihogios, J. R. Davis,
simulation conference WSC 96, Coronado, CA, USA, December 8-11, 1996, ed. S. Andradóttir,
1457-1464. [19] S. B. Prykhodko, Simulation of gaussian random variables using Johnson SU transform,
[20] K. V. Mardia, Measures of multivariate skewness and kurtosis with applications, Biometrika 57 (1970) 519 530. doi:10.1093/biomet/57.3.519. [21] D. Port, M. Korte, Comparative studies of the model evaluation criterions MMRE and PRED in software cost estimation research, in: Proceedings of the 2nd ACM-IEEE International
2008, pp. 51 60. doi:10.1145/1414004.1414015