Reliability is an important factor but is hard to asses it during the development of an application. In this paper we propose an approach to predict reliability changes of object oriented applications using object oriented metrics from multiple versions of an application. Further on, after computing this criterion, the current work investigate how the new criterion that we propose correlates with changes in object oriented metrics using statistical methods. Analyzing results from the correlation, we can see that there is a strong correlation between our criterion and WMC, RFC and LOC metrics, a moderate correlation with CBO, LCOM5 and a very weak correlation with NOC metric. The approach was applied to seven projects (a total number of 1.462.178 classes from 678 total versions), with sizes ranging from small to medium, in order to assure statistical relevance and project diversity (not resumed to a single project with several versions).
Complex systems needs to be developed within time and budget, are usually done incrementally over a long period of time and, after delivering them, developers have to maintain and improve them based on the new requirements.
Fore some of these systems (like safety critical systems for example) one of the most important quality factor to be considered is reliability, which expresses the degree to which a system is free of faults or can recover to a stable state after failure. Nina S Godbole points out that Quality goals must be clearly defined, effectively monitored and rigorously enforced [Godbole 2004] in order to deliver reliable software systems.
Being hard to asses reliability during the development cycle of a software, the assessment of reliability is usually done at the end of the cycle so that corrections to improve it cost more time and money. Also is very hard to find a metric that computes exactly the reliability of a software during the development phase and current methods of predicting reliability are mainly focusing on complexity of the software without taking into consideration the facts that object oriented programming imposes a special structure of the code and special interactions between constructs are not taken into consideration.
Since, using the current IDEs available, is easy to compute object oriented metrics for a version of a software and see how the metrics evolve between multiple versions we focused on finding a method to relate this changes in metrics to changes in reliability. After proposing our method of predicting reliability changes based on OO metrics, we applied statistical methods in order to validate if the changes in the metrics correlates with the changes in our criteria of predicting reliability changes.
The paper is organized as follows: Section 2 contains preliminary information about reliability, object oriented metrics used in our research and related work. In Section 3 we present our approach of 19:2 predicting reliability changes using OO metrics and the results of our case study. In the last section we conclude the paper and discuss future work.
Starting with the [McCall et al. 1977], Reliability has been introduced as a quality factor, and then it has been present as a software quality factor as shown in Table I.
SQ Model McCall Boehm FURPS ISO 9126 Dromey ISO 25010
Due to the fact that applications have evolved and changed over time, the subfactors that characterized it changed, in order to capture the relevant aspects and requirements of applications. As part of reliability in earlier models, accuracy, which represents the measure of required precision in computation, was present, but has lost its importance in the newer models, being replaced by maturity and recoverability.
In ISO 25010, availability is also recognized as a key characteristic of software, capturing the aspect that software should be ready to be used and accessible. Looking at nowadays mobile and web applications the importance of availability is even more important.
”Degree to which a system, product or component meets needs for reliability under normal operation”
[
Object oriented metrics provide information about code and help developers in discovering design
flows and are widely used in software development. There are multiple tools that allows us to
compute the metrics, some integrated in IDEs or frameworks and some independent, that compute OO
met
Until now a lot of metrics have been proposed and new metrics continue to appear in the
literature regularly. Some examples are the metrics proposed by [Abreu 1993], [Abreu and Rogerio 1994],
[Chidamber and Kemerer 1994], [Li and Henry 1993], and the MOOD metrics proposed in [Abreu
1995]. [Ma
In the current study we have chosen the following metrics: WMC, CBO, RFC, LCOM5, NOC and
LOC that you can find bellow as they are defined by Sou
The chosen metrics were selected starting from the metrics proposed in CK Metric
The systematic literature review provided in [Jabangwe et al. 2015] showed that the most investigated quality characteristic is reliability and most of the studies are linked to fault-proneness.
WMC, DIT, RFC, NOC and CBO from CK Metrics appear to be useful to predict class fault-proneness during the high- and low-level design phases of the life-cycle, as presented in [Basili et al. 1996].
In MetricsReloaded plugin from Intellij [
QCP RLBT Y = N + (2 N EST ) + (3 V (g)) + BRAN CH + +CON T ROL + EXEC where: N is the Halstead Length metric for a method, NEST is the maximum nesting depth of each method, V(g) is the cyclomatic complexity of each non-abstract method, BRANCH is the total number of non-structured branch statements in each method, CONTROL is total number of control statements in each method and EXEC is the total number of executable statements in each method.
Relationship between existing class level OO metrics and fault proneness over the multiple releases
is inve
[Lincke and Lowe 2006] links software quality factors to object oriented metrics, based on the semantics of the syntactical constructions that are involved in evaluation of the factor and the computation of the metric. The approach had resulted in a Compendium [Lincke and Lowe 2013] containing 37 quality attributes (factors and criteria) and 23 metrics, together with their influences.
Neural network are also used to predict software readiness. In [
This paper proposes a new method for assessing reliability changes in a project using OO metrics in object oriented software. The main focus of this research is to introduce a way to measure reliability using object oriented metrics - WRC and to see if changes of this indicator correlates with object oriented metrics that we took under consideration.
We selected the metrics that we use when defining our own criterion, based on the information from
ARISA Compendium, the experience that we gathered when trying to asses Maintainability changes
a
Pm2class C M RELIABILIT Y (m) no of methods in class C
W M C where:
M RELIABILIT Y = HP L + (2 N L) + (3
M cCC) + N OS where: HPL is the Halstead program length metric for a method; NL is the complexity of the method expressed as the depth of the maximum embeddedness of its conditional, iteration and exception handling block scopes; McCC is complexity of the method expressed as the number of independent control flow paths in it; NOS is number of statements in the method.
After computing WRC for each class from each version of the projects, we used statistical analysis
using R Studio in order to see if the variations of WRC correlates with WMC, CBO, LCOM5, RFC, NOC
and LOC met
For this the first step was to run the Shapiro-Wilk test [Royston 1982], for each of the variables
to see if they are normally distributed. Now because the Shapiro-Wilk test is limited to a number of
maximum 5000 observations, we also tested normal distribution with Anderson-Darling nor
Gathering data for study was not easy, since there are no databases from where metrics from multiple
versions of projects to be considered. Moreover, it’s hard to find data from commercial projects, and
because of this we moved our attention to the open source projects. The first thing that we did was
to select some projects from GitHub of different sizes and with different number of releases and to
retrieve the releases of those projects using our own script. Project used in our study are log4j, [
After this the next challenge that we faced was to automatically compute the OO metrics that we
needed in our research. For this we searched for a tool that can do this and after trying multiple
tools we stopped at SourceMeter since it was the most suitable for our needs. Using SourceMeter we
computed WMC, CBO, LCOM5, RFC, NOC and LOC metrics for our project and then using our own tool
computed WRC for each class. Table III shows the summary of the metrics computed by SourceMeter
and the summary for our WRC criterion computed with our own tool for log4j [
Using R Studio we gathered all the classes from all versions of the projects in one table and starting the correlation analysis. After running normal distribution tests (using the Shapiro-Wilk and Anderson-Darling normality test) for WRC, WMC, CBO, LCOM5, RFC, NOC and LOC we determined that all variables are not normally distributed (for all the p-value < 2.2e-16). For example the distribution of WMC can be seen in Figure 2.
Since the variables are not normally distributed we continue the analysis using the Spearman’s correlation and tested the correlation between WRC and object oriented metrics. Results of the analysis can be seen in Table V.
Analyzing results from the correlation, available in Table V we can see that there is a strong correlation between our method and WMC, RFC and LOC metrics, a moderate correlation with CBO, LCOM5 rho WRC-WMC 0.7428248 WRC-CBO 0.4281653 WRC-LCOM5 0.4279371 WRC-RFC 0.7427667 WRC-NOC 0.09204933 WRC-LOC 0.780197 and a very weak correlation with NOC metric. Looking at the p-value, we can see that it is lower than 0.05 so this means that the correlations are statistically significant.
Changes in WMC, RFC and LOC metrics will strongly impact the Reliability, CBO and LCOM5 will have a moderate impact and changes in NOC will have a weak impact.
In [Lincke and Lowe 2013] WMC, CBO, RFC, NOC and LOC are categorized as related to Reliability and LCOM is Highly Related. We can see that some correlations from ARISA Compendium are also reflected in the present study, but there are some changes that might result from the fact that different tools are used to compute the metrics and values differ from tool to tool, or from the fact that data from the study is limited to the projects that we managed to gather and extending the data might allow us to find a better correlation. In order to draw a stronger conclusion further study needs to be done using different tools and more data (as discussed also in the future work section).
The selection of metrics was done to ensure that key characteristics of the software are covered, but a further extension of this selection is required to be investigated in order to better capture the impact of each metric. Also since there are differences between how each tool compute the metrics and because gathering enough data is a challenge another threat to validity is the size of the sample data.
The paper propose an approach to predict reliability of object oriented applications using object oriented metrics from multiple versions of an application.
We proved that correlations can be found between Reliability and OO metrics using statistical analysis. Analyzing results from the correlation, we showed that there is a strong correlation between our method and WMC, RFC and LOC metrics, a moderate correlation with CBO, LCOM5 and a very weak correlation with NOC metric.
The approach was applied to seven projects, with sizes ranging from small to medium, in order to assure statistical relevance and project diversity (not resumed to a single project with several versions).
Regarding the future work one of the aspects that needs to be covered is extending the selection of
metrics and also the size of the sample data. Also the method of predicting reliability can be further
improved by taking in consideration the thresholds of the metrics and weighting more the values of
metrics that are outside of the thresholds. The approach can be extended to other quality factors and
this can be the subject of a different research, for example to validate and improve the approach to
assess the maintainability, using several object oriented metrics a