This paper proposed an information technology to predict code properties based on software design metrics, underscoring the critical interplay between metrics and software code properties. A meticulous case study leveraging data from 39 open-source Java projects demonstrates the efficacy of machine learning methodologies, including random forest and artificial neural networks, in predicting code properties utilizing selected design metrics. The study reveals insights into the correlation between design metrics and lines of code (LOC), suggesting the feasibility of using design metrics for LOC forecasting and, by extension, various software characteristics. The findings emphasize the importance of prioritizing generalizability over specificity to enhance the model's reliability across diverse software engineering contexts. Overall, this paper advances our understanding of the significance of design metrics in forecasting code properties, providing valuable insights into their application within software engineering practices to mitigate risks and enhance software quality. Through these contributions, this research lays a solid foundation for further exploring and utilizing design metrics in software development processes.
Software quality assurance issues are becoming increasingly important with the spread of software into different spheres of industry and life and with the increasing variety of software types. Therefore, the quest for quality assurance remains paramount. However, achieving these objectives requires more than post hoc debugging and testing; it necessitates proactive measures to predict and preempt potential issues before they manifest. This is where the concept of quantitative prediction emerges as a pivotal tool in the arsenal of software engineers. By leveraging design metrics, architectural insights, and empirical data, engineers could anticipate how a software system will behave under different conditions, enabling them to ∗ Corresponding author. † These authors contributed equally optimize its design, allocate resources judiciously, and mitigate potential bottlenecks before deployment.
The importance of performance prediction cannot be overstated in today's software development landscape. In modern IT environments, the traditional approach of reactive troubleshooting proves inadequate, if not impractical. Performance prediction offers a proactive strategy to address these challenges, empowering stakeholders to make informed decisions at every stage of the software development lifecycle.
At the heart of prediction is the premise that specific design metrics are reliable software
quality indicators [
For instance, studies have shown that high coupling levels between modules tend to increase
the propagation of defects and decrease the system's resilience to change [
Advances in machine learning and data analytics enable the development of predictive models that leverage historical data to forecast software quality metrics. We can identify underlying trends and accurately predict future outcomes by training these models on repositories of code artefacts, bug reports, and performance logs.
Predicting software performance characteristics based on design metrics holds immense significance. It offers tangible cost reduction, risk mitigation, and stakeholder satisfaction benefits. Software engineers can avoid costly rework, delays, and customer dissatisfaction by identifying potential performance issues early in the software lifecycle. Either limited or absent historical data typically hinders this endeavour. Existing datasets detailing software defects typically lack measurements about the design aspect. Secondly, datasets focusing on design components are often gathered on a per-project basis and lack extensive measurements.
This paper explores methodologies for utilizing indirect metrics to forecast software performance characteristics during the software design phase.
The paper is structured as follows. Section 2 provides an overview of the published research on which our work is based. In section 3, we describe information technology for prediction. Section 4 describes the application of the proposed information technology. Finally, the general conclusions of the work are collected in Section 5.
Given the paper's focus on software design metrics, we thoroughly analyzed articles about this category of metrics. It's important to highlight that our review exclusively included articles published after 2018.
The paper [
In [
The authors of [
The paper [
Silva et al. [
Special consideration should be directed towards the research [
The paper [
The paper [
The paper [
The paper [
In summarizing the published research, it became evident that all researchers acknowledged the dependency between design metrics and software code properties. The disparity lies in the approaches employed to leverage or model this dependency. Armed with this understanding of the correlation, we can now develop information technology for predicting code properties based on design metrics.
However, we must point out that few studies link design metrics to software characteristics. The main body of publications is devoted to using code characteristics, mainly the number of lines of program code metrics, to predict different software properties.
Lines of code (LOC) is a fundamental software code measure widely used as a proxy for
software development effort or as a normalization factor in many other software-related
measures [
The most developed topic is software defect prediction (SDP). Pradhan et al. [
However, other applications were described. The experimental results in [
Based on the analysis performed, we can assume the feasibility of LOC forecasting based on software design metrics. Predicting various software characteristics based on LOC is a wellstudied task.
Let us generalize the published insights. Information technology aims to predict significant software quality features during the design phase, reducing uncertainty in software projects.
Input data consists of measurements on metrics that describe the software design. Various
authors have proposed different sets of metrics, with the most comprehensive list presented in
[
A natural constraint on the choice of input and output data is the availability of historical data on the selected metrics. After all, a predictive machine learning model must be trained on historical data.
Information technology comprises two core procedures: model identification and prediction utilizing the model. Both procedures are widely recognized and nearly standard in machine learning. Figure 1 outlines a schematic representation of these procedures.
A sufficient accuracy level of quality feature prediction under specific conditions could always be determined. Depending on this, a particular predicting technique should be chosen.
It's important to note that while the technology itself is significant, the primary focus of our work lies in the results yielded by the case study.
The developed information technology underwent testing using data from 39 open-source Java projects. Data measurements were collected for individual components across 86 metrics within each project. Specific metrics exhibit explicit dependencies, such as the correlation between the number of LOC and kilo lines of program code (KLOC).
Figure 2 illustrates data from three randomly chosen projects plotted in the space defined by the first two principal components. It's important to highlight that the component characteristics of these projects are notably similar, indicating that the datasets from individual projects can be effectively merged.
From the available set of metrics, we used only those related to the software design, namely • • • • • • •
Number of Attributes (NOA), Number of Parameters (NOP), Number of Children (NOC), Coupling Between Objects (CBO), Depth Inheritance Tree (DIT), Response for a Class (RFC),
Lack of Cohesion of Methods (LCOM5).
The datasets provided did not include measurements of metrics directly tied to the quality of a software product, such as the quantity or presence of defects. Nevertheless, it has been established that the number of defects tends to be proportionate to the number of LOCs. This dependency was utilized in our analysis.
Initially, we explored existing linear dependencies by examining the correlation matrix. Consequently, we identified only two significant linear dependencies with LOC: CBO and RFC. Therefore, we opted not to utilize linear regression. Instead, we employed random forest (RF), AdaBoost, and artificial neural network (ANN) to model the regression dependence.
We utilized the Mean Absolute Percentage Error (MAPE) to assess the prediction quality. In our experiment, the AdaBoost method yielded the poorest result, with an error exceeding 100%. The Random Forest (RF) method achieved a MAPE of 40.64%, while the Artificial Neural Network (ANN) allowed for a reduction of MAPE to 30.49%. However, it's important to note that these results alone do not definitively establish neural networks as the superior prediction method.
It's well-known that achieving quality results with neural networks often requires training the model on extensive datasets. Our experiment validated this notion. When the neural network was trained solely on data from one project, the prediction error surpassed 100%, which is comparable to AdaBoost's MAPE. Conversely, the RF and AdaBoost methods exhibited consistent performance across different training dataset sizes—whether trained on data from one, multiple, or all projects.
In summarizing the results obtained, it became apparent that the regression model yielded forecasts of low quality. However, it's worth noting that our primary concern wasn't forecasting LOC itself. Instead, we focused on utilizing LOC to infer quality characteristics. Consequently, we could establish varying "risk" levels based on module sizes measured by LOC and transition to a classification task. Figure 3 depicts the frequency distribution of LOC values.
Most modules contain up to 75 LOC, suggesting that modules of this size typically exhibit acceptable quality. Conversely, there are a few extensive modules, which likely present challenges. Hence, we propose categorizing modules with LOC>75 as risky.
As it was realized for regression, we used RF, AdaBoost, and ANN methods for the classifier. The results obtained were almost the same (Table 1).
We observe that the ANN demonstrated slightly inferior performance, likely attributed to the inadequacy of the sample size used. For practical applications, RF is preferable due to its consistent performance across datasets of varying sizes and its simplicity compared to the AdaBoost method.
Therefore, reducing the specificity of the outcome criteria enables the development of a model with satisfactory prediction accuracy.
This inference remains valid irrespective of the programming language utilized or the type of project under consideration. By broadening the scope of outcome criteria, the model becomes more adaptable and capable of providing reliable predictions across various contexts.
This flexibility enhances the model's utility and ensures its applicability in diverse software engineering scenarios. Consequently, prioritizing generalizability over specificity enhances the model's effectiveness and reliability in predicting outcomes.
The importance of performance prediction cannot be overstated in modern software development environments. Traditional reactive troubleshooting approaches often prove inadequate or impractical. Performance prediction offers a proactive strategy to address these challenges, empowering stakeholders to make informed decisions at every stage of the software development lifecycle.
Advances in machine learning and data analytics have facilitated the development of predictive models that leverage historical data to forecast software quality metrics. We can identify underlying trends and accurately predict future outcomes by training these models on repositories of code artefacts, bug reports, and performance logs.
Predicting software quality based on design metrics offers tangible benefits, including cost reduction, risk mitigation, and stakeholder satisfaction. By identifying potential performance issues early in the software lifecycle, software engineers can avoid costly rework, delays, and customer dissatisfaction.
Reviewing the literature, we find a wealth of research exploring the relationship between design metrics and software quality. Various studies have investigated the efficacy of different metrics and modelling techniques in predicting software reliability, identifying design smells, and assessing the impact of design patterns on software metrics.
Our information technology aims to generalize these insights and provide a framework for predicting significant quality features during the design phase, thereby reducing uncertainty in software projects. By utilizing historical data on design metrics, we can develop predictive models that offer valuable insights into the future performance of software systems.
In our case study, we tested the developed information technology using data from 39 opensource Java projects. By analyzing metrics related to software projects and employing regression and classification techniques, we sought to predict software quality characteristics. Our results highlight the importance of training predictive models on extensive datasets and the potential limitations of specific modelling techniques.
Ultimately, our findings underscore the value of predictive modelling in software engineering, offering a proactive approach to software quality assurance. Engineers can make informed decisions and optimize software designs by leveraging design metrics and historical data for improved performance and reliability.