Technical dept metaphor is widely discussed topic in research, but there is no common model on how to manage technical dept [3]. Companies invest a lot of money in maintenance and in commercial software system maintenance it is typical to have some penalties on missing the SLA deadline on bug resolution times. Adding technical dept can be understandable, in order to be quicker in markets with new features, but in order to manage it effectively, it is mandatory to understand the risks and impacts of the interest of it. In this research machine learning technology was used to evaluate whether SonarQube technical dept KPIs can be used to predict bug resolution times. The fact that the data was collected only from open source projects was limitation, but the results were encouraging. Accuracy approximately of 90% was reached. As it was seen that number of lines of code is also a valid indicator of bug resolution times, it was concluded, that it would be best to repeat this study in environment of commercial company which maintains many projects of similar size.
Technical dept metaphor is found attractive to practitioners as it communicates to
both technical and nontechnical audiences that if quality problems are not addressed,
things may get worse. In their research, Ernst et al. concluded that the even though
technical dept is widely known and accepted, making it visible and measurable is a
big gap in practice. Tools are installed, but the complexity of configuring them or
interpreting results meant that they were unused. Only very small minority of business
managers were actively managing technical debt. [
This paper focuses to investigate whether technical depth KPIs could be used together with machine learning in order to manage technical dept in software maintenance project in controlled manner.
Dataset [
Limitations of this paper include the fact that strict SLA policies are more common
in commercial software products under maintenance than in open source projects.
Also, the open source software products which were used in dataset [
However, results indicated, that machine learning methodologies could estimate quite accurately, when bug fixing times start to have big probabilities of growing too big and outside of SLA limits. When using ROC-AUC as measure of how good the estimation is, excellent correlation was found. When simple number of lines variable was removed from variables, still good and close to excellent AUC numbers were found. 2
Software maintenance has dramatically evolved in the last four decades in order to
cope with the continuously changing development models. Maintenance is also an
increasingly popular research topic, with an increasing number of new models and
approaches being proposed. Interestingly, the number of models proposed is
increasing rather than consolidating. The fact highlights that more research effort is needed
to identify reusable and tunable models that can be applied in different contexts.[
In their work Lenarduzzi et al., presented results of a Systematic Literature
Review, highlighting the evolution of the metrics and models adopted in the last forty
years. Key findings included, that there is increase in the size of the data analyzed.
One reason is due to the availability of an enormous open source code base that can
be easily used to build maintenance models. Despite the open source ideology, it is
nearly impossible to replicate the studies, because almost all papers are based on
private data sets and/or used custom tools that are not available to other researchers to
support the replication.[
Researchers and industry are adopting various tools for static code analysis to
measure technical dept and evaluate the quality of their code [
Empirical studies on software projects are expensive because it takes a lot of time
to analyze the projects. Also, the results are difficult to compare as studies commonly
consider different projects. In their work, Lenarduzzi et al [
At the level of IT management in industry, organizations are interested in the
economic consequences of technical debt and risks that they may pose. Technical debt
can decrease efficiency of running software systems and create difficulties in
extending them. Cost overhead in fixing issues or adding new functionality caused by
technical debt is considered as the interest of technical debt. In economic perspective,
technical debt is defined as the cost of repairing quality issues in software systems to
achieve an ideal quality level. An ideal quality level is the highest achievable level of
quality defined in a quality model adopted by an organization. The amount of debt is
the gap between the current and the ideal level. Interest is defined as the extra
maintenance cost spent for not achieving the ideal quality level. Maintenance activities
include adding new functionality and fixing bugs. Maintenance and technical quality
repair action is different in that former involves visible changes and their impacts are
immediately visible. Extra effort spent on new functionality or fixing bugs are
examples of interest on technical debt. Interest of technical debt is not the same as
maintenance costs, because systems without technical issues will still spend some effort on
maintenance.[
In software development and maintenance cost point of view, the earlier bugs are
found, the cheaper it is. On the other hand, companies want to be fast in market. But
faster time-to-market and quick user feedback also implies less time for testing and
bug fixing in early stages [
Code with bad quality is more expensive to maintain, but also refactoring is risky.
It requires changes to working code that can introduce subtle bugs. Refactoring, if
done wrongly, can set you back days or weeks. Refactoring becomes riskier when
practiced informally or ad hoc.[
If there is a way to apply machine learning techniques to identify problems earlier,
there is a good motivation to include it already in CI/CD pipeline, using the
SonarQube metrics and technical Dept information to forecast problems. This could help
organizations to balance between time to refactor an be fast in time-to-market. Using
the open dataset described collected from various open source projects [
If the technical dept KPIs can predict future bugs, those can be used as good indicator for refactoring purposes. RQ: Can SonarQube technical depth KPIs be used to estimate increasing bug resolution times?
In this study, the dataset [
Technical depth, by definition, impacts to the time which is used to add features or fix
bugs. In the used dataset [
The dataset [
Since number of lines of code is easy KPI to predict longer resolution times, it was decided to execute the python scripts with lines of code included as one KPI and then collecting another set of data with same methodology and taking away the number of lines of code, in order to verify if the results differ when the most evident single KPI which predicts bug resolution times was removed. 4
This study is using big dataset [
The research also relies on monthly averages of technical dept measures while it would be best to analyze the dept KPIs from the exact moment when bug is found. As the data comes from various phases of the products lifecycle, one can find very short bug resolution times, which are due to the fact which can be seen on commit dates compared to bug opening and closing times, that some bugs are probably found during development, fixed at the same time and the bug is reported afterwards. 5
First executions of the python scripts against the data was executed three times. The dataset was balanced so, that there was just as many very long bug resolution times included as there were bugs with resolution times less than 30 days. Random forest classifier algorithm was used. Three executions gave following results:
Area under curve (AUC) shows that the random forest classifier gave predictions,
which were between 90 and 92% accurate. This gives an idea, that technical dept
KPIs do have strong impact to the bug resolution times. However, one of the used
KPIs from dataset [
One can make conclusion from the results, that with the help of machine learning classifiers one can build a model to predict SLA violations. Research question was:
tion times? Research shows that technical dept KPIs can be used for this purpose, however, technical dept KPIs by themselves did not work as good as used together with simple number of lines of code KPI. Results are encouraging, taking into concern that all bugs were treated equal and that sizes of the projects did not impact the results too much.
It is obvious that the size of the project impacts on the bug resolution times. Also, it is normal that severity of the bug impacts on SLAs on commercial products. Therefore, in following research the projects themselves should be classified by size and bugs separately based on severity. Then machine learning models could be trained against the set which includes data from similar sized projects. This way the model it could be possible to build logic in CI/CD pipeline to predict SLA violations. Further research is needed to replicate similar study in environment, which includes several similarly sized commercial software products under maintenance. As a result, it could be possible to build model which gives good reasons for refactoring.
Technical dept, by definition, impacts to the time which is used to add features or fix
bugs [
Dataset, which consist of 33 projects, approximately 1.9M SonarQube issues, 38K
code smells, and 28K faults [
Results showed, that the model can predict bug resolution times to pass or meet 30 days threshold quite accurately. With all the selected KPIs included, the model predicted correct results with 90 – 92 percentage accuracy. Lines of codes was obvious easy single KPI to predict the resolution times, since it is assumed that the bigger the project is, the more time it takes to make changes. When the impact of number of lines of codes was taken away, accuracy remained high, 86 – 88 percentages. Research concludes, that machine learning models could be used to predict software products bug resolution times.
Limitations were seen also. Even the dataset [