=Paper=
{{Paper
|id=Vol-1469/paper3
|storemode=property
|title=Toward Measuring Defect Debt and Developing a Recommender system for their Prioritization
|pdfUrl=https://ceur-ws.org/Vol-1469/paper3.pdf
|volume=Vol-1469
}}
==Toward Measuring Defect Debt and Developing a Recommender system for their Prioritization==
https://ceur-ws.org/Vol-1469/paper3.pdf
Toward Measuring Defect Debt and Developing a
Recommender system for their prioritization∗
Shirin Akbarinasaji
Data Science Lab
Ryerson University
Toronto, Canada
shirin.akbarinasaji@ryerson.ca
ABSTRACT Every project requires to be completed and delivered under
Software development managers make a release decision with- certain constraints. Project management body of knowl-
out fully resolving the defects from current and previous re- edge modeled these constraints as the iron triangle of time,
leases due to tight deadlines. Deferring the defects would cost and quality (scope). Figure 1 illustrates the project
accumulate a tremendous amount of technical debt in the management triangle. Typically, changing one side of the
system. Typically, the defect debts are defined as the type triangle has an effect on the other side of the triangle. For
of defect that should be fixed. However, due to compet- instance, increasing the quality (scope) leads to the growth
ing priorities and the limited amount of time and resources, in time and cost. Similarly, tight time (budget) constraints
they would be postponed to the next release. In order to may cause increasing of the budget (time) and reducing the
aid practitioners ,who make release decisions, to observe the quality. Complex software projects certainly need to tackle
amount of debt, there is a need for quantifying the defect the triple project constraints as well. The quality of soft-
debt. Software bug repositories roughly provide us with in- ware often diminished over time since the software mainte-
formation about the amount of time the defect debt exist in nance projects are performed under tight time and resource
the system, the time the defects are resolved and the sever- constraints. The project managers need to make balance
ity of the defect. We suggest categorizing the defect into between time and cost properties and the required quality
the regular defect and debt prone defect by analyzing this level. To achieve the system equilibrium, they might delay
information. Afterwards, we compare the regular defect and some maintenance activities such as documentation, testing
debt-prone defect to determine the principal, interest and or even fixing bugs. The consequence of delaying these tech-
interest probability of defect debt. We also propose the re- nical development activities can be interpreted as a type of
inforcement learning for scheduling which defect debt needs debt and it will affect the long term maintenance and devel-
to be paid and when they need to be paid. opment activities. The trade-off between short term benefits
of delaying these activities and long-term effect of postpon-
Categories and Subject Descriptors ing them is articulated as “technical debt”.
H.4 [Information Systems Applications]: Miscellaneous;
D.2.8 [Software Engineering]: Metrics—complexity mea-
sures, performance measures
General Terms
Theory
Keywords
Technical Debt Measurement, Defect Debt, Reinforcement
Learning, Software Maintainability
Figure 1: Project Triangle
1. INTRODUCTION
∗Copyright ©2015 for this paper by its authors. Copying Generally, the technical debt describes the delayed devel-
permitted for private and academic purposes. opment activity due to the time and resource constraints.
Likewise the financial debt, principal is an amount of effort
in terms of time or cost require paying off the debt (i.e com-
plete the task). Interests are the potential penalty in terms
of extra amount of effort required to pay as a compensation
for what is borrowed [24]. Accumulation of technical debt in
the system has tremendous effect on the quality of the sys-
tem [24]. There is always a choice between paying down the
whole principal or continuing to pay off the interest. And,
in small software projects, the managers may implicitly de-
cide on the amount of the debt to be paid off and payment
15
schedule. However, in the large projects, there is a need for quantifying the defect debt and we seek feedback on our
a comprehensive system that is able to track and manage research approach and our proposed solution.
the technical debt [19].
Additionally, technical debts may occur intentionally or un- 3. RELATED WORK
intentionally [14]. Intentional debts are the kind of debts The technical debt was first introduced two decades ago by
that the developers and managers are aware of their exis- Ward Cunningham [3]. He described that “Shipping first
tence and they occur deliberately due to the strategic and time code is like going into debt. A little debt speeds devel-
tactical mission. Unintentional debts occur due to the de- opment so long as it is paid back promptly with a rewrite”.
velopers'lack of attention or understanding and the team is Although it is a fairly recent metaphor in software engineer-
not aware of their existence and their location. Further- ing, it is highly related to well-researched issue like software
more, the debts may also be classified according to its type. decay by Lehman and Belady [11] and software aging by
According to Li et al. technical debts are classified into 10 Parnas [17] . Software decay is a complexity of software
types of debts including: requirement, architectural, design, due to continuous changes and software aging is a disability
code, test, build, documentation, infrastructure, versioning of software to meet required changes. The introduction of
and defect debt[13]. In this study we particularly focus on technical debt facilitates managing these concepts.
defect debts which refer to the defects, bugs or failures found
but not fixed in the current release[19]. Martin Fowler [7] suggested the quadrilateral categorization
of technical debt considering intention (deliberate or inad-
Since, quantifying the technical debt is a key factor in mak- vertent) versus awareness (reckless or prudent). Brown et al.
ing decision about incurring, paying off and deferring tech- [1] extended the concept of technical debt from the code level
nical debt, we will initially propose a new approach for mea- metaphor to architectural and detailed design. Since then,
suring the principals and interests of defect debt. Most of the many practitioners and researchers relied on the definition
quantifying approaches in the literature are based on the dis- of the technical debt in order to explain various cost-drive
tance between the violation of the code from the ideal code. issues in software engineering. Tom et al. [24] explored
However, our approach differentiates from the existing ap- the technical debt concept focusing on dimension of tech-
proaches since it concentrates on mining of bug repositories nical debt, the advantages and drawbacks of allowing them
and collecting data from issue tracking system. in a system and its origins. In another study, Tom et al.
[23] did a systematic literature review focusing on the state
On the other hand, software development team always faces of academic research for technical debt. Li et al. [13] did a
the high volume of defect reports and change requests in systematic mapping study on technical debt and its manage-
every cycle of release. The challenge is to figure out which ment. Seaman and Guo [19] reviewed some issues associated
instance of defects should be addressed in this release and with technical debt and proposed the framework for mea-
in which order they are required to be fixed. Therefore, an suring and monitoring the debt based on risk management
important next step is recommending a system to the de- approaches. Their framework for managing technical debt
velopers and managers to prioritize instances of defect debt was identifying debt, measuring debt and monitoring debt.
in the current release. Our proposed solution for scheduling For identifying debt, especially code-based debt, there are
of the defect debt is based on reinforcement learning. Our various tools. They are designed to detect code smell [18],
ultimate goal is finding an optimal action-selection policy modularity violation [25], grime buildup [9] and potential
for paying off the defect debt in limited time in such a way defect. Zazworka et al. [27] found that different tools did
that minimizing the amount of interest. We can summarize not reveal overlapping results, rather they each pointed out
our contribution as follows: a different problem.
Letouzney [12] proposed the SQALE (Software Quality As-
• Proposing a new approach for quantifying the defect sessment based on Lifecycle of Expectations) for evaluat-
debt, ing the technical debt. Sonar tool1 is a popular tool for
• Developing an automated framework for prioritization evaluating the technical debt but it is not perfect as it is
of defect debt widely discussed in technical debt literature [5]. Sonar ap-
plied quality heuristic for identifying code duplication, vio-
lation of code standard, lack of testing, and potential latent
The remainder of the paper is as follows: in section 2, we bugs. Nord et al. [15] presented a metrics based on architec-
will briefly explain the current research issues and what we ture and measurement approach for managing the technical
would like to get advice on. Section 3 will review the related debt. Nugroho et al. [16] redefined the technical debt as
work. In section 4, we will specifically discuss our research fixing cost of the technical issue. They performed static
objectives. Section 5 is a brief explanation of research ap- analysis to identify debt and estimated the debt principle
proach. Finally, section 6 will review next step and section based on the percentage of changed line code, code duplica-
7 is conclusion and summary. tion, dependency and parameter count and also complexity
metric. They calculated the interest estimation by assigning
2. CURRENT RESEARCH ISSUES these metrics to risk categories. Guo et al. [8] investigated
Although the technical debt attracts interests of many re- the effect of technical debt in a real software project by es-
searchers in recent years, most of the work only capture the timating the principal debt according to effort estimation.
theoretical aspect of the technical debt and there is still a Zazworka [28] applied cost-benefit analysis for prioritizing
lack of empirically based studies in the literate. In this re-
1
search, we would like to propose an empirical approach for http://www.sonarsource.org/)
16
the God class debt. He ranked the cost of paying debt and Jifeng et al. referred to the defect debt by another terminol-
the impact of debt on quality to determine which refactoring ogy as “debt prone bugs” and described it as any software
activities need to be performed initially. Singh et al. [20] bugs which remain in the system because of any immature,
monitored developer activities to estimate the debt interest. incomplete process of fixing the bugs[26].
Several studies proposed considering interest probability for
interest estimation [2], [6]. This is the probability of debt, if From the perspective of defect debt definition, the bugs are
not paid, how it would affect other tasks in a negative way divided into the bugs which constitute the technical debt
[19]. There are also several studies that are presented and and the bugs which do not form any debt. Hereafter, we
classified the technical debt by mapping study by [13]. refer to the latter as regular bugs. Therefore, two type of
bugs exist in any bug repositories:
Xuan et al. proposed the concept of debt-prone bugs and
identified three types of debt :tag bugs, reopen bugs and du-
plicate bugs. Tag bugs are the bugs which “fixme, todo, xxx” • Regular bugs which refer to the bugs that are submit-
tags annotated to them. Reopened bugs are the bugs which ted and resolved in the same release
are solved by developers but reopen later, and duplicate bugs • Debt prone bugs which refer to the bugs that are sub-
are the bugs with the same root as existing ones.They con- mitted in one release but are not resolved in that re-
ducted a case study on Mozilla to investigate effect of debt lease
prone bugs on software quality. [26].Snipes et al. identified
and categorized cost related to fixing the defects or deferring
the defects. They stated that decision factor for managing 5.2 Measuring defect debt principal and in-
defects from the technical debt perspective are: severity, an terest
existence of a workaround, an urgency of fix required by In management level of any software organization, they are
customer, the effort to implement the fix, the risk of the interested in achieving a quantitative understanding of time
proposed fix and the scope of testing required[21]. constraints for fixing the bugs. However, the time con-
straints that debt prone bugs may impose to the system
In this study,we particularly concentrate on defect debts and is totally different from the regular bug fixing time. The
we propose how to measure the principal and interest and fixing time for debt prone bugs is the summation of the
interest probability of defect debts. However, the main con- standard amount of time required for fixing the bugs and
cern of development team is how to prioritize defect debts in an extra amount of time required to fix them as a penalty
order to minimize the total amount of interest. We propose of deferring the process. In other word, this fixing process
reinforcement learning to schedule debt prone bugs pay off. for debt prone bugs may exceed the standard fixing time for
For future step, we plan to collect data from bug reposi- regular bugs because the accumulation of unresolved bugs
tory of a real project to show the feasibility of our proposed in the system would make it more complicated to repair the
model. defects. Will Snipes et al. identified the following type of
time (cost) for fixing defects[21]:
4. RESEARCH OBJECTIVE
The general objective of this study is to propose an approach
that provides the developers and practitioners to make a de- • Investigation time or the time of diagnosing, verifying
cision under uncertainty during project management regard- and finding the alternative solution for defects.
ing the defect debt. In particular, the goals of this proposal • Modification time which refers to the time of applying
can be summarized as following: the solution to fix defects
• Work around time that deals with providing the bypass
• To propose a simple and straightforward approach for for defects which are not resolved immediately
measuring the principal and interest of any defect prone
bug instance: In most of the previous studies, the tech- • Customer support time is the time of providing sup-
nical debt has been measured by comparing the current port for the customer because of the defect that exists
state of the code and the ideal target for the code. In in the current release
this study, we would like to propose a new approach to
measure the technical debt by mining the bug reposi- • Patch time or the time of finding temporary solution
tory. for fixing the bugs
• To develop a framework for automating the prioritiza- • Validation time that refers to the time of testing the
tion of defect debt: Once technical debt is measured, systems.
the next step is to provide a guide to practitioners a
decision technique and approach to determine whether
or not to pay off the debt at a particular point of time. Investigation, modification and validation time are incurred
for fixing the regular bugs. However, postponing the fix-
ing process of the bugs may impose additional time such as
5. RESEARCH APPROACH workaround, customer support time and patch time to the
5.1 Definition of Defect Debt system as well. Besides, the investigation cost may increase
Defect debts refer to the defects, bugs or failures found but because of the accumulation of more defects make the diag-
not fixed in the current release, because there are higher pri- nosis process more complicated[21]. As mentioned earlier,
ority bugs to fix or there are limited resources to fix them. technical debt principal is the amount of effort in terms of
17
time that are required for fixing bugs. And the interest is are dependent to current bugs need to be divided by the
an extra amount of time as a type of penalties which are number of all existing bugs in the current release.
required for fixing the defect. Consequently, principal value
for debt prone bugs may include three components: investi- 5.3 Applying Reinforcement Learning for Pri-
gation, modification and validation time. The interest value
may compromise additional workaround, customer support
oritization of Defect Debt
Reinforcement learning is a machine learning task of find-
time and patch time.
ing an optimal action-selection behavior of an agent in or-
der to maximize the total amount of reward[22]. Reinforce-
In order to help software development team to better esti-
ment learning is inspired by a concept in psychology “re-
mate if the defect debt can be absorbed in current release or
inforcement”. In behavioral psychology, reinforcement is a
not, a fair approximation for principal and interest are re-
consequence that will motivate the agent behavior by of-
quired. Suppose that two similar bugs with the same charac-
fering specific stimulus followed by the behavior. Basically,
teristic such as severity, same assignment, and same product
reinforcement learning is a problem of an agent who inter-
exist in bug repositories. One bug is treated as a regular bug
acts with the environment and tries to achieve an optimal
and one is treated as debt prone bug. For any regular bug,
goal. Reinforcement learning has many applications in dif-
we can easily estimate the regular fixing time by retriev-
ferent domains: robotic control, scheduling, chess playing,
ing the historical data from bug tracking system. In any
backgammon, etc. [10]. Reinforcement learning is based
bug tracking system, the bugs are submitted by develop-
on Markov Decision Process (MDP) and the components of
ers and testers with an ID, description of bugs, its version,
basic reinforcement learning model are as following:[22]:
the reported date and its severity. The developers who are
interested in working on those bugs may assign different la-
bels to bugs such as new, unconfirmed, reopened and etc. • a set of actions a ∈ A
They also contain an adequate information regarding the
last time the bugs are modified and the history of modifica- • a set of states in the environment s ∈ S
tion. The fixing time for regular bugs is equivalent to the • a set of transactions between the states
difference between reporting time of the bugs and the re-
solving time of the bugs. Therefore, we are able to build a • The rules that determine an immediate reward for any
prediction model based on the data from regular bug fixing transition (r)
time to predict the principal for debt prone bugs. The po- • The rules that determine what the agent are able to
tential input features for feeding the prediction model would observe from an environment
be severity, submitter, owner, priority, the indicator if the
bug internally discovered or externally discovered, etc.
The agent interacts with an environment over potentially
In order to evaluate the interest for debt prone bugs, ini- infinitive discrete time steps of t = 1, 2, 3, . The time step
tially the variance between the fixing time and the estimated is not necessarily fixed interval time, it can be determined
principal needs to be calculated. The interest amount is based on point of time that the state change or new action
proportional to this variance. The weighting factor for ad- has been performed[4]. At each time, the agent based on
justing that is called severity. Severity is the potential effect state of an environment chooses an action at . Then, the
of defect in the functionality of the system and customer agent receives a reward based on chosen action and will be
requirement[21]. For instance, one reason for deferring the transferred to new state st+1 . The agent can evaluate its
bugs fixation to the next release is that the bug is very triv- performance based on the received reward. The reward de-
ial. Suppose the customer is requesting changes related to fines the goal in the learning problem. The Policy is the
background color of the system. In this case, the impact rule agent uses to select actions and for each state it assigns
of the defect is a minor irritation. Therefore, the severity a probability to each possible action. The agent tries to
of the bug is very low and postponing the defect for many adjust policy in order to maximize the accumulated reward
years may not affect the functionality of the system. There- [4]. Figure (2) depicts a standard framework of reinforce-
fore, the interest amount for this particular bug is the vari- ment learning.
ance between the regular fixing time and estimated principal
times the severity of the bugs. From this point of view, we
can conclude that the interest amount is the time difference
between principal and real fixing time multiplying the co-
efficient based on severity. Fundamentally, the more severe
bugs pose more interests to the system if they are not fixed
in the same release as they reported. Therefore, an interest
amount can be calculated as below:
InterestAmount = (RealF ixingT ime−EstimatedP rincipal)
∗ Severity
Figure 2: Reinforcement learning Framework
Another component of the interest is the interest probability
which refers to the probability that defect debt, if not fixed,
will negatively affect the fixing time of other defects[19].For
Prioritization of defect debt can be modeled as a reinforce-
the sake of estimating interest probability, all the bugs that
ment learning. The main reason is that in scheduling of de-
18
fect debt, we are facing a changing environment and, there- Where B is the long-term reward in the system, π is the
fore, a traditional fixed policy schedule would not work in action (policy) the developers decide to take. r is a reward
such a fluctuating environment. Additionally, we are not at each state and 0 < γ < 1 is the discount factor which
able to anticipate the long-term consequence of debt prone needs to be determined initially. The optimum decision is
bugs in the system. In software defect detection, practition- to optimize the long-term reward:
ers are looking for recommendations beyond classification.
Identifying and recommending a course of action for defect π(s) = argmaxBπ (s)
debt is a complex task that traditional classification algo- The optimal π would return the sequence of the defect debt
rithms may not suffice. A classification algorithm takes a that developers should fix to save the maximized amount of
given data to learn and build a model for making a single interest.
prediction or decision. However, human learning takes place
in three phases, classification, memorization and procedural
by interacting with the environment in making decisions. 6. PLANNED NEXT STEPS
When building recommender systems in complex environ- In order to check the feasibility of our approach, we need
ments such as software defect debt prediction, we need to to apply it in real world projects and analyze the results.
use learning techniques such as reinforcement learning where Therefore, we plan our next steps of this research as below:
an algorithm considers the effect of actions by trying things
to learn. It is different than dynamic programming as we do The initial next step is collecting data from both open source
not know the effects of actions in the case of prioritization projects and also commercial software companies and ap-
of defect debts. Since technical debt is hidden in the code plying our proposed techniques for quantifying their defect
and we do not know its future effect, we need a learning al- debt. We also need to consult with developers in each do-
gorithm that considers this uncertainty and learns from its main to assess our approach. The second step is determin-
environment by interacting with it. Such learning is more ing which reinforcement learning method is best situated for
realistic and closer to how humans learn, but it is difficult our model such as temporal difference learning, Q-learning,
to implement as a learning algorithm. SARSA , etc. The last step is to extend our approach to
other types of technical debt.
The potential framework for prioritization of debt is as fol-
lows: Suppose that the developer is the agent in the model 7. CONCLUSION
that should adjust policy in order to maximize the amount
In this study, we described a new procedure for measur-
of interest we could save. A set of states in environment
ing the principal, interest and interest probability for defect
might be the amount of time remaining before upcoming
debt. Despite most of the existing approaches which are
release. A set of actions is whether to pay debt i in the
based on the violation of the program from the right code
upcoming release or not. Reward is equal to amount of in-
practice, our proposed method is based on mining the bug
terest we save if we pay the debt i at current time instead
repository and standardizing the measurement based on the
of postponing it to the next release. Note that after fixing
actual status of the system. Our proposed approach presents
a defect debt bugs, the target is achieving zero (minimum)
a simple heuristic for practitioners with limited knowledge
amount of debt before the release or maximizing amount of
about the architecture of the code in quantifying the defect
interest that we saved. We assume that the developers are
debt. Typically, we concentrate on the bugs that are opened
fully aware of the current state of the model. They have
and resolved in the same release and use them as an original
all the required information about how much debt is in the
required fixing time. Then, by comparing the defect debt
system and how much time they have to work on debt prone
prone to the regular defect, we calculate the interest.
bugs.
Furthermore, we would like to develop a novel learning frame-
Assume that in each release the developer team decides to
work for scheduling of the defect debt. Our proposed method
assign a specific amount of time T to fix some defect debts.
is reinforcement learning because of the dynamic nature of
At t = 1 the developer face totally N defect debts to be
the defect debt environment. We believe that determining
fixed. He needs to decide which debt to pay at time t = 1.
which defect debt to work on is a very complicated problem
Suppose that the developer decides to fix defect debt i and
and developers need an automated framework that suggest
it would take x1 minutes to fix it. He would save an interest
them a sequence of defect debt they can fix in limited time
of r(s1 ). At t = 2, the remaining time is equal to T − x1 .
that can return the maximum amount of reward to them.
As, defect debt i has been fixed in the system, it would
effect the number of defect debt we may face in the system
since it will facilitate fixing of some other bugs which are 8. ACKNOWLEDGMENT
dependent in defect debt i. For instance, the number of I would like to express my sincere thanks to Dr. Ayse Bener
defects may change to N + m − 1. Suppose, at time t = 2 and Dr. Bora Caglayan for providing me with their valuable
the developer decides to fix defect debt j and it would take advice and guidance. I am very grateful for their support
x2 . The remaining time would be T − x1 − x2 and he would and feedback that help me to improve my proposal.
save r(s2 ). The developer will continue till he reaches time
T . He needs to make the decision based on the long-term
reward the system would return to him. The expected long-
9. REFERENCES
[1] N. Brown, Y. Cai, Y. Guo, R. Kazman, M. Kim,
term rewards at each state s for policy ,π, would be:
P. Kruchten, E. Lim, A. MacCormack, R. Nord,
I. Ozkaya, et al. Managing technical debt in
Bπ (s) = Exp[r(st ) + γ ∗ r(st+1 ) + γ 2 ∗ r(st+2 ) + ...|st = s] software-reliant systems. In Proceedings of the
19
FSE/SDP workshop on Future of software engineering 1–8. ACM, 2011.
research, pages 47–52. ACM, 2010. [17] D. L. Parnas. Software aging. In Proceedings of the
[2] Y. Cai, R. Kazman, C. Silva, L. Xiao, and H.-M. 16th international conference on Software engineering,
Chen. A decision-support system approach to pages 279–287. IEEE Computer Society Press, 1994.
economics-driven modularity evaluation. [18] J. Schumacher, N. Zazworka, F. Shull, C. Seaman, and
Economics-Driven Software Architecture, 2013. M. Shaw. Building empirical support for automated
[3] W. Cunningham. The wycash portfolio management code smell detection. In Proceedings of the 2010
system. ACM SIGPLAN OOPS Messenger, ACM-IEEE International Symposium on Empirical
4(2):29–30, 1993. Software Engineering and Measurement, page 8. ACM,
[4] M. K. P. DARSINI. Application of Reinforcement 2010.
learning algorithms to software verification. PhD [19] C. Seaman and Y. Guo. Measuring and monitoring
thesis, M. Sc. Thesis, Universite Laval, 2006. technical debt. Advances in Computers, 82:25–46,
[5] R. J. Eisenberg. A threshold based approach to 2011.
technical debt. ACM SIGSOFT Software Engineering [20] V. Singh, W. Snipes, N. Kraft, et al. A framework for
Notes, 37(2):1–6, 2012. estimating interest on technical debt by monitoring
[6] C. Fernández-Sánchez, J. Dı́az, J. Pérez, and developer activity related to code comprehension. In
J. Garbajosa. Guiding flexibility investment in agile Managing Technical Debt (MTD), 2014 Sixth
architecting. In System Sciences (HICSS), 2014 47th International Workshop on, pages 27–30. IEEE, 2014.
Hawaii International Conference on, pages 4807–4816. [21] W. Snipes, B. Robinson, Y. Guo, and C. Seaman.
IEEE, 2014. Defining the decision factors for managing defects: A
[7] M. Fowler. Technical debt quadrant. Bliki [Blog]. technical debt perspective. In Managing Technical
Available from: http://www. martinfowler. Debt (MTD), 2012 Third International Workshop on,
com/bliki/TechnicalDebtQuadrant. html, 2009. pages 54–60. IEEE, 2012.
[8] Y. Guo, C. Seaman, R. Gomes, A. Cavalcanti, [22] R. S. Sutton and A. G. Barto. Reinforcement learning:
G. Tonin, F. Q. Da Silva, A. L. Santos, and C. Siebra. An introduction, volume 1. MIT press Cambridge,
Tracking technical debtâĂŤan exploratory case study. 1998.
In Software Maintenance (ICSM), 2011 27th IEEE [23] E. Tom, A. Aurum, and R. Vidgen. A consolidated
International Conference on, pages 528–531. IEEE, understanding of technical debt. In ECIS, page 16,
2011. 2012.
[9] C. Izurieta and J. M. Bieman. How software designs [24] E. Tom, A. Aurum, and R. Vidgen. An exploration of
decay: A pilot study of pattern evolution. In technical debt. Journal of Systems and Software,
Empirical Software Engineering and Measurement, 86(6):1498–1516, 2013.
2007. ESEM 2007. First International Symposium on, [25] S. Wong, Y. Cai, M. Kim, and M. Dalton. Detecting
pages 449–451. IEEE, 2007. software modularity violations. In Proceedings of the
[10] L. P. Kaelbling, M. L. Littman, and A. W. Moore. 33rd International Conference on Software
Reinforcement learning: A survey. Journal of artificial Engineering, pages 411–420. ACM, 2011.
intelligence research, pages 237–285, 1996. [26] J. Xuan, Y. Hu, and H. Jiang. Debt-prone bugs:
[11] M. M. Lehman and L. A. Belady. Program evolution: technical debt in software maintenance.
processes of software change. Academic Press [27] N. Zazworka, C. Izurieta, S. Wong, Y. Cai,
Professional, Inc., 1985. C. Seaman, F. Shull, et al. Comparing four approaches
[12] J.-L. Letouzey. The sqale method for evaluating for technical debt identification. Software Quality
technical debt. In Proceedings of the Third Journal, 22(3):403–426, 2014.
International Workshop on Managing Technical Debt, [28] N. Zazworka, C. Seaman, and F. Shull. Prioritizing
pages 31–36. IEEE Press, 2012. design debt investment opportunities. In Proceedings
[13] Z. Li, P. Avgeriou, and P. Liang. A systematic of the 2nd Workshop on Managing Technical Debt,
mapping study on technical debt and its management. pages 39–42. ACM, 2011.
Journal of Systems and Software, 101:193–220, 2015.
[14] S. McConnell. Technical debt. 10x software
development. Blog]. Available at: http://blogs.
construx.
com/blogs/stevemcc/archive/2007/11/01/technicaldebt-
2. aspx,
2007.
[15] R. L. Nord, I. Ozkaya, P. Kruchten, and
M. Gonzalez-Rojas. In search of a metric for managing
architectural technical debt. In Software Architecture
(WICSA) and European Conference on Software
Architecture (ECSA), 2012 Joint Working IEEE/IFIP
Conference on, pages 91–100. IEEE, 2012.
[16] A. Nugroho, J. Visser, and T. Kuipers. An empirical
model of technical debt and interest. In Proceedings of
the 2nd Workshop on Managing Technical Debt, pages
20