=Paper=
{{Paper
|id=Vol-1519/paper4
|storemode=property
|title=A Decision Support Platform for Guiding a Bug Triage for Resolver Recommendation Using Textual and Non-Textual Features
|pdfUrl=https://ceur-ws.org/Vol-1519/paper4.pdf
|volume=Vol-1519
|dblpUrl=https://dblp.org/rec/conf/apsec/SurekaSBMK15
}}
==A Decision Support Platform for Guiding a Bug Triage for Resolver Recommendation Using Textual and Non-Textual Features==
https://ceur-ws.org/Vol-1519/paper4.pdf
A Decision Support Platform for Guiding a Bug
Triager for Resolver Recommendation Using
Textual and Non-Textual Features
Ashish Sureka, Himanshu kumar Singh, Manjunath Bagewadi, Abhishek Mitra, Rohit Karanth
Siemens Corporate Research and Technology, India
Abstract—It is largely believed among researchers that the that reviews the incoming bugs and decides who to assign it to,
software engineering methods and techniques based on mining of or whether more information is required, or a bug is irrelevant,
software repositories (MSR) have the potential of providing sound or behavior needs to be observed more. As can be imagined
and empirical basis for Software Engineering tasks. But it has
been observed that the main hurdles to adoption of the techniques that the decisions made by the CCB are knowledge intensive
are organizational in nature or people centric, for example lack and it requires prior knowledge about the software system,
of access to data, organizational inertia, general lack of faith in expertise of the developer, team structure and composition and
results achieved without human intervention, and a tendency of developer workload. In some instances, in order to optimize
experts to feel that their inability to arrive at optimal decisions the time of the entire CCB, a pre-CCB is conducted by
is rooted in someone else’s shortcomings, in this case person
who files the bug. We share our experiences in developing a individual members of the board on assigned subset of bugs
use case for applying such methods to the common software and the individual recommendations are reviewed in complete
engineering task of Bug Triaging within an industrial setup. We CCB. The average time to triage a bug in such a process can
accompany the well researched technique of applying textual be captured as following:
information content in bug reports with additional measures
in order to improve the acceptance and effectiveness of the Tpre−CCB ∗ M + TCCB
system. Specifically we present: A) use of non-textual features t=
for factoring in the decision making process that a human would N
follow; B) making available effectiveness metrics that present a
basis for comparing the results of the automated systems against Here, t denotes the average time it takes to triage a bug,
the existing practice of relying on human decision making; and given M committee members taking Tpre−CCB time individu-
C) presenting reasoning or the justification behind the results so ally for assessing their subset of bugs and TCCB time together
that the human experts can validate and accept the results. We to discuss and finalize recommendation for N bugs. From the
present these non-textual features and some of the metrics and
discuss on how these can address the adoption concerns for this
above, its clear that any method that can assist in reducing
specific use case. any of M, TCCB and Tpre−CCB has potential to increase
Index Terms—Bug Fixer Recommendation, Bug Triaging, Issue overall efficiency. Research shows that manual assignment of
Tracking System, Machine Learning, Mining Software Reposito- bug reports to resolvers without any support from an expert
ries, Software Analytics, Software Maintenance system results in several incorrect assignments [1][2][3][4][5].
Incorrect assignment is undesirable and inefficient as it delays
I. P ROBLEM D EFINITION AND A IM the bug resolution due to reassignments. While there has been
Bug Resolver Recommendation, Bug Assignment or Triag- recent advancements in solutions for automatic bug assign-
ing consists of determining the fixer or resolver of an issue ment, the problem is still not fully solved [1][2][3][4][5]. Fur-
reported to the Issue Tracking System (ITS). Bug Assignment thermore, majority of the studies on automatic bug assignment
is an important activity both in OSS (Open Source Software) are conducted on OSS data and there is a lack of empirical
or CSS/PSS (Closed or Proprietary Source Software) domain studies on PSS/CSS data. In addition to lack of studies on
as the assignment accuracy has an impact on the mean time to Industrial or Commercial project data, application of non-
repair and project team effort incurred. Bug resolver assign- textual features such as developer workload, experience and
ment is non-trivial in a large and complex software setting, collaboration network for the task of automatic bug assignment
especially with globally distributed teams, wherein several is relatively unexplored. The work presented in this paper is
bugs may get reported on a daily or weekly basis increasing the motivated by the need to develop a decision support system for
burden on the triagers. One of the primary ways, identification bug resolver recommendation based on the needs of Triagers.
of resolvers for open bug reports is normally done is through The specific aims of the work presented in this paper are:
a decision by Change Control Board (CCB), members of 1) To build a decision support system for guiding and as-
which represent various aspects of the software project such as sisting triagers for the task of automatic bug assignment,
project management, development, testing and quality control. this involves application of textual (terms in bug reports)
The CCB usually works as a collective decision making body to build a classification model
3rd International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2015) 23
2) Using of non-textual features (components, developer a multi-step processing pipeline from data extraction (from
workload, experience, collaboration network, process the issue tracking system) as back-end layer to display as
map) for contextualizing the model. the front-end layer. As shown in Figure 2, we implement
3) Provide insights about the bug fixing efficiency, defect adaptors to extract data from Issue Tracking System (ITS)
proneness and trends on time-to-repair through visual used by the project teams and save into a MySQL database.
analytics and a dashboard. We create our own schema to save the data in our database
4) Build the system in user centric manner by providing and implement functionality to refresh the data based on a
the justification and reasoning behind the recommended pre-defined interval or triggered by the user. Bug reports
assignments. consist of free-form text fields such as title and description.
Rest of the paper is structured as follows: in Section II we We apply a series of text pre-processing steps on the bug
discuss and argue that user centric approach to build such report title and description before they are used for model
recommendation systems incorporates the elements necessary building. We remove non content bearing terms (called as
to address the above goals. Next we discuss some of the stop terms such as articles and propositions) and apply word
contextualization measures for the model, specifically use of stemming using the Porter Stemmer (term normalization). We
a practitioner survey results and process map. In Section IV, create a domain specific Exclude List to remove terms which
describes some of the metrics and measures that accompany are non-discriminatory (for example, common domain terms
the system are how can they be used. Section V presents early like bug, defect, reproduce, actual, expected and behavior).
results from applying the system on two sets of data obtained We create an Include List to avoid splitting of phrases into
from actual industrial projects, one active for 2 years whereas separate terms such as OpenGL Graphics Library, SQL Server
other for 9 years. and Multi Core. We first apply the Include List and extract
important phrases and then apply the domain specific exclude
II. U SER CENTERED D ESIGN AND S OLUTION list. Include and Exclude Lists are customizable from the User
A RCHITECTURE Interface by the domain expert. The terms extracted from
We create a User-Centered Design considering the objec- the title and description of the bug reports represents dis-
tives and workflow of CCB. Our main motivation is to ensure a criminatory features for the task of automatic bug assignment
high degree of usability and hence we give extensive attention (based on the hypothesis that there is a correlation between
to the needs of our users. Figure 1 shows a high-level overview the terms and the resolver). The next step in the processing
of the 4 features incorporated in our bug assignment decision pipeline is to train a predictive model based on the Machine
support system. We display the Top K recommendation (k is Learning framework. We used Weka which is a widely used
a parameter which can be configured by the administrator) Java based Machine Learning toolkit called for model building
which is the primary goal of the recommender system. In and application. We embed Weka within our system and invoke
addition to Top K recommendation, we present the justification its functionality using the Java API. We train a Random
and reasoning behind the proposed recommendation. Forest and Naive Bayes classification model and use a voting
We believe that displaying justification is important as the mechanism to compute the classification score of the ensemble
decision maker needs to understand the rule or logic behind the rather than individual scores to make the final predictions. We
inferences made by the expert system. We display the textual also extract the component of the bug report as a categorical
similarity or term overlap and component similarity between feature as we observe a correlation between the component
the incoming bug report and the recommended bug report as and the resolver.
justification to the end-user. We show developer collaboration
network as one of the output of the recommendation system. In terms of the implementation, we create an Attribute-
The node size in the collaboration network represents the Relation File Format (ARFF) that describes the list of train-
number of bugs resolved, edge distance or thickness represents ing instances (terms and components and predictors and the
the strength of collaboration (number of bugs co-resolved) resolver as the target class). As shown in the Figure 3, we
and the node color represents role. As shown in Figure 1, extract the developer collaboration network, information on
we display the developer workload and experience to the prior work experience with the project and workload from the
Triager as complementary information assisting the user to ITS. The ITS contains the number of bugs resolved by every
make triaging decisions. Figure 1 illustrates all four factors developer from the beginning of the project. The ITS also
influencing triaging decisions (Top K Recommendation, Justi- contains information about the open bugs and the assignees
fication and Reasoning, Collaboration Network and Developer for the respective open bug. We use close and open bug
Workload and Experience) which connects with the results status information and the assignees field to compute the prior
of our survey and interaction with members of the CCB in experience of a developer and the current work load with
our organization. Figure 2 shows the high-level architecture respect to bug resolution. Similarly, the collaboration network
illustrating key components of the decision support system. between developers is determined by extracting information
We adopt a platform-based approach so that our system can from the bugs lifecycle. The frond-end layer implementation
be customized across various projects using project based consists of D3.JS, JavaScript, Java Servlet and Java Server
customization and fine-tuning. The architecture consists of Pages (JSP).
3rd International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2015) 24
Fig. 1. A High-Level Overview of the Features: Top K Recommendation with Confidence or Score Value, Justification or Reasoning behind the
Recommendation, Collaboration Network between the Developers, Workload & Prior Experience Values
Fig. 2. High-Level Architecture Diagram displaying Key Components (Front-End, Back-End and Middle Tier) - A Platform-Based Architecture
III. M ODEL C ONTEXTUALIZATION AND P ROCESS product manager, solution architect, quality assurance leader,
PARAMETERS developers, and testers. The survey respondents had been in
various roles and active members of bug triaging process.
Since our goal is to solve problems encountered by the Hence the survey responses are from representatives in-charge
practitioners and model the system as closely as possible to the of various aspects such as development, quality control and
actual process and workflows of CCB, we conduct a survey of management. The objective of our survey was to gain insights
experienced practitioners to better understand their needs. We on factors influencing the change boards triaging decisions.
conduct a survey of 5 senior committee members belonging
to Change Control Board (CCB) of our organizations soft- IV. P RACTITIONER ’ S S URVEY
ware product lines. The average experience (in CCB) of the Figure 3 shows the 5 questions in our questionnaire and
respondents was 7.5 years. In our organization, a CCB consists the responses received. Each response is based on a 5 point
of members belonging to various roles: project manager, scale (1 being low and 5 being high). Figure 3 reveals that
3rd International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2015) 25
V. D ECISION S UPPORT S YSTEM U SER I NTERFACE
A. Recommendation and Settings
Figure 5 shows the snapshot of the decision support system
displaying the Top K recommendation, score for each recom-
mendation, prior work experience and the current work load
of the proposed resolver. Figure 5 also shows the collabora-
tion network of the developers. Nodes in the collaboration
network can be filtered using the check-boxes provided in
the screen. The confidence values shown in Figure 5 are
probability estimates for each of the proposed resolver. The
Fig. 3. Survey Results of Practitioners in Industry on Factors Influencing
sum of the confidence values or probability estimates across
Bug Resolver Recommendation Decision [EX: Resolver Experience with the all possible resolvers (and not just the Top K) sum up-
Project, WL: Resolver Workload, CP: Bug Report Component, TD: Bug to 1. We display the probability estimates and not just the
Report Title and Description, PS: Bug Report Priority and Severity]
rank to provide additional information on the strength of the
correlation between the resolver and the incoming bug report.
Figure 6 shows a snapshot of the settings page consisting of
five tabs: resolvers, components, and training duration, include
and exclude list and train model. We describe and provide a
there are clearly multiple factors and tradeoffs involved in screenshots for one of the tabs due to limited space in the
making a triaging and bug assignment decision. We observe paper. We apply a platform-based approach and provide a
that bug report title and description and the available resolvers configurable settings page so that the decision support system
experience with the project are the two most important factors can be customized according to specific projects. As shown
influencing the triaging decision (both having a score of 3.8 out in Figure 6, a user can add, rename and modify components
of 5). The priority and severity of the bug as well as component component names. A software system evolves over a period
assigned to the bug are also considered quite important with a of time and undergoes architectural changes. New components
score of 3.4. The current workload of the resolvers as a criteria get added, components gets merged and renamed. We provide
influencing bug triaging decision received a score of 2.4 out a facility to the user to make sure that the model built on the
of 5 which is the lowest amongst all the 5 factors. The survey training data is in-synch with the software system architecture.
results support our objective of developing a bug resolver Similar to component configuration, we provide a tab to
recommendation decision support system based on multiple customize resolver list. For example, if a developer has left
factors (such as priority and severity of the bug report and the organization then its information can be deleted through
current workload of the available resolvers) and not just based the Resolver tab and ensure that his or her name is not shown
on matching the content of the bug report with the resolved in the Top K recommendation. The training instances and the
bug reports of fixers. amount of historical data on which to train the predictive
model can also be configured. The predictive model should
be representative of the current practice and hence we provide
We present our case study on a real-world project using the
a facility for the user to re-train the model based on recent
IBM Rational ClearQuest as Issue Tracking System. Clear-
dataset.
Quest keeps track of entire bug lifecycle (from reporting to
resolution), state changes and comments posted by project B. Visual Analytics on Bug Resolution Process
team members. We consider three roles: Triager, Developer In addition to the Top K recommendation, justification,
and Tester. A comment can be posted and the state of a bug developer collaboration network [6], developer prior work
can be changed by Triager, Developer and Tester. Figure 4 experience and current workload, we also present interactive
shows 9 possible states of a bug report in ClearQuest and visualizations on the bug resolution process. Francalanci et
the 81 possible transitions. A comment (in the ClearQuest al. [7] present an analysis of the performance characteristics
Notes Log) consisting of state transition from Submitted to (such as continuity and efficiency) of the bug fixing process.
In-Work contains the Triager and the developer role (from They identify performance indicators (bug opening and closing
and to field). Similarly, In-Work to Solved state transition trend) reflecting the characteristics and quality of bug fixing
contains the developer and testers IDs. We parse ClearQuest process. We apply the concepts presented by Francalanci et al.
Notes Log and annotate each project member ID with one of [7] in our decision support system. They define bug opening
the three roles: developer, tester and triager. We then remove trend as the cumulated number of opened and verified bugs
tester and triager and consider only the developers as bug over time. In their paper, closing trend is defined as the
resolvers for the purpose of predictive model building. This cumulated number of bugs that are resolved and closed over
step of inferring developers is crucial since, triagers and testers time [7][8].
frequently commit on the bug reports and their comments Figure 7 displays the opening and closing trend for the Issue
should not skew the results. Tracking System dataset used in our case-study. At any instant
3rd International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2015) 26
Fig. 4. List of 9 States in a Bug Lifecycle and 81 Possible Transitions. Infeasible Transitions are Represented by −. Each State Transitions is Used to Infer
Roles within the Project Team. [TRG: Triager, DEV: Developer, TST, Tester]
Fig. 5. A Snapshot of the Bug Resolver Recommendation Decision Support Tool displaying the Top 10 Recommendations, Confidence Value or Score,
Workload and Past Experience with the Project
Fig. 7. Graph Depicting Bug Fix Quality as the Extent of Gap between the
Bug Opening and Bug Closing Trend or Curve
Fig. 8. A Combination of a Heat Map and a Horizontal Bar Chart displaying
Three Dimensions in One Chart: Component, Number of Bugs and Duration
to Resolve the Bug
of time, the difference between the two curves (interval) can
be computed to identify the number of bugs which are open at
that instant of time. We notice that the debugging process is of duration to resolve each reported bug. We observe that the
high quality as there is no uncontrolled growth of unresolved bug fixing time for the Atlas Valves component is relatively
bugs (the curve for the closing trend grows nearly as fast or on the lower side in comparison to the sDx component. UBE,
has the same slope as the curve for the opening trend). Volume Review and Workflow are the three components on
Figure 7 shows a combination of Heat Map and a Horizontal which maximum numbers of bugs have been reported. The
Bar Chart providing insights on the defect proneness of a information presented in Figure 8 is useful to the CCB as the
component (in-terms of the number of bugs reported) and the bug resolver recommendation decision is also based on the
3rd International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2015) 27
Fig. 6. A Snapshot of the Setting Page Consisting of 5 Tabs: Resolvers, Components, Training Duration, Include & Exclude List, Train Model. The Spanshot
displays List of Components and the Remove Option
Fig. 10. A Bubble Chart displayed as a Scatter Chart in which each Data
Point denotes Component Diversity (Bubble Size) and Number of Resolvers
Fig. 9. A Spectrum of Boxplots Depicting Descriptive Statistics on Time
(Color) across Various Quarters
Taken to Fix a Bug. Each Boxplot Corresponds to Dataset Belonging to One
Quarter
years (dataset period) and then color-code the bubble for
buggy component and the defect proneness of the component. each quarter depending on its value being above or below
Figure 9 shows a spectrum of Box plots across various years the average value. Figure 11 displays the number of state
and quarters displaying descriptive statistics and five-number transitions between any of the 81 state transitions. Figure 11
summary on time taken to fix a bug (bug resolution time). is a Heat Map in which every cell is color coded depending
The spectrum of Box plots provides insights to the CCB on on the number transitions representing the cell. The Heap
the changes in the distribution of resolution time over several Map is useful to the CCB in gaining insights on process
quarters or time periods. anti-patterns and inefficiencies. For example, Reopened bugs
Figure 10 shows a bubble chart displaying component increase the maintenance costs, degrade overall user-perceived
diversity and trends on the average number of developers quality of the software and lead to un-necessary rework by
needed to a resolve a bug across project time-line. Figure busy practitioners [9]. Figure 11 reveals several cases of bug
10 reveals that the component diversity was high in July re-opening (such Solved-to-Inwork, Terminated-to-In Decision
and October Quarter of the year 2013 which means that the transitions).
reported bugs were spread across various components. We
infer that the component diversity decreases in April and July VI. E MPIRICAL A NALYSIS AND R ESULTS
Quarter for the year 2014 which means that majority of the We conduct a series of experiments on real-world data from
bugs were reported within a small number of components. Siemens product lines to evaluate the effective of our approach.
We also present insight on average number of developers We conduct experiments on two projects to investigate the
needed to resolve a bug. We first compute the average number generalizability of our approach. One of the projects is a Image
of developers needed to resolve a bug over the entire 2 processing based product (Project A) deployed in Computed
3rd International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2015) 28
TABLE I
R ECALL @ K , P RECISION @ K AND F-M EASURE @ K FOR P ROJECT A
K=1 K=2 K=3 K=4 K=5 K=6 K=7 K=8 K=9 K=10
RECALL 0.160 0.337 0.474 0.547 0.599 0.647 0.688 0.721 0.750 0.774
PRECISION 0.324 0.353 0.324 0.287 0.254 0.232 0.214 0.199 0.186 0.174
F-MEASURE 0.214 0.345 0.385 0.376 0.357 0.342 0.327 0.313 0.298 0.284
TABLE II
R ECALL @ K , P RECISION @ K AND F-M EASURE @ K FOR P ROJECT B
K=1 K=2 K=3 K=4 K=5 K=6 K=7 K=8 K=9 K=10
RECALL 0.242 0.644 0.794 0.819 0.848 0.864 0.875 0.890 0.900 0.905
PRECISION 0.437 0.599 0.493 0.408 0.342 0.292 0.255 0.228 0.206 0.188
F-MEASURE 0.312 0.620 0.609 0.545 0.488 0.436 0.395 0.363 0.335 0.311
developers, test leads) are working on the project. The second
project (Ultra-Sound Clinical Workflow Management System)
is a relatively larger project (Project B) which started in 2005
and there are 17267 bugs reported till October 2014. Out
of 17267 reported bugs, 12438 are resolved. A total of 253
professionals have worked on the project during the past 9 to
10 years. We consider only the resolved bugs for the purpose
of conducting our experiments. N folds cross validation with
N = 10 and K = [1, 10] is used for computing the precision and
recall performance evaluation metrics. The formulae used for
calculating precision@K and recall@K in information retrieval
systems are as follows (where K is the number of developers
in the ranked list):
B
1 X |Pi ∩ Ri |
Recall@K =
Fig. 11. A Heat Map showing the Number of Transitions (during the Bug B i=1 |Ri |
Lifecycle of all Bug Reports in the Dataset) between the 8 Possible States
B
1 X |Pi ∩ Ri |
P recision@K =
B i=1 |Pi |
In the formula for Precision and Recall, B denotes the
number of bug reports, R represents the set of actual resolvers
for a bug and P is the set of Predicted resolvers for the bug.
The calculated values have been shown in Tables I and II.
We observe that the recall values increase as we increase K
(which is quite intuitive). At K = 10, Project A has a recall
of 0:734 with 345 solved bugs, whereas Project B has a recall
of 0:905 with 1000 of the latest solved bugs. We observe
that the precision values are maximum at K = 2 in both the
projects. This is because in both projects the average number
Fig. 12. Metrics to be used to track the effectiveness and usefulness of using of resolvers per bug is very close to 2.
the recommendation system. We conduct a manual analysis and visual inspection of a
large number of bug reports and identify several instances
in which a bug report is assigned to a bug fixer based on
Tomography Scan Machine. Project A started in 2012 and has prior experience, workload, recent activity and severity and
772 bugs reported till November 2014. Out of the 772 bug not just based on the closest match in terms of problem
reports present in the Issue Tracking System, 345 have been area expertise. We observe that in several cases the top
solved and validated. We found that 236 issues have been recommended resolver (by our prediction model purely based
closed due to either being duplicate bug reports or bug reports on similar content-based recommendation) does not get the
invalidated by the triager due to insufficient information to bug assigned due to factors such as workload and prior work
reproduce the bug. At the time of conducting the experiments, experience of developers with the project incorporated in our
a total 78 members (project manager, product manager, testers, decision support tool but not within the Decision Tree and
3rd International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2015) 29
Naive Bayes based classification model. In one of the bug [4] W. Wu, W. Zhang, Y. Yang, and Q. Wang, “Drex: Developer recommen-
reports (status transition from in-work to in-work), we see a dation with k-nearest-neighbor search and expertise ranking,” in Software
Engineering Conference (APSEC), 2011 18th Asia Pacific, pp. 389–396,
developers comments 2011.
• Due to workload issue, Alan was able to solve it partially [5] A. Tamrawi, T. T. Nguyen, J. M. Al-Kofahi, and T. N. Nguyen, “Fuzzy
set and cache-based approach for bug triaging,” in Proceedings of the
and it needs to update the resolver. 19th ACM SIGSOFT Symposium and the 13th European Conference on
• Since the bug is related to SRC Component, Todd has Foundations of Software Engineering, pp. 365–375, 2011.
the experience in solving SRC related bugs and assigns [6] A. Sureka, A. Goyal, and A. Rastogi, “Using social network analysis
for mining collaboration data in a defect tracking system for risk
the bug to Todd instead of Ramesh. and vulnerability analysis,” in Proceedings of the 4th India Software
• The bug is high priority and assign it to Abhishek. Engineering Conference, ISEC ’11, (New York, NY, USA), pp. 195–204,
• Assign partial work to Rashmi and partial work to Manju ACM, 2011.
[7] C. Francalanci and F. Merlo, “Empirical analysis of the bug fixing process
• Please assign this bug to me (I have been working in it
in open source projects,” in Open Source Development, Communities and
recently). Quality, pp. 187–196, 2008.
[8] S. Lal and A. Sureka, “Comparison of seven bug report types: A
Our manual inspection of several bug reports and the case-study of google chrome browser project,” in Software Engineering
threaded discussions across two active projects in our or- Conference (APSEC), 2012 19th Asia-Pacific, vol. 1, pp. 517–526, Dec
ganization demonstrates that factors in addition to content 2012.
[9] E. Shihab, A. Ihara, Y. Kamei, W. Ibrahim, M. Ohira, B. Adams,
based assignment needs to be presented to the decision maker A. Hassan, and K.-i. Matsumoto, “Studying re-opened bugs in open
(as incorporated in our proposed decision-support system) source software,” in Empirical Software Engineering, pp. 1005–1042,
for making better triaging decisions. In order to enable the 2013.
projects to track the effectiveness and benefits of using the
recommendation system, we proposed simple process metrics
as shown in figure 12. The metrics are calculated for Project
A, considering the data from the bugs that have already been
resolved.
VII. CONCLUSIONS
Our survey results demonstrate that there are multiple
factors influencing triaging decision. Terms in bug report
title and description as well as resolver experience with the
project are the two most important indicators for making bug
assignment decision. Our interaction with practitioners in our
organization reveals that justification or reasoning behind a
recommendation, developer collaboration network, developer
work experience and workload are also important and useful
information in addition to the Top K recommendation. De-
scriptive statistics, trends and graphs on bug fixing efficiency,
big opening and closing trends, mean time to repair and
defect proneness of components are also important and com-
plementary information for the Change Control Board while
making triaging decisions. We demonstrate the effectiveness
of our approach by conducting experiments on real-world
CSS/PSS data from our organization and report encouraging
accuracy results. We conclude that an ensemble of classifiers
consisting of Decision Tree and Naive Bayes learners and
incorporating factors such as workload, prior work experience,
recent activity and severity of bugs is an effective mechanism
for the task of automatic bug assignment.
R EFERENCES
[1] G. Bortis and A. v. d. Hoek, “Porchlight: A tag-based approach to
bug triaging,” in Proceedings of the 2013 International Conference on
Software Engineering, ICSE ’13, pp. 342–351, 2013.
[2] X. Xia, D. Lo, X. Wang, and B. Zhou, “Accurate developer recommen-
dation for bug resolution,” in Reverse Engineering (WCRE), 2013 20th
Working Conference on, pp. 72–81, 2013.
[3] X. Xie, W. Zhang, Y. Yang, and Q. Wang, “Dretom: Developer recom-
mendation based on topic models for bug resolution,” in Proceedings
of the 8th International Conference on Predictive Models in Software
Engineering, PROMISE ’12, pp. 19–28, 2012.
3rd International Workshop on Quantitative Approaches to Software Quality (QuASoQ 2015) 30