A major aspect of maintaining the quality of software systems is the management of bugs. Bugs are commonly fixed in a corrective manner; detected after the code is tested or reported in production. Analyzing Fix-Inducing Changes (FIC) developer code that introduces bugs - provides the opportunity to estimate these bugs proactively. This study analyzes the evolution of FICs to visualize patterns associated with the introduction of bugs throughout and within project releases. Furthermore, the association between FICs and complexity metrics, an important element of software evolution, is extracted to quantify the characteristics of buggy code. The findings indicate that FICs become less frequent as the software evolves and more commonly appear in the early stages of individual releases. It is also observed that FICs are correlated to longer Commit intervals. Lastly, FICs are found to be more present in codes with fewer lines and less cyclomatic complexity, which corresponds with the law of growing complexity in software evolution.
For this study, eight Java repositories from GitHub Osman et al.[19] extracted bug-fix patterns by mining with a total of 142,555 Commits have been analyzed. change history of 717 open source projects. They manFrom the Commits, FICs are detected, release information ually inspected the patterns to retrieve the context and are extracted and relevant metrics are calculated. Thereasons that cause those bugs. ifndings show that FICs decrease as the software evolves, So far, FICs have been analyzed to derive relationships while remaining more prevalent early in release cyclesw. ith diferent project metrics. While the evolution of Statistically analyzing the data shows that FICs containCommits has been observed, the evolutionary properties larger intervals and their code reduced LoC and CC thanof FICs have yet been studied. regular Commits.
Sliwerski et al.5[] introduced the term Fix-Inducing Changesevolve throughout the lifetime of software projects and (FIC), providing a process that detects FICs in version how these relate to complexity metrics. The methodolhistory from Concurrent Versions System (CVS) with ogy of the study is divided into three parts, described as bug reports from Bugzilla. Moreover, they showed a follows. relation between FICs and number of files changed. Antoniol et al.1[4] showed that FICs create adverse efect 3.1. Fix-Inducing Change (FIC) Detection and produce unexpected results. They presented a robust approach to detect groups of co-changing files in CVS FICs are changes to code that causes problems to the repositories. software system. FICs are the introduction of bugs or
Yin et al.[6] identified and analyzed incorrect bug fixes errors to the software, inducing fixes in the future. Hence, which introduce new ones instead. They analyzed the these can be detected from the changes that fix bugs and code of operating systems namely, FreeBSD, Linux and errors.
OpenSolaris. This approach also combined version con- This study utilizes Commits, the documented changes trol systems and bug repository to categorize changes. in software projects that are managed through version They proved that Fix Inducing Fix (FIF) can cause crashes, controlling. Commits contain the exact lines and files hangs, data corruption or security problems. where changes are made along with information of and
Bavota et al.7[] showed that 15% refactoring tasks in- message from the developer who posted these. The deduce bugs, analyzing 52 kinds of refactoring on 3 Java tection of FICs through Commits is conducted in the projects. They detected inheritance related refactoring following steps, influenced by the process of [7] as the most error-prone refactoring.
In order to analyze FICs, various works focused on different properties of change that would induce the bugs.
Levin et al.1[5] and Menzies et al.1[6] focused on source code changes of afected files. Fukushima et al. [ 17] introduced developer experience, time of day, time interval of commit and some other properties of change that would induce bugs. Sadiq et al. 8[] related FICs with change couplings to find that recent change couples provide better insight on new errors. Huq et al. 9[] showed that developer sentiment is related with FICs, where positive comments and reviews in Pull Requests can lead to buggy Commits.
Weicheng et al.1[8] explored the relation between developer Commit patterns in GitHub and software evolution. They used four metrics to measure the Commit activity of developers and code evolution: changes, interval, author and source code dependency. Moreover, this paper showed techniques to visualize these metrics for a given project. They developed a tool named Commits Analysis Tool (CAT) that finds that the changes in previous versions can afect the file which is dependent on it in the next version. 1. All Commits are fetched from GitHub reposito
ries. 2. Commit messages are extracted to detect terms such as “bug”, “fix” or “patch”. These terms signify that the aim of the Commit is related to the management of bugs. 3. Now the changes in these Commits are analyzed.
Since the study deals with Java projects, it is first checked whether the changes occur in “.java” files.
Commits with no changes to such files indicate that the Commits dealt with non-code entities of the software (configuration files, documentation etc.). Furthermore, the changes made in “.java” ifles are analyzed to see whether the changes were code comments, which also signifies the absence of code entities. 4. Then, the type of the edit made by the Commit is checked. There are three types of edit: Insert, Delete and Replace. An Insert edit means that a patch code is added onto the existing code base.
However, it does not help to track which part of the previous code was buggy. There is no way of tracking back to a Commit that introduced a
However, since Commits are automatically assigned all future tags, the labeling was conducted in two parts. First,
Commits are extracted from all tags. Then, for every tag, bug. Hence, Commits with Insert edit types are only those Commits that were posted after the previous discarded from further consideration. release tag are assigned to the current one. 5. After the filtering process, the remaining Com- As Commits are segmented into releases, analysis for mits are labeled as “Fixing Commits” or “Fixes”. Research Question (RQ) 1 is conducted. For each release, These are the Commits that removed buggy code. the number of FIC and non-FIC is calculated based on 6. Next, origin of each legitimate change in the Fix section 3.1. This segmentation is further elaborated for is tracked using the blame function, which re- RQ2, by dividing each release into three equal parts. The turns the Commit where a specified changed line three divisions are extracted to better understand the was last added or modified. These Commits are early, middle and late stages of a single release. labeled as FICs.
3.2. Evolution Analysis To analyze the relation between FICs and complexity metrics — Line of Code (LoC) and Cyclomatic ComplexTo understand the evolution of FICs, the projects’ re- ity (CC) — first the changes to code are extracted. This lease tags are analyzed. Release tags define iterative final includes the content of the changed files and numbers versions of the software in the project’s lifetime. Since of lines which are modified or deleted. In the case of file Commits are assigned these tags, it is possible to catego- contents, along with that of the current Commit, conrize Commits based on releases. tents of its parent Commit are also extracted. Parent
To analyze releases, first non-release tags are filtered Commit is referred to the Commit directly prior to the out based on naming structures. Usually the release tagscurrent Commit. The contents of the parent Commit in most projects abide by the pattern: “v #.#.#”. The provides information of the state of code before the currest are tags depicting other information like branches rent Commit’s changes. For FICs, their parents retains or experiments. However, the structure of naming tags the properties of the code where the bug was introduced. can vary with projects. For instance, patterns in projects With the contents of the current and parent Commits, like ElasticSearch or Commons-lang ar“eElasticsearch the two metrics are calculated in the following manner: #.#.#” and “commons-lang-#.#.#” respectively. Hence, tags are manually analyzed for each repository. Additionally, 1. LoC: To calculate the line of code, without considversions that are release candidates are discarded since ering comments, first the Abstract Syntax Tree these do not depict final releases. (AST) [20] of a program is generated from the changed files using JavaParser1. It is verified Table 1 whether the changes have been conducted on Repository description of the eight projects eaxreeceultiambilneacteodd.e Norenxto,t.thTehmeroedfoifierde, lbilnaenskinsptahcees Project Name Commits (LYifeeatrism) e -Cbounttorriscurrent content are checked to identify whether Apache Tomcat 19360 8.5 21 these are comments based on the AST. If none of Google Guava 4798 5 187 the changed lines are executable code, then the Mockito 5019 6.7 155 ifle is not further considered. Otherwise, the LoC Commons-lang 5396 10 115 of the parent content is calculated. Apache Hadoop 21435 4.8 191 2. CC: To calculate cyclomatic complexity, the method Selenium 23550 9.3 435 in which change was introduced is first identified. Elastic Search 44975 8.5 1216 This is done by taking each changed line of the Spring Framework 18022 6.4 369 current content and tracing its state back in the Total 142555 59.1 2689 parent content. The generated AST of the parent content is traversed for each line. Each changed line of executable code is individually assessedto the system tend to contain more instances of bug inand provided an associated method. This pro- troduction. This can be due to a rapidly changing and vides a list of changed methods for a single file. volatile initial requirement, formative and incomplete Each of their cyclomatic complexities are calcu-development processes, lack of collaborative experience lated, aggregating all possible paths (If-else, loopas,mong the developers, or an insuficiency of reviewing switch statements). and testing resources. But as the software evolves, the FICs get reduced, as an indication of bolstered testing and quality assurance processes, and project maturity. 4. Experimentation and Findings On the other hand, projects Tomcat and Hadoop in Figures 3(a, e) show the opposite trend, where FICs are Description of the dataset and the results observed for more predominant in later versions. This could happen the 4 Research Questions (RQs) are described as follows.due to a decreased level of scrutiny in reviewing eforts, a overhaul of new requirements, or other project and 4.1. Dataset personnel related events. Only Figur3e(f) showcases a slightly more uniform pattern of FICs for the project To conduct this research, eight well known Java projects Selenium. Although there are spikes of FICs occurring are chosen from GitHub’s repositories. These projects are in specific versions, there is no apparent progression in open source and use GitHub as their primary medium of the appearance of FICs. code storage and version control, enabling the extraction Such visualizations of the evolution of FICs help in of all necessary Commits. Details of the repositories are observing the history of the project in terms of buggy displayed in Table1. The projects comprise of a total changes. This can be related to other aspects of projects of 142555 Commits to analyze. All eight repositories that coincide with the decrease and increase of FICs to are used to analyze Research Questions (RQ) 1, 2 and 3.understand what afects the introduction of bugs from a RQ4, which requires the source code, utilizes the first high level perspective. ifve repositories.
The 1st RQ aims to understand how FICs evolve, in terms is observed. In Figure4, the appearance of FICs within of frequency and ratio, throughout the lifetime of soft- releases is displayed as black circles, where the size of ware projects. The graphs in Figure3 showcase the evo- the circle is determined by the proportion of FIC on the lution of FICs in the eight software repositories analyzed. total number of Commits in that stage. The releases are The diferent repositories show diferent types of patterns. divided into three stages: early, middle and late, and for In the majority of patterns, as seen in Figures3(b, c, d, g, some projects, versions are merged for visibility. h) for projects Guava, Mockito, Commons-lang, Elastic- It can be seen that for almost all the projects, FICs are search and Spring-framework respectively, FICs appear more predominant in early and middle stages of releases in the early stages of the projects’ lifetime and decreasecompared to late ones. The exceptions are Tomcat and in newer versions. This indicates that earlier changesSpring framework, where FICs are similarly or more prevailing in the late stages. The high level of appearance of
FICs in early and middle stages of a release can be con(a) Project: tomcat (b) Project: guava (c) Project: mockito (d) Project: commons-lang (e) Project: hadoop (f) Project: selenium (g) Project: elasticsearch tributed to a higher emphasis on adding and updating Table 2 features in those stages. The late stages are more focused Results for RQ3 and RQ4 on debugging and deployment eforts.
The 3rd RQ deals with the relation between Commit intervals and FICs. Table2 shows that, on average, the interval (in minutes) for FICs is longer than that for regular Commits. The p-value of< 2.2 −16 solidifies this diference as significant. This indicates that either a large amount of work relates to FICs (as shown by 5[]), or that the developer introduces bug when they are away from the code for a long time.
The finding can help in preemptively detecting buggy code. Commits posted after a longer period than average can be given extra emphasis in reviews. Additionally, developers should be suggested not to disassociate from the code for a long time.
complexity metrics of software evolution: Line of Code (LoC) and Cyclomatic Complexity (CC). It can be seen in Table2 that the average LoC of code where FICs occur is lower than that of regular Commits, with a p-value of < 2.2 −16, making this diference significant. The result says that a lower LoC relates to buggy code. Hence the smaller, less busy components need to be given more emphasis in coding correctly and reviewing for bugs.
Next, as seen in Table2, FICs occur in methods with significantly less CC than regular Commit, based on the 6.48 −10 p-value. A lower CC means that the tasks in methods are logically simpler. And yet, bugs, as statistically shown, tend to be introduced in such methods. Similar to LoC, this result prompts for a higher level of scrutiny when dealing with smaller and simpler methods.
Both of these findings support the evolution of FICs compared to complexity metrics. As described by Lehman’s law of evolution1[1], complexity rises as software evolves, hence increasing LoC and CC. Similarly, based on RQ1, FICs decrease in ratio in most cases, which is solidified by its inverse relation with the complexity metrics.
From the findings generated in this study, the following interpretations and applications can be estimated:
1 and 2, it can be seen that bugs appear mostly in the early stages of versions and release cycles. This finding solidifies the intuition that early code tends to cause more bugs than later ones
Metric Interval LoC CC
Commit Type FIC Regular FIC Regular FIC Regular
Average when maintenance starts to outrank development, and the software stabilizes. With this intuition proven through data, the finding can be applied to change the way software is developed. A more test-driven approach can be adopted in software projects from the beginning to mitigate the large influx of bugs. 2. Comparative history: By graphically extracting the evolution of FICs in software projects, the appearance of bugs can be historically analyzed.
This history can unearth valuable insight, for example periods of time or certain releases where FICs peaked in number. These exceptions can be comparatively analyzed with other metrices related to the project. The metrices can range from code properties like components developed, design patterns used etc, or project aspects like type of assignment, assigned developer, developer turnover etc. The proponents may vary from project to project, hence the historical data of FICs can be used as a constant reference to such difering metrices. 3. Intervals and bugs: RQ3 provides insight into the correlation between Commit interval and FICs, showing the tendency of larger intervals causing bugs. This finding can be applied in project management, by monitoring the absences of developers. Developers who have been absent from the development process for longer periods should be assigned to tasks that are less sensitive and their work be reviewed more intensely. Furthermore, as also observed by Sliwerski et al.5][, large amount of changes in a single Commits that cause higher time for completion should be regulated for FICs. 4. Software evolution and complexity : The last ifnding demonstrates how FICs are correlated with line of code (LoC) and cyclomatic complexity (CC). These metrices, referred to as complexity metrices in the domain of software evolution, are important in understanding the evolution of FICs. As graphically shown in RQ1, FICs tend to decrease as the software evolves. On the other hand, complexity increases with the software’s gineering Conference (APSEC), IEEE, 2019, pp. age [11]. This indicates an inverted relationship 514–521. between the two metrices, which is proven in [10] S. F. Huq, A. Z. Sadiq, K. Sakib, Is developer senRQ4 where FICs are found to be related with a timent related to software bugs: An exploratory larger LoC and CC in the code. study on github commits, in: 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER), IEEE, 2020, pp. 6. Conclusion 527–531. [11] M. M. Lehman, Laws of software evolution revisThis study analyzes GitHub repositories to extract Fix- ited, in: European Workshop on Software Process inducing Changes (FICs) — changes that introduce buggy Technology, Springer, 1996, pp. 108–124. code — and observes its evolution and characteristics. It is [12] C. F. Kemerer, S. Slaughter, An empirical approach seen that FICs tend to occur in earlier versions and stages to studying software evolution, IEEE transactions of releases. There is also a significant delay in posting on software engineering 25 (1999) 493–509. FICs than regular Commits. Lastly, when relating with [13] G. Xie, J. Chen, I. Neamtiu, Towards a better undercomplexity metrics, FICs show up in code with less LoC and less CC than regular Commits. This corresponds with the decreasing FIC and increasing complexity of software evolution. standing of software evolution: An empirical study on open source software, in: 2009 IEEE International Conference on Software Maintenance, IEEE, 2009, pp. 51–60. [14] G. Antoniol, V. F. Rollo, G. Venturi, Detecting groups of co-changing files in cvs repositories, in: Eighth International Workshop on Principles of [1] M. M. Lehman, L. A. Belady, Program evolution: Software Evolution (IWPSE’05), IEEE, 2005, pp. processes of software change, Academic Press Pro- 23–32.
fessional, Inc., 1985. [15] S. Levin, A. Yehudai, Boosting automatic commit [2] M. Monperrus, Automatic software repair: a bib- classification into maintenance activities by utilizliography, ACM Computing Surveys (CSUR) 51 ing source code changes, in: Proceedings of the (2018) 1–24. 13th International Conference on Predictive Models [3] N. Chauhan, Software Testing: Principles and Prac- and Data Analytics in Software Engineering, ACM, tices, Oxford University, 2010. 2017, pp. 97–106. [4] T. Ackling, B. Alexander, I. Grunert, Evolving [16] T. Menzies, J. Greenwald, A. Frank, Data minpatches for software repair, in: Proceedings of ing static code attributes to learn defect predictors, the 13th annual conference on Genetic and evolu- IEEE transactions on software engineering 33 (2006) tionary computation, 2011, pp. 1427–1434. 2–13. [5] J. Śliwerski, T. Zimmermann, A. Zeller, When do [17] T. Fukushima, Y. Kamei, S. McIntosh, K. Yamashita, changes induce fixes?, in: ACM sigsoft software N. Ubayashi, An empirical study of just-in-time deengineering notes, volume 30, ACM, 2005, pp. 1–5. fect prediction using cross-project models, in: Pro[6] Z. Yin, D. Yuan, Y. Zhou, S. Pasupathy, L. Bairava- ceedings of the 11th Working Conference on Minsundaram, How do fixes become bugs?, in: Pro- ing Software Repositories, ACM, 2014, pp. 172–181. ceedings of the 19th ACM SIGSOFT symposium [18] Y. Weicheng, S. Beijun, X. Ben, Mining github: and the 13th European conference on Foundations Why commit stops–exploring the relationship beof software engineering, ACM, 2011, pp. 26–36. tween developer’s commit pattern and file version [7] G. Bavota, B. De Carluccio, A. De Lucia, M. Di Penta, evolution, in: 2013 20th Asia-Pacific Software EngiR. Oliveto, O. Strollo, When does a refactoring neering Conference (APSEC), volume 2, IEEE, 2013, induce bugs? an empirical study, in: 2012 IEEE pp. 165–169. 12th International Working Conference on Source [19] H. Osman, M. Lungu, O. Nierstrasz, Mining freCode Analysis and Manipulation, IEEE, 2012, pp. quent bug-fix code changes, in: 2014 Software Evo104–113. lution Week-IEEE Conference on Software Main[8] A. Z. Sadiq, M. J. I. Mostafa, K. Sakib, On the evolu- tenance, Reengineering, and Reverse Engineering tionary relationship between change coupling and (CSMR-WCRE), IEEE, 2014, pp. 343–347. ifx-inducing changes (2019). [20] I. Neamtiu, J. S. Foster, M. Hicks, Understanding [9] S. F. Huq, A. Z. Sadiq, K. Sakib, Understanding source code evolution using abstract syntax tree the efect of developer sentiment on fix-inducing matching, in: Proceedings of the 2005 international changes: An exploratory study on github pull re- workshop on Mining software repositories, 2005, quests, in: 2019 26th Asia-Pacific Software En- pp. 1–5.