In the rapidly evolving field of software engineering, understanding the intricacies of the software development process is crucial for delivering high-quality, efficient and reliable software solutions. This paper delves into the comprehensive study of the software development lifecycle, focusing on pivotal aspects such as code quality, implementation and testing. By dissecting these elements, we aim to offer insights into optimizing the development process, ensuring that software not only meets but exceeds the rigorous demands of applications to be realized.
At the heart of any software project lies the quality of its code, which serves as the cornerstone for functionality, maintainability, and scalability. We explore methodologies and practices such as code reviews, static code analysis, and adherence to coding standards that contribute to enhancing code quality. By integrating these practices, developers can reduce bugs, facilitate easier updates, and ensure a robust foundation for the software's architecture. The phases of implementation and testing are critical for transforming conceptual designs into functioning software. Contributions. This paper examines how generative AI models have been integrated in a DevOps pipeline for helping in improving the quality of the software released. We conducted an experimentation on several repositories in Java and C# and we demonstrated that our solution is able to fix around
Developing an automatic code fixer is key for enhancing programming productivity [1] and is an active area of research [2,3,4].
This trend has gained increasing popularity in recent years. Examples include Google's Tricorder [5], Facebook's Getafix [6] and Zoncolan and Microsoft's Visual Studio IntelliCode. The techniques underlying these tools can be classified into broadly two categories: logical, rulebased techniques [5] and statistical, data-driven techniques [7,6,8]. The former uses manually written rules capturing undesirable code patterns and scans the entire codebase for these classes of bugs. The latter learns to detect abnormal code from a large code corpus using deep neural networks.
Despite great strides, however, both kinds of tools are limited in generality because they target error patterns in specific codebases or they target specific bug types. For instance, Zoncolan's rules are designed to be specifically applicable to Facebook's codebases, and deep learning models target specialized bugs in variable naming [7] or binary expressions [6]. Moreover, the patterns are relatively syntactic, allowing them to be specified by human experts using logical rulesor learnt from a corpus of programs.
In this paper, we propose an effective and scalable easyto-use framework for fixing software issues in a DevOps pipeline by means of an LLM model (i.e., GPT3.5 1 ).
In this section, we describe the LLM models that have been used, how the prompt has been engineered so that it effectively performs for our task as well as the classification of the issues based on a taxonomy that we defined.
We evaluated the following models: We used OpenAI models via API on Cloud while we finetuned the Llama 2 model. Fine-tuning has been carried out by giving examples of snippets pairs incorrect code/correct code extracted from our internal repositories.
We conducted an analysis about the distribution of issues being fixed by the proposed approach among five classes that we defined (see Figure 1). On unused fields, best practices and code structure classes the proposed solution is able to correct around 50% of the issues whilst on the remaining two classes the fixing rate is around 30%. Please fix the error in the code snippet without completing it. The code must remain incomplete and indented as in the original snippet. Please provide a JSON response.
Following an analysis of common issues observed in code returned by generative models, a series of postprocessing functions have been implemented to enhance the quality of the response both in terms of writing style and integration with actual code. This manipulation occurs before the code is inserted into files, prior to undergoing quality checks and software compilation. Autocompletion errors prevention: Generative models often tend to complete the input code, which frequently consist of incomplete fragments, such as if or for statements without subsequent blocks, or portions that lack logical coherence when considered out of context. To address this issue, lines generated as completions of these snippets can be removed, considering the error occurs midway through the original snippet. Using Greedy String Tiling, a metric employed in literature for comparing code strings, the last lines of the generated code are compared with those from the input. If a match with the original code's final line is identified, only the preceding part up to that line is retained for insertion into the file.
Indentation correction: The generated code often loses the information regarding indentation levels, resulting in snippets where the indentation style and depth may differ from the original code. This discrepancy can include variations in both indentation style (such as tabs versus spaces) and indentation depth within the snippet.
Despite the flexible rules regarding indentation in currently supported languages, a method has been implemented to address this issue. This approach, again based on Greedy String Tiling, compares lines between input and output code to identify and apply a base indentation level that aligns with the indentation found in the received snippet. This ensures improved readability and quality of the generated code snippet, which is guaranteed to have consistent indentation with the surrounding code.
In this section, we report a study on how issues are distributed and results on two languages that are the most widely used by developers.
We conducted an analysis about the distribution of issues being fixed by the proposed approach among five classes that we defined (see Figure 2). Looking at the plot, on three classes the proposed solution is able to correct around 50% of the issues whilst on the remaining two classes the fixing rate is around 30%. Exception handling: in this class there are six Sonar-Qube rules. The type of issues belonging to this class are for example: 1) either log or rethrow this exception and 2) throw a dedicated exception instead of a generic one.
Best practices/conventions: in this class there are twenty-seven SonarQube rules. The type of issues belonging to this class are for example: 1) rename this field x to match the regular expression y and 2) block of commented lines of code should be removed.
Code structure/elements: in this class there are thirty-five SonarQube rules. The type of issues belonging to this class are for example: 1) merge this if statement with the enclosing one and 2) add a x field to this class.
Code complexity: in this class there are ten Sonar-Qube rules. The type of issues belonging to this class are for example: 1) the cyclomatic complexity of this method x is greater than the authorized value and 2) remove this expression which always evaluates to x.
We report the quantitative evaluation of the proposed solution on the two languages of the experimentation. In Table 1, we summarize the results on Java language on which an average debit reduction of 29,4% has been reached, with a peak of 63.0%. The pipeline executes on average 5.5 times faster than developers with a peak of 17 times. In Table 2, we summarize the results on C# language on which an average debit reduction of 25,9% has been obtained, with a peak of 42.9%. The pipeline executes on average 2.4 times faster than developers with a peak of 4.8 times. Results on C# are slightly worst because code is more complex and for building and analyzing the code more time is required with respect to Java.
In conclusion, our comprehensive study elucidates the multifaceted nature of the software development process, offering insights into optimizing development practices to meet and exceed the demanding requirements of today's applications. The integration of generative AI models into the software development lifecycle marks a significant advancement, showcasing the potential to revolutionize how software is developed, tested, and maintained. This paper contributes to the body of knowledge by demonstrating the effectiveness of these models in improving software quality and development efficiency, setting a precedent for future research and application in the field of software engineering.