The paper relates to the use of Generative Artificial Intelligence in software engineering practice. The nature of Generative Artificial Intelligence is introduced. Drawing upon the literature of the field, the positive benefits of AI use are considered and discussed in relation to some of the drawbacks and challenges involved. The discussion then moves on to examine the views of professionals, ascertained through a longitudinal study involving interviews with experienced, senior software engineers from a number of different organizations. These short interviews were focused on participants' immediate experience of using GenAI software in their work as professional developers. All companies involved in the investigation have longstanding and significant IT development departments for in-house ICT projects, and client-oriented, collaborative business-to-business projects. The results suggest that an initial experience of significant productivity improvement from AI-supported software development was critically hampered by quality issues. There were unexpected and undesirable software issues and problems, requiring laborious and resource-intensive efforts to address them retrospectively. Significant delays and drains on resources resulted. Some key points for reflection are drawn from the results of this small study.
The purpose of this paper is to consider the advantages and challenges involved in use of
Generative AI tools in the field of software engineering, as an example of a sociotechnical
endeavour. In the 21st Century, much progress has been made in the field of Artificial
Intelligence, leading to the possibility of enhancing processes, for instance in banking and
detection of crime
Neural networks have underpinned the development of AI. These comprise interconnected
nodes (artificial neurons), structured in layers, that process and ‘learn’ from data.
Feedforward neural networks, in which data flows in one direction, can perform tasks such as
pattern recognition, choice and classification. Their structure consists of three layers: an input
layer, a hidden layer and an output layer. The hidden layer transforms the input into
something useable by the output layer, using key parameters such as weights and biases. Learning
can take place as parameters are updated in response to new data or conditions. Each
adjustment brings about an evolution in the response of the network, so that it can adapt to
different tasks or environments. Neural networks of greater complexity, comprising many layers,
have enabled further developments in the AI field through Deep Learning
Large Language Models (LLMs) are advanced machine learning models designed to understand and generate human language. The largest and most ‘capable’ LLMs are generative pretrained transformers. They are trained using vast amounts of text data, giving them the capability to perform natural language processing (NLP) tasks. While LLMs do not have agency in themselves, they are used in a variety of applications, such as generating text, translating, summarizing content, and answering questions.
There is a great deal of interest in the literature relating to this application. Much of this focuses upon the benefits of AI use. Relatively little attention to date appears to have been given to the specifics of Generative AI.
A number of benefits are put forward for those adopting GenAI to carry out professional
work, such as software development.
Thus, it is easy to see how the production process can be speeded up, and no particular professional skills are needed in order to become productive, but Software Engineering comprises more than just coding. If the necessary stress testing and the requirements of integration are considered, both the claim to democratization and to cost-saving are premature. As Feldt, et al (2018) pointed out, while AI use can bring benefits to Software Engineering life cycles, new functionality also brings new challenges. There is a need to focus upon the different ways in which AI may be applied in SE and the different types of risk that each may entail.
An overview of the early literature of this field was given by
There are undoubted drawbacks to the use of GenAI in creating software, quite apart
from the risks mentioned above. Any output from such tools can only ever be unreliable
since their underlying logic is to generate a plausible outcome, based on pattern recognition.
Gen AI assistants have no in-built conceptions of either truth or reality
A further point to consider is a benefit mentioned by
As can be seen, GenAI tools may have much to offer in helping to improve productivity in fields such as software engineering, by supporting prototyping, improving workflow and enhancing error detection. However, proponents would do well to consider the tasks that will be involved in ensuring reliability, validity and security. These must be set against the desired productivity gains.
The next section of the paper looks at a small study designed to gauge opinions of professional software engineers regarding use of GenAI tools in their work. These snapshot accounts from practice were derived from a longitudinal study conducted between February 2023 and January 2025.
The purpose of the study was to obtain snapshot views of real-world experiences and practice of AI use in software engineering, by exploring participants’ views of professional experience within individual work contexts, from their own, unique perspectives.
The authors have approached this study taking an interpretive stance. Many sociotechnical inquiries are conducted from a desire to uncover generalisable principles for good practice, focusing upon precision and clarity in expressing an issue or problem. This approach has been termed logical empiricism. However, Radnitzky (1973) points to a danger that such an approach promotes an artificial separation between observers, with their unique worldviews, and the observations of phenomena that they make. This inquiry is therefore based in an alternative philosophical paradigm of hermeneutic dialectics. Here, there is explicit recognition of uncertainty and ambiguity, as features of socially constructed perspectives on human activity. Rather than precision and clarity, focus is on transparency, emphasising individual selfawareness.
Such an interpretive stance emphasises relevance in findings, rather than rigour and generalisability. The material uncovered is considered to be illuminating in its own right. Researchers adopting an LE stance will make provision in their design of inquiry for measures of reliability and validity. Within an HD-informed study, on the other hand, the appropriate measures will be transparency and recoverability. Thus, coding is used so that it will be possible to attribute particular findings to particular subjects, and the observations recorded are fed back to the participants, so that they have an opportunity both to confirm and reflect upon them.
The research was conducted within the Ethical Framework adopted and published by the University of Portsmouth. This is grounded in the UKRIO Code of Practice for Research. An ethics review was conducted at the beginning of the project, taking into account the principles set out in this framework. These include, inter alia, assessing whether the study would maximise benefit for individuals and society, while minimising risk/harm to participants; respecting privacy, autonomy, diversity, values and dignity of individuals, group and communities; ensuring that participation is informed and voluntary, and can be withdrawn at any time; ensuring that activities are conducted with integrity and transparency, using appropriate methods. Full details of this framework can be found at https:// policies.docstore.port.ac.uk/policy-028.pdf.
In the context of this inquiry, it was possible to involve participants in discussion of ethical considerations during the first round of interviews, including decisions on such matters as the need for anonymity. The results of these discussions were incorporate into the ongoing research design. Initially, not all of the interviewees saw a need for anonymity. However, this changed over time, and by the 3rd round of interviews everyone involved agreed that anonymity had been the right choice. This was a direct reflection of their changes in perspective, becoming more and more sensitive and potentially challenging to the viewpoints of their own managers and the corporate environment. This reflection on ethical commitment also required that the identities of their employers to be kept anonymous.
It was further agreed with the interviewees no recordings should be made. All notes were therefore made on paper, and consolidated during, as well as immediately after the meetings. Personal names, company names, places, etc were explicitly excluded from any notes.
Convenience sampling was used to recruit participants in this small study, drawing on personal recommendations and connections with professionals, including some alumni of the university. The selection criteria were specifically, [a] senior professionals with significant experience as lead software developers and project managers; [b] professionals who were actively working in organizations with relatively large scale (in house) projects, in departments, teams or groups of more than 50 software developers, contracted as employees. This resulted in 7 participants, all of whom were employed at a senior level. The interviewees were the same throughout all four interviews.
Table 1 shows the range of employing organisations and the professional role of participants, providing context for their reflections.
Four rounds of informal, on-line interviews were conducted, using Teams, Google Meet, Zoom etc dependent on the convenience and preferences of the interviewees. The interviews were short (approximately 15 minutes), sometimes during lunch breaks, sometimes in the late afternoon (after 17:00 when support staff had left for the day). While areas for discussion were considered during the planning stage, the interviews took the form of informal, guided conversations, in which participants were invited to explore their own perceptions from professional practice, within the context of their own work environments. Open-ended, generic questions were used to initiate conversation, asking participants to talk about their experiences and personal views on the situation and use of GenAI tools and technologies. Notes were made and fed back to the participants, during and after the conclusion of each interview.
Each snapshot response was systematically and carefully evaluated, interpreted and compared with the others. Simple differences in wording were ignored, and some acceptance of vagueness was tolerated, as the main focus of this study is on potential issues with the overall experience, and change of experience, as described. Such issues have potential consequences for professionals using this kind of technology to enhance their effectivity. This is so even in areas with which they are intrinsically familiar and where they already have deep knowledge. There are also implications for academics.
The first round of interviews was conducted during the Spring of 2023. At this stage, the opportunity was taken to discuss ethical considerations. The participants indicated their view that anonymity was necessary in recording the results in the ongoing study (see Research Ethics section above).
Table 2 shows the reflections of participants on use, policies and practice of AI in software engineering during the first round of interviews.
As was to be expected, use of AI, and development of internal policies for its use, extended over time. While Table 2 contains brief accounts of the participants’ awareness of a role for AI in software development, actual use was not reported in all cases. Some participants were able to report on management policies; in most cases use of AI was tentative and exploratory only. It should be noted that a variety of GenAI tools were in use in the different companies represented in the study. It is not intended to reflect upon the usefulness of any of these in relation to others.
Table 3 shows the responses of participants during the second round of interviews, which took place in the autumn of 2023. It can be seen that these are considerable fuller than those recorded in the first round. This may be in part because of rapid growth in awareness and use of AI tools in professional practice, but may also be in part due to development in the participants’ interest in, and sensitivity to, the impact of such tools within their context of use as the study progresses.
The third round of interviews took place during the Spring of 2024.
Table 4 shows the contextual reflections of participants from this round. It can be seen that, once again, the amount of material recorded from each response has expanded. Here, we can clearly see that policies for use of AI tools are beginning to be laid down by managers. There are also details of perceived problems and challenges that have arisen in context and remain to be addressed.
The fourth and final round of interviews took place during the autumn of 2024.
Table 5 shows the responses of participants during conversations in round 4.
The notes have again become fuller and more elaborate, and it can be seen that participants are reporting a division among their colleagues. Junior staff are appreciating the qualities of AI tools in terms of ease of use and productivity. Senior managers appear to be keen to encourage use of these tools in order to reap their purported benefits for the company. However, at the same time, senior professionals and some managers are beginning to have severe misgivings about emergent difficulties. Perceived problems appear to coalesce around quality issues. While there are undoubted productivity gains, work is often undocumented. This means that reverse engineering is required in order to trace origins of poor functionality. Concerns are raised about sustainability and security. Where work is documented, there may be a mismatch between the documentation and the actual logic of code produced. There is recognition by senior professionals that extra work must be done to validate output produced using AI tools. However, this recognition does not always include senior managers who provide resources and, in some cases, have been keen to reap the promised productivity gains. In this last round of interviews it was clear that all participants had developed strong views and reflections on the topic.
Table 6 provides a brief summary of the factors emerging from the interviews.
There are a number of interesting points emerging from the various conversations. These are perceptions from senior professional software engineers, who may be considered well-informed about the context of practice within which AI tools are to be deployed. By the last round of interviews, it became clear that all participants had developed strong views and reflections on the topic.
Interviewees concluded that real-world cost and penalty of using existing AI tools went far beyond any benefit that an appearance of initial productivity gains would justify. Participants were concerned about a conflict between their own experience of use of AI tools and the ongoing insistence and promotion of use by their corporate leadership. They did not experience that they were being listened to, as promises of productivity gains and cost saving completely skewed the expectations of corporate management. Senior managers seemed to be prepared to turn a blind eye to the explosion of cost and delay experienced by those involved in practice. They failed to relate productivity gains to the extra resources required when reverse engineering became necessary, to identify and address undocumented flaws and problems. There appeared to be an ongoing mismatch of expectations and reality of experiences. The interviewed professionals did not see a constructive way out of the situation or a ready solution to these difficulties. They did not feel their concerns were listened to, or that any lessons were being learned or taken on board by corporate management.
As a result of the study, the authors consider that there are areas that would be worthy of deeper consideration in future research.
First, it appears that corporate management in a number of companies appeared not to be listening to senior professionals or engaging in real dialogue about deployment of GenAI tools. Ordinarily, it would be expected that the opinions of such colleagues would be relevant and influential on promotion of these or any other tools in a context of use. Promises made for GenAI, and the capabilities for which it is promoted, are very attractive within a competitive business or public service environment. Thus, wishful thinking and desire may lead people to engage with them. People who are not technical experts may become susceptible to disqualifying critical mindsets and ignoring professional experiences and misgivings. In the final stages of the study reported here, there was a clear agreement by the end that use of the tools had created more problems and issues over time, ironically while being pushed and promoted more than ever before.
There are a number of key points for consideration:
Resources or support not available for professionals on how to apply verification and validation of AI output, and how source necessary resources to address quality and security issues.
No description or advice was available as to what kind of workarounds were acceptable. There was no guidance on what to do when necessary resources to address practical problems were not available.
No feedback or discussion took place regarding development of contextually-relevant practices for quality assurance.
Professionals did not experience help or support to address actual problems. They were instead referred to simplistic statements that all output would need to be verified and validated before use.
What appears to be a significantly expanding use of GenAI by professionals is interesting but not surprising per se. One of the main, human characteristics that make us effective and efficient is our ability to choose NOT to use our intellect to address every problematic phenomenon we encounter; to do so would be extremely wasteful and, potentially, would make us so ineffective that we would be unable to adapt to our environment. Therefore, we apply our intellect and intelligence selectively, and even go out of our way to avoid using our thinking and problem-solving capabilities unless we make a judgement that this is necessary. This is, of course, a contributory reason why AI solutions are so popular. The idea of GenAI, and the abilities for which it is promoted, are very seductive to our human experience of life. Thus, we may be motivated by ‘wishful thinking’ and to accept such tools more readily than perhaps we should. We may become susceptible to disqualifying critical mindsets and ignoring our own misgivings. Having been used to placing reliance on previous generations of productivity tools, such as libraries of reusable code and knowledge based systems, it is not surprising if GenAI appears attractive to professionals.
Ironically, this point is also overgeneralized, and therefore a flawed foundation upon which most GenAI support tools are based. They are fundamentally unreliable and misleading, based on a belief that most people prioritize efficiency over relevance to objective and context, or indeed effectivity. The reason GenAI technologies are not just unreliable but also largely misleading, is the model upon which their functionality is based generating patterns based on analysis of patterns. There is no actual correlation between training data and output within any specific factual content. This is also the reason why so many people fail, in practice, to do the validation required (if they are honest). The required resources, in time and effort, are not options realistically contextualized enough to be meaningful within the space of a real-world job. If the output generated looks the same as the expected output, then there is no apparent reason, in context, to deviate from habitual prejudice and bias. This will render it emotionally, as well as intellectually difficult to discipline oneself to undertake conscious validation of output with original source, systematically and with the utmost care required. To do this manually demands not just a lot of attention to detail, but a great deal of precious time.
When considering the lack of correlation between any reality and the output from GenAI, it is sobering to reflect that the issues discussed above may become more problematic over time if we continue to abrogate responsibility to check and modify the results. Plausible, but incorrect content will be output into the world and will become a source of new training data for tools of the future. The logic of this is that, eventually, a point of total model collapse would be reached, leaving those whose work has become dependent upon GenAI in an unenviable position.
The author(s) have not employed any Generative AI tools in the writing of this work.
Taylor, J. (2025). Fake cases, judges’ headaches and new limits: Australian courts grapple with lawyers using AI. The Guardian, 9 February 2025. https://www.theguardian.com/law/ 2025/feb/10/fake-cases-judges-headaches-andnew-limits-australian-courts-grappling-with-lawyers-using-aintwnfb?CMP=Share_iOSApp_Other Yetiştiren B., Özsoy I., Ayerdem M. and Tüzün E. (2023). Evaluating the code quality of aiassisted code generation tools: An empirical study on GitHub copilot, amazon codewhisperer, and ChatGPT. arXiv preprint: https://arxiv.org/pdf/2304.10778