=Paper=
{{Paper
|id=Vol-3927/paper16
|storemode=property
|title=Effect of Large Language Model Use on Programming Project Groups
|pdfUrl=https://ceur-ws.org/Vol-3927/paper16.pdf
|volume=Vol-3927
|authors=Laura Graf
|dblpUrl=https://dblp.org/rec/conf/ectel/000324
}}
==Effect of Large Language Model Use on Programming Project Groups==
https://ceur-ws.org/Vol-3927/paper16.pdf
Effect of Large Language Model Use on Programming
Project Groups
Laura Graf
Technische Universität München, Arcisstraße 21, 80333 Munich, Germany
Abstract
The adoption of Large Language Models (LLMs) in education has prompted questions about their
impact on programming projects. This research will explore how the use of LLMs affects learning and
socio-affective outcomes on individual and group level in first software engineering projects. Existing
literature explores both potential benefits and pitfalls of LLMs in educational contexts. LLMs enhancing
readability, explaining others’ code and providing quick answers to less experienced students could
improve group work. However, there are concerns such as students’ judgment of competency, effort
and contributions created with LLM support affecting group collaboration dynamics. To address the
gap in empirical research on LLMs' impact on perceptions of teammate competency, connectedness,
self-efficacy, learning gain, and professional identification we will analyze not only self-reported
measures but also work with process data from collaborative coding platforms to extract meaningful
measures of collaboration behavior and issues in group code prominent when LLMs are being used.
Keywords
Large Language Models, Computer Science Education, Collaborative learning, Process Mining, Socio-
affective measures 1
1. Introduction ensure that integration does not deprive students from
developing much needed higher order thinking.
After the vast adoption of Large Language Models Another reason why a focus on solely optimizing
(LLMs) universities have raised questions about the student use of LLMs is insufficient in educational
adequacy of curriculum and assessment in response to settings is that technologies continue to evolve. The
student use of computer-generated output during their models change to support humans better, and this
studies (Kasneci et al., 2023). These questions are process cannot be expected to stabilize on a certain
particularly pertinent for future programming pattern (Joksimovic et al., 2023).
professionals, since LLMs are effective in generating Students will need to continue working together,
code chunks (Kazemitabaar et al., 2023). The use of LLM- solving problems and communicating effectively,
generated code can not only speed up programming regardless of the specific cognitive tool they may use.
tasks but substantially offload thinking processes to the Hence, it is also important to understand the
machine. In light of potential automation of coding relationship between the use of cognitive tools and
tasks, it has become unclear as to what skills should be educational outcomes that reach beyond domain
taught to future programming professionals to enable knowledge. Frequent use of tools like LLMs may shape
effective integration of LLMs into the human-led students in a way that affects them profoundly, so
process. stakeholders need a clearer understanding of how
The focus on optimization of productivity enabled broader educational outcomes are affected by those tools
by machines has thus far been central to research on so that instructional practices can be adapted to preserve
how to integrate LLMs into human cognitive practices the focus on developing skills essential for humans.
(Wang et al., 2019). Yet such a focus is only partially To address this pressing need, my project will
relevant in educational settings. Educational outcomes investigate how LLMs affect educational outcomes in a
target students' cognitive development and higher-order collaborative setting where future programming
thinking in relation to the domain they study. Tools professionals practice a broad set of skills. Collaborative
supporting cognitive processes can benefit the learner in work is commonly part of software engineering projects
offloading some parts of such a process and allowing the in computer science curricula. Collaborative work is an
learner to focus on higher- order thinking (Salomon, essential part of professional software engineering and
2003). At the same time, improper use of the tool can interpersonal skills are among the most significant for
lead to a reduced, shallow understanding (ibid.). the effectiveness of software engineers (Boyatzis et al.,
Therefore, it is important to understand the relationship 2017). Groups have been shown to innovate faster,
between student use of cognitive tools such as LLMs and identify mistakes more quickly, and find better solutions
learning outcomes related to their domain knowledge to
Proceedings of the Doctoral Consortium of the 19th European Conference l.graf@tum.de (L. Graf)
on Technology Enhanced Learning, September 16-20, 2024, Krems, Austria © 2025 Copyright for this paper by its authors. Use permitted under
Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
to problems; all while reporting a higher job satisfaction tasks may affect educational outcomes. This includes
(Duhigg, 2019). individual outcomes, such as learning, self- efficacy, and
Moreover, project-based assignments where professional identity, as well as group-related socio-
students practice collaborative work are a catalyst for affective outcomes, such as trust in teammates
bonding and social learning, facilitating social capital competency and connectedness. I also explain why the
among future professionals and affecting professional process of how learners collaborate when they
identity. Collaboration experiences can also create individually use LLMs must be considered.
precedents for exclusion and negatively affect belonging
and diversity in STEM (Miller-Young et al., 2023). This 2.1. Effect of LLMs on Learning
richness of educational outcomes makes collaborative Individuals
programming assignments a suitable context to examine
LLMs offer a diverse range of applications for
the effect of cognitive tools, such as LLMs.
enhancement of learning experiences, personalized to
Current literature is limited in explaining the effects
the student (Kasneci et al., 2023). In introductory
that LLMs can have on broader educational outcomes in
programming assignments, LLM-based coding tools
programming group work. Existing research suggests
currently already perform at the level that outscores the
that such effects could be both positive and negative.
average student (Finnie- Ansley et al., 2022). Most
LLMs have the potential to support participation of
students prefer using a LLM, especially to get a starting
students with less programming skills in code
point, even when they often face difficulties in
production and help to understand others’ code,
understanding, editing, and debugging generated code
important for positive collaborative work. However, it
(Vaithilingam et al., 2022). Studies have also shown that
could also amplify issues of unequal effort distribution
programmers tend to defer tasks related to
through the option to auto-generate code, which is
comprehension to the LLM, even though this can steer
known to create negative experiences (Nguyen et al.,
them in the wrong direction (Nam et al., 2024). Some
2023). LLMs can, for instance, facilitate shared
scholars also suggests that students use LLMs for
understanding by improving code readability and
requesting explanations of code and general questions
documentation as well as reduce the need for any group
more often than for code generation (Kazemitabaar et
members to spend large parts of the time on lower-level
al., 2024).
tasks such as generating test cases, which might change
To circumvent challenges associated with LLM
previously common role distributions. With LLMs
use, chatbots have been developed to offer hints to
affecting most parts of the programming projects, an
mimic human tutoring, instead of giving students full
influence on the social aspects of group work is likely
solutions (Bassner et al., 2024).
and deserves attention.
Literature so far has shown that integrating LLMs
This research gap calls for empirical examination of
into practices around learning and studying can affect
the effects of LLMs in programming assignments on
individual learning gains. A major concern here is that
domain- specific knowledge and the effect on group
when students regularly offload to technology, they may
processes, as well as longer-term imprint on students,
not actually learn how to perform the task on their own.
such as the formation of professional identity. My thesis
(Darvishi et al. 2024) found that when using LLM,
will focus on addressing this gap. I will employ mixed
students seem to be finishing tasks well, but once the
methods research design. First, I will analyze the effect
LLM was removed, they did not replicate the new
of LLM integration into programing group projects, in
strategies used by the LLM that were helpful with the
relation to student perceptions of learning, their socio-
tasks. Another study showed that learning gains from
affective attitudes towards teammates and their
using an LLM in learning programming languages vary
evolving identification with the domain. Second, I will
with context and task complexity (Aviv et al., 2024).
investigate the relationship between these perceptions
Researchers observed that LLMs did not reduce
and process data from code progression, as student
metacognitive difficulties for students with limited
perceptions are largely mediated by the code-based
programming abilities and even introduced new ones
communication on GitHub. The thesis is in its planning
(Prather et al., 2024).
stage.
In addition to learning gains, integrating LLMs
into learning practices can impact self-perceptions, such
2. Related Work as self- efficacy and professional identity. Studies on
When students use LLM-based tools for their tasks LLMs’ effects on students’ self-efficacy found that LLM-
in a course, this affects how and what they learn. To supported review of course topics improved students’
advance the goal of understanding the effect of LLMs in self- efficacy and motivation (Lee et al., 2022). This effect
collaborative programming assignments, this section on self-efficacy appeared because LLM helped students
explains how LLM use in collaborative programming become active during learning, as it provided a safe way
to explore questions (Y.-F. Lee et al., 2022). A study processes. The effects of such individual use within a
where interactions with an LLM supported student collaborative task have not yet been explored.
thinking deeply about a topic showed improved self- Previous research suggests that AI can affect
efficacy and learning achievements (Chang et al., 2022). collaboration in unintended ways (Wang et al., 2022),
(Wang et al., 2023) found that AI based on good and this can also be expected when students integrate
technology combined with technological skills in a LLMs to support individual programming needs within
higher education program improve students’ self- a collaborative task. For instance, group-related socio-
efficacy, mediating performance. Perception of self- affective outcomes, such as trust in teammates’
efficacy can also benefit from having a starting point in competency and feeling of connectedness with the
coding (Vaithilingam et al., 2022). group, may be affected. Students’ perceptions of
Self-efficacy further plays an important role in teammates’ contributions may change when individuals
securing diversity, equity, and inclusion in STEM. submit auto-generated code without transparency of
Minorities and women feel less included in the how it was created. Engagement in collaborative work is
engineering groups in general, but female students’ who strongly connected to trust in team members, and
plan to persist in this male-dominated domain also show motivation to perform collaborative tasks may diminish
high self-efficacy (Marra et al., 2009). It therefore also when this is compromised (Dirks, 1999). Studies in
may be important to ensure that integrating LLMs into software engineering emphasize the role of perceived
collaborative work, where many of the exclusionary transparency for trust (P. T. Y. Lee et al., 2024) as well as
practices occur (e.g. William M. Hall, Toni Schmader, the role of perceived task-related competency of another
Elizabeth Croft, 2015), maintains positive impact on team member (Mayer et al., 1995). Presumably, when
long- term professional orientation, mediated by group team members use LLMs to generate code that in its
experiences. form resembles more advanced programmers’ code,
their competency is much harder to judge, especially in
2.2. Effect of LLMs on Learning Groups the earlier stages of a project and by novices. As team
members progress in collaborative tasks, building on
When it comes to collaborative learning settings
others’ code is necessary and requires judgment of the
as in group programming assignments, LLM use reaches
quality of that code. Studies about the relationship of
beyond the effects on the individual, such as learning,
LLM to the judgment of competency show that LLM use
self-perceptions, and future identification. Both socio-
can lead novice programmers to misaligned confidence
cultural (Vygotsky, 1978) and socio-cognitive theories of
regarding their skills and understanding (Prather et al.,
learning (Dillenbourg, 1990) highlight the influence of
2024). Research has not yet addressed if the difficulty
the environment on learning, often enacted through
associated with the judgement of competency also
peer interactions. LLM use by individual learners can
applies to group-related judgement.
potentially influence peer interactions, mediated by
The difficulty in judging contributions might also
technology, and further impact group- related socio-
affect the connectedness of the group. Previously, social
affective outcomes, such as trust in group members and
connectedness, defined via measurements of frequency
connectedness.
of social contact, task assistance and compassion, as well
Research on the use of LLMs in collaborative
as sense of belonging, has been shown to be related to
learning has been limited to the development of tools
well-being (Frieling, M., Peach, E. K., & Cording, J.,
that target collaborative processes at the group-level.
2018). Connectedness can be defined as an affective
For example, Kasneci et al. (2023) speculate that these
outcome of group processes developed directly from
tools can facilitate group discussions by providing
interactions, such as mutual support, but also
feedback and personalized guidance to students to
impressions of others, from their contributions against
improve group participation or give editing
the context of own work on the common project.
recommendations to support collaborative writing.
LLMs could help avoid common faults in the group
processes by integrating information or promoting
2.3. The Role of Process in Collaborative
knowledge convergence and decision-making – all Programming
group-level processes essential for effective I have argued that LLMs used by individuals in
collaboration (Westby & Riedl, 2023; Järvelä & Hadwin, collaborative programming may affect learning and
2013; Khakurel & Blomqvist, 2022). It is noteworthy that socio-affective outcomes. A sole focus on outcomes in a
many of these existing propositions are limited to the collaborative learning scenario is insufficient.
LLM-based tools specifically designed to support group Dillenbourg et al. (1996) argued that process variables
work. However, group members can also choose to use must also be considered when studying collaboration.
LLMs for individual needs, rather than to support group This is because interaction effects between the many
process-related mediators of collaboration outcomes
would prevent reliable causal inference. Given the which contributions fix or keep problems in the code as
dearth of research on process variables related to LLM- indicated by build logs (Chen et al., 2022). These process
mediated contributions in a collaborative process, a measures have been analyzed in relation to student
relationship between the indicators of the process data performance, but not in relation to student perceptions
with learners’ perceptions of the members and the group of each other and the group, as well as with LLM-
need to be established. ingestions within the contributions.
For this, individual and group-level team code
submissions need to be transformed to appropriate
interaction process indicators. Log data from 3. Research Questions
programming projects is different from conversation
data often applied in collaboration research, though To investigate the effects of individual use of LLM-based
conceptual similarities exist. For example, students who tools on group-based learning in programming projects
work on programming projects regularly merge their in higher education, this project poses the following
modified versions of the software into a common research questions:
version. How the students amend the versions and who
does this gives insight into success of previous 1. How does the use of LLM in collaborative
coordination as well as (perceived) value of the programming affect individual outcomes, such as
individual members’ contributions and who maintains learning gain, self-efficacy, and professional
overview of the group’s code. In some groups, major identification, and group-related socio- affective
conflicts result from not being able to amend different attitudes, such as perceptions of teammate
versions to a working product (Tushev et al., 2018). In competency and connectedness?
sum, group-level patterns of logs can make an
impression on student perceptions of others and the 2. What is the relationship between process
group itself. indicators describing individual and group- level
Moreover, depending on the type of contributions, team code with the perceptions of competency
individual roles in relation to the group may also be among team members and group-related socio-
visible in GitHub traces. For example, previous work has affective attitudes?
talked about the phenomenon of “cowboy
programming”, where a group member took over the
management of the relevant parts of the software 4. Research Design
development without including others (Tushev et al., To address these research questions, a series of authentic
2018). Similarly, “free riders” and “social loafers” studies in a university scenario are planned. I plan to
describe common patterns of individual roles students collect the data from group-based programming projects
take on, bringing out negative group work dynamics from the courses targeting novice programmers. This
(Nguyen et al., 2023). choice is due to the focus on the first experiences with
Existing research on process indicators in programming group projects within higher education.
programming creates a foundation for analyzing both The courses are expected to give us a participant number
individual and group processes. According to code of between 600 and 1800 students in 120 to 450 groups.
collaboration project research, best- performing teams We will collect data via interviews, self- reports
show equal contributions, not necessarily the highest with established tools three times throughout the course
total number of commits, but parallel main work times and log data of the evolving code and all GitHub
and work on separate branches of the code (Tushev et activities. The interviews are employed to potentially
al., 2018). Team roles can be visible as one person show causalities between our researched variables.
contributing documentation while another contributes Including a self- and team assessment part in group
the code (Tushev et al., 2018). (Gitinabard et al., 2020) projects is a common strategy for their grading, and we
look at teamwork features on GitHub projects and will extend the usual questions with additional
classify the student teams into three groups, questions. A final presentation of the artifacts is part of
collaborative, cooperative, or solo-submit, the project’s examination and questions about the
differentiating each contribution into types such as bug team’s code show each students’ understanding of the
fix, documentation, test case, or implementation. They software.
look at how many lines of code the members change, The first study will address the question of how
across how many different files, how much they delete, the use of LLM in collaborative programming affects
and informativeness of commit messages written by a individual outcomes, such as learning gain, self-efficacy,
contributor. Another line of work goes deeper into and professional identification, and group-related socio-
contribution quality from code analysis and considers affective attitudes, such as perceptions of teammate
competency and connectedness, also considering Achievement. Educational Technology & Society,
perceived equality of contributions. We will use self- 25(4), 245-256.
reports for established measurement tools and [5] Chen, H.-M., Nguyen, B., & Dow, C.-R. (2022).
interviews to capture these. Code-quality evaluation scheme for assessment of
In the second study, we will analyze process data student contributions to programming projects.
to evaluate relations to perceived effort distribution, Journal of Systems and Software, 188, 111273.
accuracy of judgment in team members’ competency, https://doi.org/10.1016/j.jss.2022.111273
trust, differences in patterns in code contributions, [6] Darvishi, A., Khosravi, H., Sadiq, S., Gašević, D., &
mutual support, prevalent issues in engineering the Siemens, G. (2024). Impact of AI Assistance on
software, perceived gain of skill, perceived appreciation Student Agency. Computers & Education, 210.
from others and identification with the field of study and 104967.
future professions. https://doi.org/10.1016/j.compedu.2023.104967
Since I am at an early stage of this project, I am not [7] Dillenbourg, P. (1990). What do you mean by
set on the methods of analysis yet. We will tentatively “collaborative learning”? In Collaborative
analyze the first data and develop process measures Learning: Cognitive and Computational
appropriate to the theory of group dynamics in Approaches (pp. 1-19). Pergamon.
programming projects and relate them to self-report [8] Dillenbourg, P., Baker, M., Blaye, A., & O'Malley,
measures. The process measures will be sourced from C. (1996). The Evolution of Research on
log data of AI interactions, submitted milestone Collaborative Learning. In Learning in Humans
planning documents for coordination traces and features and Machine: Towards an Interdisciplinary
of the frequently logged contributions (using analytics Learning Science (pp. 189– 211). Elsevier.
of and building on previous code contribution [9] Dirks, K. T. (1999). The effects of interpersonal
evaluation metrics). The intra-group interactions will trust on work group performance. Journal of
include information of who built on top of or modified Applied Psychology, 84(3), 445–455.
whose code, who fixed code cohesion or formatted the https://doi.org/10.1037/0021-9010.84.3.445
others’ code, who contributed comments or [10] Duhigg, C. (2019). What Google Learned From Its
documentation, and where were errors introduced and Quest to Build the Perfect Team. The New York
resolved. Times.
[11] Finnie-Ansley, J., Denny, P., Becker, B. A., Luxton-
Acknowledgements Reilly, A., & Prather, J. (2022). The Robots Are
Coming: Exploring the Implications of OpenAI
I would like to thank my supervisor, Professor Sasha Codex on Introductory Programming.
Poquet, for her extraordinary support in my research Australasian Computing Education Conference,
work and learning, also in the advice given for this 10–19. https://doi.org/10.1145/3511861.3511863
project. [12] Frieling, M., Peach, E. K., & Cording, J. (2018). The
Measurement of Social Connectedness and Its
References Relationship to Wellbeing. Ministry of Social
[1] Bassner, P., Frankford, E., & Krusche, S. (2024). Development New Zealand.
Iris: An AI-Driven Virtual Tutor For Computer [13] Gitinabard, N., Estrada, T., & Bowers, A. (2020).
Science Education. Teamwork Features and Their Impact on Student
https://doi.org/10.1145/3649217.3653543 Collaboration in GitHub Projects. Journal of
[2] Begel, A., & Nagappan, N. (2009). Coordination in Learning Analytics, 7(3), 25–45.
Large-Scale Software Development: Helpful and https://doi.org/10.18608/jla.2020.73.3
Unhelpful Behaviors. [14] Hall, W. M., Schmader, T., & Croft, E. (2015).
https://www.microsoft.com/en- Engineering Exchanges: Daily Social Identity
us/research/publication/coordination-in-large- Threat Predicts Burnout Among Female
scale-software-development-helpful-and- Engineers.
unhelpful-behaviors/ https://journals.sagepub.com/doi/abs/10.1177/
[3] Boyatzis, R., Rochford, K., & Cavanagh, K. V. 1948550615572637
(2017). Emotional Intelligence Competencies in [15] Järvelä, S., & Hadwin, A. F. (2013). New Frontiers:
Engineer’s Effectiveness and Engagement. Career Regulating Learning in CSCL. Educational
Development International, 22(1), 70–86. Psychologist, 48(1), 25–39.
https://doi.org/10.1108/CDI-08-2016-0136 https://doi.org/10.1080/00461520.2012.748006
[4] Chang, R., & Fang, Z. (2022). Effect of AI- Assisted [16] Kasneci, E., Seßler, K., Kuechemann, S., Bannert,
Tutoring on Student Self-Efficacy and M., Dementieva, D., Fischer, F., Gasser, U., Groh,
G., Günnemann, S., Hüllermeier, E., et al. (2023). https://doi.org/10.1145/3597503.3639187
ChatGPT for Good? On Opportunities and [25] Prather, J., Reeves, B., Leinonen, J., Macneil, S.,
Challenges of Large Language Models for Randrianasolo, A., Becker, B., Kimmel, B., Wright,
Education. Learning and Individual Differences, J., & Briggs, B. (2024). The Widening Gap: The
103, 102274. Benefits and Harms of Generative AI for Novice
https://doi.org/10.1016/j.lindif.2023.102274 Programmers.
[17] Kazemitabaar, M., Hou, X., Henley, A., Ericson, B. [26] Salomon, G. (2003). Distributed Cognitions:
J., Weintrop, D., & Grossman, T. (2023). How Psychological and Educational Considerations.
Novices Use LLM-Based Code Generators to Solve Cambridge University Press.
CS1 Coding Tasks in a Self-Paced Learning [27] Tushev, A., Gross, S., & Schaarschmidt, M. (2018).
Environment. Collaborative Programming Dynamics in Software
arXiv. http://arxiv.org/abs/2309.14049 Engineering Education: Insights from Student
[18] Kazemitabaar, M., Ye, R., Wang, X., Henley, A. Z., Project Logs. ACM Transactions on Computing
Denny, P., Craig, M., & Grossman, T. (2024). Education, 18(3), Article 14.
CodeAid: Evaluating a Classroom Deployment of https://doi.org/10.1145/3134822
an LLM-based Programming Assistant that [28] Vaithilingam, P., Zhang, T., & Glassman, E. L.
Balances Student and Educator Needs. (2022). Expectation vs. Experience: Evaluating the
Proceedings of the CHI Conference on Human Usability of Code Generation Tools Powered by
Factors in Computing Systems, 1–20. Large Language Models. CHI Conference on
https://doi.org/10.1145/3613904.3642773 Human Factors in Computing Systems Extended
[19] Khakurel, J., & Blomqvist, K. (2022). Artificial Abstracts, 1–7.
Intelligence Augmenting Human Teams. A https://doi.org/10.1145/3491101.3519665
Systematic Literature Review on the [29] Vygotsky, L. S. (1978). Mind in Society:
Opportunities and Concerns. In H. Degen & S. Development of Higher Psychological Processes.
Ntoa (Eds.), Artificial Intelligence in HCI (pp. 51– Harvard University Press.
68). Springer International Publishing. [30] Wang, L., Gao, R., Váncza, J., Krüger, J., Wang, X.
https://doi.org/10.1007/978-3-031-05643-7_4 V., Makris, S., & Chryssolouris, G. (2019).
[20] Lee, Y.-F., Hwang, G.-J., & Chen, P.-Y. (2022). Symbiotic human-robot collaborative assembly.
Impacts of an AI-based chatbot on college CIRP Annals, 68(2), 701–726.
students’ after-class review, academic https://doi.org/10.1016/j.cirp.2019.05.002
performance, self-efficacy, learning attitude, and [31] Wang, Q., Camacho, I., Jing, S., & Goel, A. K.
motivation. Educational Technology Research and (2022). Understanding the Design Space of AI-
Development, 70(5), 1843–1865. Mediated Social Interaction in Online Learning:
https://doi.org/10.1007/s11423-022-10142-8 Challenges and Opportunities. Proceedings of the
[21] Mayer, R. C., Davis, J. H., & Schoorman, F. D. ACM on Human-Computer Interaction,
(1995). An Integrative Model of Organizational 6(CSCW1), 1–26.
Trust. The Academy of Management Review, https://doi.org/10.1145/3512977
20(3), 709–734. [32] Wang, Shaofeng, Zhuo Sun, & Ying Chen. (2023).
https://doi.org/10.2307/258792 Effects of Higher Education Institutes’ Artificial
[22] Marra, R. M., Rodgers, K. A., Shen, D., & Bogue, B. Intelligence Capability on Students' Self-Efficacy,
(2009). Women Engineering Students and Self- Creativity, and Learning Performance. Education
Efficacy: A Multi-Year, Multi-Institution Study of and Information Technologies, 28(5), 4919–
Women Engineering Student Self-Efficacy. Journal 4939. https://doi.org/10.1007/s10639-022-11246-8
of Engineering Education, 98(1), 27–38.
https://doi.org/10.1002/j.2168- 9830.2009.tb01003.x
[23] Miller-Young, J., Jamieson, M., & Beck, S. (2023).
Diverse Experiences and Belonging in an Online,
First-Year, Team-Based Engineering Design
Course. Teaching in Higher Education, 1–17.
https://doi.org/10.1080/13562517.2022.2162816
[24] Nam, D., Macvean, A., Hellendoorn, V., Vasilescu,
B., & Myers, B. (2024). Using an LLM to Help With
Code Understanding. Proceedings of the
IEEE/ACM 46th International Conference on
Software Engineering, 1–13.