This paper describes an in-progress examination of the diferences in observed perceptions and usability of generative AI (GAI) in MassDigi's Summer Innovation Program (SIP) from 2023 to 2024. SIP is a professional development program where teams of interns create mobile games in the Unity game engine over an 11-week period. Following a previous study using ethnographic methods to examine this topic in summer 2023, a similar study conducted in summer 2024 found dramatically diferent results. Where previously, interns broadly resisted the adoption of methods that incorporated GAI, the 2024 cohort was found to embrace the tools. This paper presents the diferences in findings, explores possible explanations for these diferences, and presents potential paths of research to further investigate the trajectory of GAI adoption amongst creative professionals. We also present four variables that may influence GAI adoption: experience, cultural context, proportion of artists within a community, and shifting societal paradigms.
The continued adoption of generative AI (GAI) practices
across a range of industries is resulting in increased need for
research to better understand the contextual impact of this
technology in specific workplaces. Particularly, workers in
creative industries have been reported to be seeing
disruption as companies seek to replace tasks traditionally held
by artistic practitioners, or otherwise implement these tools
into existing workflows [
This paper focuses on two studies of MassDigi’s
Summer Innovation Program (SIP)- one from the summer of
2023, and one work-in-progress study from the summer of
2024. SIP is a long-established professional development
program where young professionals seeking to enter the
games industry work in teams to develop games at a rapid
pace. Our 2023 study examined the perceptions and
usability of GAI in this program [
The presence of GAI in creative work has prompted
significant concern. Bender et al. acknowledge the potential
social and environmental impact of LLMs, as well as concern
regarding the underlying politics of AI design,
implementation, and advocacy [
Despite all of the fear, concern, and potential for harm
presented by GAI systems, many recognize its potential
value for enhancing creative work. Vimpari et al. find that
alongside feeling concerned and overwhelmed, games
industry professionals are also excited about the prospect of
introducing image synthesis into their workflows [
We recognize the historical role games have played in
the development of harmful technologies. For example,
Cook highlights the long history of games research’s link
to imperialism, war, and pressures from the interests of
capitalism, calling for researchers to resist such connections
[
Our 2023 study sought to identify GAI’s potential impact on games industry workflows and challenges of integrating GAI into creative processes. The study also focused on interns’ general perspectives, especially regarding the ethical implications of GAI’s presence in the industry. Our research questions investigated interns’ assumptions about GAI, their general perspectives, if and how they were using GAI for their work in the program, and what impact incorporating GAI into workflow had.
This 2023 study collected qualitative data using a combination of semi-structured interviews and observation through ifeldwork to answer these questions. Throughout the program, individuals from the program volunteered for a total of 20 short interviews with an average duration of about 10 minutes. Our embedded researcher was present for much of the programs duration, attending events, conversing with teams, and observing the working environment. Thematic coding was used to conduct qualitative data analysis on interviews and field notes.
Our 2024 study was a part of a separate research project focusing on assessing the situated learning experience. The curiosity towards the change in viewpoints of the adoption of GAI originated from researchers’ observation: while the cohort was provided with the same level of guidance and encouragement on using or critiquing GAI, a significantly higher portion of the developers were using GAI. A portion of the interviews were dedicated to the same GAI-focused interview questions covered in the 2023 study.
Each of the 2024 teams participated in a 60-minute session about game design decisions, team collaboration, and communication. Although GAI was not the main topic in the team interviews, they were prompted to talk about its use in the team. Each individual also took part in a 30-minute session, with 20 minutes focusing on the situated learning experience, and the final 10 minutes addressing their general perspectives and assumptions about GAI, if and how they were using GAI for their work in the program, ethical considerations regarding its adoption, and the impact of incorporating GAI into their workflow. Upon our researcher’s observation of the disparity between GAI adoption rates from 2023 to 2024, the 2024 study incorporated the interview questions from 2023 into this process to provide a better point of comparison.
Interviews with research participants and field work were the most efective research methods for both studies due to the highly contextual nature of the working environment, and the games industry more generally. Workflows and practices vary greatly not only across workplaces, but across teams or individuals within a particular site. Because of the integrated nature of our researchers in these studies, as well as the shared program of study’s consistent track record, we were able to gather valuable insights into the diferences of these cohorts despite the distinct research paths of each study.
While SIP has a long-established record, showing its consistency for hiring interns and producing games, we have identified three primary variables that distinguish the 2023 and 2024 cohorts. First, the 2023 group was primarily pulled from applicants attending accredited colleges in the United States, while the 2024 group comes from students or recent graduates attending schools in the United Arab Emirates.
The second key diference was the number of artists in each group. While both cohorts had close to 25 members overall, the 2023 group had 2-3 artists per team (across 6 teams), while the 2024 group had 2-3 artists with suficient professional training in the entire collective – meaning 2024 had about one-sixth as many artists.
Finally, our third observed distinction is that the program director claimed the 2024 interns were generally less experienced. They held fewer degrees, and did not have as much practice developing games. Our study revealed 46.2% had never developed a game before, and 61.5% had never used the Unity game engine before though we do not have similar metrics for the 2023 group.
Despite these diferences, we consider these studies worthwhile to compare. The purpose of this paper is to identify a path towards a future research endeavor, rather than to produce conclusive results. Highlighting these diferences allows us to identify variables that are worth investigating further as we consider the cause of potential increased or decreased adoption of GAI systems within populations.
The observation that incited our interest for this paper, which we view as the most interesting finding we present here, is the increased adoption of GAI amongst the 2024 SIP participants compared to the previous year’s cohort. Every 2024 team made use of GAI for some aspect of the process including brainstorming, programming assistance, grammar checking (most of the 2024 participants were not native English speakers), references for art, and searching for educational or development resources. Additionally, many of the artistic assets in the published versions of the games are sourced from AI-generated content. This was a stark contrast to the 2023 cohort, where none of the published games used AI-produced art in the final product in any capacity and integration of GAI practices in development workflows was both minimal and heavily resisted.
Despite the increased integration of GAI workflows, the artists of the 2024 program still reacted negatively to the technology, in a manner similar to our observations from 2023. This led to conflict amongst team members at times. For instance, towards the end of the 2024 program, an artist refused to make a requested change to some artistic assets, and the team’s producer had to use an AI-generated placeholder until the day before the game launched, leading to negative impact for team relations. Both years, we also observed that SIP participants frequently had private conversations about GAI throughout the duration of the program where they were more willing to share strong opinions, both negative and positive, but were more reserved in team meetings.
From our observations across these two years of this program, we identify four variables that may influence adoption and perceptions of GAI within communities of creative practitioners: 1) experience, knowledge, and technical training; 2) cultural context of communities; 3) proportion of artists within a community; and 4) shifting societal paradigms. These variables will serve as coordinates for a larger study, providing a starting point to establish a framework for analyzing GAI adoption across multiple cohorts.
Through observation from our in-progress 2024 study
of the SIP interns, as well as our previous 2023 study, we
recognize the pressures of a professional games industry
environment to produce quality content at a rapid pace.
While the 2023 group found that generative tools did not
meaningfully increase either quality or eficiency of their
work, they also had a stronger foundation of knowledge to
work from. Especially amongst artists, they had established
artistic practices that allowed them to produce what they
wanted, and they had the knowledge to evaluate specific
places in the development pipeline where AI-generated
content would cause issues. With fewer artists and less
practice, the 2024 cohort did not have this benefit- especially
for the production of art assets, and therefore may have
had more to gain from the introduction of GAI into their
practices than their 2023 counterparts. Supporting this
possibility, Ling et al. highlight how creative practitioners
without artistic backgrounds find more value in integrating
generative systems into their workflows [
We also recognize that the experience of these interns is
situated in the context of culture. The 2024 SIP participants
from the UAE were not only navigating a professional
game development environment for the first time, but also
adjusting to life in a foreign country- obtaining kitchen
supplies, commuting, and communicating are all examples
of day-to-day tasks these interns were dealing with on
top of their professional development concerns. While it
is unclear the exact role this may have played regarding
the interns’ adoption of GAI practices in their
workflowssuch clarity is beyond the scope of this paper- we observed
the members of the program did use generative tools to
assist with this transition. This observation reiterates the
value of investigating niche contexts of GAI
applicationnot only across a variety of domains, but across a variety
of populations within a domain, and even within an
individual community. Avci identifies a correlation
between demographic elements and acceptance of GAI
technologies [
Our 2023 study- and others that consider the perceptions
and reactions artists have to GAI in creative spaces [
We also recognize that the development of GAI technologies and systems is continuing to grow rapidly. Indeed, in the year that has passed from one study to the next, much has changed. The perception and adoption of GAI are situated not only within the culture of a community, but more broadly within global socio-technical paradigms. As GAI continues to grow- especially as many generative systems are advertised for use by the general publicexposure to generative systems becomes more likely, potentially influencing developers interest and willingness to make use of it.
These variables highlight the trajectory of GAI perception and adoption amongst a population over the course of time as a valuable site of future study. Therefore, we plan to conduct a follow-up study of the SIP in the summer of 2025, using these four variables to trace this trajectory over time.
This comparison of our 2023 and 2024 studies of SIP reiterates the need for investigation of GAI’s influence in a variety of niche domains of study, as well as the value of investigating a multitude of populations within a particular domain. Interviews with program participants across two years revealed a dramatic shift in willingness to introduce generative systems into game development practices, highlighting the contextual influences of GAI adoption and perception.
The two studies we discuss in this paper both focus on the same program. However, these studies were neither designed to investigate the same research questions, nor to have their findings compared. The scope of this inquiry presents valuable insights from our observations regarding future research paths, but cannot provide clear correlation or rigorously informed analysis.
We plan to continue our investigation of SIP, focusing on the continued trajectory of GAI adoption. We hope to see future studies that highlight the inuflence of experience, culture, presence of artists, and social paradigms on the adoption and reception of generative systems.
This material is based upon work partially supported by the National Science Foundation (NSF) under Grant No DGE1922761. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF. We would like to thank Monty Sharma, Tim Loew, Walt Yarbrough, and all of the participants of MassDigi’s 2023 and 2024 Summer Innovation Program.