Past work has investigated the degree to which human listeners may be prejudiced against music knowing that it was created by artificial intelligence (AI). While these studies did not find a statistically significant relationship, the listening experiments were performed with music genres such as contemporary classical music or free jazz which are fairly welcoming of technology. In this work, we explore this prejudice in a context where strong opinions on authenticity and technology are typical: Irish traditional music (ITM). We conduct a listening experiment with practitioners of ITM asking each subject to first listen to a human performance of music generated by a computer in the style of ITM (this provenance is unknown to the listener), and then rate how much they like the piece. After rating all six pieces, each subject listens to each again but rates how likely they believe it is composed by a computer. The results of our pilot study suggest ITM practitioners tend to rate belief in AI authorship lower the more they rate liking a tune.
One’s experience of music can involve numerous factors, some of which are related to what
one senses (e.g., skill and efort, materials, setting), and some related to what one knows (e.g.,
programmatic information, historic context, authenticity). Music appreciation is influenced by
musical properties [
When applying artificial intelligence (AI) to music creation, how does one’s knowledge about
the involvement of AI impact their appreciation of the resulting music? Mofat and Kelly [
In this paper, we investigate the extent to which bias against AI is present in a context where computer authorship is considered at odds in the musical practice. To what extent does the culture and context of the music to which AI is being applied matter when it comes to a listener’s perception of the results? Inspired by these prior studies, we investigate the relationship between liking for a musical piece and the belief that it is AI-composed in the context of Irish traditional music (ITM). Participants of our experiments are active practitioners of ITM, and are drawn from traditional music programs in Ireland. Our hypothesis is that the context of ITM is one in which authenticity is so heavily human-centred that practitioners will show a bias against liking music they believe is authored by AI, or alternatively against believing music that they like is AI-composed.
In the next section, we review the details of past experiments in this area [
One of the challenges of studying bias against AI involvement in music is that there is no standard methodology on how to proceed. Diferent experimental designs have been proposed, as described for the following three studies.
Mofat and Kelly [
Pasquier et al. [
Moura and Maw [
Irish traditional music (ITM) is a complex genre encompassing many practices, and is an
important part of Irish culture and identity [
The practice of ITM is accompanied by values that emphasize authenticity, etiquette and often
nationalism [
Considering the context of ITM and its values, there exists tension around innovation, as
2https://www.antoireachtas.ie/
well as how ITM should be used and presented [
The hypothesis we want to test is the following: an ITM practitioner will exhibit an inverse relationship between liking a piece of music and believing it is authored by AI in the context of ITM.
We drew participants from traditional music programs at the University of Limerick, Ireland. The first cohort (E1) includes 20 participants (12 women and 8 men, aged 18–64 years, M= 36.75, SD=16.02), who are students, teachers and technicians from the BLAS International Summer School of Irish Traditional Music and Dance.3 The second cohort (E2) includes 26 participants (20 women and 6 men, aged 19–60 years, M=25.73, SD=10.14), who are students of degree programs ofered at the Irish World Academy of Music and Dance. 4 Each participant was compensated with a €20 gift card.
Since we expected participants to be very knowledgeable about ITM, we decided not to use
existing traditional tunes as stimuli for this experiment to prevent familiarity issues. Instead,
we hired professional Irish accordionist Padraig O’Connor5 to select six tunes that he likes from
large collections generated using a particular AI system [
O’Connor recorded himself playing each tune on solo accordion, with stylistic ornamentation, variation, bass, and harmonic content added as he saw fit. We permitted O’Connor to make minor changes to the notation of each tune as he wanted, but very few changes were made.
4https://https://www.irishworldacademy.ie/
5http://www.paudieoconnor.com/
6O’Connor participated as one of four judges in the AI Music Generation Challenge 2020, which focused on generating
plausible Irish double jigs [
Having a professional ITM practitioner select and perform the stimuli ensures that they are presented in a realistic setting with an authentic performance. The resulting six stimuli are about the same duration (M=75.5 seconds, SD=1.51) and tempo (M=106 bpm, SD=1.84). All stimuli were recorded by O’Connor at his home with the same accordion and single microphone audio setup to avoid discrepancies in the audio quality. The stimuli were encoded with mp3 format (MPEG ADTS, layer III, v1, 128 kbps, 44.1 kHz, stereo) and are available online.7
For both cohorts (E1 and E2), the experimental sessions took place in a media lab at the University of Limerick. We designed and hosted the experiments with a web-based interface built using jsPsych [29]. All participants were briefed and provided their informed consent before taking part. Participants used the same computer and headset models. A soundcheck using a recording of an Irish traditional tune allowed the participants to adjust the volume setting to a comfortable level.
The experimental paradigm comprised two tasks in series: the “liking task” and then the “authorship task”. · In the “Liking task”, the participant is asked, “How much do you like the tune?”, and is given a 5-point Likert scale anchored by the labels (from left to right) “Don’t like it at all”, “Don’t like it”, “Neutral”, “Like it” and “Like it a lot”. · In the “Authorship task”, the participant is asked, “How likely do you believe that the tune is composed by a computer?”, and is given a 5-point Likert scale anchored by the labels (from left to right) “Not likely at all”, “Not likely”, “Neutral”, “Likely” and “Very likely”. These tasks were completed in this order to avoid liking ratings being influenced by a prior mention of AI. For each task, on-screen instructions were provided. In both tasks, the participants were encouraged to use the full range of the scale. The rating scale only appeared once the stimulus had finished playing. The order of stimuli in each task was randomized for each participant.
After completing the two tasks, the participants filled out a short questionnaire about demographics, musical practice, and familiarity with ITM. They answered the following questions: • age: “How old are you in years?” • gender: “What is your gender?” • nationality: “What is your nationality?”
• education: “What is the highest level of education you have achieved?” • mus_pro: “Are you or have you been a professional musician?” • irish_fam: “How many Irish traditional tunes can you play/sing from memory?”, with options “0–10”, “11-50”, “51–100” and “100+” • instrument: “What is (are) your main instrument(s)” As a final question, the participant was asked to freely describe any strategies they used to determine whether a tune was composed by a human or a computer.
Each trial of this experiment lasted about 20 minutes. The experiment was reviewed and approved by the Ethics Committee of KTH (V-2021-0615).
We code the demographic data as follows: age is left as an interval variable; gender and mus_pro are nominal with two levels; nationality is nominal with two levels based on being Irish; instrument is nominal with two levels based on whether accordion is specified; education and irish_fam are both ordinal with four levels.
A two-sample -test on demographic data collected during the experiment shows that E1 and E2 difer significantly in age ( = 2.79, df = 44, < .008). A Mann-Whitney U test shows a significant diference between E1 and E2 only in mus_pro ( = 171.0, < 0.03), but does not show a significant diference in education ( = 315.5, > 0.16), irish_fam ( = 293.5, > 0.44), gender ( = 304.0, > 0.22), nationality ( = 196.0, > 0.06), or instrument ( = 269.0, > 0.74).
Our analytical approach involves two stages. In the first stage we model the bivariate responses of the participants using linear mixed-efects models [ 30]. In the second stage we regress on the individual coeficients of the model in the first stage using the demographic covariates mentioned above.
Denote by the value of the “Liking” by the -th participant in the trial they evaluate the -th tune, and the value of their belief that the same tune was composed by an AI (AIC). One model relating these responses is = + + + + (1) where and are the intercept a participant-based slope, respectively; is a fixed efect of tune and is a random efect of participant; and is the residual error. An alternative mixed-efects model of the bivariate responses casts AIC as a function of reported liking: = + + + + . (2) All random quantities are considered independent with zero means and variances 2 and 2, respectively. Each model considers tune to be a factor with six levels (and thus a fixed efect), because we only wish to draw conclusions about the population of participants. In other word, we wish to generalize our conclusions to ITM practitioners for these six stimuli, and not to the population of tunes O’Connor would curate and perform from a large collection.
The diferences in interpretation of these models is important. Model (1) poses liking as a function of the belief of being AI-composed, which can be motivated by current thinking about how aesthetic appreciation is a function of many factors, such as the value a stimulus has for a participant, the context of the perception of the stimulus, and the physiological state of the participant [31]. Model (2) seeks to determine how the factors considered by a participant in rating their belief a tune is AI-composed relate to or modulate factors they considered in rating their liking of the tune. If is significantly diferent from zero then one might conclude there to be a significant overlap of these factors. However, since the number of stimuli are few and the time between the two tasks is short, we expect there to be some contribution of memory informing AIC for a participant. In other words, their memory of having rated their liking of a tune a particular way could inform their AIC rating of the tune. Model (2) can thus be motivated by a more machine-learning oriented goal where one wants to predict a participant’s AIC rating from their rating of their liking of the tune.
In the second stage, we attempt to explain the participant coeficients of the first stage using participant covariates. To this end, denote by the vector of covariate measurements, available for the -th participant. The model is a standard regression = 0 + + (3) where 0 and are the model coeficients and is the usual independent error terms assumed normal with variance ′2. In the above formula, for simplicity, we write to mean the fitted coeficient stemming from the first stage of the analysis.
Finally, we analyze the free-form response about strategies using the method of constant comparison [32] to identify recurring themes and convergences in the strategies of participants. This allows us to explore potential explicit bias and identify which elements of ITM practice are expected to be diferent when composed by a computer. These, in turn, might reflect a listening focus in ITM practice that may inform future research in empirical musicology and generative systems.
We now fit our models to data of each cohort individually, which is motivated first by the regression on the coeficients of Model (1), and second a diference in sampling participants between them at the University of Limerick. More specifically, E1 consists of students of a twoweek-long summer school focused on Irish traditional music, as well as several staf (professors, administrators, etc.). E2 consists of students enrolled in longer-term educational programs about Irish traditional music. These cohorts may difer in other unmeasured ways as well, e.g., motivations to attend a two-week summer school are diferent from those to attend a longer educational program at a university. This could explain why cohort is a significant factor in the regression of the efects of Model (1) for the pooled data.
For Model (1) we see that E1 has a much larger number of significant negative terms (14 of 20) relative to the number for E2 (4 of 26). For Model (2), E1 has 18 terms that are significantly less than zero, of which E2 has 7. For E1, when regressing on the coeficients of
Model (1), we find significant positive contribution from irish_fam ( = 0.049, = 2.69, = 12, < 0.02) and age ( = 0.003, = 2.45, = 12, < 0.031). The adjusted 2 of this model is 0.301. When regressing on the coeficients of Model (2) for E1, we find significant contributions from all covariates except mus_pro: age ( = 0.005, = 3.72, = 12, < 0.003), education ( = − 0.048, = − 2.30, = 12, < 0.04), irish_fam ( = 0.078, = 4.42, = 12, < 0.001), gender ( = − 0.12, = − 2.28, = 12, < 0.042), nationality ( = − 0.145, = − 2.87, = 12, < 0.015), and instrument ( = − 0.248, = − 3.9, = 12, < 0.003). The adjusted 2 of this model is 0.676. When regressing on the coeficients of either model for E2, we find no significant contributions from the covariates (adjusted 2 < 0.18).
The explanatory variables had interesting imbalances between cohorts which we briefly describe. Concerning variable age, E2 was younger than E1, but both cohorts covered about the same range of age; that is, E2 has a more skewed distribution than E1 in age. Although we can test equality of means and reject it strongly between cohorts ( = 4 × 10− 10), the shape of distributions is such that the influence in the analysis is much deeper than just the location. Concerning binary variables, mus_pro had opposing proportions between cohorts; whereas variables gender, nationality had less pronounced diferences between cohorts, only variable instrument had similar patterns between E1 and E2.
At the conclusion of the listening test, each participant is asked “What strategies did you use to determine if a tune was composed by a human or a computer?”. Our qualitative analysis of the free-form responses done across both cohorts find five main strategies:
∙ 17 participants (37.0%) (5 from E1, 12 from E2) reported listening to repetitions and patterns. Tunes that were deemed “overly repetitive” were associated with computer authorship.
∙ 12 participants (26.1%) (8 from E1, 4 from E2) reported listening to structure and harmony. In particular, participants listened to how phrases were linked together as well as chord progressions and cadences. “Unnatural” harmonic and melodic structures or chords following a
Adjectives/Descriptives used for computers/AI for humans inorganic, unnatural, uncanny val- catchy, flowed, clarity, fluid, like ley feeling, simple, weird, rigid, out speaking language, alive, emotive, orof place, robotic, unusual, logical, ganic, usual, finicky, surprising, susgeneric, predictable, algorithmic taining interest, with purpose, familiar, creative, thought out, natural structure deemed “very rigid” were associated with computer authorship.
∙ 11 participants (23.9%) (4 from E1, 7 from E2) reported listening to variation and ornamentation. Tunes with more variations in the melody and with more ornamentation/embellishment were associated with human authorship.
∙ 10 participants (21.7%) (4 from E1, 6 from E2) reported listening to familiarity. Tunes that were deemed to fit with the style of Irish music and sounded familiar in that aspect were associated with human authorship. By contrast, tunes deemed too similar to existing tunes or too “generic” were associated with computer authorship.
∙ 7 participants (15.2%) (5 from E1, 2 from E2) reported listening to instrumental technique. Tunes with elements deemed “unnatural” to play on the accordion, such as “unusual” phrasing or range were associated with computer authorship.
It is also noteworthy that 5 participants tried to use strategies based on audio quality, thinking that some of the tunes were synthesized (although all tunes were human-performed in the same conditions). This shows some potential confusion on what is meant by “computer-composed”.
We also looked at the adjectives/descriptives used by the participants when talking about the tunes they believed to be AI- or human-composed (see Table 2). in order to have a basic sentiment analysis of their report. When talking about what they believe to be computer generated, the descriptives used by participants were almost exclusively negative, with a few neutral ones. Conversely, when talking about what they believe to be human-composed, the descriptives used were for the large majority positive. This diference points to a potential conscious prejudice regarding what a subject believes the capacities of AI or computers are compared to humans when it comes to ITM composition. Anecdotal but still amusing, one of the participants even claimed that their strategy was assuming that “the tunes [they] liked better were composed by humans and the ones [they] disliked were composed by the computer.”
Our quantitative analysis points to a plausible bias among ITM practitioners for these six AI-composed tunes: they tend to like more the tunes they deem hardly likely to be composed by an AI. Alternatively, the more they report liking a tune the less they report believing the tune is AI-composed. The diference in results with the previous studies could validate our hypothesis on the importance of the context (both in terms of musical culture and participants) when it comes to observing such a bias. However, as it stands, this study can only be considered a pilot study before performing a power analysis of the experiment to determine its likelihood of producing a Type-I error. Nonetheless, this pilot study provides us with valuable information regarding further testing of our hypothesis. In particular, expertise and professionalism must be explicitly defined, accounting for diferences between instrumental or vocal traditions within ITM. Our wording of the professionalism question (mus_pro) makes no distinction between career performers and a musician who has been paid to play at some event.
The analysis of the self-reported strategies of the participants suggests a conscious bias
against AI authorship with more limited expectations on computer capabilities when it comes
to some musical criteria, and an overall negative sentiment when describing tunes they believed
to be computer composed. It is interesting to compare that attitude with the reception of
technology in general in ITM. Cawley [
It is therefore dificult to argue that a bias against AI authorship emerges only from a general
negative attitude toward technology. The diference in attitude that arises in the reception
of AI music generation shows that this practice might raise some specific issues. A potential
explanation, suggested by our qualitative results, could simply be the belief that current AI
systems are not skilled enough to reach human capabilities and we are observing a similar bias
as in Kroger and Margulis [
It is also worth noting that AI music generation is not the only use of technology where tension becomes apparent in ITM. For instance, Tunetracker [36] – a software designed to “surveil” local sessions in a pub by documenting which tunes were played which days and in which configuration with other tunes – was met with concerns from practitioners going from the sheer presence of a microphone in a pub (although it was explicitly not used to store or broadcast recordings), to more complex issues such as the idea of the homogenization of the practice, or being caught by the Irish Music Rights Organisation seeking to collect royalties on modern, copyright protected tunes.
Overall, the results of our pilot study are encouraging. First, for these six tunes and our ITM practitioners, our quantitative analysis shows evidence that they tend to have a negative association between liking a tune and believing that it is AI-composed. Second, our qualitative analysis points to a conscious prejudice against the application of AI systems to ITM. The results from this pilot study encourages us to continue our investigation, and study larger populations. It would be interesting to conduct an online version of this listening test in order to test our hypothesis in the general population, and observe how bias might appear in non-practitioners of ITM. It would also be interesting to test whether such diferences hold in other musical traditions that value authenticity in ways similar to ITM. However, given the potential importance of the cultural context for the observation of bias, it is dificult to define a standardised experimental design. Appropriate decisions should be taken to adapt to the diferent musical traditions and to the participants. We also want to conduct a more in-depth qualitative study involving ITM professionals and students using process-oriented research methodology such as Think-Aloud Protocols involving introspection and retrospection [37] as well as participant-oriented research with semi-guided interviews [38]. This research would help us have a better understanding on the prejudice ITM practitioners may have with the involvement of AI in music composition or AI-assisted co-composition, and help inform future work on the ethics of this field.
This paper is an outcome of MUSAiC, a project that has received funding from the European Research Council under the European Union’s Horizon 2020 research and innovation program (Grant agreement No. 864189). We would like to thank P. Cotter, S. Joyce, N. Keegan, and A. Dormer for letting us conduct our study at the BLAS Summer School at the Irish World Academy of Music and Dance, University of Limerick, Ireland. We would like to thank P. O’Connor for taking part in the creation of the corpus of stimuli and for his interpretation of the tunes. We would like to thank A. Clemente for discussions about the psychological aspects of this work. in the style of O’Neill’s “1001”, Journal of Creative Music Systems (2021). [28] B. L. T. Sturm, 2021, 58,105 irish-style double jigs, URL: http://kth.diva-portal.org/smash/ record.jsf?pid=diva2%3A1562396. [29] J. R. de Leeuw, A javascript library for creating behavioral experiments in a web browser,
Behavior research methods 47(1) (2015) 1–12. [30] J. J. Hox, M. Moerbeek, R. Van de Schoot, Multilevel analysis: Techniques and applications,
Routledge, 2017. [31] M. Skov, Aesthetic appreciation: The view from neuroimaging, Empirical Studies of the
Arts 37(2) (2019) 220–248. [32] K. Henwood, N. Pidgeon, Grounded theory in psychological research, in: P. Camic, J. Rhodes, L. Yardley (Eds.), Qualitative Research in Psychology: Expanding Perspectives in Methodology and Design, American Psychological Association, 2003, pp. 131–156. [33] B. Duggan, B. O’Shea, Tunepal: searching a digital library of traditional music scores,
OCLC Systems & Services: International digital library perspectives 27(4) (2011) 284–297. [34] R. Pope, Creativity: Theory, History, Practice, Routledge, 2005. [35] C. G. Johnson, The creative computer as Romantic hero? Computational creativity systems and creative personae, in: Proc. of the International Conference on Computational Creativity, 2012, pp. 57–61. [36] B. Duggan, N. M. Su, TuneTracker: Tensions in the surveillance of traditional music, in:
Proc. of the ACM Conference on Designing Interactive Systems, 2014, pp. 845–854. [37] T. Boren, J. Ramey, Thinking aloud: Reconciling theory and practice, IEEE Transactions on Professional Communication 43(3) (2000) 261–278. [38] G. Saldanha, S. O’Brien, Research methodologies in translation studies, Routledge, 2014.