This paper explores the attributes necessary to determine the creative merit of intermediate artifacts produced during a computational co-creative process (CCC) in which a human and an artificial intelligence system collaborate in the generative phase of a creative project. In an active listening experiment, subjects with diverse musical training (N=43) judged unfinished pieces composed by the New Electronic Assistant (NEA). The results revealed that a two-attribute definition based on the value and novelty of an artifact (e.g., Corazza's efectiveness and novelty) sufices to assess unfinished work leading to innovative products, instead of Boden's classic three-attribute definition of creativity (value, novelty, and surprise). These ifndings reduce the creativity metrics needed in CCC processes and simplify the evaluation of the numerous unfinished artifacts generated by computational creative assistants.
intermediate products, named proto-artifacts. Traces of these dimensions can be found in practitioners’ accounts The increasing learning and predictive capabilities of of their experiences with CCC in the arts, spontaneously computational agents consistently find new ways of par- alluding to concepts traditionally discussed in creativticipating in the arts, design, and humanities, such as in ity literature such as surprise: “working with an AI is architecture [1], creative writing [2], music composition not dissimilar as working with a human being, given its [3], and video game design [4], speeding the generation of capacity to surprise you, because that is where the art alternatives, suggesting concepts, generating variants, or comes in. That is where the magic comes in, in any kind automating arrangements. Computational co-creativity of performance or working with anybody or anything. (CCC) is the field within the domain of computational [Once surprise comes in] you are able to intervene in creativity (CC) which deals with the collaborative pro- what has been generated” [8]. cess between humans and computational agents aiming We argue that the proto-artifacts in human-agent coat producing creative artifacts. Such collaborative open creation processes are essential factors of CCC not acprocess in often modeled in terms of sequences of sub- counted in traditional CC. Thus, we propose the adopprocesses, rather than automated generative pipelines tion of Corazza’s definition of dynamic creativity as it [5, 6], and many of such models characterize it as iter- addresses both the process and the potential of protoations over a generative phase and an evaluative phase artifacts in hybrid collaborative structures. This paper [7]. explores the validity of using a two-dimensional defi
As this type of human-agent collaborative partner- nition of creativity (based on originality and efectiveship takes hold, the interests of CCC researchers have ness) [9, 10], instead of Boden’s three-dimensional one shifted from studying the creative quality of a system’s (based on value, novelty and surprise) [11] to assess protooutput (the classic CC approach) to examining how cre- artifacts generated by computational agents. ativity evolves during the ongoing stream of co-creative The method used is a subject-based empirical evalsubprocesses, raising new questions such as: what di- uation of a musical CCC system that exemplifies the mensions should be used to assess a creative process, generic CCC process observed in creative disciplines. how should they be interpreted, what information do Subjects with diferent levels of musical training were these dimensions provide about the creative quality of integrated into the co-creative workflow of producing an album and appraised the creative quality of intermeJAouinsttrParlioaceedings of the ACM IUI Workshops 2023, March 2023, Sydney, diate pieces (proto-artifacts). The experimental method $ jsal@illinois.edu (J. Salamanca); daniel.gomez@upf.edu takes into account CCC practitioners’ reflections on the (D. Gómez-Marín); sergi.jorda@upf.edu (S. Jordà) value of creating with AI systems and relates Boden’s © 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License [11] and Corazza’s [10] attributes of creativity. The folCPWrEooUrckReshdoinpgs IhStpN:/c1e6u1r3-w-0s.o7r3g ACttEribUutRion W4.0oInrtekrnsahtioonpal (PCCroBYce4.0e).dings (CEUR-WS.org) lowing sections briefly introduce the field of CCC assessment, describe the apparatus used to generate musical pieces, discuss the experiment, and present the results.
The paper concludes with reflections about why a twodimensional assessment of creativity might sufice to characterize CCC processes, as well as its applications and limitations.
CCC encompasses creative processes where two or more participants actively collaborate, and at least one of them is a computational agent [12]. While humans adjust the parameters of conceptual spaces, computational agents efectively explore such conceptual spaces reflecting the human ability to select and refine the best ideas [13]. As a result, humans and agents make creative, mutually influential contributions to an artefact [ 14]. This process coincides with the TOTE behavioral model (test,operate,test,exit) that accounts for how humans execute plans to pursue goals, recursively evaluating the incongruity between the state of machine-generated intermediate artifacts and the intended goal [15]. An interesting observation is that a positive afect is often a cue that a person is moving toward the goal and negative afect signals the opposite[ 16]. The open ended nature of a CCC process entails that such evaluations are carried out by a human estimator who tries to maximize the chances of achieving something valuable with no certainty of success [10]. In this paper, we are interested in such evaluative dynamics that determine how creative are proto-artifacts in a CCC process.
In traditional CC, generative processes are commonly described as iterative structures [17, 18, 19, 20, 21, 22] and can be roughly characterized as two primary subprocesses: producing artifacts and judging them. However, it is not clear how or what to measure in each subprocess. More so if they entail collaborative work in the arts and humanities [23]. Such indetermination has been a known challenge for CC evaluation frameworks such as FACE [6] and SPECS [5] because subjects fail to understand the concepts of creativity used in the assessment or because the assessment relies on subjects’ knowledge of the inner workings of the systems being evaluated [7]. Nowadays, the question of how to evaluate computational creativity remains open and continues to mature as technical and social applications of systems evolve.
A key concern of CCC assessment is the definition of a framework to judge uncompleted work leading to a creative product. Corazza’s definition of dynamic creativity is a good starting point: “[c]reativity requires potential originality and efectiveness” [ 10, p. 262]. The word ‘potential’ conveys the openness of the co-creative process and the latent merit of its intermediate products.
A dynamic creative process has a mutable focus, incorporates the intermediate assessment of proto-artifacts, and provides feedback in response to contextual conditions. Corazza suggests that assessing a collaborative creative process implies a dynamic evaluation of an agent’s production of unfinished artifacts by another agent that takes the role of an estimator. The intermediate outcome of the process is filtered as estimators foresee the consequences of adopting or rejecting such proto-artifacts.
Furthermore, Corazza suggests the provocative idea that "[d]iscrepancies between [multiple] estimators’ assessments are a sign of potentially disruptive novelties, generating the necessary energy for transformation of a domain” [10, p. 265].
In CC and CCC, estimators (the humans assessing proto-artifacts) are usually required to use two-attribute or three-attribute definitions of creativity as inputs to assess and curate the results of a co-creative process.
To mention some of the most prevalent definitions, the standard definition of creativity [9, 24] and the dynamic definition of creativity [ 10] use originality and efectiveness attributes. Boden defines creativity as the capacity to obtain surprising, valuable, and novel ideas [25].
Her definition has been further expanded and formalized mathematically [e.g. 26], remaining as the prevalent model in CC literature as it applies to humans or generative devices without preference for either. The following definitions intend to clarify some of these attributes and how they overlap, but are not an exhaustive account of the literature.
Originality. The originality of an artifact accounts for its authenticity in terms of the independence from precedent realizations, or for the ingenious repositioning of an existing work in an new domain, as does readymade art. Originality is closely related to novelty because it derives its appraisal from being the first to occupy an unclaimed space in a domain or being the first exemplar of a new domain.
Efectiveness . This term describes the ability to produce a result. It is used in the standard definition of creativity as a criterion for eliminating trivial instances that may qualify as original or novel. Some definitions of creativity use usefulness, fit, or appropriateness to convey the same meaning. For Runco and Jaeger [9], it also takes the form of value when creative pieces are appreciated in a market. Corazza uses efectiveness and value interchangeably to convey meaningfulness.
Novelty. The novelty of an artifact can be fully appreciated by the domain-experts as they know a vast portion of the cognitive space of the domain. Therefore, they are observed and dissociated in electroencephalogram sigresponsible for identifying if an artifact extends the do- nals. It is suggested that “humans use surprise as a signal main’s boundaries or, even more, transforms the domain to decide when to adapt their behavior, while they use itself. For Boden, novelty has two meanings: when some- novelty to decide where and what to explore—to eventhing is new to its creator (psychological creativity), and tually develop an improved world-model [...] novelty when something comes to life for the first time in hu- is more related to memory-recall and surprise is more man history (historical creativity) [25]. Thus, historical related to predictions.” [33, p. 1]. creativity corresponds to originality. In summary, Corazza and Boden propose attribute as
The concept of novelty has been approached scientifi- sessment frameworks with diferent number of
dimencally in AI and technology literature, perhaps more than sions. Originality and novelty are overlapping concepts
others such as value and surprise [
Surprise. Assessing surprise during a CCC process edges they are separate concepts but argues that surprise gives clues to the level of fulfillment or intensity of the and novelty could be part of a mental process of joint creative process experienced by a subject. This attribute feeling-appreciation. has been used to measure creativity in finished artifacts None of them can be measured in absolute terms as (e.g., [28]), as a synonyms of non-obviousness in patent they are sensitive to how, when, and by who the assessevaluation [20], and as a proxy of the quality of the CCC ment is made. To the best of our knowledge these frameprocess from a practitioner’s perspective. Although the works have not been evaluated in practice and this study subjective nature of surprise could be a shortcoming attempts to shed light on the interplay between them when judging a finished artifact, it might serve to assess specifically in the assessment of CCC processes. The the performance of a computational agent’s creativity, following sections elaborate and operationalize a generaespecially when the estimator is not a domain expert. tive musical assistant and assess the co-creative process Boden specifies three causes of surprise: when unlikely judging the value, novelty and surprise of proto-artifacts things happen, when unexpected ideas fit know concepts, in a musical setting. and when new ideas break the boundaries of established conceptual spaces.
Value. Creativity researchers generally agree that do- 3. Computational Co-Creation main experts classify an artifact as creative when they With The New Electronic positively evaluate its value within a field and its subse- Assistant (NEA) quent domain. Boden argues that the concept of value, unlike novelty, is elusive [25]. The usefulness of value We use the domain of music to exemplify a CCC context in defining creativity is not straightforward, as social in which a computational agent generates real artifacts judgments of value change over time, as in the case of to be evaluated by human subjects. The complete exartworks that prove to be valuable years after they were periment is presented in Section 4, while this section ifrst presented, or artists who are considered creative describes the CCC generative system and its inner workafter they have died [29]. Both value and novelty are ings. subject to the scrutiny of appraisers. They look for virtue The New Electronic Assistant (NEA) is a music system in the former dimension, while they look for originality capable of analyzing a musical style from a symbolic corin the latter. The value of a potentially creative artifact pus and generating short musical fragments in such style cannot be measured until it is deployed and evaluated (melody and chord accompaniment). Once a melody and by estimators. Only domain experts or gatekeepers ulti- its accompaniment are generated, NEA allows a user to mately judge the value of an artifact [19, 30]. excerpt real-time transformations at four diferent levels:
While novelty and surprise are subjects of the cogni- rhythmic, dynamic, pitch and density1. tive sciences and neuroscience, value is a social construc- NEA is designed as a loop generator for music compotion [31, 32]. Corazza claims that "It is, however, evident sition and performance. A single NEA instance is suited that novelty and surprise are not disjointed dimensions, to complement pre-recorded or real-time performed mabecause if an item is expected, both surprise and concep- terial or even to conform an ensemble of multiple NEA tual novelty are denied." [10, p. 259]. Moreover, recent research claims that novelty and surprise are human re- 1Basic functionality videos can be found in this link: actions independent but close to each other that can be https://youtube.com/playlist?list=PLD3SOdFCvDNkOBA9Gh3FTAncouL6xLw1
instances. This latter configuration of instances can be used to achieve rich polyphonic musical arrangements, specially when diferent NEAs generate complementary melodic styles (e.g., bass lines, main melodies, vocal melodies, etc.) and share information among them (i.e. the chord progression of a generated melody can be trans- As presented in the section 2, a creative activity can be ferred to other instances, unifying the whole set of gener- roughly simplified as a two-stage process: generation of ated melodies). Therefore, provided the performer makes multiple ideas and refinement of the best ones. In typical a correct selection of training styles and real-time set- CC experiments, humans or computational agents meatings, this parallel propagation of information allows for sure the result of the process, reflecting on how creative highly creative mixes of musical material with low efort. the generated output is. Instead, this study is interested
The nature of recent generative interactive systems in how creative are the proto-artifacts produced during such as NEA suggests a shift in the traditional workflow the idea-generation phase. To that aim, an active listenof composition and production. Traditionally, a music ing experiment was designed to examine to what extent performance process has required precise embodied co- subjects apprehend the concepts of novelty, surprise, and ordination in a constant listening/performing cognitive value and apply them to assess CCC-generated protoloop, where music is listened to by a performer as she con- artifacts. currently plays the exact movements on a gesture-to-note Subjects with diverse musical training levels were reinstrument contributing to the composition. But inter- cruited and contextualized as participants in a musical acting with a generative system such as NEA requires a album production. Their task was to listen to musical diferent share of skills. Clicking on a graphical interface stimuli, score them, and decide which ones progress to of knobs and sliders replaces the motor skills needed to the next iteration of the creative process. The stimuli execute the instrument. The user of NEA seeks to gener- were a sample of autonomously generated musical pieces ate a variety of unfinished new melodies in real-time and produced by a multi-NEA system. judge their potential to become something greater until the "right one" is supplied and then transformed. The 4.1. Method classic procedure of playing note-by-note in an instrument is replaced by fast critical filtering and real-time parameter tweaking to obtain high-level transformations of intermediate artifacts. There is a shift from motor reaction to fast acoustic discrimination: agile, coordinated embodiment resigns to collected, prospective assessment.
To fulfill the purpose of NEA as a musical generative system, it uses synthesizers and a mixer to convert notes to sound so that the music is perceived. The synthesizers stand out for their ability to model sound in flexible and resourceful ways, especially the sound property known as timbre. Timbre is the character or identity of the source reproducing the notes (i.e., the timbre of a guitar is diferent than that of a trumpet even though both play the same notes). The mixer, on the other hand, allows control of the intensity of a sound from mute to very loud.
At both the synthesizing and mixing stages two complementary systems have been developed allowing for easy prototyping of new material using multiple instances of NEA. Describing these systems is beyond the scope of this paper but in general terms they complete the experience of a NEA’s user seeking to automate certain aspects of real-time music creation.
Materials. Eighteen iterations of the multi-NEA system yielded a constant stream of evolving electronic music pieces scattered throughout the conceptual space of ambient music. One-minute fragments were selected from each of them and used as stimuli. In addition, Two randomly selected control stimuli were duplicated to evaluate the consistency of subject’s responses. Specifically, stimuli 1 and 8 are the same, as well as 11 and 19. In total, twenty stimuli were arranged in four diferent sequences to prevent biases. Each participant listened to one sequence.
The multi-NEA system used to generate the pieces was trained with classical and pop music melodic styles while timbre and structure were managed by the systems briefly explained in section 3 .
Participants. The experiment had 43 participants, 37.2% (16) identified as female, 46.5% (20) identified as male, and 16.3% (7) undeclared their gender. Their musical training is homogeneously distributed throughout professional musicians to amateur music producers and performers range. The average musical training is 3.3 (sd = 1.7) on a scale of 1 to 6, with 1 being no-training and 6 being a professional musician. Among participants, 81.4% (41) have a medium to high knowledge of electronic music, and only 4.6% (2) reported no knowledge of electronic music.
Procedure. Participants were primed with the following script: For this listening session you are going to play the role of a music producer part of a creative group working on a new album. Your task is to listen to several pieces and assess each so that it continues in the production process or not. The music production process will continue but the essence of the piece will remain close to what you are listening to. Before starting to listen to each of the pieces, the following text was presented: After listening carefully to this piece please answer: how surprising do you find it? How valuable is it to be published in the album? How novel does it seem to you? Do you have any comments on the piece? Participants answered the same questions for each of the twenty pieces. For the ifrst three questions, a six-step Likert scale was ofered with the following ranges: from “It is not surprising” to “It is completely surprising”; from "It is not valuable" to "This piece is very valuable and should be part of the album"; from "It is not a novel piece" to "It is a revolutionary piece". To give a precise sense of the process, subjects were contextualized in an on-going activity. They were unaware the stimuli were made by a machine.
scored the same control stimuli with a diference of more than 3 points for all three attributes (value, surprise, and novelty). Therefore, all their responses were discarded due to inconsistency.Figure 1 depicts the distribution of responses.
Discernibility of Boden’s three dimensions. A one-way analysis of variance (ANOVA) carried out to evaluate the diference between the three score sets reveals a statistically significant diference between at least two groups (F(2) = 14.9, p < 0.005). A Tukey’s HSD Test for multiple comparisons found that the mean of value scores (mean = 4.12) was significantly diferent than the means of novelty and surprise scores (mean = 3.72, p = 0.001, and mean = 3.86 p = 0.0018, respectively). However, there was no statistically significant diference between the means of novelty and surprise scores (p = 0.119).
Efect of musical training on creativity scores. Subjects were segmented in three levels –low, mid, and high– according to their reported musical training to observe if the musical training has a significant efect on creativity scores. For this analysis creativity attributes are considered as treatments and training groups are analyzed as independent categories.
A one-way ANOVA followed by the corresponding Post-Hoc Tukey tests for multiple comparisons revealed that all training levels gave significantly diferent scores to value and novelty dimensions; only highly trained subjects gave significantly diferent scores to value and surprise dimensions; and all training levels gave no significant diferent scores to novelty and surprise dimensions. Moreover, the scores of novelty surprise and value for all pieces by highly trained subjects have higher standard deviations than those of low trained subjects, and those of mid trained subjects (see Table 1). Echoing [34], results reveal that a highly surprising artifact for an expert might pass as routinary to a novice .
Similarity of creativity scores across training groups. A complementary analysis was carried out having musical training as treatments and creativity attributes as categories (see Figure 2 and Table 2). This serves to discern to what extent the training level refines estimator’s creativity appraisal.
A series of one-way ANOVAs, one for each creativity attribute, followed by corresponding Tukey’s HSD Test for multiple comparisons showed that the scores of mid and low trained subjects are significantly diferent for all three attributes. High and low trained subjects have significantly diferent value scores, while high and mid trained groups have significantly diferent novelty scores (see 2). The dispersion of novelty and surprise scores are very similar for all subject segments, but value scores are more dispersed than those of novelty and surprise in high and mid trained subjects (sd= 1.679 vs 1.608, 1.607 and sd= 1.242 vs 1.092, 1.081 respectively). In the case of low-trained subjects the opposite efects is observed: value scores are less disperse than those of novelty and surprise (sd= 1.406 vs 1.449, 1.460).
The efect of training in value scores has proven to be statistically significant between mid and low training. The analysis reveals that the standard deviation of scores of highly trained subjects is significantly greater than the ones of the rest of the subjects.
2This is compliant with [33] as they suggest surprise and novelty are cognitive processes that operate closely. generative phase of co-creative processes and how to in- process. terpret such metrics. While value scores are significantly Is potential creativity a two or three-attribute diferent from surprise and novelty scores, surprise and space? The empirical results obtained show that novelty novelty ones are close to each other. Further compara- and surprise responses are not statistically distinguishtive analysis between pairs of scores reveals three clear able, suggesting that these attributes, although diferinsights: value stands out as a diferent concept from ent in meaning, have a joint appraisal in experimental novelty, novelty and surprise appear as non-discernible conditions. This resonates with the two-dimensional coconcepts 2, and value and surprise appear discernible creativity assessment models proposed by the standard to highly trained subjects but mid and low trained sub- definition of creativity, and Kantosalo et al. (value and jects have similar mental constructs for value and sur- novelty, plus the quality of user interaction) [35]. Conseprise. Consequently, one could cautiously argue that two- quently, in evaluating proto-artifacts during a co-creative attribute models of creativity could sufice expert estima- process a two-attribute model of value and originality tors to assess unfinished artifacts during a co-creative could account for Boden’s three-attribute model.
Reducing the dimension of the attribute space while preserving the assessment quality simplifies human or agent estimator’s tasks. Such reduction has practical im- termine the value of unfinished artifacts in terms of the plications in multiple real-life scenarios that require filter- foreseen potential to evolve into more refined pieces or ing large sets of artifacts created during CCC processes. branch out novel variations worth exploring. Indeed, as CCC processes permeate human creative activities, the number and quality of potentially creative artifacts that need assessment will most likely grow ex- 6. Conclusions ponentially, demanding efective and adequate metrics to carry out estimation tasks. To assess the creative potential of creating with such computational assistants, one would need to measure the creative quality of protoartifacts produced during the idea generation phase.
However, one could argue that the observed proximity between the novelty and surprise concepts can result from the experimental conditions. On the relation between experiencing surprise and rating novelty, Xu et al. explain how “humans use surprise as a signal to decide when to adapt their behavior, while they use novelty to decide where and what to explore—to eventually develop an improved world-model.” [33, p.1] This idea suggests that both attributes are used in conjunction to adjust expectations dynamically. They operate independently yet contribute to broader cognitive processing. It is necessary to investigate whether the closeness of these concepts stems from the unfinished nature of stimuli that confounds their subjective assessment or from the training level of estimators participating in the study.
The efect of domain training in assessing protoartifacts. There is a plausible efect of domain knowledge in scores of the three creativity attributes. The higher the training the greater the significance of the diferences between value and surprise, and value and novelty (see Table 1 columns 4 and 7). But the inverse efect is observed between value and surprise: the higher the training the lower the significance between novelty and surprise (see Table 1 column 10). This evidence shows that as training becomes more specialized, subjects are more confident gauging value, yet they learn that not every valuable artifact is surprising. In particular, highly trained subjects encounter more pieces with extreme value scores than mid or low trained subjects (see Figure 2). That is, experts used the whole semantic range of the evaluation scale, while non-experts concentrate their scores around the second third. A triangulation of Tukey Post Hoc test of eefcts for experts reinforces the claim that novelty and surprise are not discernible, while value is the only attribute with statistically significant diference between high and low trained subjects. This suggests that training has a higher positive efect on the ability to appreciate value than novelty or to experience surprise. In other words, domain expertise is especially expressed when assessing value and not so much when assessing novelty or surprise. A potential explanation is that training builds a more nuanced domain-specific cognition and reinforces the estimator’s capacity to deThis paper argues for the adoption of a dynamic framework to judge uncompleted work (deemed protoartifacts) leading to a creative product in the context of computational co-creation (CCC) processes. Such approach derived from Corazza’s dynamic definition of creativity, recognizes that artists engaged in computational co-creation not only estimate the creative merit of their work once the piece is finished, but assess the creative potential of intermediate proto-artifacts at each iteration of the generative process. Intermediate assessments depict how a CCC process may go about and put forward the potential anticipation of creative outcomes from the early stages. Hence, a suitable computational assistant should maximize the creative potential of the process, either by enhancing the human’s generative capacity or by facilitating recurrent proto-artifacts assessments.
The findings of an active listening experiment conducted to determine the creative quality of unfinished musical pieces generated by NEA (New Electronic Assistant) suggest that in an experimental setting subjects’ appraisal of novelty and surprise is not discernible. Thus, a two-attributes definition of creativity could account for Boden’s three-attributes definition. Even though novelty and surprise represent diferent creative attributes, originality could account for both of them because novelty and surprise tend to blend in subjective assessments of creativity, while value is certainly diferentiable, especially for domain experts.
For the time being, a two dimensional creativity assessment of proto-artifacts is not invalidated, and may simplify assessment procedures with subjects. We suggest using the dimensions of value and originality (rather than Corazzas’ efectiveness and originality). Value is preferred to efectiveness because it conveys meaningfulness in a variety of fields, including the arts, better than the functional notion of efectiveness. On the other hand, the responses of subjects with three levels of expertise in the domain studied showed that novelty and surprise are two diferent but coupled mental operations. The former is related to memory and the ability to forget and the latter is related to the stability of short-term predictions. This suggests that the assessment of one could be a proxy for the other. For practical research purposes, it makes more sense to use fewer dimensions to conduct large-scale experiments, especially with lay subjects for whom these concepts generally remain fuzzy.
Finally, as AI permeates human creative activities of all sorts the generation of proto-creative material flourishes.
That is, an unavoidable bi-product of assisted creativity [11] M. A. Boden, Creativity and art: Three roads to is the proliferation of unfinished artifacts that must be surprise, Oxford University Press, 2010. assessed not only by humans but also by AI agents. Such [12] A. Jordanous, Four pppperspectives on computaincrease in potentially creative outcomes calls out for the tional creativity in theory and in practice, Connecimplementation of assertive assessment methods. The tion Science 28 (2016) 194–216. results presented here might prove useful to define fur- [13] T. Lubart, How can computers be partners in the ther methodologies for efective human and agent-based creative process: classification and commentary on assessment of creative artifacts in CCC scenarios. the special issue, International Journal of HumanComputer Studies 63 (2005) 365–369. [14] N. M. Davis, Human-computer co-creativity: BlendReferences ing human and computational creativity, in: Ninth Artificial Intelligence and Interactive Digital Enter[1] W. Huang, H. Zheng, Architectural drawings recog- tainment Conference, 2013, pp. 9–12. nition and generation through machine learning, in: [15] G. Miller, E. Galanter, K. Pribram, Plans and the Proceedings of the 38th Annual Conference of the Structure of Behavior, Martino Publishing, USA, Association for Computer Aided Design in Archi- 1960. tecture (ACADIA), CumInCad, 2018, pp. 156–165. [16] C. Carver, M. Scheier, Attention and Self-Regulation doi:10.52842/conf.acadia.2018.156. : A Control-Theory Approach to Human Behavior, [2] H. Osone, J.-L. Lu, Y. Ochiai, Buncho: ai supported New York: Springer-Verlag, 1981. story co-creation via unsupervised multitask learn- [17] G. Wallas, The art of thought, volume 10, Harcourt, ing to increase writers’ creativity in japanese, in: Brace, 1926.
Extended Abstracts of the 2021 CHI Conference on [18] E. Sadler-Smith, Wallas’ four-stage model of the creHuman Factors in Computing Systems, 2021, pp. ative process: More than meets the eye?, Creativity 1–10. Research Journal 27 (2015) 342–352. [3] M. Avdeef, Artificial intelligence & popular mu- [19] M. Csikszentmihalyi, Flow and the psychology of sic: Skygge, flow machines, and the audio un- discovery and invention, HarperPerennial, New canny valley, Arts 8 (2019) 130. doi:10.3390/ York 39 (1997).
arts8040130. [20] D. K. Simonton, Creativity and discovery as blind [4] V. Volz, J. Schrum, J. Liu, S. M. Lucas, A. Smith, variation: Campbell’s (1960) bvsr model after the S. Risi, Evolving mario levels in the latent space of a half-century mark, Review of General Psychology deep convolutional generative adversarial network, 15 (2011) 158–174. in: Proceedings of the genetic and evolutionary [21] T. B. Ward, S. M. Smith, R. A. Finke, Creative cognicomputation conference, 2018, pp. 221–228. tion, in: R. J. Sternberg (Ed.), Handbook of Creativ[5] A. Jordanous, A standardised procedure for eval- ity, Cambridge University Press, 1998, p. 189–212. uating creative systems: Computational creativity evaluation based on what it is to be creative, Cog- [22] dTo.Ai:1m0a.b1il0e1,C7/omCBpOo9n7e8n0ti5al1t1h8e0o7ry91of6c.r0e1a2ti.vity, Harnitive Computation 4 (2012) 246–279. vard Business School Boston, MA, 2011. [6] S. Colton, J. W. Charnley, A. Pease, Computational [23] L.-C. Yang, A. Lerch, On the evaluation of gencreativity theory: The face and idea descriptive erative models in music, Neural Computing and models., in: ICCC, Mexico City, 2011, pp. 90–95. Applications 32 (2020) 4773–4784. [7] C. Lamb, D. G. Brown, C. L. Clarke, Evaluating com- [24] M. I. Stein, Creativity and culture, The journal of putational creativity: An interdisciplinary tutorial, psychology 36 (1953) 311–322.
ACM Computing Surveys (CSUR) 51 (2018) 1–34. [25] M. A. Boden, The creative mind: Myths and mecha[8] H. Herndon, M. Dryhurst, Latent visions, nisms, Routledge, 2004.
promptism and the future of ai art with ad- [26] G. A. Wiggins, A preliminary framework for
deverb [audio podcast episode], NPR, 2021. scription, analysis and comparison of creative
sysURL: https://interdependence.fm/episodes/ tems, Knowledge-Based Systems 19 (2006) 449–458.
latent-visions-promptism-and-the-future-of-ai-art-with-adverb.
[9] M. A. Runco, G. J. Jaeger, The standard definition Sdyosi:t1e0m.s1. 016/j.knosys.2006.04.009, creative
of creativity, Creativity research journal 24 (2012) [