=Paper=
{{Paper
|id=Vol-2903/IUI21WS-HAIGEN-4
|storemode=property
|title=Image Co-Creation by Non-Programmers and Generative Adversarial Networks
|pdfUrl=https://ceur-ws.org/Vol-2903/IUI21WS-HAIGEN-4.pdf
|volume=Vol-2903
|authors=Doron Friedman,Dan Pollak
|dblpUrl=https://dblp.org/rec/conf/iui/FriedmanP21
}}
==Image Co-Creation by Non-Programmers and Generative Adversarial Networks==
https://ceur-ws.org/Vol-2903/IUI21WS-HAIGEN-4.pdf
Image Co-Creation by Non-Programmers and Generative
Adversarial Networks
Doron Friedmana , Dan Pollaka
a The Advanced Reality Lab, The Interdisciplinary Center, Herzliya, Israel
Abstract
Generative models such as generative adversarial networks are now being studied extensively. Eventually, however, many of
them are intended for non-programmers to work with, e.g. designers, artists, or other content creators. What happens when
such individuals are confronted with using GANs? We present a case study – a new course intended for non-programmer MA
students in human-computer interaction, aimed at training them in authoring content using generative models. As their final
assignment, the students were asked to train generative adversarial networks in order to generate images from a predefined
category of their choice. The students either used a graphical user interface (GUI)-based software or modified preexisting
python code using simplified Google Colab notebooks. We present several lessons learned from this course. First, we analyze
the joint human-AI creation process and recognize points where students could intervene, with anecdotal examples of how
they creatively explored these opportunities. Interestingly, while the majority of algorithmic research is focused on how to
make models more controllable (e.g., via conditioning or latent space disentanglement), the students found ways to obtain
their creative needs by mostly exploring the dataset level (as opposed to the model architecture). Additionally, we present
the results of a short survey, comparing the two modes of work (GUI vs code).
Keywords
GAN, co-creation, Style-GAN
1. Introduction Our goal is to explore how such individuals can
work with novel generative models such as genera-
We are witnessing a rapid advance of “artificial in- tive adversarial networks (GANs). The opportunity
telligence” (AI) and machine learning (ML) technolo- came up in the form of a course for MA students in
gies, and these techniques are penetrating into an human computer interaction (HCI), most of them non
increasing range and diversity of aspects of every- programmers. As their final assignment in the course
day life. It seems important that the responsibility they were guided in developing a project in which they
for these systems would not only lie on the shoul- train a GAN to generate a specific set of images, in the
ders of programmers, but that additional professions spirit of “This X does not exist”1 . Following the success
would be involved in intelligent system design, devel- of StyleGAN to generate highly photo-realistic human
opment, and evaluation. As the current zeitgeist is faces, a website showing such images called “thisper-
that of data-driven methods with “deep” neural net- sondoesnotexist” went viral. This was followed by
works, explainability has become a major concern [1]. attempts in training GANs specializing in generating
In generative AI, rather than just explainbility (or in- cats, rental apartments, snack, and the list of project
stead?), we often strive for ‘controllability’, or the de- seems to be still growing. The task is similar to the
gree to which humans can control and shape the re- Little Prince’s request: “draw me a sheep”. Unlike the
sults generated by the system. Indeed, there is am- majority of human-AI co-creation tasks, in which the
ple work on on reversible generative models [2] or la- human is expected to be creative and the machine is
tent space disentanglement [3]. Nevertheless, in ad- expected to assist, in this task the machine is expected
dition to these computational efforts, sociological fac- to be creative and generate interesting samples of X,
tors are expected to play an important part. Eventu- and the human is only expected to assist. The students
ally, these systems are not intended for programmers; were given the choice whether to use programming or
rather, they would more likely be used by designers, a GUI-based software, and following the course, they
artists, writers, or other professionals of the so-called were asked to answer some questions.
‘creative industries’. This paper’s contribution is from lessons learned
from teaching generative models and synthetic media
Joint Proceedings of the ACM IUI 2021 Workshops
" doronf@idc.ac.il (D. Friedman); dandan888@gmail.com (D.
to non-programmers, anecdotal lessons learned from
Pollak) their projects, an analysis of the human intervention
points that they ‘discovered’, and results from a sur-
© 2021 Copyright for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
http://ceur-ws.org
ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) 1 http://thisxdoesnotexist.com
Proceedings
vey in which they provided some feedback2 . Interest- that there are also important differences. For example,
ingly, while the overwhelming majority of algorithmic in those artist groups the importance of the individu-
research is focused on how to make models more con- als comprising the group was much reduced; however,
trollable (e.g., via conditioning or latent space disen- in our case we are still interested in distinguishing the
tanglement), the students found ways to obtain their role and contribution of the machine and the human.
creative needs by mostly exploring the dataset level (as Bidgoli et al. actually suggest that the machines (at
opposed to the model architecture). least in this point in time) do not have an identity
to contribute; rather, it is the identity of the people
who designed it. “From this point of view, co-creation
2. Background is a special case of collaboration where the tool acts
as a “machinic surrogate” to represent the identity of
The “draw me a sheep” scenario raises several ques-
its “toolmakers”.” We suggest that this, again, under-
tions. The first is: can machines be creative and if so
mines the important differences between humans and
what does it mean? Generative algorithms are able
machines.
to generate a huge number of outcomes; however, it
Interestingly, the current legal perspective seems to
could be that most or even all of the results are uninter-
agree that the results of human-AI collaboration may
esting. Boden [4] stressed that in order for a process to
result in emergent properties that cannot be attributed
be considered creative its result needs to be both use-
to any of the stakeholders; Eshraghian [7] looks at
ful and novel. Satisfying the requirement for novelty
ownership of AI-art from a legal point of view, point-
is typically easy (e.g., throw in some noise, or increase
ing out that the stakeholders include programmers,
the exploration temperature). Whether the result is
style trainers, private datasets, and features extracted
useful is hard to define, in general. Additionally, GANs
from information collected from the general public.
are (arguably) not designed to be creative; on the con-
Most often, there is a sociological gap between al-
trary, they are designed to learn the training set prior
gorithm researchers and the target audience for us-
distribution and come up with the most prototypical
ing these tools, which are, in general, non program-
and ‘non-creative’ examples.
mers. We anticipate that as ‘generative AI’ will play
Thus, we suggest that machine creativity is not the
a larger role in an increasing number of domains, this
most appropriate framework for our case study, but
gap will need to be bridged, and ideally it should be
rather the appropriate framework is human-AI collab-
addressed by the research community, not only by in-
oration. Here we can distinguish between two ap-
dustry teams working on specific products. In the do-
proaches. The first approach is more human-centric,
main of music generation, Huang et al. [8] report on
where AI is expected to augment human creativity. For
a survey carried out with musician/developer teams,
example, Jarrahi [5] discusses the AI job crisis in gen-
and the same team evaluated a tool they developed for
eral, stressing the need for a discussion on how hu-
co-creation in song writing [9].
mans and AI can work together, and the role of AI
The human-AI co-creation model has drawn in-
in human intelligence augmentation rather than re-
creasing interest recently [10]. The main requirement
placement. However, Jarrahi also presents examples
is that the outcome cannot be attributed to either hu-
of human-machine symbiosis in chess and medical di-
man or machine alone; we suggest that our case study
agnosis, and discusses the synergy in organizational
adheres to this requirement, as illustrated in the re-
decision making.
sults section. Otherwise, however, our case study is
This leads us to the second, more recent approach,
very different from other research projects in the field
where there is growing interest in human-AI collabo-
of human-AI co-creation, and we suggest that the as-
ration as a new entity, where the whole is larger than
sumptions and range of questions that addressed by
the sum of the parts. Bidgoli et al. [6] draw historical
this field may be extended. In other words, we suggest
lessons from collaborative art, where groups of artists
that the field might be too ‘human-centric’, in both the
created new identities that attracted significant atten-
end goal (the focus is on tools that enhance human ac-
tion in the 1960s and the 1970s. This historical analogy
tivities) and the means (the focus is on real time inter-
can shed light on contemporary human-AI projects.
action techniques).
For example, cultural diversity was highly encouraged
For example, Karimi et al. [11] suggest a frame-
in those artist groups; clearly, humans and machines
work for evaluating human-AI collaborative creativity.
are inherently very different from each other, which
Their taxonomy suggests three types of systems: fully
may lead to superior results. We suggest, however,
autonomous, creativity support tools, and co-creative
2 See video: https://bit.ly/3qe2mf5 systems. However, their definition of co-creativity re-
quires the AIs to have ‘their own conceptualization
of creativity’, which we suggest is not a necessary
component. Most work in the field is based on tools
whereby humans and AI interact directly, in real time:
Yannakakis et al. [12] demonstrated joint exploration
of the search space, and other studies also present in-
teractive tools for painting or sketching [10, 13]. How-
ever, they typically go too far (we suggest) in requir-
ing the AI to have explicit mental models and natu- Figure 1: Programming experience of 18 students who filled
ral language communication abilities. We suggest that in survey questionnaire.
this is at most desired, but there are more fundamen-
tal questions in human-AI co-creation. Llano et al. [14]
suggest that creativity can be enhanced by improved given in the third (summer) semester. Figure 1 de-
communication between human and machine; the end scribes their level of programming experience. All stu-
goal is two-way communication. We suggest that at dents provided written consent for their material to
this stage this is at most a probable hypothesis. appear in writing and video.
Co-creativity has also been framed as mixed initia-
tive interaction [12, 15]. The mixed initiative frame- 3.2. The Course
work, in turn, has been borrowed from conversation
analysis, and includes three parts: task initiative – The course was focused on “synthetic media”, i.e., au-
deciding the topic, speaker initiative – deciding who tomatically generated media and art content4 . The
speaks, and outcome initiative – deciding when the first two lessons provided a historical view of AI, in-
outcome is ready [16]. In our case, all three decisions troducing key concepts and themes. The third les-
are exclusively made by the human (although, see the son provided an overview of AI used for generating
results section for a caveat). Nevertheless, we suggest media and art content; while the course focused on
that the “draw me a sheep” scenario still falls under deep neural networks, in this stage we also let the stu-
the original requirement – the whole is more than the dents explore genetic algorithms hands-on, allowing
sum of the parts, and it is difficult to disentangle the us to discuss generative projects such as Karl Sims’
relative contribution of human and machine when an- 3D evolving creatures [17], and explaining that neural
alyzing the result. networks are one technique among many. Next, we
Instead, we suggest that an appropriate framework provided a brief introduction to ‘’classic ML’ and deep
is to view the human-AI collaboration as a single pro- neural networks, and the second half of the course was
cess. The analysis we perform (in Section Results) is dedicated to a more in depth discussion of GANs and
aimed at identifying the sub tasks in this creative pro- some additional topics (“deep fake”, sequences, lan-
cess, and who is in charge of each sub task. In our case guage models). We have introduced some mathemat-
the machine would ideally be autonomous, but we see ical notation, for example, discussing loss functions,
that there are quite a few points at which human in- but the course was intended for non-mathematicians
tervention is desired or even required. and most of the discussion was at the level of popu-
lar science. Due to the Covid-19 pandemic most of the
lessons were hybrid, with half of the students in class
3. Method and the rest at home over Zoom.
In their assignments, the students explored both an
3.1. The Students easy-to-use software – RunwayML5 , and a simple Co-
lab notebook with python code. RunwayML is a com-
The course was an elective as part of an MA degree mercial product intended for artists and creators to ap-
on human-technology relationship. Thirty two stu- ply a wide range of “deep learning” models. It pro-
dents enrolled in the course, most of them with a back- vides a relatively easy to use graphical user interface
ground in design or social science, and only a minor- (GUI) that serves as a front end to existing implemen-
ity with programming experience or computer science tations of deep neural networks. It allows you to run
background. All students learned basic programming the trained models on your input, to train models on
(with p5js3 ) in the first semester, and the course was new datasets, and to concatenate models; i.e., you can
4 See video: https://bit.ly/3qe2mf5
3 http://p5js.org 5 http://runwayml.com
concatenate networks A and B if the type of output of
network A is consistent with the type of input of net-
work B. The payment is mostly per cloud GPU run-
time. Such services raise interesting and important
questions around copyright and ownership of intellec-
tual property, which are out of the scope of this paper.
As a first generative assignment the students were
asked to explore “deep dream” [18], using both Run-
way ML and a Colab python notebook. In both cases
the level of exploration was very minimal – the stu- Figure 2: The software used by teams for main assignment.
dents could only select the image to modify and the In-
ception layers whose activation would be maximized.
For the final assignment the “brief” was to come up sults matched their expectation in quality, and iii) the
with images in the spirit of thisXdoesnotexist6 . The results matched their expectation in intent.
students were provided with a Colab python note-
book for scraping images online. Next, they were
told to choose between two options: a notebook with 4. Results
documented deep convolutional GAN (DC-GAN) [19] Figure 2 describes whether the students used Run-
implementation, and RunwayML. It was explained wayML, python, or both. Interestingly, some non-
that the code allows for more freedom, whereas us- programmers (i.e., students whose only very basic pro-
ing StyleGAN [20] or StyleGAN-2 [21] on RunwayML gramming experience was in p5js, from a previous
would produce better results. Nevertheless, we ex- course) preferred to use Colab.
plained that RunwayML is also constrained by cost
and we warned the students that the extra cost we
could cover per team is limited. DC-GAN is close 4.1. Questionnaire Results
enough to a ‘vanilla’ GAN so that there are simple im- In order to find out the differences in responses to
plementations that can be documented and explained. using RunwayML versus using python and Colab we
The implementation of StyleGAN, on the other hand, ran paired-samples t-tests. The students liked Run-
is only accessible to experienced programmers (wrap- wayML significantly more than using code (t=3.073,
pers are available as simple notebooks that allow you df=17, p=0.007). Perceived quality was significantly
to run the model or even train it, but this is function- higher with RunwayML (t=2.309, df = 8, p=0.05) and
ally almost equivalent to using RunwayML, and does RunwayML was significantly easier to use (t=-3.274,
not serve any educational purpose of understanding df=7, p=0.014). However, there was no significant dif-
neural network coding). ference in the extent to which the user’s intent was
The programmers were assigned to separate teams, captured by the model, when comparing the two plat-
and they were not allowed to use RunwayML. They forms (t=1.0, df = 8, p=0.347). Nevertheless, the corre-
were told to explore a range of techniques and hyper- lation between perceived quality and captured intent
parameters and provide documentation of training was high and significant for both RunwayML (r=0.76,
runs, such as plots of generator and discriminator loss. df=12, p=0.003) and Colab (r=0.66, df=13, p=0.01).
Most of them started with DC-GAN but moved on to
StyleGAN-2 in order to improve their results.
4.2. Lessons Learned from Student
Projects
3.3. Questionnaire
We analyze the projects in terms of the points where
A week after the submission of the final project, the
the human users could intervene, with focus on the
students were asked to fill in a questionnaire, and 18
non-programmers (including students with some very
students responded. We asked about programming
basic programming experience). The first human in-
experience and what option they used for their fi-
tervention point was in determining the goal – al-
nal project. Next we asked them to rate, for either
legedly, this was completely determined by the hu-
python code or RunwayML, the extent to which: i)
mans. However, it could be argued that the machine
they liked using it, ii) they found it difficult, iii) the re-
‘took part’ in this stage as well, since about half of
6 http://thisxdoesnotexist.com
the teams modified their project goal after some at-
tempts. Typically, after realizing that their initial ex-
pectations were not realistic, the students converged
on more specific outcomes, since they realized that
the datasets have to be homogeneous for obtaining
good results. Eventually, the non-programmer teams
decided to generate: human head statues (protomes),
holocaust victims, Gaudi-style architecture, animal-
electronic device hybrids, city maps, Simpson char-
acters, politicians, butterflies, protected flowers, Dis-
ney characters, smartphone application icons, and bi-
cycles. The programmer teams decided to gener-
ate: Marvel superheros, Yayoi Kusama art, best of art
paintings (unspecified), and McDonald’s toys (mixed
team).
The students quickly learned that the main way for
them to affect the results is by modifying the train-
ing set. Although this was discussed in class, many
were surprised that the datasets need to be very homo- Figure 3: Generated politician images reveal politiican im-
age distribution properties: typical upper body composition,
geneous and that “the AI” could not deal with simple
formal dress, US flags. Results obtained with StyleGAN-2 in
invariants such as location of the main object within RunwayML, by Nurit Belorai.
the frame. Some surprising artifacts were discovered;
for example, the Gaudi team had to clean cranes from
pictures of the Sagrada Familia, which has been un-
der construction for a long time. Sometimes such bi-
ases were considered undesired, but in other cases the
students were excited to see these biases emerge from
the model’s results. For example, it was considered
a success to witness that the GAN had incorporated
flags into many images of generated politicians, and
often placed them in a ‘speech stance’, thus capturing
stereotypical features of politician pictures that distin-
guish them from other pictures of humans (see Figure
Figure 4: Generated Disney characters; trying to generate
3). characters with bodies using a model pre-trained on ani-
The next choices were technical: what algorithm mals failed in producing clean results. Results obtained with
to use (the students were pointed to DC-GAN, Style- StyleGAN-2 in RunwayML, by Hadas David and Shani Tal.
GAN, and StyleGAN-2), and automatic image pre-
processing, which still left decisions regarding image
size and crop type. tively easy to generate beautiful butterflies, but using
For Style-GAN based projects the networks were a model pre-trained on faces resulted in more symmet-
always pre-trained, and an important decision was ric butterflies (Figure 6a) than when using a model pre-
what pre-trained model to use. Sometimes the deci- trained on objects (Figure 6b).
sion was obvious, but in some cases students explored One team deliberately trained a model pre-trained
what happened when they override the most reason- on one category (cats or horses) with a very different
able decision. For example, the Disney team wanted to dataset (toasters or kitchen aid devices), with the aim
generate whole body images and avoid photo-realistic of creating a hybrid animal and electronic device (e.g.,
faces, so they opted for a model pre-trained on ani- toaster cats in Figure 7). They realized that training
mals rather than realistic human faces (Figure 4). As over many steps resulted in interpolating the model
another example, the Gaudi architecture team realized from one category to another, and they have specifi-
that using a model pre-trained on skyscrapers resulted cally tried to find the point in time (number of train-
in no green (trees and plants), so they preferred the ing steps) where the balance is of interest, systemat-
results obtained with a model pre-trained on objects ically testing when the horses disappear into kitchen
(Figure 5). The butterfly team realized that it is rela-
Figure 5: Generated Gaudi-style buildings. Training on a model pre-trained on objects preserved green (trees) (a) in
the images (a), while training on a model pre-trained on skyscrappers (b) did not. Results obtained with StyleGAN-2 in
RunwayML, by Nachshon Ben and Carmel Slavin Brand.
Figure 6: Butterflies generated by a model pre-trained on faces were symmetric (a) while butterflies trained on objects
were often not symmetric in shape and color (b). Results obtained with StyleGAN-2 in RunwayML, by Gizem Odemir and
Karin Bar-Gefen.
aid (they were happy with the results obtained after on the nature of the creative process, making it essen-
680 steps; Figure 8). While there has been computa- tially ‘non-interactive’ and limiting the scope of the
tional efforts to train GANs on such mixed datasets, work to a small number of iterations.
the goal in those cases was to teach the GANs to learn The next choice was the number of training steps or
separate modes [22]; the question of deliberate “mode epochs. The students realized that more training is not
mixing” seems novel. necessarily better; while most often the convergence is
Dataset level user manipulations are not popular in to a flat curve (i.e., the model keeps generating similar
the field; one of the main reasons is most likely that images regardless of additional training), sometimes
re-training is resource (time) consuming. Each ‘exper- earlier results are preferred. The only information
iment’, even if it only involves fine tuning, typically they could use in RunwayML, other than manually in-
takes at least several hours. Clearly, this has an effect specting the resulting images, are the Frechet Incep-
Figure 7: Toaster cats, an example of attempts to create hybrid animal and electronic device images. Results obtained
with StyleGAN-2 in RunwayML, by Eden Offer and Adi Frug.
Figure 8: An example of attempts to create hybrid animal
and electronic device images – horses and kitchen aid. Re-
sults obtained with StyleGAN-2 in RunwayML, by Eden Of-
fer and Adi Frug.
tion distance (FID) scores (only programmers learned
to plot and analyze additional information, such as
learning curves).
One team realized that they can keep fine tuning the
network more than once, over multiple steps. First,
they generated butterflies form models pre-trained on
Figure 9: Fine tuning butterflies with different patterns:
faces or objects. Next, they wanted to further shape patterns from training set (a) and resulting butterflies with
the results, with the aim at generating butterflies with tiger stripes (b). Results obtained with StyleGAN-2 in Run-
fractal patterns (failed) or animal patterns. They con- wayML, by Gizem Odemir and Karin Bar-Gefen.
tinued training their fine-tuned butterfly model for
a smaller number of steps with pictures of animal
skin. Moreover, they realized that they could easily cut After the training process, the students learned the
those pattern pictures into a rough shape of a butter- controversial art of ‘cherry picking’. While in most
fly, so that the results will not lose the butterfly shape academic contexts this practice should be avoided, we
(Figure 9).
programmers or by slightly modifying pre-exisiting
simple code. Questionnaire results indicate that using
the GUI with StyleGAN-2 was easier than code and re-
sulted in better perceived quality. Interestingly, there
was no significant difference in the perceived degree
to which the results matched the original students’ in-
tent. This is not because intent was very low, because
the mean reported intent is higher than the average,
and perceived quality and intent are highly correlated.
Nevertheless, our assessment in this respect is limited,
because we are not only comparing two tools, but in
many cases we are also comparing two models – most
students who opted to use code used DC-GAN, whose
results are inferior as compared to StyleGAN. We sug-
gest that our course is not only useful for academic in-
stitutions but may also be useful in industry, for train-
ing non-programmers to co-create with AI.
We have analyzed the human intervention points
in the creative process. The students were required
to intervene in several sub-tasks that were not imple-
mented in software. Additionally, some students in-
tervened in order to refine the results. While we per-
ceived the task in the context of almost autonomous
AI, at least two teams interpreted the task in terms
of AI assisting human creativity: one project aimed
at augmenting the design of electronic devices with
Figure 10: Automatically generated applications icons: the inspiration from animals, and the other project used
results were low quality (left) so they were recreated man- the GAN to come up with preliminary sketches for ap-
ually (right). Results obtained with DC-GAN in Colab, by plication icons, which were then finalized by the hu-
Maor Bluman and Bat Primo. mans. Allowing non-programmers more control and
intervention points is clearly desired. However, while
it is relatively straightforward to expose a few more
suggest that in the context of AI-aided design or art hyper-parameters into software such as RunwayML or
this is quite legitimate. It is important to understand ‘friendly’ code wrappers, the challenge is in providing
the implications of the systems being generative, i.e., non-experts with intuitions about the expected way to
once developed they can generate a practically infinite deal with these hyper-parameters.
number of results. If one of the results is what you Finally, a very active area of current computational
are looking for, and what you are looking for is very research is how to make generative models such as
special and difficult to obtain, “cherry picking” is rea- GANs more ‘controllable’, using latent space algebra,
sonable. Finally, one team of non-programmers who latent space disentanglement and more (e.g., [23, 24,
opted to use DC-GAN were rightly disappointed from 25, 26]). We suggest that it is both interesting and
the results, which were noisy. Nevertheless, they went important to see what happens when such tools are
on to manually clean the results, suggesting that even “unleashed” to the hands of non programmers. As we
if the AI ‘artist’ or ‘designer’ is not as competent as show here, they may discover new ways to improve
the human, it can nevertheless be used as a source of results, which were not planned for by the algorithm
inspiration (Figure 10). designers. Notably, and contrary to the majority of al-
gorithmic effort, students tried to obtain their goal by
modifying the training data set – either by selecting a
5. Discussion counter-intuitive pre-trained option, or by modifying
Non-programmers were able to grasp the main con- their own datasets that were used for fine tuning the
cepts and train GANs to obtain interesting results, models.
using either a GUI-based software intended for non-
Acknowledgments [12] G. N. Yannakakis, A. Liapis, C. Alexopoulos,
Mixed-initiative co-creativity (2014).
The authors wish to thank all students taking part in [13] P. Karimi, J. Rezwana, S. Siddiqui, M. L. Maher,
the course, many of whom are mentioned in the text, N. Dehbozorgi, Creative sketching partner: an
and especially Gizem Odemir for also making some analysis of human-ai co-creativity, in: Proceed-
suggestions on the survey. ings of the 25th International Conference on In-
telligent User Interfaces, 2020, pp. 221–230.
[14] M. T. Llano, M. d’Inverno, M. Yee-King, J. McCor-
References mack, A. Ilsar, A. Pease, S. Colton, Explainable
[1] A. Adadi, M. Berrada, Peeking inside the black- computational creativity, in: Proc. ICCC, 2020.
box: A survey on explainable artificial intelli- [15] J. Zhu, A. Liapis, S. Risi, R. Bidarra, G. M.
gence (xai), IEEE Access 6 (2018) 52138–52160. Youngblood, Explainable ai for designers: A
[2] D. P. Kingma, P. Dhariwal, Glow: Generative human-centered perspective on mixed-initiative
flow with invertible 1x1 convolutions, in: Ad- co-creation, in: 2018 IEEE Conference on Com-
vances in neural information processing systems, putational Intelligence and Games (CIG), IEEE,
2018, pp. 10215–10224. 2018, pp. 1–8.
[3] C. P. Burgess, I. Higgins, A. Pal, L. Matthey, [16] D. G. Novick, S. Sutton, What is mixed-initiative
N. Watters, G. Desjardins, A. Lerchner, Under- interaction, in: Proceedings of the AAAI spring
standing disentangling in 𝛽-vae, arXiv preprint symposium on computational models for mixed
arXiv:1804.03599 (2018). initiative interaction, volume 2, 1997, p. 12.
[4] M. A. Boden, Creativity, in: Artificial intelli- [17] K. Sims, Evolving virtual creatures, in: Pro-
gence, Elsevier, 1996, pp. 267–291. ceedings of the 21st annual conference on Com-
[5] M. H. Jarrahi, Artificial intelligence and the fu- puter graphics and interactive techniques, 1994,
ture of work: Human-ai symbiosis in organiza- pp. 15–22.
tional decision making, Business Horizons 61 [18] A. Mordvintsev, C. Olah, M. Tyka, Inceptionism:
(2018) 577–586. Going deeper into neural networks (2015).
[6] A. Bidgoli, E. Kang, D. C. Llach, Ma- [19] A. Radford, L. Metz, S. Chintala, Unsupervised
chinic surrogates: Human-machine relationships representation learning with deep convolutional
in computational creativity, arXiv preprint generative adversarial networks, arXiv preprint
arXiv:1908.01133 (2019). arXiv:1511.06434 (2015).
[7] J. K. Eshraghian, Human ownership of artificial [20] T. Karras, S. Laine, T. Aila, A style-based gener-
creativity, Nature Machine Intelligence (2020) 1– ator architecture for generative adversarial net-
4. works, in: Proceedings of the IEEE conference on
[8] C.-Z. A. Huang, H. V. Koops, E. Newton-Rex, computer vision and pattern recognition, 2019,
M. Dinculescu, C. J. Cai, Ai song contest: pp. 4401–4410.
Human-ai co-creation in songwriting, arXiv [21] T. Karras, S. Laine, M. Aittala, J. Hellsten, J. Lehti-
preprint arXiv:2010.05388 (2020). nen, T. Aila, Analyzing and improving the im-
[9] R. Louie, A. Coenen, C. Z. Huang, M. Terry, C. J. age quality of stylegan, in: Proceedings of the
Cai, Novice-ai music co-creation via ai-steering IEEE/CVF Conference on Computer Vision and
tools for deep generative models, in: Proceedings Pattern Recognition, 2020, pp. 8110–8119.
of the 2020 CHI Conference on Human Factors in [22] O. Sendik, D. Lischinski, D. Cohen-Or, Unsuper-
Computing Systems, 2020, pp. 1–13. vised k-modal styled content generation, arXiv
[10] C. Oh, J. Song, J. Choi, S. Kim, S. Lee, B. Suh, I preprint arXiv:2001.03640 (2020).
lead, you help but only with enough details: Un- [23] A. Jahanian, L. Chai, P. Isola, On the”steerability"
derstanding user experience of co-creation with of generative adversarial networks, arXiv
artificial intelligence, in: Proceedings of the 2018 preprint arXiv:1907.07171 (2019).
CHI Conference on Human Factors in Comput- [24] E. Härkönen, A. Hertzmann, J. Lehtinen, S. Paris,
ing Systems, 2018, pp. 1–13. Ganspace: Discovering interpretable gan con-
[11] P. Karimi, K. Grace, M. L. Maher, N. Davis, Evalu- trols, arXiv preprint arXiv:2004.02546 (2020).
ating creativity in computational co-creative sys- [25] J.-Y. Zhu, P. Krähenbühl, E. Shechtman, A. A.
tems, arXiv preprint arXiv:1807.09886 (2018). Efros, Generative visual manipulation on the nat-
ural image manifold, in: European conference on
computer vision, Springer, 2016, pp. 597–613.
[26] T. Park, M.-Y. Liu, T.-C. Wang, J.-Y. Zhu, Gaugan:
semantic image synthesis with spatially adaptive
normalization, in: ACM SIGGRAPH 2019 Real-
Time Live!, 2019, pp. 1–1.