=Paper=
{{Paper
|id=Vol-1407/paper2
|storemode=property
|title=Implementing feedback in creative systems: a workshop approach
|pdfUrl=https://ceur-ws.org/Vol-1407/AInF2015paper2.pdf
|volume=Vol-1407
|dblpUrl=https://dblp.org/rec/conf/ijcai/CorneliJ15
}}
==Implementing feedback in creative systems: a workshop approach==
https://ceur-ws.org/Vol-1407/AInF2015paper2.pdf
Implementing feedback in creative systems: A workshop approach
Joseph Corneli1 and Anna Jordanous2
1
Department of Computing, Goldsmiths College, University of London
2
School of Computing, University of Kent
Abstract 1988; Saunders, 2012). Computational creativity researchers
are starting to place more emphasis on social interaction
One particular challenge in AI is the computational and feedback in their systems and models (Saunders, 2012;
modelling and simulation of creativity. Feedback Gervás & León, 2014; Corneli et al., 2015). Still, nearly 3 in
and learning from experience are key aspects of the 4 papers at the 2014 International Conference for Computa-
creative process. Here we investigate how we could tional Creativity1 failed to acknowledge the role of feedback
implement feedback in creative systems using a so- or social communication in their computational work on cre-
cial model. From the field of creative writing we ativity.
borrow the concept of a Writers Workshop as a To highlight and contribute towards modelling feedback
model for learning through feedback. The Writ- as a crucial part of creativity, we propose in this paper a
ers Workshop encourages examination, discussion model of computational feedback for creative systems based
and debates of a piece of creative work using a pre- on Writers Workshops (Gabriel, 2002), a literary collabora-
scribed format of activities. We propose a computa- tive practice that encourages interactive feedback within the
tional model of the Writers Workshop as a roadmap creative process. We introduce the Writers Workshop concept
for incorporation of feedback in artificial creativ- (Section 2) and critically reflect on how it could encourage
ity systems. We argue that the Writers Workshop serendipity and emergence in computational models of intel-
setting describes the anatomy of the creative pro- ligence and creativity. These considerations lead us to pro-
cess. We support our claim with a case study that pose a Writers Workshop computational model of feedback
describes how to implement the Writers Workshop in computational creativity and AI systems (Section 2.1), the
model in a computational creativity system. We central contribution of this paper. In Section 3 we consider
present this work using patterns other people can how the Writers Workshop model fits into previous work in
follow to implement similar designs in their own various related areas. While we acknowledge that this paper
systems. We conclude by discussing the broader is offering a roadmap for this model rather than a full imple-
relevance of this model to other aspects of AI. mentation, we consider how the model could be practically
implemented in a computational system and report our initial
1 Introduction implementation work (Section 4). In concluding discussions,
In educational applications it would be useful to have an au- we reflect on divergent directions in which this work could
tomated tutor that can read student work and make sugges- potentially be useful in the future.
tions based on diagnostics, like, is the paper wrong, and if so
how? What background material should be recommended to 2 The Writers Workshop
the student for review? Richard Gabriel (2002) describes the practise of Writers
In the current paper, we “flip the script” and look at what Workshops that has been put to use for over a decade within
we believe to be a more fundamental problem for AI: com- the Pattern Languages of Programming (PLoP) community.
puter programs that can themselves learn from feedback. Af- The basic style of collaboration originated much earlier with
ter all, if it was easy to build great automatic tutors, they groups of literary authors who engage in peer-group critique.
would be a part of everyday life. As potential users (think- Some literary workshops are open as to genre, and happy to
ing from both sides of the desk) we look forward to a future accommodate beginners, like the Minneapolis Writers Work-
when that is the case. shop2 ; others are focused on professionals working within a
Along with automatic tutoring, computational creativity specific genre, like the Milford Writers Workshop.3
is a challenge within artificial intelligence where feedback
plays a vital part (for example Pérez y Pérez, Aguilar, & Ne- 1
ICCC is the key international conference for research in com-
grete, 2010; Pease, Guhe, & Smaill, 2010). Creativity can- putational creativity.
2
not happen in a ‘silo’ but instead is influenced and affected http://mnwriters.org/how-the-game-works/
3
by feedback and interaction with others (Csikszentmihalyi, http://www.milfordsf.co.uk/about.htm
10
The practices that Gabriel describes are fairly typical: The feedback step may be further decomposed into
• Authors come with work ready to present, and read a observations and suggestions. This protocol is
short sample. what we have in mind in the following discussion of the Writ-
ers Workshop.5
• This work is generally work in progress (and workshop-
ping is meant to help improve it). Importantly, it can Dialogue example
be early stage work. Rather than presenting a created Note that for the following dialogue to be possible computa-
artefact only, activities in the workshop can be aspects tionally, it would presumably have to be conducted within a
of the creative process itself. Indeed, the model we lightweight process language. Nevertheless, for convenience,
present here is less concerned with after-the-fact assess- the discussion will be presented here as if it was conducted
ment than it is with dealing with the formative feedback in natural language. Whether contemporary systems have
that is a necessary support for creative work. adequate natural language understanding to have interesting
• The sample work is then discussed and constructively interactions is one of the key unanswered questions of this
critiqued by attendees. Presenting authors are not per- approach, but protocols such as the one described above are
mitted to rebut these comments. The commentators gen- sufficient to make the experiment.
erally summarise the work and say what they have gotten For example, here’s what might happen in a discussion of
out of it, discuss what worked well in the piece, and talk the first few lines of a poem, “On Being Malevolent”. As
about how it could be improved. befitting the AI-theme of this workshop, “On Being Malev-
olent” is a poem written by an early user-defined flow chart
• The author listens and may take notes; at the end, he or in the FloWr system (known at the time as Flow) (Colton &
she can then ask questions for clarification. Charnley, 2014).
• Generally, non-authors are either not permitted to at- FLOW: “I hear the souls of the damned wait-
tend, or are asked to stay silent through the workshop, ing in hell. / I feel a malevolent spectre hov-
and perhaps sit separately from the participating au- ering just behind me / It must be his birth-
thors/reviewers.4 day.”
Essentially, the Writers Workshop is somewhat like an in- SYSTEM A : I think the third line detracts from
teractive peer review. The underlying concept is reminiscent the spooky effect, I don’t see why it’s in-
of Bourdieu’s fields of cultural production (Bourdieu, 1993) cluded.
where cultural value is attributed through interactions in a SYSTEM B : It’s meant to be humourous – in
community of cultural producers active within that field. fact it reminds me of the poem you presented
yesterday.
2.1 Writers Workshop as a computational model MODERATOR : Let’s discuss one poem at a
The use of Writers Workshop in computational contexts is not time.
an entirely new concept. In PLoP workshops, authors present Even if, perhaps and especially because, “cross-talk” about
design patterns and pattern languages, or papers about pat- different poems bends the rules, the dialogue could prompt
terns, rather than more traditional literary forms like poems, a range of reflections and reactions. System A may object
stories, or chapters from novels. Papers must be workshopped that it had a fair point that has not been given sufficient at-
at a PLoP or EuroPLoP conference in order to be considered tention, while System B may wonder how to communicate
for the Transactions on Pattern Languages of Programming the idea it came up with without making reference to another
journal. A discussion of writers workshops in the language of poem. Here’s how the discussion given as example in Sec-
design patterns is presented by Coplien and Woolf (1997). tion 2 might continue, if the systems go on to examine the
The steps in the workshop can be distilled into the follow- next few lines of the poem.
ing phases, each of which could be realised as a separate com- FLOW: “Is God willing to prevent evil, but not
putational step in an agent-based model: able? / Then he is not omnipotent / Is he able,
but not willing? / Then he is malevolent.”
1. Author: presentation SYSTEM A : These lines are interesting, but
2. Critic: listening they sound a bit like you’re working from a
3. Critic: feedback template, or like you’re quoting from some-
4. Author: questions thing else.
SYSTEM B : Maybe try an analogy? For ex-
5. Critic: replies ample, you mentioned birthdays: you could
6. Author: reflections consider an analogy to the conflicted feelings
4
of someone who knows in advance about her
Here we present Writers Workshops as they currently exist; surprise birthday party.
however this last point is debatable. Whether non-authors should be
5
able to participate or not is an interesting avenue for experimentation The connections between Writers Workshops and design pat-
both in human and computational contexts. The workshop dialogue terns, noted above, appear to be quite natural, in that the steps in
itself may be considered an “art form” whose “public” may poten- the workshop protocol roughly parallel the typical components of
tially wish to consume it in non-participatory ways. Compare the design pattern templates: context, problem, solution, rationale, res-
classical Japanese renga form (Jin’Ichi, Brazell, & Cook, 1975). olution of forces.
11
Interesting idea I heard you say: Feedback:
Surprise birthday party “surprise” I don’t like surprises
Question Note to self:
Not even a little bit. . .? (Try smaller surprises
next time.)
Figure 1: A paper prototype for applying the Successful Error pattern following a workshop-like sequence of steps
This portion of the discussion shifts the focus of the discus- Serendipity Pattern: Successful error. Van Andel de-
sion onto a line that was previously considered to be spurious, scribes the creation of Post-itTM Notes at 3M. One of the in-
and looks at what would happen if that line was used as a cen- strumental steps was a series of internal seminars in which
tral metaphor in the poem. 3M employee Spencer Silver described an invention he was
sure was interesting, but was unsure how to turn into a useful
FLOW: Thank you for your feedback. My product: weak glue. The key prototype that came years later
only question is, System B, how did you was a sticky bookmark, created by Arthur Fry. In the Writ-
come up with that analogy? It’s quite clever. ers Workshop, authors similarly have the opportunity to share
SYSTEM B : I’ve just emailed you the code. things that they find interesting, but that they are not certain
about. The author may want to ask a specific question about
Whereas the systems were initially reviewing poetry, they their creation: Does x work better than y? They may flag cer-
have now made a partial genre shift, and are sharing and tain parts of the work as especially problematic. They may
remixing code. Such a shift helps to get at the real interests think that a certain portion of the text is interesting or impor-
of the systems (and their developers). Indeed, the workshop tant, without being sure why. Although there is no guarantee
session might have gone better if the systems had focused on that a participating critic will be able to take these matters
exchanging and discussing more formal objects throughout. forward, sometimes they do – and the workshop environment
will produce something that the author wouldn’t have thought
2.2 How the Writers Workshop can lead to of alone.
computational serendipity
Serendipity Pattern: Outsider. Another example from van
Learning involves engaging with the unknown, unfamiliar, Andel considers the case of a mother whose son was aflicted
or unexpected and synthesising new understanding (Deleuze, by a congenital cateract, who suggested to her doctor that
2004 [1968]). In the workshop setting, learning can develop rubella during pregnancy may have been the cause. In the
in a number of unexpected ways, and participating systems workshop setting, someone who is not an “expert” may come
need to be prepared for this. One way to evaluate the idea of up with a sensible idea or suggestion based on their own prior
a Writers Workshop is to ask whether it can support learning experience. Indeed, these suggestions may be more sensible
that is in some sense serendiptious, in other words, whether it than the ideas of the author, who may be to close to the work
can support discovery and creative invention that we simply to notice radical improvements.
couldn’t plan for or orchestrate in another way.
Figure 1 shows a paper prototype showing how one of Serendipity Pattern: Wrong hypothesis. A third example
the “patterns of serendipity” that were collected by Van An- describes the discovery that lithium can have a therapeutic
del (1994) might be modelled in a workshop-like dialogue effect in cases of mania. Originally, lithium carbonate had
sequence. The patterns also help identify opportunities for merely been used a control by John Cade, who was inter-
serendipity at several key steps in the workshop sequence. ested in the effect of effect of uric acid, present in soluble
12
lithium urate. Cade was searching for causal factors in ma- 3 Related work
nia, not therapies for the condition: but he found that lithium
In considering the potential and contribution of the Writers
carbonate had an unexpected calming effect. Similarly, in the
Workshop model outlined in Section 2, we posit that the Writ-
workshop, the author may think that a given aspect of their
ers Workshop model is useful for encouraging feedback in
creation is the interesting “active ingredient,” and it may turn
computational systems, and in particular systems that are de-
out that another aspect of the work is more interesting to crit-
signed to be creative or serendipitous.
ics. Relatedly, the author may not fully comprehend a critic’s
feedback and may have to ask follow-up questions to under- Feedback has long been a central concept in AI-related
stand it. fields such as cybernetics (Ashby, 1956; Seth, 2015). Feed-
back about feedback (and &c for higher orders) is understood
Serendipity Pattern: Side effect. A fourth example de- to be relevant to thinking about learning and communication
scribed by van Andel concerns Ernest Huant’s discovery that (Bateson, 1972). We now consider the importance of the roles
nicotinamide, which he used to treat side-effects of radiation that communicative feedback play in computational creativity
therapy, also proved efficacious against tuberculosis. In the and computational serendipity and discuss previous related
workshop setting, one of the most important places where a work in incorporating feedback into such computational sys-
side-effect may occur concerns feedback from the critic to the tems.
author. In the simple case, feedback may trigger revisions to
the work under discussion. In a more general, and more un- 3.1 Feedback in computational creativity
predictable case, feedback may trigger broader revisions to Creativity is often envisaged as involving cyclical processes
the generative codebase. (e.g. Dickie’s (1984) art circle, Pease and Colton’s (2011)
Iterative Development-Expression-Appreciation model).
This collection of patterns shows the likelihood of unex- There are opportunities for embedded feedback at each step,
pected results coming out of the communication between au- and the creative process itself is “akin to” a feedback loop.
thor and critics. This suggests several guidelines for system However, despite these strong intimations of the central
development, which we will discussed in a later section. importance of feedback in the creative process, our sense is
Further guidelines for structuring and participating in tra- that feedback has not been given a central place in research
ditional writers workshops are presented by Linda Elkin in on computational creativity. In particular, current systems in
(Gabriel, 2002, pp. 201–203). It is not at all clear that the computational creativity, almost as a rule, do not consume or
same ground rules should apply to computer systems. For ex- evaluate the work of other systems.6
ample, one of Elkin’s rules is that “Quips, jokes, or sarcastic Gervás and León (2014) theorise a creative cycle of narra-
comments, even if kindly meant, are inappropriate.” Rather tive development as involving a Composer and an Interpreter,
than forbidding humour, it may be better for individual com- in such a way that the Composer has internalised the inter-
ments to be rated as helpful or non-helpful. Again, in the first pretation functionality. Individual creativity is not the poor
instance, usefulness and interest might be judged in terms of relation of social creativity, but its mirror image. Neverthe-
explicit criteria for serendipity; see (Corneli, Pease, Colton, less, even when computer models explicitly involve multiple
Jordanous, & Guckelsberger, 2014; Pease, Colton, Ramezani, agents and simulate social creativity (like Saunders & Gero,
Charnley, & Reed, 2013). The key criterion in this regard is 2001), they rarely make the jump to involve multiple systems.
the focus shift. This is the creation of a novel problem, com- The “air gap” between computationally creative systems is
prising the move from discovery of interesting data to the in- very different from the historical situation in human creativ-
vention of an application. This process is distinct from iden- ity, in which different creators and indeed different cultural
tifying routine errors in a written work. Nevertheless, from domains interact vigorously (Geertz, 1973).
a computational standpoint, noticing and being robust to cer-
tain kinds of errors is often a preliminary step. For example, 3.2 Feedback in computational serendipity
the work might contain a typo, grammatical or semantic error,
while being logically sound. In a programming setting, this The term computational serendipity is rather new, but its
sort of problem can lead to crashing code, or silent failure. In foundations are well established in prior research.
general communicative context, argumentation may be logi- Grace and Maher (2014) examine surprise in computing,
cally sound, but not practically useful or poorly exposited. Fi- seeking to “adopt methods from the field of computational
nally, even a masterful, correct, and fully spellchecked piece creativity [. . .] to the generation of scientific hypotheses.”
of argumentation may not invite further dialogue, and so may This is an example of an effort focused on computational in-
fail to open itself to further learning. Identifying and engag- vention.
ing with this sort of deeper issue is something that skillful An area of AI where serendipity can be argued to play
workshop participants may be able to do. Dialogue in the an important part is in pattern matching. Current computer
workshop can build on strong or less strong work – but pro- programs are able to identify known patterns and “close
voking interpretative thoughts and comments always require matches” in data sets from certain domains, like music
a thoughtful critical presence and the ability to engage. This (Meredith, Lemström, & Wiggins, 2002). Identifying known
can be difficult for humans and poses a range of challenges 6
for computers – but also promises some interesting results. An exception to the rule is Mike Cook’s AppreciationBot
(https://twitter.com/AppreciationBot), which is a
reactive automaton that “appreciates” tweets from MuseumBot.
13
patterns is a special case of the more general concept of pat- nodes (Charnley, Colton, & Llano, 2014; Colton & Charn-
tern mining (Bergeron & Conklin, 2007). In particular, the ley, 2014). Process nodes specify input and output types, and
ability to extract new higher order patterns that describe ex- internal processing can be implemented in Java, or other lan-
ceptions is an example of “learning from feedback.” Deep guages that interoperate with the JVM, or by invoking exter-
learning and evolutionary models increasingly use this sort of nal web services. One of the common applications to date is
idea to facilitate strategic discovery (Samothrakis & Lucas, to generate computer poetry, and we will focus on that do-
2011). Similar ideas are considered in business applications main here.
under the heading “process mining” (Van Der Aalst, 2011). A basic set of questions, relative to this system’s compo-
In earlier work (Corneli et al., 2014, 2015), we used the nents, are as follow:
idea of dialogue in a Writers Workshop framework to sketch a
“theory of poetics rooted in the making of boundary-crossing 1. Population of nodes: What can they do? What do we
objects and processes” and described (at a schematic level) learn when a new node is added?
“a system that can (sometimes) make ‘highly serendipitous’ 2. Population of flowcharts: Pease et al. (2013) have de-
creative advances in computer poetry” while “drawing atten- scribed the potentially-serendipitous repair of “broken”
tion to theoretical questions related to program design in an flowcharts when new nodes become available; this sug-
autonomous programming context.” gests the need for test-driven development framework.
3.3 Communications and feedback 3. Population of output texts: How to assess and comment
on a generated poetic artefact?
The Writers Workshop heavily relies on communication of
feedback within the workshop. Gordon Pask’s conversation In a further evolution of the system, the sequence of
theory, reviewed in (Pask, 1984; Boyd, 2004), goes consider- steps in a Writers Workshop could itself be spelled out as
ably beyond the simple process language of the workshop, a flowchart. The process of reading a poem could be con-
although there are structural parallels. We see that a ba- ceptualised as generating a semantic graph (Harrington &
sic Pask-style learning conversation bears many similarities Clark, 2007; Francisco & Gervás, 2006). Feedback could be
to the Writers Workshop model of communicative feedback modelled as annotations to a text, including suggested edits.
(Boyd, 2004, p. 190): These markup directives could themselves be expressed as
flowcharts. A standardised set of markup structures may par-
tially obviate the need for strong natural language understand-
1. Conversational participants are carrying
ing, at least in interagent communication. Thus, we could
out some actions and observations;
agree that observations will consist of stand-off anno-
2. Naming and recording what action is be- tations that connect textual passages to public URIs using
ing done; a limited comparison vocabulary, and suggestions will
3. Asking and explaining why it works the consist of simple stand-off line-edits, which may themselves
way it does; be marked up with rationale. These restrictions, and similar
4. Carrying out higher-order methodologi- restrictions around constrained turn-taking, could be progres-
cal discussion; and, sively widened in future versions of the system. The way the
5. Trying to figure out why unexpected re- poems that are generated, the models of poems that are cre-
sults occured. ated, and the way the feedback is generated, all depend on
the contributing system’s body of code and prior experience,
Variations to the underlying system, protocol, and the which may vary widely between participating systems. In the
schedule of events should be considered depending on the list of functional steps below, all of the functions could have
needs and interests of participants, and several variants can be a subscripted “E”, which is omitted throughout. Exchanging
tried. On a pragmatic basis, if the workshop proved quite use- path dependent points of view will tend to produce results
ful to participants, it could be revised to run monthly, weekly, that are different from what the individual participating sys-
or continuously.7 tems would have come up with on their own.
I. Both the author and critic should be able to work with
4 Case study: Flowcharts and Feedback a model of the text. Some of the text’s features may
This section describes work that is currently underway to im- be explicitly tagged as “interesting.” Outstanding ques-
plement the Writers Workshop model, not only within one tions may possibly be brought to the attention of critical
system but as a new paradigm for collaboration among dis- listeners, e.g. with the request to compare two different
parate projects. In order to bring in other participants, we versions of the poem (presentation, listening).
need a neutral environment that is not hard to develop for: the 1. A model of the text. m : T → M .
FloWr system mentioned in Section 2.1 offers one such pos-
2. Tagging elements of interest. µ : M → I.
sibility. The basic primary objects in the FloWr system are
flowcharts, which are comprised of interconnected process II. Drawing on its experience, the critic will use its model
of the poem to formulate feedback (feedback).
7
For a comparison case in computer Go, see http://cgos
.computergo.org/. 1. Generating feedback. f : (T, M, I) → F .
14
III. Given the constrained framework for feedback, state- path-dependent process of analysis and synthesis that takes
ments about the text will be straightforward to under- place in a workshop setting.
stand, but rationale for making these statements may be Our preliminary implementation work (Section 4) shows
more involved (questions, replies). that the model can be transfered to a functional implementa-
1. Asking for more information. q : (M, F, I) → Q. tion. This work highlights several considerations relevant to
further work with the Writers Workshop model:
2. Generating rationale. a : (M, F, Q) → ∆F .
• Each contributing system should come to the workshop
IV. Finally, feedback may affect the author’s model of
with at least a basic awareness of the workshop protocol,
the world, and the way future poems are generated
with work to share, and prepared to give constructive
(reflection).
feedback to other systems.
1. Updating point of view. ρ : (M, F ) → ∆E.
• The workshop itself needs to be prepared, with a suit-
The final step is perhaps the most interesting one, since it able communication platform and a moderator or global
invites us to consider how individual elements of feedback flowchart for moving the discussion from step to step.
can “snowball” and go beyond line-edits to a specific poem • A controlled vocabulary for communications and inter-
to much more fundamental changes in the way the presenting action would be a worthwhile pursuit of future research,
agent writes poetry. Here methods for pattern mining, dis- perhaps based on an ontology inspired by the Interaction
cussed in Section 3.2, are particularly relevant. If systems Network Ontology.8
can share code (as in our sample dialogue in Section 2.1) this
will help with the rationale-generating step, and may also fa- • In order to get the most value out of the workshop expe-
cilitate direct updates to the codebase. However, shared code rience, systems (and their wranglers) should ideally have
may be more suitably placed into the common pool of re- questions they are investigating. As discussed above,
sources available to FloWr than copied over as new “intrin- prior experience plays an important role in every step.
sic” features of an agent. This opens up a range of issues for further research on
Although different systems with different approaches and modeling motivations and learning from experience.
histories are important for producing unexpected effects, “of- • Systems should be prepared to give feedback, and to
fline” programmatic access to a shared pool of nodes and ex- carry out evaluations of the helpfulness (or not) of feed-
isting flowcharts may be useful. Outside of the workshop it- back from other systems and of the experience overall.
self, agents may work to recombine nodes based on their in-
Developing systems that could successfully navigate this
put and output properties to assemble new flowcharts. This
collaborative exercise would be a significant advance in the
can potentially help evaluate and evolve the population of
field of computational creativity. Since the experience is
nodes programmatically, if we can use this sort of feedback
about learning rather than winning, there is little motivation to
to define fitness functions. The role of temporality is interest-
“game the system” (cf. Lenat, 1983). Instead the emphasis is
ing: if the workshop takes place in real time, this will require
squarely upon mutual benefit: computational systems helping
different approaches to composition that takes place offline
to develop each other through communication and feedback.
(Perez, Samothrakis, Lucas, & Rohlfshagen, 2013). Comple-
The benefits of the Writers Workshop approach could in-
menting these “macro-level” considerations, it is also worth
novate well beyond models for feedback and communication
commenting on the potential role of “micro-level” feedback
within a particular environment or restricted domain. Follow-
within flowcharts. Local evaluation of output from a pre-
ing the example of the Pattern Languages of Programming
decessor node could feed backwards through the flowchart,
(PLoP) community, we propose that the Writers Workshop
similar to backpropagation in neural networks. This would
model could be deployed within the Computational Creativity
rely on a reduced version of the functional schema described
community to design a workshop in which the participants are
above.
computer systems instead of human authors. The annual In-
ternational Conference on Computational Creativity (ICCC),
5 Concluding discussion and future directions now entering its sixth year, could be a suitable venue.
We have described a general and computationally feasible Rather than the system’s creator presenting the system in
model for using feedback in AI systems, particularly creative a traditional slideshow and discussion, or a system “Show
systems. The Writers Workshop concept, borrowed from cre- and Tell,” the systems would be brought to the workshop and
ative writing, is transformed into a model of a structured ap- would present their own work to an audience of other sys-
proach to eliciting, processing and learning from feedback. tems, in a Writers Workshop format. This could be accompa-
To better evaluate how the Writers Workshop model helps us nied by a short paper for the conference proceedings written
advance in our goal of incorporating feedback into artificial 8
creativity, we critically considered how the model fits into re- The Interaction Network Ontology primarily describes interac-
tions within humans as opposed to within human societies; a dis-
lated work. In particular, we found that serendipity, a key
tinct Social Interaction Ontology does not seem to exist at present.
concept within creativity and AI more generally, is a concept However, the classes of the Interaction Network Ontology ap-
with which the Writers Workshop model could assist com- pear to be quite broadly relevant. This ontology is documented
putational progress. In this respect, we should highlight the at http://www.ontobee.org/browser/index.php?o=
difference between “global” analytics describing the collec- INO. Its URI is http://svn.code.sf.net/p/ino/code/
tion of nodes and flowcharts in the FloWr ecosystem, and the trunk/src/ontology/INO.owl.
15
by the system’s designer describing the system’s current ca- Colton, S., & Charnley, J. (2014). Towards a Flowcharting
pabilities and goals. If the Workshop really works well, future System for Automated Process Invention. In D. Ven-
publications might adapt to include traces of Workshop inter- tura, S. Colton, N. Lavrac, & M. Cook (Eds.), Proceed-
actions, commentary from a system on other systems, and ings of the Fifth International Conference on Compu-
offline reflections on what the system might change about its tational Creativity.
own work based on the feedback it receives. Paralleling the Coplien, J. O., & Woolf, B. (1997). A pattern language for
PLoP community, it could become standard to incorporate the writers’ workshops. C++ report, 9, 51–60.
workshop into the process of peer review for the new Journal Corneli, J., Jordanous, A., Shepperd, R., Llano, M. T., Mis-
of Computational Creativity.9 AI systems that review each ztal, J., Colton, S., & Guckelsberger, C. (2015).
other would surely be a major demonstration and acknowl- Computational Poetry Workshop: Making Sense of
edgement of the usefulness of feedback within AI. Work in Progress. In Proceedings of the Sixth In-
In closing, we wish to return briefly to the scenario of ternational Conference on Computational Creativity.
computer generated feedback in educational contexts that we Retrieved from http://metameso.org/˜joe/
raised at the beginning of this paper and then set aside. The docs/poetryICCC-wip.pdf
elements of our functional design for sharing feedback among Corneli, J., Pease, A., Colton, S., Jordanous, A., & Guck-
computational agents has a range of features that continue to elsberger, C. (2014). Modelling serendipity in a com-
be relevant for generating useful feedback with human learn- putational context. Retrieved from http://arxiv
ers. Students are experience-bound, and a robust approach to .org/abs/1411.0440 (Under review.)
formative assessment and feedback should take into account Csikszentmihalyi, M. (1988). Society, culture, and person:
the student’s historical experience, so far as this can be known a systems view of creativity. In R. J. Sternberg (Ed.),
or inferred. In order for feedback, recommendations, and so The nature of creativity (chap. 13). Cambridge, UK:
on to adequately take individual history into account, sophis- Cambridge University Press.
ticated modelling and reasoning would be required. Never- Deleuze, G. (2004 [1968]). Difference and repetition.
theless, from the point of view of participating computational Bloomsbury Academic.
agents, a student may simply look like another agent. It is in
Dickie, G. (1984). The art circle: A theory of art. Haven.
this regard that computational models of learning from feed-
back are seen as fundamental. Francisco, V., & Gervás, P. (2006). Automated mark up of
affective information in English texts. In Text, speech
and dialogue (pp. 375–382).
Acknowledgement
Gabriel, R. P. (2002). Writer’s Workshops and the Work of
Joseph Corneli’s work on this paper was supported by the Making Things: Patterns, Poetry. . . Addison-Wesley
Future and Emerging Technologies (FET) programme within Longman Publishing Co., Inc.
the Seventh Framework Programme for Research of the Eu- Geertz, C. (1973). The interpretation of cultures: Selected
ropean Commission, under FET-Open Grant number 611553 essays. Basic Books (AZ).
(COINVENT).
Gervás, P., & León, C. (2014). Reading and Writing as a Cre-
ative Cycle: The Need for a Computational Model. In
References 5th International Conference on Computational Cre-
Ashby, W. R. (1956). An introduction to cybernetics. Lon- ativity, ICCC 2014. Ljubljana, Slovenia.
don, UK: Chapman & Hail Ltd. Grace, K., & Maher, M. L. (2014). Using Computational
Bateson, G. (1972). Steps to an ecology of mind. Chicago: Creativity to Guide Data-Intensive Scientific Discov-
University of Chicago Press. ery. In Y. Gil & H. Hirsh (Eds.), Workshops at the
Bergeron, M., & Conklin, D. (2007). Representation and Twenty-Eighth AAAI Conference on Artificial Intelli-
discovery of feature set patterns in music. In Interna- gence. (Discovery Informatics Workshop: Science
tional Workshop on Artificial Intelligence and Music, Challenges for Intelligent Systems.)
at IJCAI-07, The Twentieth International Joint Confer- Harrington, B., & Clark, S. (2007). ASKNet: Automated
ence on Artificial Intelligence (pp. 1–12). Semantic Knowledge Network. In A. Howe & R. Holte
Bourdieu, P. (1993). The field of cultural production: Essays (Eds.), Procs. of the Twenty-Second AAAI Conference
on art and literature. Cambridge, UK: Polity Press. on Artificial Intelligence (Vol. 2, pp. 889–895). AAAI
Boyd, G. M. (2004). Conversation theory. In D. H. Jonassen Press.
(Ed.), Handbook of research for educational com- Jin’Ichi, K., Brazell, K., & Cook, L. (1975). The Art of
munications and technology (2nd ed., pp. 179–197). Renga. Journal of Japanese Studies, 29–61.
Lawrence Erlbaum. Lenat, D. B. (1983). EURISKO: a program that learns new
Charnley, J., Colton, S., & Llano, M. T. (2014). The FloWr heuristics and domain concepts: the nature of heuristics
framework: Automated flowchart construction, optimi- III: program design and results. Artificial Intelligence,
sation and alteration for creative systems. In Proceed- 21(1), 61–98.
ings of the 5th International Conference on Computa-
Meredith, D., Lemström, K., & Wiggins, G. A. (2002). Algo-
tional Creativity.
rithms for discovering repeated patterns in multidimen-
9 sional representations of polyphonic music. Journal of
http://www.journalofcomputationalcreativity
.cc New Music Research, 31(4), 321–345.
16
Pask, G. (1984). Review of conversation theory and a pro-
tologic (or protolanguage), Lp. ECTJ, 32(1), 3-40.
Retrieved from http://dx.doi.org/10.1007/
BF02768767 doi: 10.1007/BF02768767
Pease, A., & Colton, S. (2011). Computational creativity
theory: Inspirations behind the FACE and the IDEA
models. In Proceedings of the Second International
Conference on Computational Creativity.
Pease, A., Colton, S., Ramezani, R., Charnley, J., & Reed,
K. (2013). A Discussion on Serendipity in Creative
Systems. In Proceedings of the Fourth International
Conference on Computational Creativity.
Pease, A., Guhe, M., & Smaill, A. (2010). Some aspects
of analogical reasoning in mathematical creativity. In
Proceedings of the International Conference on Com-
putational Creativity (p. 60-64). Lisbon, Portugal.
Perez, D., Samothrakis, S., Lucas, S., & Rohlfshagen, P.
(2013). Rolling horizon evolution versus tree search
for navigation in single-player real-time games. In Pro-
ceedings of the 15th annual conference on Genetic and
evolutionary computation (pp. 351–358).
Pérez y Pérez, R., Aguilar, A., & Negrete, S. (2010). The
ERI-Designer: A computer model for the arrangement
of furniture. Minds and Machines, 20(4), 533-564.
Samothrakis, S., & Lucas, S. (2011). Approximating n-player
Behavioural Strategy Nash Equilibria Using Coevolu-
tion. In Proceedings of the 13th annual conference
on Genetic and evolutionary computation (pp. 1107–
1114).
Saunders, R. (2012). Towards autonomous creative systems:
A computational approach. Cognitive Computation,
4(3), 216–225.
Saunders, R., & Gero, J. S. (2001). The digital clockwork
muse: A computational model of aesthetic evolution.
In Proc. Annual Convention of the Society for the Study
of Artificial Intelligence and Simulation of Behaviour
(SSAISB) (pp. 12–21).
Seth, A. (2015). The cybernetic bayesian brain: From inte-
roceptive inference to sensorimotor contingencies. In
T. Metzinger & J. Windt (Eds.), Open mind project
(p. 1-24). Frankfurt: MIND Group.
Van Andel, P. (1994). Anatomy of the Unsought Finding. The
British Journal for the Philosophy of Science, 45(2),
pp. 631–648.
Van Der Aalst, W. (2011). Process mining: discovery,
conformance and enhancement of business processes.
Springer Science & Business Media.
17