=Paper=
{{Paper
|id=Vol-1907/12_mici_morreale
|storemode=property
|title=Constraining Control in Mixed-Initiative Musical Interfaces
|pdfUrl=https://ceur-ws.org/Vol-1907/12_mici_morreale.pdf
|volume=Vol-1907
|authors=Fabio Morreale,Raul Masu,Andrew McPherson
|dblpUrl=https://dblp.org/rec/conf/chi/MorrealeMM17
}}
==Constraining Control in Mixed-Initiative Musical Interfaces==
https://ceur-ws.org/Vol-1907/12_mici_morreale.pdf
Constraining Control in
Mixed-Initiative Musical Interfaces
Fabio Morreale Raul Masu Abstract
Centre for Digital Music interAction Lab Research on musical instrument design suggests that de-
School of EECS DISI liberate constraints can offer new creative experiences to
Queen Mary University of University of Trento
the performer. At times, design constraints are physically
London, UK Italy
embedded in the instrument to limit the interaction possibil-
f.morreale@qmul.ac.uk raul.masu@unitn.it
ities; in other cases the constraints are given by delegating
Andrew McPherson
part of the control to the machine. In this paper we pro-
Centre for Digital Music
pose a case study related to the latter form of constraints:
School of EECS
Queen Mary University of
in Chimney the musician delegates control on timing to an
London, UK autonomous agent. Elaborating on opinions collected from
a.mcpherson@qmul.ac.uk musicians interacting with Chimney, we reflect on the con-
sequences of delegating part of the control to the machine.
Without the possibility of influencing the temporal evolution
of the piece, the human performer is pushed to find creative
workarounds and to surrender to a more balanced collabo-
ration with the machine.
Author Keywords
digital musical instruments; creativity; constraints; interface
design
ACM Classification Keywords
Copyright ©2017 for this paper is held by the author(s). Proceedings of MICI 2017:
H.5.5 [Sound And Music Computing]: Systems
CHI Workshop on Mixed-Initiative Creative Interfaces.
Introduction
In Mixed-Initiative Creative Interfaces humans and ma-
chines collaborate to produce creative outputs. Efforts are
usually made for the two parts to dialogue, each employ- ported that constraints themselves were conducive to ex-
ing their own assets and delegating their limitations to the ploring subtle musical variations.
other. However, efforts can also be made to specifically
constrain human control on the creative process. Con- Another form of constraint in musical interaction is given by
straints are not ruthless attempt to limit human creativity; delegating part of the control to a machine. This is the case
rather the opposite can be true as advocated by Margaret for conductor systems, which are essentially tools that allow
A. Boden in her seminal book on human creativity [1] : musicians to control the playback of pre-recorded music.
Performers only control timing, dynamics, and other ex-
“People often claim that talk of ’rules’ and ’constraints’...must pressive factors and the machine worries about getting the
be irrelevant to creativity, which is an expression of human notes right [5]. These systems have attracted the interest of
freedom. But far from being the antithesis of creativity, con- the general public (Guitar Hero1 , Magic Fiddle 2 and similar
straints on thinking are what make it possible...Constraints applications are forms of conductor programs) by offering
map out a territory of structural possibilities which can then a simple interaction that does not require extensive training
be explored, and perhaps transformed to give another one” on the instrument [2].
In the musical domain, physical constraints contribute to This paper proposes another example in which part of the
characterise the instrument expressive scope and influence control is delegated to the machine. Chimney, a software
its playing possibilities and the development of a personal instrument developed by two of the authors [6], forces the
style [8]. For instance, the constraints of the piano define musician to delegate timing decisions to a non-responsive
its expressive character, which, in turn, defines the pianist’s autonomous agent. As a consequence, the human agent
style of playing and the compositions possibilites [4]. has limited control on the temporal evolution of the piece.
The behaviour of Chimney is in a sense the opposite kind of
New digital musical instruments (DMIs) also sometimes in- delegation compared to a conductor system. With conduc-
tegrate design constraints for creative explorations. Such tor systems a musicians can mainly get time control; with
constraints can limit the number of inputs and the outcomes Chimney timing is the main thing the musician lacks. The
that are mapped to those inputs. For example, Gurevich specific behaviour of the agent is secondary to the scope
and colleagues designed a purposely over-simplified in- of this paper, which rather focuses on the consequences
strument, in which performer interaction is very limited [3]. of divesting control on timing to an unpredictable algorith-
Despite its limitations, or perhaps because of them, the per- mic agent. The implications on the experience of the per-
formers exhibited a significant degree of stylistic diversity in former of delegating control on timing to the machine are
their interaction with the instrument. presented in the Discussion section.
In a similar study, Zappi’s and McPherson’s Cube Instru-
ment was designed to show the role of interaction con-
straints in encouraging “creative (mis)uses of technology”
[8]. Several performers that used the Cube Instrument re-
Figure 1: Screenshot of Chimney. The white path is the trace left
by the algorithmic walker. The circles are the sonic material added and mutes it. As the random walker enters a circle, the am-
by the human performer. plitude of the sound connected to that circle increases. The
maximum level is reached when the walker is at the centre
of the circle.
Player interaction is then reduced to deciding the sonic ob-
jects, and their likelihood of being played. Under these con-
ditions, the musician cannot organise the temporal struc-
ture, which is entirely controlled by the algorithmic agent.
An informal evaluation was conducted at a public concert in
which Chimney was perfomed in a duo with a trumpet [6].
Both the musician that was controlling Chimney (the second
author of this paper) and the trumpeter were interviewed at
the end of the performace.
The lack of control over timing caused interesting experi-
ences for both musicians. The performer who was control-
ling Chimney reported emotional reactions that usually do
not belong to the palette of emotions in music. Unable to
Chimney: Delegating Timing
control every aspect of the music, he could only wish for the
An example of DMI in which the musician divests control on
algorithm to answer in a particular way. As a consequence,
timing to the machine is offered by Chimney [6]. In this vir-
he experienced hope, surprise, frustration, and resignation.
tual DMI the control on the temporal evolution of the piece
is delegated to a computer agent, an algorithmic random Other interesting comments were collected from the trum-
walker. The walker autonomously roams throughout a vir- peter. Understanding all the details of the music played by
tual space following a specific statistical distribution mod- Chimney was particularly demanding. In particular, as op-
elled on an adapted version of the Perlin Noise [7]. posed to traditional improvisation, he did not feel free to
propose new musical material. He was always answering to
The musician can interact with the composition by adding to
Chimney proposals. The only exception to such approach
the virtual space pre-recorded sonic material. These sound
concerned big changes in dynamics (e.g. very intense
sources are displayed as circles that can be resized, reposi-
crescendo or diminuendo). In this cases, the trumpeter
tioned, or deleted in real time by the musician. When a new
explicitly communicated to the Chimney performer to add
circle is added to the canvas, the system sets it in idle state
sonic material to the canvas to obtain a crescendo, and to
remove some for the diminuendo. Furthermore, the trum-
1
peter commented that the specific behaviour of Chimney
https://www.guitarhero.com/uk/en/
2
https://www.smule.com/sunset/magicfiddle of being non-responsive fostered him to find novel musi-
cal strategies. For instance, in those situations in which he Finally, it should be noted that in typical MICIs, the com-
would have preferred a higher complexity in the music, he puter acquires the status of creative agent by using forms
compensated for this perceived deficiency by increasing the of artificial intelligence that actively take decisions on tasks
rhythm complexity or the loudness of his instrument. that are typically performed by humans. Although the arti-
ficial intelligence element is limited or absent in the exam-
Discussion ples presented in this paper, we believe that our reflections
This paper reviewed and compared different ways in which about the consequences of divesting control to a machine
constraining the interaction possibilities of a peformer can can be of interest for the disussions at the workshop.
be a vector of new forms of creativity. In particular, our pre-
liminary investigations with Chimney suggest that delegat- References
ing some initiative to the machine produces a set of creative [1] Margaret A Boden. 2004. The creative mind: Myths
reactions and feelings that are at times similar and at times and mechanisms. Psychology Press.
richer than simply constraining the interaction possibilies. [2] Elaine Chew and Andrew McPherson. Forthcoming.
Performing music: humans, computers and electron-
For instance, [8] found that players who got more degrees ics. In The Routledge Companion to Music Cognition,
of freedom on the Box Instrument (they could control pitch) R Ashley and R Timmers (Eds.). NY: Taylor and Fran-
exhibtited less unexpected techniques than those with a cis.
single degree of freedom (no control on pitch). In this case, [3] Michael Gurevich, Paul Stapleton, and Adnan
players who could not control such an important musical Marquez-Borbon. 2010. Style and Constraint in Elec-
feature needed to came up with original approaches to tronic Musical Instruments.. In NIME. 106–111.
playing. Similarly, the musicians that performed with Chim- [4] Thor Magnusson. 2010. Designing constraints: Com-
ney, even if they could control every other aspect of the per- posing and performing with digital musical systems.
formance, lacked control of timing, which fundamentally Computer Music Journal 34, 4 (2010), 62–73.
changed how the performance was structured. [5] Max V Mathews. 1991. The radio baton and conductor
program, or: Pitch, the most important and least ex-
One difference that we found is presence of a range of
pressive part of music. Computer Music Journal 15, 4
affective states like surpise and hope that are present in
(1991), 37–46.
Chimney and other MICIs but seem to be absent from sim-
[6] Fabio Morreale and Raul Masu. 2016. Renegotiating
ple DMIs and conductor systems. The self-frustration expe-
Responsibilities in Human-Computer Ensembles. Pro-
rienced by the performer that operated Chimney is a typical
ceedings of Colloquio di Informatica Musicale (2016),
feeling experienced in music - for instance by instrumen-
41–45.
talists when they cannot play a passage right. However, in
[7] Ken Perlin. 1985. An image synthesizer. ACM Sig-
this case, the frustration was accredited to the impossibility
graph Computer Graphics 19, 3 (1985), 287–296.
of precisely expressing one’s personal musical initiative, no
[8] Victor Zappi and Andrew McPherson. 2014. Design
matter how hard he could try.
and use of a hackable digital instrument. In NIME.