=Paper=
{{Paper
|id=Vol-1907/11_mici_sicchio
|storemode=property
|title=Layering the Choreographic Process: Making Dance Work with Machine Learning
|pdfUrl=https://ceur-ws.org/Vol-1907/11_mici_sicchio.pdf
|volume=Vol-1907
|authors=Kate Sicchio
|dblpUrl=https://dblp.org/rec/conf/chi/Sicchio17
}}
==Layering the Choreographic Process: Making Dance Work with Machine Learning==
https://ceur-ws.org/Vol-1907/11_mici_sicchio.pdf
Layering the Choreographic Process:
Making Dance Work with Machine
Learning
Kate Sicchio, PhD Abstract
New York University This paper discusses a new performance by the author
Tandon School of Engineering that was made in part by machine learning algorithms.
Brooklyn, NY 11237, USA Working with the t-SNE algorithm to visualize data, a
sicchio@nyu.edu choreographic score can become performed through a
layer of images, live coding and an improviser
performer. The final performance aims to produce new
possibilities for live performance through using code to
traverse clusters of media that the algorithm has
produced.
Author Keywords
Choreographic score; machine learning; dance
improvisation; t-SNE
ACM Classification Keywords
ACM now Copyright © 2017 for this paper is held by the author(s). H.5.m. Information interfaces and presentation (e.g.,
Proceedings of MICI 2017: CHI Workshop on Mixed-Initiative Creative HCI): Miscellaneous
Interfaces.
Introduction
When working with dance and technology it is common
for work to be developed where the dancer and their
movement is used to reflect the technology, or even
demonstrate the technology. But when choreographers
use machine learning techniques, there are interesting
results when creating layered approaches, or mixed-
initiative creative interfaces to influence the movement of these types of movements remain in the piece, in the
vocabulary in new ways. form of photographs.
For example, Wayne McGregor has worked with the The second layer is the camera. Time lapse
Choreographic Language Agent, an AI agent used to photography was used to capture the original
augment his process since 2004. The one iteration of movement. During this process a GoPro photographed
this was Becoming, a tool developed with Marc Downie two photos per second. Over 3000 images from a
and Nick Rothwell. The aim is to provoke new dance improvisation session were captured. Some
movement creation in the dance studio, which is later photos were clear shapes, others are bizarre half
shaped by McGregor to become a layer process in moments of movement. The movement is transformed
creating a performance output. Both of these pieces in this layer without any machine learning just by trying
rely on the computation as part of the choreographic to make it static.
practice. This layered approach does not celebrate the
computer as the choreographer but simply uses it in a
way to inspire or create new movements that may be
have been part of the choreographer or dancer’s
cannon.
Untitled Algorithmic Dance 2
Untitled Algorithmic Dance 2 is a dance performance
where the choreography is generated through a series
of interactions between humans and machines. It is a
layered approach that creates a system with various
places for interpretation, by both people and
algorithms. This is no simple mapping of input to
output, but several processes that layer to become a Figure 1: Example of clustering of images as performed by the
dance. t-SNE.
The first layer in this process is human. A dancer
improvises to create movement. This becomes the
The next layer is the actual layer of machine learning
content that is layer used to produce a choreographic
using Gene Kogan’s code for the t-SNE algorithm
score for the performance. Some of the movement
(originally by van der Maaten) in OpenFrameWorks to in
material is just warming up, some is dancing for the
effect cluster the images. This is an untrained machine
sake of movement, some is purely gestural such as me
learning algorithm used to visual large amounts of
grabbing a water bottle, and some responds to the
data. The algorithm finds the similarities between
prompt of creating movement for a camera. For now all
photos and places them into a grid. What you see right working with machine learning that are still human
away is a new dance. A score made just through centered and uses this process as just a tool.
recognition and reorganization.
Conclusions
However, one does not have to simply use this grid,
and certainly using the grid as a score is only one Untitled Algorithmic Dance 2 explores the use of the t-
possibility with this re-imaged plot of movement. SNE algorithm to create live choreographic scores from
Another human layer becomes part of the system a collection of time lapse photos. It works with a
within this piece. The navigation of the clusters of complex layering of human and machine interactions to
captured movement is done live by the choreographer. create a performance in which machine learning
In this first version of the piece it is very simply done influences the final piece. Through this process an
through navigating the grid with a mouse. The interruption of a choreographic process via AI has
choreographer controls the path of the score, the begun and will be developed further with different
timing of the score and most importantly can respond initiatives such as motion capture.
to the dancer performing the score.
References
Which brings me to the final layer of the system, 1. Kate Sicchio. 2014. Hacking Choreography: Dance
another human. There is an interpreter at the end of and Live Coding. Computer Music Journal. Vol. 38, Iss.
this system, which is a dancer. The dancer sees the 1 (March 2014 ) pp31 – 39.
image that the choreographer has currently selected
and responds. For this performance, the dancer has 2. Studio Wayne McGregor. 2004. Choreographic
agency to respond however they like – they may copy Language Agent.
the shape of the body in the image, or make a http://waynemcgregor.com/research/choreographic-
contrapuntal shape. They may move in a direction language-agent
indicated by the image or move in the opposite way.
The dancer has the final say in the movement being 3. Laurens van der Maaten. 2014. Accelerating t-SNE
generated. The human dancer has the ultimate control. using Tree-Based Algorithms. Journal of Machine
Learning Research 15 (2014) pp 1-21.
And while this layered approach works in creating a
composition that we all recognize as a dance piece, it is 4. Gene Kogan. 2016.
infused throughout with new tools to push the https://github.com/genekogan/ofxTSNE
choreographic possibilities of a single choreographer.
This resonates with McGregor in that it is an influence
on the movement and not the full development of the
movement. Perhaps this also demonstrates ways of