=Paper=
{{Paper
|id=Vol-1628/Creative_paper4
|storemode=property
|title=None
|pdfUrl=https://ceur-ws.org/Vol-1628/Creative_paper4.pdf
|volume=Vol-1628
}}
==None==
https://ceur-ws.org/Vol-1628/Creative_paper4.pdf
Hypertext and Cultural Autobiography:
Talk with Your Hands Like an Ellis Island Mutt
Steven Wingate
South Dakota State University
English Department, Box 2218
Brookings, SD 57007
1-720-256-8435
Steven.Wingate@sdstate.edu
ABSTRACT
Talk with Your Hands Like an Ellis Island Mutt is an interactive
video, built using Korsakow cinema database software, made of
interlocking fragments. It consists of 157 short clips (most 8-12
seconds in length) including altered images of my own hand
gestures, archived records and objects of my immigrant ancestors,
and archival footage of newcomers arriving at Ellis Island—
through which the majority of European immigrants passed as
America’s population exploded between 1890 and 1920. It offers
multiple unique playthroughs in an order determined both by
interactor choice and the computational operation of its database.
CCS Concepts
Figure 1. Various hand gestures from Talk with Your Hands
• Applied computing Arts and HumanitiesMedia Arts.
Like an Ellis Island Mutt.
Keywords working class immigrants and their descendants. (One expected
Interactive cinema; interactive video; database cinema; Korsakow; result of successful assimilation is the loss of this habit, which I
hypertext; visual hypertext; cultural autobiography. did not achieve.) The idea for HandMutt came during the editing
process for my digital lyric memoir daddylabyrinth, which
1.1 Description of the piece premiered at the International Conference on Interactive Digital
Talk with Your Hands Like an Ellis Island Mutt is an interactive Storytelling in 2014 at the ArtScience Museum of Singapore. In
video project that uses new media tools to explore cultural identity its video footage I found many hand gestures that led me to reflect
in a way that analog tools would not permit, suggesting that on my own heritage and on the multi-generational assimilation
identity is the result of multiple lifelong collisions between process of American immigrants. This resulted in the birth of
elements of personality that we have inherited with or without HandMutt as a separate project.
knowing it. The technology-enabled fragmentation of story, image, and
Designed for performative screening or interactive exhibition and consciousness in HandMutt reflects the fragmentation of cultural
built in the Korsakow cinema database system, HandMutt identity itself, which is often so dispersed in our lives that we can
embodies my experience as a second-generation American whose scarcely identify it. The work generates a nuanced, intimate
ancestors came through Ellis Island from Europe (Poland, understanding of the immigration and assimilation experience that
Hungary, and Scotland between 1907 and 1914), and whose would not be possible with traditional monolinear narrative and
identity is formed by a multi-generational assimilation process non-interactive media.
that often feels perpetual. With three of my four grandparents The video clips in HandMutt come from four distinct sources, and
being born outside the US in non-English speaking environments each is paired with short voice-over commentary by me. The
(and the fourth born here not long after his own parents arrived), I largest group is a set of sixty-four of my hand gestures harvested
have been aware of my own otherness all my life, particularly in from the video selfies in my digital work daddylabyrinth. Each of
regard to my working class and Eastern European heritage. these gestures is isolated from its communicative context and
Part of the expression of that heritage is talking with one’s broken down into micromotions—often only a few frames in
hands—a habit attributed in America almost exclusively to length—that are repeated, reversed, and subjected to
manipulations in frame rate using Final Cut Pro X editing
software. This visual idea is an homage to my film mentor Ken
Jacobs, whose performative Nervous System films of the 1980s
and 90s created a similar “stutter” effect (one that we now see all
around us, in altered form, thanks to GIF culture).
The second group of clips I worked with was harvested from
public domain archival footage of immigrants arriving at Ellis
Island in 1903 and 1906, which are subject to the same Ken
Jacobs-esque manipulations. I left these untouched in terms of
Figure 2. Still with archival Ellis Island footage.
Figure 3. A view of the Korsakow database.
color to keep the historical record of immigration as cleanly
represented as possible. These images, which were shot with a managing multiple perspectives and combinations of narrative
primitive stationary camera, are blown up to capture the micro- elements. The most important aspect of digital arts to me is the
dramas embedded within the larger frame—a squabbling couple, a ability to develop stories that are not pre-arranged into traditional
frightened child, an old woman uncertain of the life ahead of her. Aristotelian act structures. I am interested in creating what Judy
Malloy calls “narrabases” and Mark Amerika calls “narrative
HandMutt also includes clips of tangible family relics and
environments”—bodies of related emotional material that can be
photographs from my maternal and paternal families. I have few
encountered in various configurations, each resulting in a unique
possessions from either side, and many have achieved near-
aesthetic experience that relies on interactors’ choices and calls on
iconographic status for me no matter how simple or utilitarian their innate ability to co-create stories from discrete elements.
they are. The final group is simple text animations of keywords
that are used in the Korsakow database that resonated with themes
To achieve this I have sought out software systems designed to
which emerged from the other three categories of clips. These offer multiple possible relationships between the nodes in my
represent the emotions and psycho-social forces that have shaped
narratives. Thus far I have worked with Scalar for daddylabyrinth:
my relationship with my ancestry. The overall result of these
a digital lyric memoir and Korsakow for HandMutt. In the future I
multiple strands in conversation with one another is that of many am particularly interested in working with software that allows for
stories weaving together to form a narrative experience without
unconscious interactor choice (using biometric input, for
the benefit of Aristotelian coherence—a mosaic portrait of
instance), which would enable me to develop ambient or proto-
American assimilation and its multi-generational challenges.
narrative experiences built up from thousands or even tens of
Interactors can experience HandMutt via installation or web thousands of narrative stimuli.
browser. They navigate through a series of brief clips, choosing
the next one from a set of thumbnail images that appear at the end 1.3 Hardware requirements, length, format,
of each—all representing other nodes in the video database. The HandMutt can be run on any computer via local files on a web
short length of the clips forces interactors into an active viewing browser. It has typically run on a Mac in Google Chrome. No
mode in which they must make frequent choices about which clip special software is required to be installed. I can set up the project
to watch next without knowing how that choice will alter their on my own machine (MacBook Pro) in the exhibition space and
movement through the material. Choosing a given thumbnail deliver a ten-minute presentation on the work. Alternately, I can
affects the next set of choices, but does not set the interactor on a provide an external hard drive with the video. An interactive setup
“path,” as there are no predetermined navigations in the work. of HandMutt is simple and flexible in terms of space; it requires
Instead, the next set of clips is determined by the keywords used only a computer, a monitor, a VGA or HDMI cord, a mouse or
to describe the one that has just played; while the number of touchpad, and headphones (if desired), and can be spatially
variations is by no means infinite, the computational operation of configured in a variety of ways based on exhibition needs. The
the Korsakow database makes it extremely difficult to replicate project is designed for interactors to traverse it in half an hour.
the same navigation twice.
1.4 Exhibition history
1.2 Artistic statement HandMutt was first exhibited in a partial screening at the
I came to the electronic arts in mid-career as an evolution of my Electronic Literature Organization’s 2015 conference in Bergen,
training and work in literature and film—particularly the Norway. It will premiere in full at the 22nd International
experimental traditions of both. My first artistic education was in Symposium on Electronic Art in Hong Kong in May 2016.
experimental cinema; I was a student of Ken Jacobs and later
worked in a department with Stan Brakhage. I was also mentored 1.5 Links
by two experimental fiction writers, Ronald Sukenick and Steve A web version is at http://ellisislandmutt.korsakow.tv (login
Katz, who were at the forefront of the metafiction community in , password ). Please note that after
the 1970s that later spawned hypertext fiction. Because of this it premieres in full at ISEA, the login will no longer be required.
twin background I have always been invested in hybrid genre A two-minute video describing the project is at
work and invented forms. It was not until this decade that I turned https://vimeo.com/134786308.
toward digital expression, to which I was drawn because it offers
vastly more powerful tools than print and film alone in terms of