=Paper=
{{Paper
|id=Vol-2811/Paper11
|storemode=property
|title=Sound Art and Technology: Exploring the Affective Intensities of Embodied Audibility
|pdfUrl=https://ceur-ws.org/Vol-2811/Paper11.pdf
|volume=Vol-2811
|authors=Andromachi Vrakatseli,Nikos Bubaris
}}
==Sound Art and Technology: Exploring the Affective Intensities of Embodied Audibility==
https://ceur-ws.org/Vol-2811/Paper11.pdf
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
SOUND ART AND TECHNOLOGY: EXPLORING THE
AFFECTIVE INTENSITIES OF EMBODIED AUDIBILITY
Andromachi Vrakatseli1, Nikos Bubaris2
1 PhD Candidate, Department of Cultural Technology and Communication,
University of the Aegean, andromachiv@hotmail.com
2 Assistant Professor, Department of Cultural Technology and Communication,
University of the Aegean
Abstract
The aim of this paper is to investigate the affective intensities of embodied audibility in
sound art by using technology. Embodied audibility forms an expanded listening
practice through the body’s capacity to perceive, to be affected and to interact with
audible, non - audible and non-sensuous sound waves as well as vibrational movements
(i.e. affective intensities). This research extends the considerable interest in sound art
theory, especially Steve Goodman’s approach on the inaudible sound and its interaction
with the body. Drawing on Ihde’s post-phenomenological approach, the paper explores
the ways in which technology unfolds the affective dimensions of embodied audibility
in sound artworks presented in Greece, in order to demonstrate how the coupling of
technology with sound art may enhance a multi-layered relation between humans and
the environment.
Keywords: affect, embodied audibility, postphenomenology, sound art, technology
Introduction
This paper draws together theories from phenomenology and the notion of affect in
order to investigate the affective intensities of embodied audibility with reference to
examples in which sound art encounters technology. In contemporary cultural theory
there is a growing interest in the ways human bodies interact with sonic materialities.
This theoretical turn marks a move away from the production of meaning which is based
on symbolic and linguistic representations. Resonating the work of Pierre Schaeffer,
philosopher Christopher Cox(2011) points out that sound is in constant flux and has to
be experienced beyond its context. Along this line of thought, many sound artists give
prominence to a more sensorial mode of listening demonstrating the multiple ways of
what Bernhard Leitner (1998:175) calls “haptic acoustics”, that is, the ways with which
sound can be felt with the whole body.
The human body can interact with sounds that are beyond the audible range. For
example, sound combined with ultrasound can affect human brain in non-conscious ways
(Oohashi, 2000). English sound theorist and dj Steve Goodman (2010: xx) introduces
the term “unsound” to describe the affective intensities of sounds that are inaudible or
Copyright © 20218 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0) DCAC 2018.
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
not-yet audible. Unsound has two dimensions: a) it refers to sonic phenomena at the
peripheries of auditory perception, such as infra- and ultra-sounds which respectively
affect the body and neurological functions, and b) it includes sonic rhythms, textures
and compositions that are audible but they have not been actualised yet due to cultural,
technological or other reasons (for example, scratching was an unsound before its
establishment as a characteristic of hip-hop technique).
Goodman analyses different ways of how unsound is used in contemporary life
within a context of a sonic warfare capable of modulating feeling, mood and the physical
dimensions of the human body. For example, infrasound can be used as an acoustic
weapon to control citizens, creating unpleasant feelings and harmful effects on the body,
such as anxiety and nosebleeds. In the course of this paper, we discuss techniques of
turning the two types of unsound into an audible experience with reference to the sound
artworks “Hearing the magnetic storm”, “I/E Elefsis”, “Inhibition” and “Micropolitics
of Noise”. Our analysis mainly draws on the postphenomenological work of Don Ihde
on technological mediation and particularly on the multiple interrelations of humans,
technology and the world through sound.
Postphenomenology
Phenomenology is concerned with the way humans experience the world particularly
through perceptual bodily experiences. In classical phenomenology, technology is
considered a functional tool of human experience. The post-phenomenologist thought
of Don Idhe broadens this focus by investigating how technology contributes to the
organisation of human practices and perceptions of the world. For Ihde, people,
technology and the world are not predetermined entities but they are mutually constituted
as they are related. Ihde discusses various modes of relations that highlight the role of
technology as a mediator between human knowledge and subjectivity (Rosenberger &
Verbeek, 2015). For example: in “embodiment relations”, humans and technologies
form a unity (as in the case of microscope) which is directed at the world. In this way
people perceive the world through technologies (Idhe, 1990:230). In “hermeneutic
relations”, technologies form a unity with the world creating representations (as in the
case of mapping). In these relations the world is transformed into code, text etc. Humans
experience this transformation “via the direct experience and interpretation of
technology itself” (Rosenberger & Verbeek, 2015).
Idhe argues that recent technologies broaden the human sensorium as they datafy
phenomena and in this way bring them into the realm of human perceivable experience.
This technology is called “translational instrument” (Idhe, 2017:39). Idhe (2017:40)
outlines two translational capacities of technology: “intra sensory translations (for
example sound turns into image and vice versa) and translations from non perceivable
dimensions into sensory ones (for example magnetic lines or x- rays are visualized).”
This study highlights the need that the discipline of phenomenology of sound
which investigates “existential possibilities of auditory experience” (Idhe, 2007:23) is
established.
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
I/E Elefsis
In the sound artwork “I/E Elefsis”, (Aeschylia festival, Elefsina, 2015) sound artist Tarek
Atoui and field recordist Chris Watson focus on the materiality of sound of Elefsina
City. In particular, they reached the hidden layers of the soundscape of Elefsina aiming at
revealing the micro and macro vibrations of the city. In so doing, the artists used various
recording devices and techniques. The sounds recorded were both audible and inaudible
and they transcribe the historic identity of the place from ancient times until nowadays.
For example, the artists recorded the vibrations of an ancient temple, the reverberation
of a thunderstorm inside an abandoned factory. In the catalogue of the artwork, we read
that the use of audio instruments such as recorders create a dialogue with the site “as
sound matter” because artists explore “the acoustics, refractions and reverberations of
the architecture of the space (the vibrations of the city)” (Carras & Hatzidaki, 2015).
In this paper we pay attention to the recorded sounds that the human ear cannot
reach due to their volume level or their source’s position. These sounds, even though
there are within the limits of human audition, are masked by other sounds of the
environment. The artists make these sounds hearable through technological mediation,
such as contact microphones and hydrophones. The above instruments provide translated
perception for people as they capture and amplify sounds that humans are not able to
hear without amplification. For example, contact microphone brings into perceptual
experience the sounds produced inside a column. That means that these technologies
sonify the inaudible and imperceptible sounds and represent the hidden soundscape of a
place. Thus, an aspect of the world that is silent due to masking phenomena, becomes
audible and available to human knowledge and experience.
“I/E Elefsis” acts in a way that does not only describe the unintentional relations
between the human body and the site itself as a living sonic matter but also reconfigures
them. Through the use of the aforementioned technology humans can learn auditorily the
“unsound” and the unseen world. According to Massumi, human body accesses more of
its potential (Zournazi, 2002) and expands the affective dimensions of its embodied
audibility. By mediating capacities of technologies in sound artworks “our experience
of listening itself is being transformed, and included in this transformation are the ideas
we have about the world and ourselves” (Idhe, 2007:5).
Hearing the magnetic storm
“Hearing the magnetic storm” is a digital interactive environment created by
Emmanouel Rovithis and Fiori Metallinou (Athens Science Festival, Athens 2016). The
work translates the changes in the magnetic field of the Earth during magnetic storms
into auditory sensory data. Solar storms have a dynamic impact on the Earth’s magnetic
field as they cause magnetic storms. This field surrounds the Earth and protects it from
cosmic and solar radiation. It has a significant role in its habitability (Magnetospheres,
n.d. para.1) and also in human life. For example, it is proven that magnetic storms can
deregulate driverless cars (Macdonald, 2018). Magnetic storms are not directly
experiencable, but the human body can interact with them unintentionally. This virtual
presence of magnetic storm becomes audible in “Hearing the magnetic storm”. The
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
aim of this project is to educate and promote awareness of the physical world and solar
activity (Rovithis, Metallinou & Floros, 2016).
Rovithis and Metallinou design an application in order to present and make us aware
of this phenomenon by turning non-perceivable astronomic data into audible data. In
doing so, they first represent sonically the magnetic field with the use of two sinusoidal
sounds of almost the same frequency. Secondly, they represent the perturbations of the
magnetic field by modulating the frequency and depth of the two sounds. Modulations
represent the varying degrees of the phenomenon as it is being recorded with Dst index,
which measures geomagnetic activity per hour. This technique of sonifying the magnetic
field is called “Parameter mapping sonification” and “involves the association of
information with auditory parameters for the purpose of data display” (Grond & Berger,
2011). In “Hearing the magnetic storm”, the severity of magnetic storms modulates the
frequency and the timbre of sound. In addition, it changes the density and frequency of
a third high tone that represents the movement of charged particles (Rovithis, Metallinou
& Floros, 2016).
Although the data of Dst index that feed the application are scientifically
documented, the selected variables and the sounds used to signify the magnetic field,
emerge out of the creators’ creative thought. Implementing techniques and principles of
sound design, the two sounds that represent the two poles of the magnetic field were
chosen to be 110 and 110,5 Hz because low- frequency sounds help creating a warm
and immersive drone. When the creators wanted to sonify moments that the magnetic
storm was becoming severe and dangerous, then the sounds were becoming more high-
pitched.
Inhibition
Artist and researcher Marinos Koutsomichalis designed a headset that is called
“Inhibition” (Onassis Cultural Centre, Athens 2016-7) which has the capacity to monitor
neurophysical activity in real time and to translate this activity into sound and rhythms.
The headset is equipped with sensors and uses the method of electroencephalography
(EEG) to capture brainwave data. Through algorithmic sound synthesis, it generates
sounds that change and “detune” the cerebral rhythms of the user and inhibit his/her
focus.
The hardware of the original headset records two-channel EEG activity via C++
programming language. Koutsomichalis used a Drive Right Leg (DRL) circuit which
eliminates interference noise and three filters that cut specific frequencies. As a result,
the hardware achieves an excellent performance at the 1-20Hz frequency range with low
electric hum (Koutsomichalis, 2016 para. 4). A software driver, designed by the artist,
interacts with a hardware analogue-to- digital convertor and samples the signal of the two
channels in different time and sample rate. In the original headset the sample rate was
4000 (M. Koutsomichalis, personal communication, May 27, 2018).
The artist used SuperCollider platform to create a system for audio synthesis. The
sound sources are pairs of sine oscillators and wavetable oscillators that scan data
derived from electroencephalography and reproduce them in a different frequency.
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
Frequency or amplitude modulation is generated between the oscillators. The parameters
of the dominant frequency generated by the brain and spectral features recorded by an
artificial intelligence module define the above system. The basic concept is that, if the
dominant frequency changes often and the spectral features are almost the same, the
audio parameters will be unstable otherwise they will change. There are also random
parameters in order to avoid static texture (M. Koutsomichalis, personal communication,
May 27, 2018).
The above technology sonifies non audio and perceivable data that refer to cognitive
processes. Users of “Inhibition” headset are able to perceive and listen to an interpretation
of their cerebral activity according to parameters defined by the artist.
Micropolitics of Noise
Being inspired by Steve Goodman’s approaches about infrasound as a form of violence
and technique of affective mobilization sound artist Lambros Pigounis focuses on the
relationship between body and sonic materiality. In the sound performance “Micropolitics
of Noise”, (Benaki Museum, Athens, 2016) Pigounis addressed sound vibration as a
phenomenon of contact “at the level of the enfolding of affects into the body” (Goodman,
2010: 135) before its cognitive appropriation. Visitors can experience the conditions of
a sonic war that took place in Gaza Strip. In this case, the low altitude flight of Israeli
military aircrafts broke the sound barrier causing damages such as broken windows and
health problems to the people. Pigounis wanted to show how these subsonic bombs are
corporeally felt.
In order to liken these conditions of the infrasound vibrational field the artist used
hardly audible and inaudible sounds and frequencies that are within human audible
range. He created a sound space of high density and rich harmonic content produced by
four subwoofer speakers of 10 Hz frequency and 6 kW volume. In this case non audible
sound becomes audible. Visitors could walk and lie on a white platform with a slomping
ramp that covered the largest part of the gallery space. To heighten the vibrations,
Pigounis used practices related to the vibration of the platform itself. Through Max
/ Msp language programming he produced a low noise frequency of 0-45 Hz and
sinusoidal waveforms of 10-60 Hz. He found the dominant frequency of the platform and
he produced sinusoidal waveforms according to this. Pigounis achieved the maximum
vibration of the platform. Hence, visitors could not listen to the infrasound with their ears
but feel it instead with their whole body.
The immersion into that kind of high density sonic space creates a haptic sonic
experience of sound and provokes resonance within the body. Sound was corporeally
felt by creating effects on the body such as stomach ache. Body facilitates a transducer
of vibrational affective intensities and understands its placement in relation to the
unsound world. In this way translational instrumentation in sound art becomes a means
of embodied audibility that modifies the acoustic sense of the body.
Conclusion
The above sound artworks which present sonically sounds that are inaudible in everyday
life and worldly phenomena which are beyond the range of human hearing, use
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
technological mediation to organise and enhance our perception and knowledge of the
world. Drawing on the post-phenomenological work of Don Ihde we could notice that
the artworks relate human audition, technological functions and the world by using
different techniques of instrumental translational perception. These sound artworks
present phenomena that are not only “unsound”, following Goodman’s definition, but
also what we could call “non-sound”, that is temporal phenomena such as waves and
vibrations that can be translated into sound through the transcription of their properties
(frequency, phase, rhythm, intensity, etc).
In “I/E Elefsis”, humans listen through technology to the world. That is, the artists
(i.e. the humans) use audio devices (i.e technology) in order to enhance aurally otherwise
imperceptible events like the underground movement of the ants (i.e. the world). Similar
to Marshall McLuhan’s thesis that media are extensions of the human sensorium,
humans coupled with embodied audio instruments are directed at the world performing
an augmented instrumentally translational perception. This human-technology relation
is based aurally and metaphorically on what Ihde calls “amplification” and is isomorphic
to the senses.
In “Hearing the magnetic storm”, a two-level expanded hermeneutic relation
of human, technology and the world unfolds. In the case of Dst index, which feeds
the application with data, a technology (i.e. datafication) couples with the world (i.e.
magnetic storms) providing humans with raw material for producing meaning of a cosmic
phenomenon. However, meaning producing undergoes a second - order technological
mediation: drawing on their knowledge on sound design, the artists (i.e. humans) recall
acoustic experiences with positive or negative psychological effects (i.e. the world) in
order to select artificial sounds and the technics for their manipulation (i.e. technology).
In “Inhibition”, humans listen through technology to themselves. The artwork blends
and re- directs the embodiment and hermeneutic relations that are at play in the
two artworks we discussed above. In “Inhibition”, the coupling of the users (i.e.
humans) with the headset (i.e the embodied technology) is not directed to the world
but turns back to itself providing to humans with a new field of self-awareness (i.e.
the sonification of their brain activity). In other words, humans experience the ways
technology perceives and interprets them. In so doing, technological mediation results
in a “reflexive intentionality” through which people may also immerse in technology’s
intentionality (Rosenberger & Verbeek, 2015: 22).
“Micropolitics of Noise” creates a hermeneutic relation. Speakers, platform and
Max/Msp programming language (i.e. technology) relates to the world (i.e. sonic bombs)
in order visitors (i.e. humans) to experience the unsound. This kind of experience is
produced in two ways: through hardly audible low frequency sounds and through the
vibration of the platform (i.e. non – sound). In the latter case, sound is not rendered,
however non - sound constitutes a form of sonification because it focuses on the
experience of “haptic acoustics”. Hence, in “Micropolitics of Noise” the two capacities
of technology are applied, following Idhe (2017:40) a) intra sensory translations
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
because what is corporeally felt is also hearable with the ears, b) translations from non
-perceivable dimensions into sensory as the inaudible becomes audible.
In “I/E Elefsis” the ways in which the unsound becomes audible are analogous to
the phenomenon and to the human senses. Through technology, sonic materiality
enhances the phenomenon which, under any other conditions, would not be hearable. In
this case, sound art deals with the phenomenon itself. In “Hearing the magnetic storm”
and “Inhibition”, however, the instrumental translation of the unsound into audible
experience is not isomorphic to the cosmic phenomenon and the cerebral activity as
much as it is analogous to pleasant/unpleasant acoustic experiences on planet earth and
artistic explorations. With the above sonification technique, sound art deals with the
representation of the phenomena. “Micropolitics ofNoise” combines the above because
it deals with the representation of the phenomena but it is isomorphic to the human
senses as the sonic vibration is still corporeally felt.
In conclusion, the sound artworks presented in this paper employ different
techniques of making audible the dynamic phenomena that are beyond audible range.
They develop different forms of technological mediation reconnecting people with the
world by helping people understand how the acoustic experience may be related to an
extended vibrational spatiality and sonic materialities in which affective intensities
become augmented and conscious.
References
Carras, M.Th., & Hatzidaki, O. (2015). I/E Elefsis. Exhibition catalogue. Athens: Locus
Athens.
Cox, C. (2011). Beyond Representation and Signification: Toward a Sonic Materialism.
Journal of Visual Culture, Vol 10(2), 145–161. DOI 10.1177/1470412911402880
Goodman, S. (2010). Sonic Warfare: Sound, Affect, and the Ecology of Fear. Cambridge:
MIT Press.
Grond, F., & Berger, J. (2011). Parameter Mapping Sonification. In T. Hermann, A.
Hunt, & J.G. Neuhoff, (Eds.), The Sonification Handbook (pp. 363-397).
Berlin,:Logos Verlag.
Ihde, D. (1990). Technology and the Lifeworld: From Garden to Earth. Bloomington
and Indianapolis: Indiana University Press
Idhe, D. (2007). Listening and Voice. Phenomenologies of Sound. Albany:State
University of New York Press.
Ihde, D. (2017). Acoustic Technics (Postphenomenology and the Philosophy of
Technology). London: Lexington Books.
Koutsomichalis, M. (2016). Technical. Retrieved from http://inhibition- eeg.
com/2016/10/24/technical/
Leitner, B. (1998). P.U.L.S.E.,Räume der Zeit/Spaces in time. Berlin: ZKM and Hatje
Cantz. Macdonald, C. (2018, March, 16). Driverless cars could be stopped in
their tracks by solar storms, Dailymail, Retrieved from https://www.dailymail.
co.uk/sciencetech/article-5511491/Driverless-cars-stopped-tracks-solar-
storms.html
Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology
Magnetospheres, (n.d). Retrieved from https://science.nasa.gov/heliophysics/focus-
areas/magnetosphere-ionosphere
Oohashi, T. et al., (2000). Inaudible High-Frequency Sounds Affect Brain Activity:
Hypersonic Effect. Journal of Neurophysiology. 83: 6, 3548-3558.
Rosenberger, R. & Verbeek, P-P. (2015). A postphenomenological field guide. In R.
Rosenberger, & P-P Verbeek (Eds.), Postphenomenological Investigations: Essayson
Human– Technology Relations (pp. 9-41). London: Lexington Books.
Rovithis, E., Metallinou, F., & Floros, A. (2016). Hearing the Magnetic Strorm an
educational interactive audio environment. Retrieved from https://www.
researchgate.net/publication/324808539_Akougontas_te_Magnetike_
Kataigida_ena_ekpaideutiko_diadrastiko_echetiko_periballon_Hearing_the_
Magnetic_Strorm_an_edu cational_interactive_audio_environment
Zournazi, M. (2002). Navigating Movements. Interview with Brian Massumi. Retrieved
from https://archive.org/stream/InterviewWithBrianMassumi/intmassumi_
djvu.txt