Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology SOUND ART AND TECHNOLOGY: EXPLORING THE AFFECTIVE INTENSITIES OF EMBODIED AUDIBILITY Andromachi Vrakatseli1, Nikos Bubaris2 1 PhD Candidate, Department of Cultural Technology and Communication, University of the Aegean, andromachiv@hotmail.com 2 Assistant Professor, Department of Cultural Technology and Communication, University of the Aegean Abstract The aim of this paper is to investigate the affective intensities of embodied audibility in sound art by using technology. Embodied audibility forms an expanded listening practice through the body’s capacity to perceive, to be affected and to interact with audible, non - audible and non-sensuous sound waves as well as vibrational movements (i.e. affective intensities). This research extends the considerable interest in sound art theory, especially Steve Goodman’s approach on the inaudible sound and its interaction with the body. Drawing on Ihde’s post-phenomenological approach, the paper explores the ways in which technology unfolds the affective dimensions of embodied audibility in sound artworks presented in Greece, in order to demonstrate how the coupling of technology with sound art may enhance a multi-layered relation between humans and the environment. Keywords: affect, embodied audibility, postphenomenology, sound art, technology Introduction This paper draws together theories from phenomenology and the notion of affect in order to investigate the affective intensities of embodied audibility with reference to examples in which sound art encounters technology. In contemporary cultural theory there is a growing interest in the ways human bodies interact with sonic materialities. This theoretical turn marks a move away from the production of meaning which is based on symbolic and linguistic representations. Resonating the work of Pierre Schaeffer, philosopher Christopher Cox(2011) points out that sound is in constant flux and has to be experienced beyond its context. Along this line of thought, many sound artists give prominence to a more sensorial mode of listening demonstrating the multiple ways of what Bernhard Leitner (1998:175) calls “haptic acoustics”, that is, the ways with which sound can be felt with the whole body. The human body can interact with sounds that are beyond the audible range. For example, sound combined with ultrasound can affect human brain in non-conscious ways (Oohashi, 2000). English sound theorist and dj Steve Goodman (2010: xx) introduces the term “unsound” to describe the affective intensities of sounds that are inaudible or Copyright © 20218 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0) DCAC 2018. Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology not-yet audible. Unsound has two dimensions: a) it refers to sonic phenomena at the peripheries of auditory perception, such as infra- and ultra-sounds which respectively affect the body and neurological functions, and b) it includes sonic rhythms, textures and compositions that are audible but they have not been actualised yet due to cultural, technological or other reasons (for example, scratching was an unsound before its establishment as a characteristic of hip-hop technique). Goodman analyses different ways of how unsound is used in contemporary life within a context of a sonic warfare capable of modulating feeling, mood and the physical dimensions of the human body. For example, infrasound can be used as an acoustic weapon to control citizens, creating unpleasant feelings and harmful effects on the body, such as anxiety and nosebleeds. In the course of this paper, we discuss techniques of turning the two types of unsound into an audible experience with reference to the sound artworks “Hearing the magnetic storm”, “I/E Elefsis”, “Inhibition” and “Micropolitics of Noise”. Our analysis mainly draws on the postphenomenological work of Don Ihde on technological mediation and particularly on the multiple interrelations of humans, technology and the world through sound. Postphenomenology Phenomenology is concerned with the way humans experience the world particularly through perceptual bodily experiences. In classical phenomenology, technology is considered a functional tool of human experience. The post-phenomenologist thought of Don Idhe broadens this focus by investigating how technology contributes to the organisation of human practices and perceptions of the world. For Ihde, people, technology and the world are not predetermined entities but they are mutually constituted as they are related. Ihde discusses various modes of relations that highlight the role of technology as a mediator between human knowledge and subjectivity (Rosenberger & Verbeek, 2015). For example: in “embodiment relations”, humans and technologies form a unity (as in the case of microscope) which is directed at the world. In this way people perceive the world through technologies (Idhe, 1990:230). In “hermeneutic relations”, technologies form a unity with the world creating representations (as in the case of mapping). In these relations the world is transformed into code, text etc. Humans experience this transformation “via the direct experience and interpretation of technology itself” (Rosenberger & Verbeek, 2015). Idhe argues that recent technologies broaden the human sensorium as they datafy phenomena and in this way bring them into the realm of human perceivable experience. This technology is called “translational instrument” (Idhe, 2017:39). Idhe (2017:40) outlines two translational capacities of technology: “intra sensory translations (for example sound turns into image and vice versa) and translations from non perceivable dimensions into sensory ones (for example magnetic lines or x- rays are visualized).” This study highlights the need that the discipline of phenomenology of sound which investigates “existential possibilities of auditory experience” (Idhe, 2007:23) is established. Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology I/E Elefsis In the sound artwork “I/E Elefsis”, (Aeschylia festival, Elefsina, 2015) sound artist Tarek Atoui and field recordist Chris Watson focus on the materiality of sound of Elefsina City. In particular, they reached the hidden layers of the soundscape of Elefsina aiming at revealing the micro and macro vibrations of the city. In so doing, the artists used various recording devices and techniques. The sounds recorded were both audible and inaudible and they transcribe the historic identity of the place from ancient times until nowadays. For example, the artists recorded the vibrations of an ancient temple, the reverberation of a thunderstorm inside an abandoned factory. In the catalogue of the artwork, we read that the use of audio instruments such as recorders create a dialogue with the site “as sound matter” because artists explore “the acoustics, refractions and reverberations of the architecture of the space (the vibrations of the city)” (Carras & Hatzidaki, 2015). In this paper we pay attention to the recorded sounds that the human ear cannot reach due to their volume level or their source’s position. These sounds, even though there are within the limits of human audition, are masked by other sounds of the environment. The artists make these sounds hearable through technological mediation, such as contact microphones and hydrophones. The above instruments provide translated perception for people as they capture and amplify sounds that humans are not able to hear without amplification. For example, contact microphone brings into perceptual experience the sounds produced inside a column. That means that these technologies sonify the inaudible and imperceptible sounds and represent the hidden soundscape of a place. Thus, an aspect of the world that is silent due to masking phenomena, becomes audible and available to human knowledge and experience. “I/E Elefsis” acts in a way that does not only describe the unintentional relations between the human body and the site itself as a living sonic matter but also reconfigures them. Through the use of the aforementioned technology humans can learn auditorily the “unsound” and the unseen world. According to Massumi, human body accesses more of its potential (Zournazi, 2002) and expands the affective dimensions of its embodied audibility. By mediating capacities of technologies in sound artworks “our experience of listening itself is being transformed, and included in this transformation are the ideas we have about the world and ourselves” (Idhe, 2007:5). Hearing the magnetic storm “Hearing the magnetic storm” is a digital interactive environment created by Emmanouel Rovithis and Fiori Metallinou (Athens Science Festival, Athens 2016). The work translates the changes in the magnetic field of the Earth during magnetic storms into auditory sensory data. Solar storms have a dynamic impact on the Earth’s magnetic field as they cause magnetic storms. This field surrounds the Earth and protects it from cosmic and solar radiation. It has a significant role in its habitability (Magnetospheres, n.d. para.1) and also in human life. For example, it is proven that magnetic storms can deregulate driverless cars (Macdonald, 2018). Magnetic storms are not directly experiencable, but the human body can interact with them unintentionally. This virtual presence of magnetic storm becomes audible in “Hearing the magnetic storm”. The Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology aim of this project is to educate and promote awareness of the physical world and solar activity (Rovithis, Metallinou & Floros, 2016). Rovithis and Metallinou design an application in order to present and make us aware of this phenomenon by turning non-perceivable astronomic data into audible data. In doing so, they first represent sonically the magnetic field with the use of two sinusoidal sounds of almost the same frequency. Secondly, they represent the perturbations of the magnetic field by modulating the frequency and depth of the two sounds. Modulations represent the varying degrees of the phenomenon as it is being recorded with Dst index, which measures geomagnetic activity per hour. This technique of sonifying the magnetic field is called “Parameter mapping sonification” and “involves the association of information with auditory parameters for the purpose of data display” (Grond & Berger, 2011). In “Hearing the magnetic storm”, the severity of magnetic storms modulates the frequency and the timbre of sound. In addition, it changes the density and frequency of a third high tone that represents the movement of charged particles (Rovithis, Metallinou & Floros, 2016). Although the data of Dst index that feed the application are scientifically documented, the selected variables and the sounds used to signify the magnetic field, emerge out of the creators’ creative thought. Implementing techniques and principles of sound design, the two sounds that represent the two poles of the magnetic field were chosen to be 110 and 110,5 Hz because low- frequency sounds help creating a warm and immersive drone. When the creators wanted to sonify moments that the magnetic storm was becoming severe and dangerous, then the sounds were becoming more high- pitched. Inhibition Artist and researcher Marinos Koutsomichalis designed a headset that is called “Inhibition” (Onassis Cultural Centre, Athens 2016-7) which has the capacity to monitor neurophysical activity in real time and to translate this activity into sound and rhythms. The headset is equipped with sensors and uses the method of electroencephalography (EEG) to capture brainwave data. Through algorithmic sound synthesis, it generates sounds that change and “detune” the cerebral rhythms of the user and inhibit his/her focus. The hardware of the original headset records two-channel EEG activity via C++ programming language. Koutsomichalis used a Drive Right Leg (DRL) circuit which eliminates interference noise and three filters that cut specific frequencies. As a result, the hardware achieves an excellent performance at the 1-20Hz frequency range with low electric hum (Koutsomichalis, 2016 para. 4). A software driver, designed by the artist, interacts with a hardware analogue-to- digital convertor and samples the signal of the two channels in different time and sample rate. In the original headset the sample rate was 4000 (M. Koutsomichalis, personal communication, May 27, 2018). The artist used SuperCollider platform to create a system for audio synthesis. The sound sources are pairs of sine oscillators and wavetable oscillators that scan data derived from electroencephalography and reproduce them in a different frequency. Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology Frequency or amplitude modulation is generated between the oscillators. The parameters of the dominant frequency generated by the brain and spectral features recorded by an artificial intelligence module define the above system. The basic concept is that, if the dominant frequency changes often and the spectral features are almost the same, the audio parameters will be unstable otherwise they will change. There are also random parameters in order to avoid static texture (M. Koutsomichalis, personal communication, May 27, 2018). The above technology sonifies non audio and perceivable data that refer to cognitive processes. Users of “Inhibition” headset are able to perceive and listen to an interpretation of their cerebral activity according to parameters defined by the artist. Micropolitics of Noise Being inspired by Steve Goodman’s approaches about infrasound as a form of violence and technique of affective mobilization sound artist Lambros Pigounis focuses on the relationship between body and sonic materiality. In the sound performance “Micropolitics of Noise”, (Benaki Museum, Athens, 2016) Pigounis addressed sound vibration as a phenomenon of contact “at the level of the enfolding of affects into the body” (Goodman, 2010: 135) before its cognitive appropriation. Visitors can experience the conditions of a sonic war that took place in Gaza Strip. In this case, the low altitude flight of Israeli military aircrafts broke the sound barrier causing damages such as broken windows and health problems to the people. Pigounis wanted to show how these subsonic bombs are corporeally felt. In order to liken these conditions of the infrasound vibrational field the artist used hardly audible and inaudible sounds and frequencies that are within human audible range. He created a sound space of high density and rich harmonic content produced by four subwoofer speakers of 10 Hz frequency and 6 kW volume. In this case non audible sound becomes audible. Visitors could walk and lie on a white platform with a slomping ramp that covered the largest part of the gallery space. To heighten the vibrations, Pigounis used practices related to the vibration of the platform itself. Through Max / Msp language programming he produced a low noise frequency of 0-45 Hz and sinusoidal waveforms of 10-60 Hz. He found the dominant frequency of the platform and he produced sinusoidal waveforms according to this. Pigounis achieved the maximum vibration of the platform. Hence, visitors could not listen to the infrasound with their ears but feel it instead with their whole body. The immersion into that kind of high density sonic space creates a haptic sonic experience of sound and provokes resonance within the body. Sound was corporeally felt by creating effects on the body such as stomach ache. Body facilitates a transducer of vibrational affective intensities and understands its placement in relation to the unsound world. In this way translational instrumentation in sound art becomes a means of embodied audibility that modifies the acoustic sense of the body. Conclusion The above sound artworks which present sonically sounds that are inaudible in everyday life and worldly phenomena which are beyond the range of human hearing, use Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology technological mediation to organise and enhance our perception and knowledge of the world. Drawing on the post-phenomenological work of Don Ihde we could notice that the artworks relate human audition, technological functions and the world by using different techniques of instrumental translational perception. These sound artworks present phenomena that are not only “unsound”, following Goodman’s definition, but also what we could call “non-sound”, that is temporal phenomena such as waves and vibrations that can be translated into sound through the transcription of their properties (frequency, phase, rhythm, intensity, etc). In “I/E Elefsis”, humans listen through technology to the world. That is, the artists (i.e. the humans) use audio devices (i.e technology) in order to enhance aurally otherwise imperceptible events like the underground movement of the ants (i.e. the world). Similar to Marshall McLuhan’s thesis that media are extensions of the human sensorium, humans coupled with embodied audio instruments are directed at the world performing an augmented instrumentally translational perception. This human-technology relation is based aurally and metaphorically on what Ihde calls “amplification” and is isomorphic to the senses. In “Hearing the magnetic storm”, a two-level expanded hermeneutic relation of human, technology and the world unfolds. In the case of Dst index, which feeds the application with data, a technology (i.e. datafication) couples with the world (i.e. magnetic storms) providing humans with raw material for producing meaning of a cosmic phenomenon. However, meaning producing undergoes a second - order technological mediation: drawing on their knowledge on sound design, the artists (i.e. humans) recall acoustic experiences with positive or negative psychological effects (i.e. the world) in order to select artificial sounds and the technics for their manipulation (i.e. technology). In “Inhibition”, humans listen through technology to themselves. The artwork blends and re- directs the embodiment and hermeneutic relations that are at play in the two artworks we discussed above. In “Inhibition”, the coupling of the users (i.e. humans) with the headset (i.e the embodied technology) is not directed to the world but turns back to itself providing to humans with a new field of self-awareness (i.e. the sonification of their brain activity). In other words, humans experience the ways technology perceives and interprets them. In so doing, technological mediation results in a “reflexive intentionality” through which people may also immerse in technology’s intentionality (Rosenberger & Verbeek, 2015: 22). “Micropolitics of Noise” creates a hermeneutic relation. Speakers, platform and Max/Msp programming language (i.e. technology) relates to the world (i.e. sonic bombs) in order visitors (i.e. humans) to experience the unsound. This kind of experience is produced in two ways: through hardly audible low frequency sounds and through the vibration of the platform (i.e. non – sound). In the latter case, sound is not rendered, however non - sound constitutes a form of sonification because it focuses on the experience of “haptic acoustics”. Hence, in “Micropolitics of Noise” the two capacities of technology are applied, following Idhe (2017:40) a) intra sensory translations Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology because what is corporeally felt is also hearable with the ears, b) translations from non -perceivable dimensions into sensory as the inaudible becomes audible. In “I/E Elefsis” the ways in which the unsound becomes audible are analogous to the phenomenon and to the human senses. Through technology, sonic materiality enhances the phenomenon which, under any other conditions, would not be hearable. In this case, sound art deals with the phenomenon itself. In “Hearing the magnetic storm” and “Inhibition”, however, the instrumental translation of the unsound into audible experience is not isomorphic to the cosmic phenomenon and the cerebral activity as much as it is analogous to pleasant/unpleasant acoustic experiences on planet earth and artistic explorations. With the above sonification technique, sound art deals with the representation of the phenomena. “Micropolitics ofNoise” combines the above because it deals with the representation of the phenomena but it is isomorphic to the human senses as the sonic vibration is still corporeally felt. In conclusion, the sound artworks presented in this paper employ different techniques of making audible the dynamic phenomena that are beyond audible range. They develop different forms of technological mediation reconnecting people with the world by helping people understand how the acoustic experience may be related to an extended vibrational spatiality and sonic materialities in which affective intensities become augmented and conscious. References Carras, M.Th., & Hatzidaki, O. (2015). I/E Elefsis. Exhibition catalogue. Athens: Locus Athens. Cox, C. (2011). Beyond Representation and Signification: Toward a Sonic Materialism. Journal of Visual Culture, Vol 10(2), 145–161. DOI 10.1177/1470412911402880 Goodman, S. (2010). Sonic Warfare: Sound, Affect, and the Ecology of Fear. Cambridge: MIT Press. Grond, F., & Berger, J. (2011). Parameter Mapping Sonification. In T. Hermann, A. Hunt, & J.G. Neuhoff, (Eds.), The Sonification Handbook (pp. 363-397). Berlin,:Logos Verlag. Ihde, D. (1990). Technology and the Lifeworld: From Garden to Earth. Bloomington and Indianapolis: Indiana University Press Idhe, D. (2007). Listening and Voice. Phenomenologies of Sound. Albany:State University of New York Press. Ihde, D. (2017). Acoustic Technics (Postphenomenology and the Philosophy of Technology). London: Lexington Books. Koutsomichalis, M. (2016). Technical. Retrieved from http://inhibition- eeg. com/2016/10/24/technical/ Leitner, B. (1998). P.U.L.S.E.,Räume der Zeit/Spaces in time. Berlin: ZKM and Hatje Cantz. Macdonald, C. (2018, March, 16). Driverless cars could be stopped in their tracks by solar storms, Dailymail, Retrieved from https://www.dailymail. co.uk/sciencetech/article-5511491/Driverless-cars-stopped-tracks-solar- storms.html Digital Culture & Audiovisual Challenges: Interdisciplinary Creativity In Arts And Technology Magnetospheres, (n.d). Retrieved from https://science.nasa.gov/heliophysics/focus- areas/magnetosphere-ionosphere Oohashi, T. et al., (2000). Inaudible High-Frequency Sounds Affect Brain Activity: Hypersonic Effect. Journal of Neurophysiology. 83: 6, 3548-3558. Rosenberger, R. & Verbeek, P-P. (2015). A postphenomenological field guide. In R. Rosenberger, & P-P Verbeek (Eds.), Postphenomenological Investigations: Essayson Human– Technology Relations (pp. 9-41). London: Lexington Books. Rovithis, E., Metallinou, F., & Floros, A. (2016). Hearing the Magnetic Strorm an educational interactive audio environment. Retrieved from https://www. researchgate.net/publication/324808539_Akougontas_te_Magnetike_ Kataigida_ena_ekpaideutiko_diadrastiko_echetiko_periballon_Hearing_the_ Magnetic_Strorm_an_edu cational_interactive_audio_environment Zournazi, M. (2002). Navigating Movements. Interview with Brian Massumi. Retrieved from https://archive.org/stream/InterviewWithBrianMassumi/intmassumi_ djvu.txt