=Paper=
{{Paper
|id=Vol-2299/paper1
|storemode=property
|title=Exploration of Auditory Augmentation in an Interdisciplinary Prototyping Workshop
|pdfUrl=https://ceur-ws.org/Vol-2299/paper1.pdf
|volume=Vol-2299
|authors=Katharina Groß-Vogt,Marian Weger,Robert Höldrich
|dblpUrl=https://dblp.org/rec/conf/fmt/Gross-VogtWH18
}}
==Exploration of Auditory Augmentation in an Interdisciplinary Prototyping Workshop==
https://ceur-ws.org/Vol-2299/paper1.pdf
Exploration of Auditory Augmentation in an
Interdisciplinary Prototyping Workshop
Katharina Groß-Vogt, Marian Weger, Robert Höldrich
Institute for Electronic Music and Acoustics (IEM)
University of Music and Performing Arts
Graz, Austria
Email: vogt@iem.at, weger@iem.at, hoeldrich@iem.at
Abstract—Auditory augmentation has been proposed as a II. W ORKSHOP LAYOUT
specific, ambient sonification method. This paper describes an
interdisciplinary workshop exploring this method by designing
The workshops of the Science by Ear series are structured
prototypes. Three of the prototypes are presented and discussed. as follows. About 20 people with different backgrounds are in-
Concluding on the workshop’s results, the authors suggest a vited to participate: sound and sonification experts, researchers
broader definition and deduce limitations of auditory augmenta- of a certain domain science, and composers or sound artists.
tion. During the workshop, they take the roles of programmers,
sound experts, data experts, and others (e.g., moderators of
I. I NTRODUCTION the teams). One workshop takes place on three or four days.
After some introduction, participants are split into three to
Sonification, in the authors’ definition, is the translation of four teams of about five people. Each team is working on one
information for auditory perception, the acoustic equivalent to sonification task with a given data set for three hours. Team
data visualization [1]. Our institute pursues the Science by members always include a moderator who also takes notes, and
Ear (SBE) workshop series on sonification. Since 2005, four one or two dedicated programmers who implement the ideas
of these workshops took place. They explored sonification in of the team. The prototyping sessions combine brainstorming,
an interdisciplinary, small group of attendees, and each had data listening, verbal sketching, concept development, and
a different focus: The first workshop (SBE1) explored data programming. After each session, the teams gather in plenum
sets in a variety of disciplines (from sociology to physics).1 to show and discuss their prototypic sonifications. Besides
The workshop set-up has proven convincing, even if the the hands-on character of the workshops, there is a certain
large variety of disciplines and the scientific dialects of their challenge between teams to produce interesting results within
communities were demanding for the attendees. SBE2 focused three hours. Data sets, tasks, and software are prepared by the
on physics’ data2 and SBE3 on climate data3 . organizers in order to ensure that technical obstacles can be
One of the lessons learned from hosting this series is to overcome within the limited time.
carefully balance the interdisciplinary, creative setting with Within SBE4, the fourth edition of the workshop series,
well-prepared tasks. If this is achieved, the setting provides the method of auditory augmentation has been explored. This
a fruitful development of prototypes, as shown in this paper. implied another level of complexity, as not only the data
The layout of the workshop is discussed in Sec. II. In the and the software needed to be prepared by the organizers
fourth edition (SBE4), the focus was less on a specific data and understood by the participants, but also possibilities and
domain but instead on exploration of a specific sonification restrictions of the provided hardware systems had to be
method: Auditory augmentation has been proposed by Bover- communicated.
mann et al. [2] as altering the characteristics of structure- Including the authors, 19 participants took part. Eleven
borne sounds for the purpose of supporting tools for data of these can be counted to the sonification community (but
representation. Besides this term, many notions and systems with varying backgrounds in sciences, arts, and humanities),
following a similar idea have been published, as discussed in while the rest included two media and interaction experts,
Sec. III. As exemplary data set used in this exploration, we two composers, two sound engineers, one musicologist, and
chose data of in-home electric power consumption. Section IV one sociologist. Participants divided in seven at pre-doc and
introduces our working definition of auditory augmentation twelve at post-doc level or above; in three female and 16 male
and three of the prototypes developed on three hardware participants. Not all of them took part throughout the whole
platforms. Finally, in Sec. V, we discuss some answers to our workshop, leading to varying group sizes of three to six for
research questions, and conclude in Sec. VI. the prototyping sessions.
1 SBE1: http://sonenvir.at/workshop/
III. R ELATED WORK ON AUDITORY AUGMENTATION
2 SBE2: http://qcd-audio.at/sbe2.html The concept of auditory augmentation has been proposed by
3 SBE3: https://sysson.kug.ac.at/index.php?id=16451 Bovermann et al. [2] as “building blocks supporting the design
10
Exploration of Auditory Augmentation in an Interdisciplinary Prototyping Workshop
of data representation tools, which unobtrusively alter the au-
ditory characteristics of structure-borne sounds.” One of these
authors’ examples eliciting the concept is ‘WetterReim’ [3].
An ordinary laptop keyboard is equipped with a contact INTERACTION SOUND
microphone, recording the typing sounds. The microphone
signal is filtered with varying parameters that depend on the OBJECT
Physical
Environment
weather condition. The output is played back in real time - Table
- Room
and fuses with the original sound to a new auditory gestalt. - Object X with BRIX
In short: depending on the weather outside, typing on the
keyboard sounds different.
The concept of auditory augmentation has been discussed
to be extended to the more general blended sonification [4] Auditory Augmentation
which “describes the process of manipulating physical inter-
action sounds or environmental sounds in such a way that
the resulting sound carries additional information of interest Live Recorded
Virtual Environment
Data of Electrical Power Sonification
while the formed auditory gestalt is still perceived as coherent Consumption (/Mapping)
auditory event.” Blended sonifications “blend into the user’s Recorded Live
Sound Input
environment without confronting users with any explicitly
Interaction data
perceived technology”. In consequence, they provide an am- (Sensors)
bient sonification channel. Blended sonification is similar to
or even encompassing auditory augmentation but takes into
account environmental sounds, in addition to structure-borne
sounds. A different generalization of auditory augmentation
is given by Weger et al. [5] who “define augmented auditory
Fig. 1. Sketch of our working definition of auditory augmentation. Three
feedback as the artificially modified sonic reaction to physical mandatory elements are shaded in gray: an object delivering either sound
interaction”. Augmented auditory feedback can become an input and/or interaction input, data, and sonification of these data which is
auditory augmentation if it conveys additional information. auditorily augmenting the object in a closed feedback loop.
For the context of SBE4 we decided to stick to the original
term auditory augmentation with a new working definition that
used to convey three levels of intensity of the notification.
incorporates the prepared platforms and tasks (see Sec. IV-A).
High intensity is mapped to a dry, loud sound, while low
Looking at a broader context, auditory augmentation is part
intensity is saturated by reverb, giving the impression of a
of Sonic Interaction Design (SID) which has been defined by
far sound. Finally, a similar project has been realized by
various authors with different focuses: Rocchesso et al. [6]
Brazil and Fernström [10] who explored a basic system for
defined that it “explores ways in which sound can be used to
an ambient auditory display of a work group. The presence of
convey information, meaning, aesthetic and emotional qualities
colleagues is sonified as a soundscape of individual sounds,
in interactive contexts.” Franinović and Serafin [7] set the
each time a person enters or leaves the workplace. The
focus more on phenomenology and perception: “Sonic interac-
proposed system utilizes auditory icons [11, p. 325-338] for
tion design [...] [considers] sound as an active medium that can
creating a soundscape and is not based on auditory feedback
enable novel phenomenological and social experiences with
as was the case in WISP.
and through interactive technology.” Auditory augmentation is
certainly part of sonic interaction design, and it is within the IV. S CIENCE BY E AR 4
scope of this paper to elicit the specificities about it. A. Working definition of auditory augmentation
We found a few more SID systems in the literature that are
SBE4 provided a set-up for exploring auditory augmentation
closely related to auditory augmentation, especially regarding
and defined that
its focus being an ambient display. For instance, Ferguson [8]
developed a series of prototypes which are similar to the ones Auditory augmentation is the augmentation of a
that emerged from our workshop. One example is a wardrobe physical object and/or its sound by sound which
door that plays back a sonification of the daily weather forecast conveys additional information.
when opened; Ferguson uses the term ambient sonification As sketched in Fig. 1, three elements are needed for auditory
systems. augmentation:
Kilander and Lönnqvist developed Weakly Intrusive Ambient 1) Physical objects in a physical environment. In our work-
Soundscapes for Intuitive State Perception (WISP) [9] in order shop setting, these were a table, a room, or any sensor-
to “convey an intuitive sense of any graspable process” rather equipped physical object. These objects may produce
than a direct information display. In a ubiquitous service a sound, or not; users might interact physically with
environment, individual notifications are presented with a them, or not. In some cases, the sound is a result of
sound associated to the user. Playback volume and reverb are the interaction, but does not have to be. Either of these
11
Exploration of Auditory Augmentation in an Interdisciplinary Prototyping Workshop
inputs has to be there: real-time sound input or real-time 1) REAL-TIME: The REAL-TIME data set was provided
data input from the interaction. as a real-time power measurement of five appliances at our
2) Data that are sonified. These can be real-time data or institute’s kitchen (see [15] for how this system has been
recorded data; in the setting of the workshop, we chose used before). Alternatively, the teams could attach the mea-
data sets of electric power consumption. The sonification surement plugs to any other appliances during the workshop.
may use ‘natural’ sound input (real-time sound or field The sampling interval was about one second; data was sent
recordings), or may be based on sound synthesis. Further over OSC, with a measurement range between zero and
input for parametrizing the sonification can stem from 3 000 Watt. Measured kitchen appliances were dish washer,
interaction data. coffee machine, water kettle, microwave, and fridge.
3) The sonification is played back in the physical envi- 2) PLUGWISE: The PLUGWISE data set stems from a
ronment, auditorily augmenting the physical object we private household where nine appliances’ loads have been
started from. measured for one year with a sampling interval of 10 minutes.
In short, for auditory augmentation we need an object which Measured appliances comprise: kitchen water kettle, ceiling
produces sound or is being interacted with, data, and their light and media station in the living room, fridge, toaster,
sonification. dehumidifier, dishwasher, washing machine, and TV.
B. Interaction platforms 3) IRISH: The IRISH data set stems from a large survey
of smart meter data in Ireland, collecting data of 12 000 Irish
The various possibilities of auditory augmentation have
households over 1.5 years with a sampling interval of 30 min-
been explored on three different platforms during the work-
utes [16]. We extracted 54 households for each combination of
shop.
three family structures (single, couple, family), two education
1) ROOM: The ROOM is situated in the institute’s main
levels, and two housing types (apartment vs. detached house).
performance and lecture hall, equipped with a 24-channel
loudspeaker array on a half-sphere for ambisonic sound D. Resulting prototypes
spatialization [12]. Furthermore, there are five microphones
During the three days of the workshop, four parallel proto-
mounted permanently to allow for a virtual room acoustics. For
typing sessions took place; one for each of the three data sets
SBE4 we prepared to work with both live sound input from the
and an open session on the last day where chosen prototypes
microphones of the virtual acoustics system and additionally
were refined. An overview of all the resulting prototypes is
added ambient sounds.
2) TABLE: The TABLE is an experimentation platform shown on the SBE4 website4 . In this paper, we only focus on
developed within an ongoing PhD project (see [5]). Techni- three exemplary cases: sonic floor plan, writing resonances,
cally, it incorporates a wooden board or table (depending on and standby door, realized with the three different platforms
its orientation in space) equipped with hidden contact micro- respectively (see Fig. 2).
phones and exciters or additional loudspeakers; a marker-based 1) Sonic floor plan (ROOM): The real-time data set used in
optical tracking system locates the position of any object or this scenario provided data of electric power consumption of
hand interacting with the surface. Any sound produced on the different devices in one specific household. The team devel-
TABLE is recorded and can be augmented in real time through oped a scenario with an assumed floor plan of the household
a filter-based modal synthesis approach. The prepared setting (see Fig. 2a). Feedback on energy consumption is played back
for the workshop allows to change the perceived materiality in one room (e.g., a media room) on a surround sound system.
in a plausible way while, e.g., knocking or writing on it. A sound occurs periodically after a specified time, as well as
3) BRIX: Our co-organizers (see Acknowledgment) pro- when a person enters the room. The appliance that currently
vided their BRIX system [13], [14]. This system has been de- consumes the most energy defines the direction for sound
veloped to allow for simple prototyping of interactive objects. spatialization. Environmental sounds from outside the building
In the prototyping sessions with the BRIX, the team could are captured by a microphone. These are played back in the
choose an interaction scenario with any object, equipping it room with loudness and position depending on the level of
with BRIX sensors and/or with microphones. Sensors that energy consumption. As only the power consumption of the
have been prepared for the workshop include accelerometer appliance with the highest load is sonified, even small standby
and gyroscope, as well as light, proximity, humidity, and consumption may attract attention, e.g., when no major energy
temperature sensors. consumer is active.
2) Writing resonances (TABLE): Writing resonances is the
C. Data sets
most ‘classical’ example of auditory augmentation during
Next to the three hardware platforms described above, the workshop, because it is based on structure-borne sounds
we prepared three data sets of electric power consumption. and therefore clearly fulfills the initial definition of Bover-
The data sets are of different nature (real-time vs. recorded mann et al. The scenario is to provide feedback of in-home
data) and therefore propose different tasks, i.e., real-time power consumption through an auditorily augmented writing
monitoring as opposed to data exploration. The teams had to desk (see Fig. 2b), based on the system presented in [5]. The
develop scenarios that support the saving of electric energy
consumption. 4 SBE4: https://iem.at/sbe4
12
Exploration of Auditory Augmentation in an Interdisciplinary Prototyping Workshop
table is augmented through additional resonances, based on a
physical model. The size of the modeled plate is controlled in
real time by the total amount of electric power consumption,
employing the metaphor of a larger table (i.e., deeper sound)
when more power is consumed. With the augmentation being
only a modulation of the existing interaction sounds, the soni-
fication only gets audible through interaction with the table;
the feedback is therefore calm and unobtrusive. The primary
task is writing, but also an active request of information by
knocking is possible. This prototype has been extended in the
fourth, open session, allowing for a modulation of individual
partials in order to additionally convey information concerning
the different appliances’ individual power consumption.
3) Standby door (BRIX): This scenario augmented our
institute’s inside entry door (see Fig. 2c for the prototypic (a) Sonic floor plan.
mockup). Most potential for saving energy lies in the reduction
of standby consumption. Therefore, when the door is opened,
e.g., in the morning, the standby consumption over the recent
past is communicated through a simple parameter mapping
sonification. The playback speed depends on the assumed
stress level of the user, derived from the opening speed of
the door. In the open prototyping session, this approach has
been extended to be able to sonify individual appliances. For
intuitive discrimination between them, the sonification is based
on synthesized speech. When opening the door, an emotionless
computer-voice says ‘coffee machine’, ‘microwave’, and the
like, with timbre, loudness, or other parameters controlled by
how much electricity this specific appliance has used over
night.
V. F EEDBACK AND A NALYSIS (b) Writing resonances.
We explored four main research questions by applying
various analysis methods on the results of the workshop.
Plenary sessions with discussions of the presented prototypes
have been recorded and partially excerpted. Additional inputs
are written notes, code, and demo videos resulting from the
prototyping sessions. All these inputs led to fundamental
considerations on auditory augmentation and how it can be
used, but also to general feedback concerning sonification,
workshop setting, or design issues.
A. Peculiarity of sound in augmented reality
Augmented reality usually refers to a mainly visual system.
But if this concept is transferred to audio, why then is listening
to the radio not ‘auditorily augmented reality’? Or is it? The
underlying question is: what are the peculiarities of sound in (c) Standby door.
the context of augmented reality? Fig. 2. Three of the prototypes that have been developed during the workshop.
Concerning the radio, the answer is relatively clear. In the
visual domain, augmented reality usually does not include an
overlay of a video on top of the visually perceived scene if A more profound analysis of this question is without the
there is no direct connection between them [17]. Applying scope of this paper. Still, a few things came up during the
this argumentation to the auditory domain, the radio (being an discussions in the workshop. On the one hand, sonification
overlay to the acoustic environment) does not directly interact is challenging because visual and auditory memory work
with the physical environment of the user and therefore can differently, and therefore two sonifications are more difficult
not be regarded as augmented reality. to compare than two static visualizations. These and other
13
Exploration of Auditory Augmentation in an Interdisciplinary Prototyping Workshop
challenges are well known, see for instance [18]. On the other
hand, designing for sound creates new perspectives, e.g., on the EITHER/OR
User interaction Sound input
quality of an interaction. One participant reported, his group
had discussed the nature of an interaction in their scenario
(in the standby door prototype, the quickness of opening the Auditorily
door is related to the user’s mood). Even if sound is not augmented
involved directly, thinking about the design is different with object
sound “deep in our minds” (participant P10).
To conclude, ‘auditorily augmented reality’ clearly behaves
External Data Sound design
different from its visual counterpart, and this fact deserves
more systematic research.
Fig. 3. An auditorily augmented object is influenced by three elements: either
B. Definition and limitations of auditory augmentation of the user interaction or the object’s sound, external data, and the sound
design used in the sonification.
One purpose of the workshop was to develop and test our
working definition which is deliberately wide to incorporate
different platforms. The question is, if this definition is more For comparison between the three exemplary prototypes
useful. (Sec. IV), we analyzed the qualitative behavior of input ele-
Our working definition of auditory augmentation has not ments on auditory augmentation. Abstracting from Fig. 1, we
been questioned within the workshop; therefore, we would like identify four of these elements: user interaction, input sound,
to propose it for future applications. Still, one aspect deserves external data, and sound design. This more abstract concept
more attention than has been discussed above, as came up of auditory augmentation is shown in Fig. 3. We assume that
during the final discussion round. The definition starts from more coherent relations between user interaction, input sound,
an object that is being interacted with, i.e., a primary task of external data, and sound design lead to higher naturalness and
the user with the object is pre-assumed. “Having a concrete intuitiveness of the auditory augmentation system. Multiple
task helps to design” (P1), and helped the teams to elaborate dependencies are possible, even though not all are needed for
their scenarios during the sessions. During the final feedback auditory augmentation:
round, however, it has turned out that the task is an ill-defined • User interaction may influence external data, e.g., turning
entity. Does it relate only to the interaction between user and on the water kettle while its electric power consumption
object? And if the object is augmented, and its sound conveys serves as data. The task of writing by hand, however,
additional information, is there such a thing as a monitoring does not directly modulate in-home electric power con-
task? Does only a goal-oriented activity comprise a task or sumption.
can it be a by-product of daily “state of being” (P7)? • Data may have a close link to sound design, utilizing
We conclude that auditory augmentation always involves a either direct sonification (e.g., audification [11, p. 301-
primary task, may it be goal-oriented or not (e.g., writing on 324]) or a fitting metaphor, e.g., a larger desk with lower
a desk in the writing resonances prototype or just being in pitch representing a larger energy consumption.
the media room of the sonic floor plan). This task should not • User interaction may directly influence sound design, in
be disturbed by the augmentation, but rather the augmentation all cases where sounds are augmented that have been
adds a secondary task of monitoring. produced by the interaction.
• Sound input may not be stemming from the interaction
C. Relationship between sound, data, and augmented object but from an external source; still it may influence sound
In addition, we aimed at exploring the qualitative factors design, e.g., when using environmental sounds from
between object and sound in the context of auditory augmen- outside the windows as in the sonic floor plan prototype.
tation. Which qualities are needed to (fully) describe their In conclusion, it seems that a more natural prototype of
relationship? For example: auditory augmentation has more coherent relationships be-
tween user interaction and/or sound input on the one hand,
• Is the sound, in relation to the object, plausible? Is the
and external data and sound design on the other hand.
mapping of data to sound intuitive/metaphoric?
• Is the augmented object more useful, or more fun than D. Perceptual factors of data and sound
the original one? Does its affordance change and is the As a final research question, we aimed at sketching out
original interaction, i.e. the primary task of the user with perceptual factors of data and sound concerning auditory
the object, disturbed? augmentation systems. What is the capacity of information
A central issue that has been raised throughout the workshop that can be conveyed with auditory augmentation? Which data
was in how far objects change in perception when they are are suitable for it? Which factors play a role for blending
auditorily augmented. This experience, “you can only get it if the augmented object in the environment: is it unobtrusive
you interact yourself” (P1). but salient enough in order to be perceived? Of course, a full
14
Exploration of Auditory Augmentation in an Interdisciplinary Prototyping Workshop
analysis of perceptual factors can only be a result of evaluation be important to cultivate a beginner’s mind – something, he
that was not within the scope of the workshop. However, some stated, that has been well achieved with the interdisciplinary
ideas emerging from the final discussions may serve as a basis workshop setting for most of the participants. Another general
for future investigations. design-issue concerns the difference between prototypes and
As one participant articulated, auditory augmentation works final products. The realized prototypes are meant to be listened
best with low-dimensional data – “otherwise you are not to for a longer period of time, but designers only hear them for
augmenting the object but creating a nice sonification” (P7). a short period of time (P8). Furthermore, for the purpose of
The information capacity, i.e., the level of information that demonstration and presentation, prototypes need to exaggerate,
can be conveyed, is rather low for the examined platforms. while in final products the appropriate settings are usually less
Especially for ROOM and TABLE, only the reverberation salient. Participants who worked with iconic sounds of audio
of the room – respectively the resonances of the structure- recordings stated that some cartoonification is needed (e.g.,
borne sounds – can be changed, with only few levels that through post-processing or re-synthesis of the sounds), because
can be differentiated perceptually. The BRIX system is more “for ordinary people they all sound the same” (P9).
flexible in design, therefore no general conclusion can be VI. C ONCLUSION
drawn. Sonifying is difficult under these conditions, because
Within this paper, we conclude on results from an in-
“it boils down to the question which dimensions you chose
terdisciplinary workshop exploring the concept of auditory
and which ones you leave out” (P11).
augmentation. The workshop resulted in nine prototypes and,
In the writing resonances prototype, the developing team
among others, recorded discussions that have been analyzed.
found a borderline for perception. Depending on the quickness
Concluding on this material, we propose to use the term
of parameter changes, sounds lost their gestalt identity with
auditory augmentation with a new, broader definition: auditory
the interaction, i.e., sometimes the sounds were perceived
augmentation is the augmentation of a physical object and/or
as separate auditory events played from the loudspeakers –
its sound by sound which conveys additional information.
despite measured round-trip latency below 5 ms. This example
General considerations for auditory augmentation are sum-
shows that perception is very sensitive and systems need to be
marized as follows.
well evaluated.
Auditory augmentation requires a primary task of a user
E. General feedback with an object; this task is not explorative data analysis. One
reason is that data for auditory augmentation needs to be low-
Next to finding some insights on the aforementioned re-
dimensional. Another reason is the differentiation between au-
search questions, the analysis of SBE4 provided some feed-
ditory augmentation and sonification. By augmenting an object
back on the workshop setting itself, as well as on general
auditorily, a secondary task of monitoring in the background
design issues.
turns up. This task must not interfere with the primary task.
A general issue of sonification is its right to exist – referring
There seems to be a quality of ‘naturalness’ (affecting
to Supper’s thesis “about community’s struggles to have listen-
also the ‘intuitivity’) of systems of auditory augmentation.
ing to scientific data accepted as a scientific approach” [19].
The most natural systems have several coherent relationships
Useful and meaningful sonifications are difficult to come up
between the four possible input factors, user interaction and/or
with, which raised the question, “why sonify at all”? It was
sound input, with external data and sound design. We envisage
part of the workshop design to develop useful scenarios for
exploring this hypothesis further.
the pre-defined platforms. This worked well for the prototypes
There are borderline cases of perception, where the fusion
presented in this paper, but not for all.
of auditory gestalts between the original sound and the aug-
Generally, feedback on the workshop design was positive.
mented one does not work anymore. The influencing factors
The interdisciplinary, hands-on sessions were “so enriching”
need to be explored systematically.
(P1). Participants further reported that they had “really time
Finally, the analysis of the final discussions during the
to try out something” (P2) – even if a certain approach did
workshop proved that the developed workshop setting is con-
not work out in the end, there was learning even by dead
vincing. It establishes an interdisciplinary, playful atmosphere
ends. However, it was remarked that the prepared platforms
of research by design. The balance of possible ingenuity and
had “narrowed down” (P3) possible paths of design. Some
well-prepared tasks, platforms, and data sets are crucial for a
participants articulated the wish to be more free in designing
successful event.
interaction scenarios independently from a platform (P3, P5),
but the prepared setting was very time-efficient. Most proto- ACKNOWLEDGMENT
types have reached a promising state after the three hours’ We would like to thank our co-organizers from CITEC,
sessions: “there are nine prototypes that are really worthwhile Bielefeld University, and all participants of SBE4:
considering and working on in the future” (P4). Lorena Aldana Blanco, Luc Döbereiner, Josef Gründler,
One participant (P1) raised the issue that designing ambient Thomas Hermann, Oliver Hödl, Doon MacDonald, Norberto
displays means designing for the background, while in the Naal, Andreas Pirchner, David Pirrò, Brian Joseph Questa,
designer’s mind the sound is in the foreground: “we have an Stefan Reichmann, Martin Rumori, Tony Stockman, Leopoldo
excitement for sound and sonic display”. Therefore, it would Vargas, Paul Vickers, and Jiajun Yang.
15
Exploration of Auditory Augmentation in an Interdisciplinary Prototyping Workshop
R EFERENCES
[1] K. Vogt, “Sonification of simulations in computational physics,” Ph.D.
dissertation, Graz University, 2010.
[2] T. Bovermann, R. Tünnermann, and T. Hermann, “Auditory Augmen-
tation,” International Journal on Ambient Computing and Intelligence
(IJACI), vol. 2, no. 2, pp. 27–41, 2010.
[3] T. Bovermann. (2010) Auditory augmentation – wetterreim. [Online].
Available: https://vimeo.com/19581079
[4] R. Tünnermann, J. Hammerschmidt, and T. Hermann, “Blended sonifi-
cation – sonification for casual information interaction,” in Proceedings
of the International Conference on Auditory Display (ICAD), 7 2013.
[5] M. Weger, T. Hermann, and R. Höldrich, “Plausible auditory aug-
mentation of physical interaction,” in Proceedings of the International
Conference on Auditory Display (ICAD), 2018.
[6] D. Rocchesso, Explorations in Sonic Interaction Design. COST Office
and Logos Verlag Berlin GmbH, 2011.
[7] K. Franinovic and S. Serafin, Sonic Interaction Design, K. Franinovic
and S. Serafin, Eds. MIT Press, 2013.
[8] S. Ferguson, “Sonifying every day: Activating everyday interactions
for ambient sonification systems,” in Proceedings of the International
Conference on Auditory Display (ICAD), 2013.
[9] F. Kilander and P. Lönnqvist, “A whisper in the woods – an ambient
soundscape for peripheral awareness of remote processes,” in Proceed-
ings of the International Conference on Auditory Display (ICAD), 2002.
[10] E. Brazil and M. Fernström, “Investigating ambient auditory information
systems,” in Proceedings of the International Conference on Auditory
Display (ICAD), 2007.
[11] T. Hermann, A. Hunt, and J. G. Neuhoff, The sonification handbook.
Logos Verlag Berlin, Germany, 2011.
[12] J. Zmolnig, W. Ritsch, and A. Sontacchi, “The iem-cube.” International
Conference on Auditory Display (ICAD), 2003.
[13] S. Zehe, “Brix2 – the xtensible physical computing platform,”
https://www.techfak.uni-bielefeld.de/ags/ami/brix2/, 2014.
[14] ——, “BRIX2 – A Versatile Toolkit for Rapid Prototyping and Educa-
tion in Ubiquitous Computing,” Ph.D. dissertation, Bielefeld University,
2018.
[15] K. Groß-Vogt, M. Weger, R. Höldrich, T. Hermann, T. Bovermann, and
S. Reichmann, “Augmentation of an institute’s kitchen: An ambient audi-
tory display of electric power consumption,” in International Conference
on Auditory Display (ICAD), 2018.
[16] Commission for Energy Regulation (CER). (2012) CER smart metering
project – electricity customer behaviour trial, 2009-2010 [dataset].
Irish Social Science Data Archive. SN: 0012-00. [Online]. Available:
www.ucd.ie/issda/CER-electricity
[17] R. T. Azuma, “A survey of augmented reality,” Presence: Teleoperators
& Virtual Environments, vol. 6, no. 4, pp. 355–385, 1997.
[18] G. Kramer, B. Walker, T. Bonebright, P. Cook, J. Flowers, N. Miner,
J. Neuhoff, R. Bargar, S. Barrass, J. Berger et al., “The sonification
report: Status of the field and research agenda. report prepared for the
national science foundation by members of the international community
for auditory display,” International Community for Auditory Display
(ICAD), Santa Fe, NM, 1999.
[19] A. Supper, “The search for the ”killer application”: Drawing the bound-
aries around the sonification of scientific data.” in The Oxford Handbook
of Sound Studies., K. E. Pinch, T. & Bijsterveld, Ed. Oxford University
Press, 2011.
16