Visualization of business process models with large numbers of process activities and running instances in di erent execution states shows various limitations. Hence, using soni cation methods becomes a promising idea to enable users to gain insight into process information that cannot be conveyed by using purely visual means. This visionary short paper aims to envision the usage of soni cation methods in order to represent business process-related data in all phases of the process life cycle. Soni cation methods are presented and analyzed in terms of their potential suitability for representations of process data. Overall, this paper aims at breaking new ground for designing and applying multimodal approaches for making process information more accessible to users.
During the individual life cycle phases of business processes (design & simulation, operation and analysis), di erent kinds of process model- and process instancerelated data accumulate. For all phases, especially for the design & simulation phase, graphical user interfaces and visualization methods are widespread. However, visualization techniques can reach their limits. As an example, process models in the design and simulation phase can have a huge number of events and activities, which can make it di cult to visually identify deadlocks or weaknesses in process models. In the operation and evaluation phases, processes can have thousands of simultaneously running process instances, which can make it hard to nd deviations from regular process execution paths or monitor the health or processes and process instances using only visual means.
The usage of data soni cation as an enhancement to process visualizations
might be able to tackle some of these challenges. Soni cation can be de ned
as the "presentation of data using sound" [
This visionary short paper provides an analysis of soni cation methods with respect to their suitability in the area of business processes. Existing basic soni cation methods are discussed and their potential to convey information in consideration of the business process life cycle is analyzed. This is a rst step towards a multi-modal approach in order to make information related to business processes more accessible to users.
The paper is structured as follows. Section 2 presents basic soni cation methods. Possible solutions regarding how soni cation methods can be used to make process-related data more accessible to users are discussed in Section 3. Finally, Section 4 concludes the paper and gives an outlook on future work. 2
Nowadays, there is a growing amount of research in the elds of areas of
applications, methods, and perception of soni cation. Regarding the application of
soni cation there is, among others, research in the elds of astronomy, volcano
activity, ice glaciers, RNA structures, brain activities or weather data. Other
examples are soni cations of software code and sonically enhanced data mining.
Furthermore, there are a few examples in the elds of social sciences: soni cations
for population developments and election outcomes, sport sciences or economics
(e.g., soni cation of stock market data [
After this brief introduction into the eld of soni cation, in the following the
four probably most widely used and researched soni cation methods (audi
cation, auditory icons, earcons and parameter mapping) will be introduced.
Audi cation. An audi cation is the direct conversion of data points into sound.
It interprets data sequences as an audio waveform by mapping the data to sound
pressure levels. Therefore, a very high number of data sets is needed in order to
produce audible results, which is limiting the eld of possible modes of operation.
[
Earcons. Earcons are non-verbal audio messages consisting of motives, which
are short rhythmic sequences of pitched tones with variable timbre, pitch and
amplitude. Timbre describes the basic properties of sounds and is a subjective
characteristic that enables the di erentiation of two sounds, even though they
might have the same loudness and pitch [
The concept of earcons is similar to that of auditory icons with the di erence, that auditory icons are everyday sounds that directly represent the event that is being soni ed, whereas earcons can be abstract symbols that are not similar to the real world sound of the represented event or object.
Parameter Mapping. Parameter mapping is the mapping (either direct or
by scaling) of data values to speci c attributes of sound. These attributes are
typically volume, pitch, panning (the position of a sound in the stereo eld)
or timbre. Other possibilities to map parameters of sound are repetitions and
pauses between distinct sound events in loops. Other approaches are to lter
out speci c ranges of frequencies according to data. Due to these characteristics
parameter mapping is often being said to be the sonic pendant to a scatter plot
diagram. [
Even though a substantial amount of soni cation research has accumulated, so
far there seems to be no research concerning the application of soni cation in the
di erent life cycle phases of business processes. One of the few examples of the
usage of soni cation concerning processes in business environments is Grooving
Factory [
This section will discuss possible solutions in terms of how soni cation can be used to make process-related data more accessible to users and to reduce the potential limitations of process visualizations. Therefore, the respective process life cycles phases are analyzed in terms of which soni cation techniques might be suited best to support the tasks users typically have to perform in those phases. Soni cation in Process Design. Audi cation relies on a huge number of quantitative data, which makes this technique seem unsuitable in cases where no or little quantitative data needs to be soni ed, but instead merely events. Auditory icons, on the other hand, seem very suitable for soni cations in the process design phase: for process instances that are created during the simulation of process models, the sonic pendants of the involved activities and events could be played back upon their incidences. Depending on the industry and the type of processes, there is often a variety of self-explanatory sounds that can be used in order to sonify the respective events and activities. In auditory-based soni cations, the relevant events and activities could be soni ed by using sounds that naturally represent these events and activities as accurately as possible. As an example, the sound of a shopkeepers bell could signify the reception of a new order. Analogous, the process event "customer has payed his invoice" could be conveyed by playing the sound of a cash register being opened, while the activity "delivery" could be soni ed by applying motor sounds. Fig. 1 shows a schematic overview of how such a soni cation of a process instance could be realized. The x-axis is the time axis, whereas each row on the y-axis contains a sound le that represents one activity or event. These sound les are played back sequentially from left to right.
In this example, the sounds that convey events have been assigned a xed length of 1.5 seconds (as the time axis in the lower part of the gure shows). The lengths of the audio signals that convey activities, on the other hand, represent the actual duration of the represented activities. Thus, a motor sound with a duration of three seconds could, depending on the scaling, for example signify that the transport took three days. Analogous, a silence between activities and events could signify a waiting period. If, for example, there is a gap of two seconds between the playback of the sounds that represent the activities "production" and "packaging", one can conclude that there has been a waiting period of two days between the production of goods and the transport of said. This could be a hint that there are ine ciencies in the process.
Audio les of three di erent instance soni cations of this example process are available online1. The audio le "Example one - average process instance"2, shows a soni cation of an average process instance. The audio le "Example two - no payment"3 is a soni cation of a process instance, in which no incoming payment has been registered. The audio le "Example three - Production and transport delayed"4 is a soni cation of a process instance, in which the activities production and transport have been delayed (which can be recognized by the pauses before the respective audio signals). This simple example tries to show, that auditory icons are able to point out deviations in process instances. 1 http://soundcloud.com/tobias hildebrandt/ 2 direct link: http://soundcloud.com/tobias hildebrandt/business-process-soni cation 3 direct link: http://soundcloud.com/tobias hildebrandt/business-process 4 direct link: http://soundcloud.com/tobias hildebrandt/business-process-1 However, due to its simplicity, this example cannot serve as a comprehensive demonstration of the strengths of soni cation (like its ability to convey complex or irregular data). Further prototypes that combine visual and auditory means and base on more complex process models will help in evaluating, if the inherent features of the auditory perception (like the sensitivity to rhythm and its ability to recognize even small changes in sounds over time) can help to convey process instance-related information better than purely visual means.
Earcons are in a similar fashion suitable for process data soni cations but more exible. For some process events it could prove di cult to nd real-worldsonic analogies. For example, it could be a challenge to nd sounds that are sonic analogies to the states "customer is already registered" and "new customer". This di erentiation would therefore be hard to convey using auditory icons, so the usage of earcons might solve that problem (even though studies suggest that earcons are harder to recognize than auditory icons). By using parameterized auditory icons or earcons, not only the information can be conveyed that a certain event has occurred, but also one or several quantitative data attributes that are connected to that event. For example, one could imagine an auditory icon that conveys the occurrence of an event "incoming payment", while the sum of the payment is mapped to the pitch of that auditory icon.
Parameter mapping might not be the most obvious choice for the process design phase - parameter mapping relies on quantitative data that varies over time, rather than on information on events and their sequences of occurrence, as it is typically the case for process instance-related data.
Soni cation in Process Operation and Analysis. Soni cations that aim to assist users during the process operation phase and the process analysis phase probably have to ful ll similar requirements. In both, potential users might want to obtain aggregated information about processes and associated instances and analyze conspicuous phenomena in detail.
Audi cation does not seem not very suitable for the soni cation of data that is related to the process operation and analysis phases, as it seems in exible in terms of sound design and the structure and format of input data (high amounts of quantitative data that lie within a speci c range).
Auditory icons should o er the possibility to recognize deviations of process instances from the process model by the fact that the respective sounds are being played in a di erent order, or in a di erent rhythm while monitoring and analyzing individual process instances. The same is true for (parameterized) earcons, analogous to what has been said for the process design phase.
Parameter-mapping soni cations might be especially useful during the process operation and the process analysis phases. During these phases, usually quantitative data accumulates that might be mapped to one or several sound streams. These sound streams might then, for example in the process operation phase, be played back continuously which should make it feasible for the user to recognize patterns and modi cations as well as to get an overview of the general health of individual processes or a complete system. During process operation, an advantage over the usage of purely visual means would be that users would not need to focus their visual attention to speci c displays. Thus, they would be able to work on other things while at the same time they could be informed about background activities. Of course, such a soni cation would have to be designed in a non-disruptive way.
In the process analysis phase one could imagine a sonic summary of a certain time period (for example a shortened soni cation of the last 24 hours). In such a soni cation it should, after a learning phase, be possible to detect deviations or critical situations during the execution of process instances. 4
Soni cation is gaining more and more importance in various disciplines. Due to the existing limitations of visualizations (e.g., keeping track of high numbers of running process instances in di erent execution states), the authors of this paper propose to introduce it as an enhancement to visualization methods to convey business process information. To achieve this goal it is rst necessary to understand how soni cation methods can be used to represent business process-related data. The motivation of this paper was to give a rst overview of di erent sonication methods. Possible directions and solutions concerning how soni cation can be used to make process-related data more accessible to users were discussed.
Of the presented soni cation methods, parameterized earcons and parameterized auditory icons seem to be best suited for soni cations during the process design phase, while the monitoring in the operation phase and the analysis in the evaluation phase seem to be well suited for a parameter mapping soni cation.
A multi-modal combination of visualization and soni cation should consider the individual strengths and weaknesses of both methods. In general, it should use the abilities of visualization to convey exact information and of soni cation to convey changes in temporal developments.
In future work, we will develop and combine soni cation methods with visualization methods { particularly for scenarios where visualization techniques come to their limits { that can best be combined into an integrated multi-modal approach for each phase of the business process life cycle. Further, we plan to conduct user studies to evaluate the design of the multi-modal approach and the ndings of the evaluations will in uence the further design process to improve our approach iteratively.
The research was partly funded by COMET K1, FFG - Austrian Research Promotion Agency.