Interdisciplinary Classification of Audio Effects in the Audio Effect Ontology Thomas Wilmering and Mark B. Sandler Centre for Digital Music (C4DM) Queen Mary University of London, London E1 4NS, UK {t.wilmering, mark.sandler}@qmul.ac.uk Abstract. This paper discusses an extension to the Audio Effect On- tology (AUFX-O) for the interdisciplinary classification of audio effect types. The ontology extension implements a unified classification system that draws on knowledge from different music-related disciplines and is designed to facilitate the retrieval of audio effect information based on low-level and semantic aspects. It extends AUFX-O enabling communi- cation between agents from different disciplines within the field of music creation and production. After briefly discussing the ontology, we show how it can be used to efficiently classify and retrieve effect types. 1 Introduction In modern music production, composers and producers can choose from a large number of audio effect implementations of a variety of effect types. Identify- ing suitable effects for specific tasks typically remains a time-consuming manual process. Moreover, the aspects by which audio effects are described and classi- fied can differ depending on the discipline in the field of music production and composition. This paper introduces an extension to the Audio Effect Ontology (AUFX-O) we presented in [4], for the interdisciplinary classification of audio effect types. AUFX-O is designed to describe audio effect implementations, i.e. sound transformation devices, and their use in music production process. While the core ontology already allows for the annotation of effect types with commonly used tags, this ontology extension introduces defined concepts for effect descrip- tion. It implements a unified classification system that draws on knowledge from different music-related disciplines and is designed to facilitate the retrieval of audio effect information based on low-level and semantic aspects. Its design is aimed at enabling communication between agents adhering to data models with conceptualisations informed by and supporting different disciplines, both tech- nical and artistic. After briefly discussing the ontology model, we show how an effect type can be described based on low-level and high-level descriptors.1 1 Since AUFX-O and its extension are domain-specific we refer the reader to [7] for a detailed discussion of audio effect types and their implementation. 2 2 Audio Effect Classification Digital audio effects are used by composers, performers and sound engineers. They can be classified based on different aspects that are relevant for these in- dividual disciplines. For instance, a software developer would put emphasis on the signal processing techniques applied in the effect implementation, whereas a musician may prefer to describe effects by their perceptual qualities or by high- level semantic descriptors. An example for such a discipline-specific classification is given in [3, 2], where effects are categorised based on the perceptual attributes most affected by a given sound transformation. The perceptual axes are loud- ness, pitch, time, space, and timbre.2 Audio effects may also be classified by the way its parameters are controlled, for instance by way of manual input, low fre- quency oscillators (LFOs), or noise generators. Lower level descriptions include the signal processing strategies applied in a given effect type’s implementation and their characteristics, such as the processing domain (i.e. time or frequency domain)3 , and the algorithm or model implemented for their realisation. Figure 1 illustrates layers of discipline-specific classifications ranging from low-level to high-level descriptors. In an effort to unify these classification approaches an in- terdisciplinary classification model has been proposed in [3, 2], which forms the basis for the AUFX-O classification extension. Digital implementation Processing Applied Control Perceptual Semantic technique Domain processing Type Attribute descriptors low level high level features features Fig. 1. Layers of discipline-specific classifications ranging from low-level to high-level features. 3 The AUFX-O Classification Extension The main classes and properties of the AUFX-O extension are shown in Fig- ure 2. The extension incorporates the classification systems discussed in Section 2, including the classification based on perceptual attributes which we evalu- ated by conducting listening tests [6]. The extension links to AUFX-O via the :effect_type property that links instances of the aufx:Fx class conceptualising audio effects on an abstract level (as physical phenomena) to instances of the EffectType class. Semantic descriptors and the common name of an effect type are linked via annotation properties, while object properties relate EffectType to PerceptualAttribute, ControlType and AppliedProcessing. The latter can be further described with object properties linking the Implementation Technique which in turn is linked to its ProcessingDomain. While the ontol- ogy contains individuals of the PerceptualAttribute and ProcessingDomain classes, other concepts contain subclass structures for further specialisation. 2 for a detailed discussion of music perception see [1] 3 Time domain refers to the variation of a signal’s amplitude over time; in the frequency domain signals are represented by a magnitude and phase as a function of frequency. 3 aufx:Fx xsd:string :name :effect_type :PerceptualAttribute :semantic_descriptor :perceptual_attribute :ControlType :control_type :EffectType :ProcessingDomain :applied_processing :processing_domain :AppliedProcessing :implementation_technique :ImplementationTechnique Fig. 2. Main classes and properties in the AUFX-O classification extension. We illustrate how effect types are modelled with the example of the chorus ef- fect. A chorus effect artificially simulates the doubling of voices occurring when several musicians play the same melody in unison and slight pitch, dynamic, rhythm and timbre differences arise because the instruments are not physically identical, nor are perfectly tuned and synchronised. This widely used effect pro- vides some ”warmth” to a sound, and can be considered an effect on timbre. While its usual implementation involves one or many delay lines, with modulated length and controlled by a white noise, more realistic sounding implementations use several slightly pitch-shifted and time-scaled versions of the same sound with refined models and mixing them together. Listing 1 shows how this knowledge is encoded in the ontology. The Chorus class is defined as a subclass of the EffectType class, as well as of several anonymous classes reflecting the descrip- tors discussed in Section 2. Anonymous classes further describe the processing applied and the techniques needed for the effect’s implementation. For instance, the ontology reflects that for the implementation of the chorus effect there are three possible processing strategies, AddingDelayedSignal, TimeScaling and Transposition. Applying transposition, i.e. the changing of pitch of a sound, can be achieved by three different implementation techniques. These include the PhaseVocoder technique in the FrequencyDomain and the SOLA technique in the TimeDomain [7]. 4 Conclusions and Future Work We discussed the ontological representation of an interdisciplinary classification system for audio effects implemented as an extension to AUFX-O. We showed how the extension can add value to the ontology by allowing users and agents to retrieve audio effect implementations based on different characteristics that are important for different disciplines in the field of music composition and pro- duction. Future work includes extending the ontology with a more thorough coverage of effects used in music production, and further development of the linked data service we presented in [5], exposing metadata about audio effect implementations. Acknowledgments. This paper has been supported by EPSRC Grant EP/ L019981/1, Fusing Audio and Semantic Technologies for Intelligent Music Pro- duction and Consumption. Mark B. Sandler acknowledges the support of the Royal Society as a recipient of a Wolfson Research Merit Award. 4 Class: Chorus Annotations: semantic_descriptor "warm sound", semantic_descriptor "several performers" , name "Chorus" SubClassOf: EffectType, applied_processing some (AddingDelayedSignal or TimeScaling or Transposition), control_type some WhiteNoiseControl, perceptual_attribute value Timbre Class: DelayLine SubClassOf: ImplementationTechnique and (processing_domain value TimeDomain) Class: PhaseVocoder SubClassOf: ImplementationTechnique and (processing_domain value FrequencyDomain) Class: SOLA SubClassOf: ImplementationTechnique and (processing_domain value TimeDomain) Class: TimeScaling SubClassOf: AppliedProcessing and (implementation_technique some (AdditiveModel or PhaseVocoder or SOLA)) Class: Transposition SubClassOf: AppliedProcessing and (implementation_technique some (AdditiveModel or PhaseVocoder or SOLA)) Class: DelayLine SubClassOf: ImplementationTechnique and (processing_domain value TimeDomain) Class: AddingDelayedSignal SubClassOf: AppliedProcessing and (implementation_technique some DelayLine) Class: AdditiveModel SubClassOf: ImplementationTechnique and (processing_domain value FrequencyDomain) Listing 1. Excerpt from the AUFX-O classification extension relating to the effect type Chorus. References 1. Deutsch, D. (ed.): The Psychology of Music, 3rd Edition. Academic Press (2013) 2. Verfaille, V., Holters, M., Zölzer, U.: Introduction. In: Zölzer, U. (ed.) DAFX - Digital Audio Effects. 2nd Edition, J. Wiley & Sons (2011) 3. Verfaille, V., Guastavino, C., Traube, C.: An interdisciplinary approach to audio ef- fect classification. Proceedings of the 9th International Conference on Digital Audio Effects (DAFx-06), Montreal, Canada (2006) 4. Wilmering, T., Fazekas, G., Sandler, M.B.: AUFX-O: Novel methods for the rep- resentation of audio processing workflows. Proceedings of the 15th International Semantic Web Conference (ISWC 2016) (2016) 5. Wilmering, T., Fazekas, G., Allik, A., Sandler, M.B.: Audio effects data on the semantic web. Presented at the 139th Audio Engineering Society Convention, New York, NY, USA (2015) 6. Wilmering, T., Fazekas, G., Sandler, M.B.: Audio effect classification based on au- ditory perceptual attributes. presented at the AES 135th Convention, New York, NY, USA. (2013) 7. Zölzer, U. (ed.): DAFX - Digital Audio Effects. J. Wiley & Sons, 2nd edn. (2011)