The Shape of Music - The Music of Shape

-

Musicians make use of the notion of shape in relation to music performance, teaching, composition, improvisation; they often think and talk about the shape of a musical phrase or melody, the shape of a single sound event, the shape of a longer section or piece, the expressive shaping of music during performance. But how can a primarily spatial term such as shape (meaning “the external form, contour, or outline of someone or something” according to the Oxford dictionary), be applied in the temporal domain of music? How can one talk of shape in regards to a temporally emitted sequence of sound events?

Many researchers have studied the correspondence between space and time in human conceptualisation, and more specifically the use of spatial metaphors in temporal reasoning [ 1, 2 ]. Johnson and Larson [ 3 ] discuss the relation between music motion and space, and, more specifically, the employment of two basic spatialisation metaphors of time in the conceptualisation of musical time and motion. Frequency/pitch is also commonly conceptualised in terms of spatial metaphors, namely along the high/low spatial axis [ 4, 5 ]. The combination of the x-axis spatial representation of time with an orthogonal high/low pitch y-axis gives rise to the most common 2D representation of music (standard score notation, piano-roll notation, spectrogram, various graphic scores) – see [ 6 ] for a cross-cultural study on visual representation schemes of melodic shape. The correspondence between time and space is so strong that it is difficult to conceptualise music without thinking about shape or pattern.

Music becomes intelligible to a great extent through its inner self-referential structural relations. New unheard musical passages relate to previously heard material giving rise to meaningful musical units (such as motives, themes, rhythm patterns, harmonic progressions). The emergence of musical patterns via repetition/similarity is paramount in making sense and understanding music. As music does not have explicit denotative meaning (as language has) musical meaning is more multi-faceted, ambiguous and, in a certain sense, richer. It has been debated for centuries whether musical meaning is intra- or extra-musical (absolutists vs referentialists), whether music can evoke emotions (formalists vs expressionists), what music signifies [ 7 ]. In this paper we focus on structural patterns in music: in intra-musical structural relations per se that allow the emergence and perception of salient musical shapes, patterns, forms, structures, and, additionally, on aspects of music performance that shape music by bringing out various elements of musical structure giving this way ‘life’ to a music score (relations of musical performance to physical gestures, embodiment, emotions are not discussed).

Pattern matching/extraction techniques are crucial in discovering salient recurring musical patterns. Yet, despite the efforts made towards the systematic description of musical pattern patterns, musical similarity remains an elusive concept, resisting robust formalisation. Why does the introduction of well-established powerful pattern matching techniques (exact or approximate) in the musical domain, usually ends up with rather limited/partial/fragmentary results? Why is it so difficult to create a general model of musical similarity that may capture musically and cognitively plausible patterns?

We focus on three sources of difficulty in describing musical similarity. Firstly, it is not always easy, to get a musical sequence per se on which to apply pattern matching techniques; especially in non-monophonic music (i.e., most music), it is anything but trivial to derive cognitively meaningful auditory streams within which patterns may emerge (musical patterns emerge within musical streams, not across them). Secondly, it is most important to decide how a sequence of musical events may be represented; representation in music is complex due to the multi-dimensional and hierarchic nature of musical data. For instance, encoding a melodic line as pitches or pitch-intervals makes a great difference on the patterns that may be discovered. Thirdly, it is vital to define the nature of a certain similarity process, as special models may have to be devised (rather than use of standard off-the-shelf algorithms). In this address, examples and techniques from current research on musical pattern discovery for musical analysis and performance, primarily in melodic contexts, are presented to highlight the importance of looking in detail at the musical and cognitive aspects of music pattern discovery tasks before attempting to use/develop specific pattern matching algorithms.

Pattern matching methods are commonly employed to capture musical variation, especially melodic variation [ 8, 9, 10 ]. Dynamic programming techniques, often based on various types of edit distance, are used to find patterns in melodic strings. It is maintained that techniques using standard edit distance operations (replacement, insertion, deletion, along with consolidation and fragmentation) applied on strings of notes [ 11 ] are limited and have inherent shortcomings. Instead, the problem of matching is redefined in a way that is appropriate for strings of melodic intervals (not notes). To this aim, the replacement, insertion and deletion operations are abolished, and only consolidation and fragmentation operations are retained which are adapted to the interval domain. It is shown that this new description of the problem of melodic matching enables more reliable matches and is also transposition invariant [ 12 ].

In recent years there is a significant amount of research on patterns of music expression that relate to musical structure. Empirical studies have examined the way performers shape musical structure [ 13, 14, 15 ]. Additionally, rules of performance have been encoded computationally and tested [ 16 ] or machine learning techniques have been employed aimed at learning patterns of music expression from actual music performances [ 17 ]; such shapes of expression can be utilised in the context of performance visualisation aids [ 18 ] or, more so, in the actual rendering of novel computer performances of new scores.

Shape and pattern are basic notions by which musicians and listeners conceptualise music. The link between music and shape is so strong that, not only music is often understood in terms of shape, but also shape can be understood and realised as music.

References

[1] Gentner , D. ( 2001 ). Spatial metaphors in temporal reasoning. In Spatial schemas and abstract thought . MIT Press, pp. 203 - 222 .

[2] Lakoff , G. , & Johnson , M. ( 2008 ). Metaphors we live by . University of Chicago press.

[3] Johnson , M. L. , & Larson , S. ( 2003 ). " Something in the Way She Moves"- Metaphors of Musical Motion . Metaphor and symbol , 18 ( 2 ), 63 - 84 .

[4] Zbikowski , L. ( 2002 ). Conceptualizing Music. New York: OUP.

[5] Antovic , M. ( 2011 ). Musical Metaphor Revisited: Primitives, Universals and Conceptual Blending . Available online at SSRN. http://ssrn.com/abstract=1763503. Accessed 24 October 2015 .

[6] Athanasopoulos , G. , & Moran , N. ( 2013 ). Cross-cultural representations of musical shape . Empirical Musicology Review , 8 ( 3-4 ), 185 - 199 .

[7] Nattiez , J. J. ( 1990 ). Music and discourse: Toward a semiology of music . Princeton University Press.

[8] Cambouropoulos E. Crawford T. and Iliopoulos C.S. ( 2001 ) Pattern processing in melodic sequences: Challenges, caveats and prospects . Computers and the Humanities , 35 ( 1 ): 9 - 21 .

[9] Ferraro , P. , Hanna , P. and Robine M. ( 2007 ) On optimising the editing algorithms for evaluating similarity between monophonic musical sequences . Journal of New Music Research , 36 ( 4 ): 267 - 279 .

[10] Hewlett , W. and Selfridge-Field , E. ( 1988 ) Melodic Similarity: Concepts, Procedures, and Applications . MIT Pess, Cambridge (Ma).

[11] Mongeau M. and Sankoff D. ( 1990 ) Comparison of musical sequences . Computers and the Humanities , 24 : 161 - 175 .

[12] Barton , C. , Cambouropoulos , E. , Iliopoulos , C. S. , & Lipták , Z. ( 2012 ). Melodic string matching via interval consolidation and fragmentation . In Artificial Intelligence Applications and Innovations (pp. 460 - 469 ). Springer Berlin Heidelberg.

[13] Gabrielsson , A. ( 1987 ). Once again: the theme from Mozart's piano Sonata in A Major (k. 331). Action and perception in rhythm and music , 55 , 81 - 103 .

[14] Repp , Bruno H. ( 1995 ) Expressive timing in Schumann's 'Träumerei:'An analysis of performances by graduate student pianists . The Journal of the Acoustical Society of America 98 .5: 2413 - 2427 .

[15] Prior , H. M. ( 2011 ). Exploring the experience of shaping music in performance. In CMPCP Performance Studies Network International Conference . Faculty of Music, University of Cambridge.

[16] Friberg , A. , Bresin , R. , & Sundberg , J. ( 2006 ). Overview of the KTH rule system for musical performance . Advances in Cognitive Psychology , 2 ( 2-3 ), 145 - 161 .

[17] Widmer , G. , & Goebl , W. ( 2004 ). Computational models of expressive music performance: The state of the art . Journal of New Music Research , 33 ( 3 ), 203 - 216 .

[18] Dixon , S. , Goebl , W. , & Widmer , G. ( 2005 ). The" Air Worm": An Interface for Real-Time Manipulation of Expressive Music Performance . Proc. ICMC'05 , 614 - 617 .