Opening musical creativity to non-musicians

Abstract

This paper gives an overview of my PhD research that aims at contributing toward the de nition of a class of interfaces for music creation that target non-musicians. In particular, I am focusing on the di erences on design and evaluation procedures with respect to traditional interfaces for music creation that are usually intended to be used by musicians. Supported by a number of preliminary ndings we developed the rst interactive system: The Music Room is an interactive installation which enables people to compose tonal music in pairs by communicating emotion expressed by moving throughout a room.

Keywords

• Musical interfaces
• user-experience
• performing art
• active listening

Our contribution

The rst interface we developed is The Music Room, an installation that provides a space where people can compose music expressing their emotions through movements. The visitors experience the installation in couple by informing the system on the emotions they intend to elicit. The couple directs the generation of music by providing information about the emotionality and the intensity of the music they wish to create. To communicate emotions, the analogy with love is used: the proximity between them a ects the pleasantness of the music, while their speed a ects the dynamic and intensity. We decided to limit the interaction dimensions to closeness and speed in order to keep the experience as simple and intuitive as possible. Proxemics information is acquired by a vision tracking system. It is then converted into emotional cues and nally passed to the musical engine. These intuitive compositional rules provide everybody with unlimited musical outcomes. As regard the generation of music, we developed Robin, an algorithmic composer that composes original tonal music in piano 1.

1 Examples of pieces generated at The Music Room can be listened at goo.gl/Ulhgz

Related works

This project spans several research areas. The adoption of the metaphor of gestures and emotions is partially in uenced by previous collaborative interactive systems for music generation. The rules of the compositional system are founded research on music perception, while Robin is inspired by existing approaches for algorithmic composition. 3.1

Interactive Musical System

Research on the design of interactive systems for generative music has been growing in the last decade. A number of tangible musical interfaces such as the Reactable [ 1 ], Jam-O-Drum [ 17 ], and GarageBand for the iPad, target users that have at least a minimum musical training as sonic and musical inputs are adopted. A category of interfaces addresses users to collaborate. In particular, several works exploit the concept of active listening, an approach where listeners can interactively control the music content by modifying it in real-time while listening to it [ 18, 19 ]. TouchMeDare aims at encouraging strangers to collaborate for reaching a common creative goal: pre-composed music samples are triggered when both simultaneously touch a canvas [ 22 ]. In the Urban Musical Game, users manipulate pre-composed music by playing with sensors-equipped foam balls [ 21 ]. With Sync'n'Move music is also experienced by collaborative means [ 23 ]. Two users freely move their mobile phones and the level of music orchestration depends on the synchronization of their movements. In Mappe per A etti Erranti, a group of people can explore pre-composed music by navigating a physical and emotional space [ 20 ]. Once again, collaborative situations are encouraged as music can only be listened to in its full complexity if the participants cooperate. 3.2

Eliciting emotions in music

Related works suggest that the perception of emotions in music depends on compositional parameters (e.g. tempo, melody direction, mode) and performance behaviors (articulations, timing, dynamics) whose combinations elicit di erent emotional responses in the listener [5{7]. Measurement and classi cation of emotions in music, most of the works in the music domain are operates on Russell's Circumplex model [ 8 ]. This model describes emotions as a continuum along two dimensions: valence and arousal. In 1937, Hevren identi ed the most important compositional factors in terms of emotions elicitation by labelling them on the music's expressive character [ 9 ]. Juslin and Sloboda later reviewed this categorization by representing the emotions along the valence/arousal dimensions [ 10 ]. There is a consensus that at the compositional level, mode and rhythm are responsible for valence, while tempo and dynamics are responsible for arousal. Despite the remarkable amount of works on this area, no signi cant study has been tried to understand to which extent expertise has a role on judging, appreciating and perceiving musical pieces. How do non-musicians perceive and

describe music? What are the musical parameters and semantic elements that are more relevant for them? 3.3

Algorithmic Composition

Generative music composition has been widely explored in the last decade. The most common approaches are: rule-based, learning-based and evolutionary composition [ 10 ]. In rule-based approach, algorithmic rules inspired from music theory are manually coded into the system [ 11, 12 ]. In learning-based approach, the system is trained with existing musical excerpts and a number of rules are automatically included [ 13, 14 ]. Even though this method manages to decrease the reliance on designer skills on music theory, the output heavily depends on the training set. Lastly, evolutionary algorithms allow the creation of original and complex melodies by means of computational approaches inspired by biological evolution [ 15 ]. The generated music is original and unpredictable but it might sound unnatural and lack ecological validity if compared to rule-based systems that are generally superior in contexts of tonal music [ 16 ]. 4

Results achieved

A number of personal contributions for each of the three research areas were recently published. At the Interactivity session of CHI 2013, we demoed The Music Room [ 24 ], whose objectives, development, ndings and evaluation are better discussed on the forthcoming publication at the Special Issue of Pervasive and Ubiquitous Computing on Designing Collaborative Interactive Spaces.

The role of expertise on the evaluation of induced emotions in music was analysed in a experiment we conducted in 2012 whose results were recently published on Proceedings of ICME3 [ 25 ].

The details on the ideation and implementation of Robin, the algorithmic composer, are going to be published at Proceedings of SMC2013 [ 26 ]. 5

Future works

The last year of my PhD will be mainly devoted toward a formal de nition of interactive systems for music creation that target non-musicians. The rst objective is to investigate similarities and di erences with traditional digital musical interfaces. By date, just a few studies highlighted the di erences between interfaces for artistic experience and for musical expression but these works didn't have a follow-up in the last decade [ 27 ]. However, we believe that a number of relevant di erences exist. By combining personal intuitions with related literature ndings, we propose a list of potential di erences between the two sets. Possibly, the output of this study will consist of a categorization of musical interfaces. The idea is to exhaustively describe musical interfaces by means of a model composed of a space of number of dimensions such as:

{ Target user { Ultimate goal { Learning curve { Interaction metaphor { Level of direction { Musical possibilities { Role of the audience

The successive step would consist in testing the proposed dimensions with a series of existing interfaces. Once validated, I will elaborate on de ning a series of evaluation principles for each dimension. This will allow interface designers to position their projects on the model and to evaluate them consequently.

I'd also wish to tackle a number of challenges regarding The Music Room. Even though the current implementation received a lot of interest, there is room for several improvements. A number of innovations to music engine are currently under development: the quality of music will be enhanced by introducing new genres and instruments as well as by teaching Robin new compositional rules. I also aim at further elaborating on the communication of intended emotions to the system. Temporal aspects will be taken into consideration in order to determine a general evolution of the experience, by considering recurrence of patterns of moving close and getting far. Also, we are likely to introduce head pose tracking in order to have information whether the two people are looking at each other. This additional information will be used to di erentiate the situations in which the user are facing or turned and direct the music generation consequently.

References

1. Jorda , S. , Geiger , G. , Alonso , M. , & Kaltenbrunner , M. ( 2007 ). The reacTable: exploring the synergy between live music performance and tabletop tangible interfaces . Proceedings of the 1st international conference on Tangible and embedded interaction (pp. 139 - 146 ). ACM.

2. Camurri, a., Volpe , G. , De Poli, G. , & Leman , M. ( 2005 ). Communicating expressiveness and a ect in multimodal interactive systems . IEEE Multimedia , 12 ( 1 ), 43 - 53 .

3. Balkwill , L.-L. , Thompson , W. F. , & Matsunaga , R. I. E. ( 2004 ). Recognition of emotion in Japanese, Western, and Hindustani music by Japanese listeners . Japanese Psychological Research , 46 ( 4 ), 337 - 349 .

4. Fritz , T. , Jentschke , S. , Gosselin , N. , Sammler , D. , Peretz , I. , Turner , R. , Friederici , A.D. , Koelsch , S. ,( 2009 ). Universal Recognition of Three Basic Emotions in Music, Current Biology , Volume 19 , Issue 7 , 14 April 2009 , Pages 573 - 576 , ISSN 0960- 9822

5. Juslin P.N. , & Sloboda J.A. ( 2009 ). Handbook of music and emotion: theory, research, applications . Oxford University Press.

6. Temperley , D. ( 2001 ). The Cognition of Basic Musical Structures . Mit Press.

7. Gabrielsson , A. , Lindstrm , E. , ( 2001 ). The in uence of musical structure on emotional expression . Music and emotion: Theory and research . Series in a ective science., (pp. 223 - 248 )

8. Russell , J. ( 1980 ). A circumplex model of a ect . Journal of personality and social psychology.

9. Hevner , K. ( 1937 ). The A ective Value of Pitch and Tempo in Music . The American Journal of Psychology , 49 ( 4 ), 621 - 630 CR - Copyright & 169 ; 1937 University of Illinois Press.

10. Todd , P. M. , & Werner , G. M. ( 1999 ). Frankensteinian methods for evolutionary music composition. Musical networks: Parallel distributed perception and performance , 313 - 339 .

11. Rader , G. ( 1974 ). A method for composing simple traditional music by computer . IT press, Cambridge, MA

12. Steedman , M. ( 1984 ). A Generative grammar for jazz chord sequences . Music Perception Vol. 2 , No. 1, Fall , 1984

13. Simon , I. , Morris , D. , & Basu , S. ( 2008 ). MySong: automatic accompaniment generation for vocal melodies. Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems (pp. 725 - 734 ). ACM.

14. Hiller , L. , Isaacson , L. ( 1992 ). Musical composition with a high-speed digital computer , Machine models of music.

15. Wiggins , G. , Papadopoulos , G. , Phon-amnuaisuk, S. , & Tuson , A. ( 1998 ). Evolutionary Methods for Musical Composition . United Kingdom 06 - Biological and medical sciences.

16. Nierhaus , G. ( 2009 ). Algorithmic Composition: Paradigms of Automated Music Generation . Springer

17. Blaine , T. , & Perkis , T. ( 2000 ). The Jam-O-Drum interactive music system: a study in interaction design . In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques (DIS '00) . ACM.

18. Rowe , R. ( 1993 ). Interactive music systems: Machine listening and composition . MIT Press, Cambridge MA, 1993 .

19. Camurri , A. , Canepa , C. , & Volpe , G. ( 2007 ). Active listening to a virtual orchestra through an expressive gestural interface: The Orchestra Explorer . Proceedings of the 7th international conference on New interfaces for musical expression (pp. 56 - 61 ).

20. Camurri , A. , Varni , G. , & Volpe , G. ( 2010 ). Towards analysis of expressive gesture in groups of users: computational models of expressive social interaction. Gesture in Embodied Communication and Human-Computer Interaction , ( 1947 ), 122 - 133 .

21. Van Boerdonk K. , Tieben Rob , Klooster S., Van den Hoven E. , ( 2009 ). Contact through canvas: an entertaining encounter . Personal and Ubiquitous Computing . Springer.

22. Rasamimanana , N. , Bevilacqua , F. , Bloit , J. , Schnell , N. , Flty , E. , Cera , A. , Frechin , J.-L. , et al. ( 2012 ). The urban musical game: using sport balls as musical interfaces . CHI 2012 Proceedings , 1027 - 1030 .

23. Varni , G. , Mancini , M. , Volpe , G. , & Camurri , A. ( 2010 ). Sync'n'Move: social interaction based on music and gesture . User Centric Media , 31 - 38 .

24. Morreale , F. , Masu , R. , De Angeli , A. , Rota , P. ( 2013 ). The Music Room . CHI'13 Extended Abstracts on Human Factors in Computing Systems , 3099 - 3102 ( 2013 ).

25. Morreale , F. , Masu , R. , De Angeli , A. , Fava , P. ( 2013 ). The E ect of Expertise in Evaluating Emotions in Music . Proceedings of the 3rd International Conference on Music & Emotion.

26. Morreale , F. , Masu , R. , De Angeli , A. ( 2013 ). Robin: An Algorithmic Composer For Interactive Scenarios . Proceedings of Sound And Music Computing.

27. Blaine , T. , Fels , S. Contexts of collaborative musical experiences . In Proceedings of the 2003 conference on New interfaces for musical expression (NIME '03).