=Paper=
{{Paper
|id=None
|storemode=property
|title=Opening Musical Creativity to Non-Musicians
|pdfUrl=https://ceur-ws.org/Vol-1065/paper2.pdf
|volume=Vol-1065
|dblpUrl=https://dblp.org/rec/conf/chitaly/Morreale13
}}
==Opening Musical Creativity to Non-Musicians==
https://ceur-ws.org/Vol-1065/paper2.pdf
Opening musical creativity
to non-musicians
Fabio Morreale
Experiential Music Lab
Department of Information Engineering and Computer Science
University of Trento, Italy
Abstract. This paper gives an overview of my PhD research that aims
at contributing toward the definition of a class of interfaces for music
creation that target non-musicians. In particular, I am focusing on the
differences on design and evaluation procedures with respect to traditional
interfaces for music creation that are usually intended to be used by mu-
sicians. Supported by a number of preliminary findings we developed the
first interactive system: The Music Room is an interactive installation
which enables people to compose tonal music in pairs by communicating
emotion expressed by moving throughout a room.
Keywords: Musical interfaces, user-experience, performing art, active
listening.
1 Research questions
I am a third year PhD candidate at the HCI group of the University of Trento,
guided by Antonella De Angeli. The focus of my study is to design interactive
systems to allow everybody to experience musical creativity. So far, the inherent
complexity of music composition limits the access of traditional musical inter-
faces to musicians due to the extensive presence of musical notation. Novel tech-
nologies (e.g. tabletops, mobile apps, motion capture sensors) have been adopted
to replace traditional instruments with more intuitive devices [1, 2] and lead a
new set of design issues. As musical notation fails on giving everybody access to
music creation, what kind of interaction paradigm can be used in order to ease
this art to a wider and lay audience?
In order to answer this question, we explored new interaction metaphors that
have to meet a series of requirements: they have to be available to everybody,
intuitive, with a proper level of affordance and naturally connected with music.
Emotion seems to be the element that best meets these requirements. Music is
one of the arts that can most effectively elicit emotions [3, 4] and it has always
been connoted as emotional [5]. In interactive systems, emotions need to be
mediated by specific artefacts in order to be communicated to the system. Bodily
movements, which, in the different declinations of dancing and conducting, are
traditionally associated to music, can be the most appropriate medium through
which emotions can be conveyed [5].
Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org).
Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
2 Our contribution
The first interface we developed is The Music Room, an installation that provides
a space where people can compose music expressing their emotions through
movements. The visitors experience the installation in couple by informing the
system on the emotions they intend to elicit. The couple directs the generation
of music by providing information about the emotionality and the intensity of
the music they wish to create. To communicate emotions, the analogy with
love is used: the proximity between them affects the pleasantness of the music,
while their speed affects the dynamic and intensity. We decided to limit the
interaction dimensions to closeness and speed in order to keep the experience as
simple and intuitive as possible. Proxemics information is acquired by a vision
tracking system. It is then converted into emotional cues and finally passed to
the musical engine. These intuitive compositional rules provide everybody with
unlimited musical outcomes. As regard the generation of music, we developed
Robin, an algorithmic composer that composes original tonal music in piano 1 .
Fig. 1. The Music Room.
1
Examples of pieces generated at The Music Room can be listened at goo.gl/Ulhgz
Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org).
Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
3 Related works
This project spans several research areas. The adoption of the metaphor of ges-
tures and emotions is partially influenced by previous collaborative interactive
systems for music generation. The rules of the compositional system are founded
research on music perception, while Robin is inspired by existing approaches for
algorithmic composition.
3.1 Interactive Musical System
Research on the design of interactive systems for generative music has been
growing in the last decade. A number of tangible musical interfaces such as the
Reactable [1], Jam-O-Drum [17], and GarageBand for the iPad, target users
that have at least a minimum musical training as sonic and musical inputs are
adopted. A category of interfaces addresses users to collaborate. In particular,
several works exploit the concept of active listening, an approach where listeners
can interactively control the music content by modifying it in real-time while
listening to it [18, 19]. TouchMeDare aims at encouraging strangers to collab-
orate for reaching a common creative goal: pre-composed music samples are
triggered when both simultaneously touch a canvas [22]. In the Urban Musical
Game, users manipulate pre-composed music by playing with sensors-equipped
foam balls [21]. With Sync’n’Move music is also experienced by collaborative
means [23]. Two users freely move their mobile phones and the level of music
orchestration depends on the synchronization of their movements. In Mappe per
Affetti Erranti, a group of people can explore pre-composed music by navigating
a physical and emotional space [20]. Once again, collaborative situations are en-
couraged as music can only be listened to in its full complexity if the participants
cooperate.
3.2 Eliciting emotions in music
Related works suggest that the perception of emotions in music depends on com-
positional parameters (e.g. tempo, melody direction, mode) and performance
behaviors (articulations, timing, dynamics) whose combinations elicit different
emotional responses in the listener [5–7]. Measurement and classification of emo-
tions in music, most of the works in the music domain are operates on Russell’s
Circumplex model [8]. This model describes emotions as a continuum along two
dimensions: valence and arousal. In 1937, Hevren identified the most important
compositional factors in terms of emotions elicitation by labelling them on the
music’s expressive character [9]. Juslin and Sloboda later reviewed this cate-
gorization by representing the emotions along the valence/arousal dimensions
[10]. There is a consensus that at the compositional level, mode and rhythm are
responsible for valence, while tempo and dynamics are responsible for arousal.
Despite the remarkable amount of works on this area, no significant study has
been tried to understand to which extent expertise has a role on judging, ap-
preciating and perceiving musical pieces. How do non-musicians perceive and
Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org).
Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
describe music? What are the musical parameters and semantic elements that
are more relevant for them?
3.3 Algorithmic Composition
Generative music composition has been widely explored in the last decade. The
most common approaches are: rule-based, learning-based and evolutionary com-
position [10]. In rule-based approach, algorithmic rules inspired from music the-
ory are manually coded into the system [11, 12]. In learning-based approach, the
system is trained with existing musical excerpts and a number of rules are au-
tomatically included [13, 14]. Even though this method manages to decrease the
reliance on designer skills on music theory, the output heavily depends on the
training set. Lastly, evolutionary algorithms allow the creation of original and
complex melodies by means of computational approaches inspired by biological
evolution [15]. The generated music is original and unpredictable but it might
sound unnatural and lack ecological validity if compared to rule-based systems
that are generally superior in contexts of tonal music [16].
4 Results achieved
A number of personal contributions for each of the three research areas were
recently published. At the Interactivity session of CHI 2013, we demoed The
Music Room [24], whose objectives, development, findings and evaluation are
better discussed on the forthcoming publication at the Special Issue of Pervasive
and Ubiquitous Computing on Designing Collaborative Interactive Spaces.
The role of expertise on the evaluation of induced emotions in music was
analysed in a experiment we conducted in 2012 whose results were recently
published on Proceedings of ICME3 [25].
The details on the ideation and implementation of Robin, the algorithmic
composer, are going to be published at Proceedings of SMC2013 [26].
5 Future works
The last year of my PhD will be mainly devoted toward a formal definition of
interactive systems for music creation that target non-musicians. The first objec-
tive is to investigate similarities and differences with traditional digital musical
interfaces. By date, just a few studies highlighted the differences between inter-
faces for artistic experience and for musical expression but these works didn’t
have a follow-up in the last decade [27]. However, we believe that a number of
relevant differences exist. By combining personal intuitions with related liter-
ature findings, we propose a list of potential differences between the two sets.
Possibly, the output of this study will consist of a categorization of musical in-
terfaces. The idea is to exhaustively describe musical interfaces by means of a
model composed of a space of number of dimensions such as:
Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org).
Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
– Target user
– Ultimate goal
– Learning curve
– Interaction metaphor
– Level of direction
– Musical possibilities
– Role of the audience
The successive step would consist in testing the proposed dimensions with a
series of existing interfaces. Once validated, I will elaborate on defining a series
of evaluation principles for each dimension. This will allow interface designers
to position their projects on the model and to evaluate them consequently.
I’d also wish to tackle a number of challenges regarding The Music Room.
Even though the current implementation received a lot of interest, there is room
for several improvements. A number of innovations to music engine are currently
under development: the quality of music will be enhanced by introducing new
genres and instruments as well as by teaching Robin new compositional rules. I
also aim at further elaborating on the communication of intended emotions to the
system. Temporal aspects will be taken into consideration in order to determine
a general evolution of the experience, by considering recurrence of patterns of
moving close and getting far. Also, we are likely to introduce head pose tracking
in order to have information whether the two people are looking at each other.
This additional information will be used to differentiate the situations in which
the user are facing or turned and direct the music generation consequently.
References
1. Jordà, S., Geiger, G., Alonso, M., & Kaltenbrunner, M. (2007). The reacTable:
exploring the synergy between live music performance and tabletop tangible inter-
faces. Proceedings of the 1st international conference on Tangible and embedded
interaction (pp. 139-146). ACM.
2. Camurri, a., Volpe, G., De Poli, G., & Leman, M. (2005). Communicating expres-
siveness and affect in multimodal interactive systems. IEEE Multimedia, 12(1),
43-53.
3. Balkwill, L.-L., Thompson, W. F., & Matsunaga, R. I. E. (2004). Recognition of
emotion in Japanese, Western, and Hindustani music by Japanese listeners. Japanese
Psychological Research, 46(4), 337-349.
4. Fritz, T., Jentschke, S., Gosselin, N., Sammler, D., Peretz, I., Turner, R., Friederici,
A.D., Koelsch, S.,(2009). Universal Recognition of Three Basic Emotions in Music,
Current Biology, Volume 19, Issue 7, 14 April 2009, Pages 573-576, ISSN 0960-9822
5. Juslin P.N., & Sloboda J.A. (2009). Handbook of music and emotion: theory, re-
search, applications. Oxford University Press.
6. Temperley, D. (2001). The Cognition of Basic Musical Structures. Mit Press.
7. Gabrielsson, A., Lindstrm, E., (2001). The influence of musical structure on emo-
tional expression. Music and emotion: Theory and research. Series in affective sci-
ence., (pp. 223-248)
8. Russell, J. (1980). A circumplex model of affect. Journal of personality and social
psychology.
Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org).
Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.
9. Hevner, K. (1937). The Affective Value of Pitch and Tempo in Music. The American
Journal of Psychology, 49(4), 621-630 CR - Copyright & 169; 1937 University of
Illinois Press.
10. Todd, P. M., & Werner, G. M. (1999). Frankensteinian methods for evolutionary
music composition. Musical networks: Parallel distributed perception and perfor-
mance, 313-339.
11. Rader, G. (1974). A method for composing simple traditional music by computer.
IT press, Cambridge, MA
12. Steedman, M. (1984). A Generative grammar for jazz chord sequences. Music Per-
ception Vol. 2, No. 1, Fall, 1984
13. Simon, I., Morris, D., & Basu, S. (2008). MySong: automatic accompaniment gener-
ation for vocal melodies. Proceeding of the twenty-sixth annual SIGCHI conference
on Human factors in computing systems (pp. 725-734). ACM.
14. Hiller, L., Isaacson, L. (1992). Musical composition with a high-speed digital com-
puter, Machine models of music.
15. Wiggins, G., Papadopoulos, G., Phon-amnuaisuk, S., & Tuson, A. (1998). Evo-
lutionary Methods for Musical Composition. United Kingdom 06 - Biological and
medical sciences.
16. Nierhaus, G. (2009). Algorithmic Composition: Paradigms of Automated Music
Generation. Springer
17. Blaine, T., & Perkis, T. (2000). The Jam-O-Drum interactive music system: a study
in interaction design. In Proceedings of the 3rd conference on Designing interactive
systems: processes, practices, methods, and techniques (DIS ’00). ACM.
18. Rowe, R. (1993). Interactive music systems: Machine listening and composition.
MIT Press, Cambridge MA, 1993.
19. Camurri, A., Canepa, C., & Volpe, G. (2007). Active listening to a virtual orchestra
through an expressive gestural interface: The Orchestra Explorer. Proceedings of the
7th international conference on New interfaces for musical expression (pp. 56-61).
20. Camurri, A., Varni, G., & Volpe, G. (2010). Towards analysis of expressive gesture
in groups of users: computational models of expressive social interaction. Gesture
in Embodied Communication and Human-Computer Interaction, (1947), 122-133.
21. Van Boerdonk K.,Tieben Rob, Klooster S., Van den Hoven E., (2009). Contact
through canvas: an entertaining encounter. Personal and Ubiquitous Computing.
Springer.
22. Rasamimanana, N., Bevilacqua, F., Bloit, J., Schnell, N., Flty, E., Cera, A.,
Frechin, J.-L., et al. (2012). The urban musical game: using sport balls as musi-
cal interfaces. CHI 2012 Proceedings, 1027-1030.
23. Varni, G., Mancini, M., Volpe, G., & Camurri, A. (2010). Sync’n’Move: social
interaction based on music and gesture. User Centric Media, 31-38.
24. Morreale, F., Masu, R., De Angeli, A., Rota, P. (2013). The Music Room. CHI’13
Extended Abstracts on Human Factors in Computing Systems, 3099-3102 (2013).
25. Morreale, F., Masu, R., De Angeli, A., Fava, P. (2013). The Effect of Expertise in
Evaluating Emotions in Music. Proceedings of the 3rd International Conference on
Music & Emotion.
26. Morreale, F., Masu, R., De Angeli, A. (2013). Robin: An Algorithmic Composer
For Interactive Scenarios. Proceedings of Sound And Music Computing.
27. Blaine,T., Fels, S. Contexts of collaborative musical experiences. In Proceedings of
the 2003 conference on New interfaces for musical expression (NIME ’03).
Proc. of CHItaly 2013 Doctoral Consortium, Trento (Italy), September 16th 2013 (published at http://ceur-ws.org).
Copyright © 2013 for the individual papers by the papers' authors. Copying permitted for private and academic purposes.
This volume is published and copyrighted by its editors.