“I’d like it to do the opposite”: Music-Making Between Recommendation and Obstruction Peter Knees1 , Kristina Andersen2 , and Marko Tkalčič1 1 Dept. of Computational Perception, Johannes Kepler University Linz, Austria 2 Studio for Electro-Instrumental Music (STEIM), Amsterdam, the Netherlands peter.knees@jku.at,kristina@steim.nl,marko.tkalcic@jku.at Abstract. To build new and rethink existing interfaces for producing and working creatively with electronic music, we are engaged in an on- going conversation with professional musicians. With regard to getting suggestions on musical material and progression by the machine on, e.g., samples, loops, rhythmic patterns, or musical structure, we find evidence that the notion of an algorithmic recommendation system should be ex- tended through the use of artistic obstruction. We propose the concept of “strangeness” as an addition to recommendation systems to allow the adjustment of the degree of desired otherness in the suggestions made. This marks an important difference to existing consumer-centred recom- mendation approaches, going even beyond the notion of serendipity. 1 Motivation The majority of today’s electronic music is created from pre-recorded or live- generated sound material. This process often combines sound loops and samples with synthesized and processed elements. For supporting the creator in the pro- cess of finding suited sound material to express musical ideas, content-based audio recommendation and retrieval techniques can be applied, e.g., [2]. How- ever, these methods are often tailored to music consumers rather than creators, resulting is limited uptake in the professional domain. To increase acceptance of these methods also for the creative acts of music making and composing, within the GiantSteps project,1 we are engaging in an ongoing conversation with pro- fessional musicians to tailor musical tools to their needs. From 16 open-ended interviews with young electronic musicians and produc- ers conducted in the context of the Red Bull Music Academy 2014, a number of ideas have emerged with regard to systems for collaboration, intelligent and intuitive organisation of personal sound collections, and intelligent composition support. In this paper we highlight one particular idea that emerged in the con- text of discussing the usefulness of automatic suggestions on musical material and progression by the machine, e.g., samples, loops, rhythmic patterns, or musi- cal structure, namely the desire to extend algorithmic recommendation systems with the concept of artistic obstruction. We present exemplary quotes from the 1 http://giantsteps-project.eu 2 P. Knees, K. Andersen, and M. Tkalčič interviews as evidence and contextualise the findings with respect to artistic practise and recommender systems research. Finally, we propose the concept of “strangeness” as an addition to recommendation systems to allow the adjust- ment of the degree of desired otherness in the suggestions made. This request for the creative domain marks an important difference to existing consumer-centred music recommendation approaches but might also inspire new developments in mainstream recommendation systems. 2 Recommendation for Creatives In relation to supportive and recommendation systems, i.e., to the question “how do we want the computer to ‘help’ us in our creative work process?”, beside issues of artistic control and the fear of making predictable sounds, it becomes apparent that the desired features of recommenders in a creative context go beyond the query-by-example-centred paradigm of finding similar items and even also beyond the goal of serendipitous suggestions. “Part of making music is about being lost a little bit and accidentally stumbling upon stuff that you didn’t think would work.” USER07 This first quote highlights a key element of much creative work, the ele- ment of the accidental, sometimes caused by the positive and negative effects of malfunctioning of sound-editing software. It shows that serendipity is highly important to support creative work. “It could be something like Google image search for example. [...] So if I set it to 100% precise I want it to find exactly what I am searching for and probably I will not find anything, but maybe if I instruct him for 15% and I input a beat or a musical phrase and it searches my samples for that, that could be interesting.” USER03 Starting from an established retrieval scenario, the artist expresses the pref- erence that the difference between the query and the results is an explicit factor that can be controlled. “What I would probably rather want it would do is make it complex in a way that I appreciate, like I would be more interested in something that made me sound like the opposite of me ... but within the boundaries of what I like, because that’s useful. Cause I can’t do that on my own, it’s like having a band mate basically. [...] I am happy for it to make sugges- tions, especially if I can ignore them, but for it to just make suggestions that I would have never come up with, but would wish that I had come up with.” USER07 “I’d like it to do the opposite actually, because the point is to get a pos- sibility, I mean I can already make it sound like me, it’s easy.” USER01 Music-Making Between Recommendation and Obstruction 3 These statements bear again the notion of controlled difference, including the idea of complete opposition. Instead of wanting the machine to propose a common or well understood solution, the artists would like it to perform the role of “the other”, providing an alternative or divergent view on a piece of music. So the desired functionality of the machine is to provide an alter-ego of sorts, which provides the artist with opposite suggestions, that still reside within the artist’s idea of his own personal style. Note that this is in contrast to many content- based recommendation use cases and systems that try to mimic and predict the behaviour of the user. “The listening from outside the room is such a good one, turning around and the stereo just flipped, in terms of squinting, I often do this thing of listening through, I work on a track for half an hour through like horrific filters, so like the EQ feels all fucked up and putting it through a gramophone, and then just switching it off...” USER16 Here a user lists a number of artistic strategies of “obstruction” he is already using to assess the quality of a piece, by changing the perception of the freshly edited music through changes in acoustics and hardware to render the piece “strange”. 3 Contextualisation The idea of making things different and “strange” is a well-known cornerstone in modern art strategies. The technique is known as “defamiliarisation” as defined by Shklovsky [10] and is a basic artistic strategy central to both Surrealism and Dada. It is based on the idea that the act of experiencing something occurs in- side the moment of perceiving it and that the further you confuse or otherwise prolong the moment of arriving at an understanding, the deeper or more detailed that understanding will be. This technique and the findings from the interviews can be directly translated into new requirements for recommendation engines in music making. This need for opposition goes far beyond the commonly known and often ad- dressed needs for diversity, novelty, and serendipity in recommendation system research, which has identified purely similarity-based recommendation as a short- coming that leads to decreased user satisfaction and monotony [11]. This phe- nomenon spans all domains: from news articles [7] to photos [8] to movies [6] to music recordings [4,15]. One idea proposed to increase diversity is to subvert the basic idea of collaborative filtering systems of recommending what people with similar interests found interesting by recommending the opposite of what the least similar users (the k-furthest neighbors) want [9]. Indeed it could be shown that this technique allows to increase diversity among relevant suggestions. The common theme of this research is that diversity should not significantly harm the accuracy of the approaches, i.e., items need to be relevant [12], how- ever heterogeneous. Diversity is thus merely aimed at providing “the user with optimal coverage of the information space in the vicinity of their query” [11]. 4 P. Knees, K. Andersen, and M. Tkalčič Ultimately, this still bears the notion of similarity to preserve the close context, which might constitute one of the biggest differences between media consump- tion and creation, where challenge, change of context, and questioning are the desired qualities rather than “more of the same”. However, the aspect of recommending the opposite has not received much attention, potentially caused by a lack or insufficiency of technical definitions of terms such as “unexpectedness”, “serendipity”, or “utility” (which is often defined through economic factors rather than a real value for the user — not to speak of creative output) as well as the non-trivial notion of “the other”, which first requires defining the full space of possible actions. While proximity of data items can be defined rather intuitively based on a similarity metric (e.g., the nearest neighbours), remoteness is a less applicable concept. Particularly in high- dimensional data spaces, similarity-based rankings become somewhat meaning- less at higher ranks, as many points can have comparable distances (however, in different directions, making them mutually dissimilar, thus inconsistent). In the context of experimental music creation, Collins has addressed the ques- tion of opposition in the “Contrary Motion” system [5] using a low-dimensional representation of rhythm. The system opposes a piano player’s rhythm in real time by constructing a structure located in the space of actions “where the hu- man is likely not to be” [5]. The hypothesis underlying the system is that being confronted with an oppositional music style can be stimulating for a musician. Experiments where the opposing structure is sonified using a different instrument have indeed shown that musicians start to experiment and play with the oppos- ing agent. The quotes presented in section 2 further support this hypothesis. 4 Conclusions and Proposed Concept Inspired by both the input from our expert users and the inherent difficulty in creating effective search algorithms, and picking up the idea of controlled difference to the point of complete otherness in the context of electronic music production, we propose to include “strangeness” as a controllable parameter, e.g., through a dedicated dial or slider. The concept of “strangeness” is related to concepts like serendipity and diver- sity [3] and even more to unexpectedness [1] in consumer-oriented recommender systems in that the item space and its niches need to be explored in order to find desired results. However, while end consumers of, e.g., movies, might be more or less inclined to watch something different and consider it a satisfactory result, music composers and other creatives want to be challenged. For end users the consumption ends with the item (i.e., at the end of a movie or a song) while for artists the recommendation is just the beginning of a creative process that will be assessed way after the recommendation has been given. This includes the possibility of having a first negative opinion about the recommendation (equal to a low rating in consumer-based recommenders) in order to spark a creative streak that eventually results in being content with the given recommendation (high rating). Thus, for strangeness, the goal is not to optimize for immediate Music-Making Between Recommendation and Obstruction 5 liking or usefulness but primarily in giving results that help in reflecting upon the whole process. Unexpectedness, as defined in [1], plays a role in this process, however the ideas of controlled strangeness and desired opposition also encom- pass the possibility of including “irrelevant” results, i.e., results whose “utility” value might only become apparent after some undefined time. “Strange results” — as well as complete opposition, as the extreme form of strangeness — could therefore also help in appreciating other results in a different way, after being ex- posed to the strange version. This could be understood as a set recommendation problem, where the user’s utility function is uncertain, because of the vagueness of the feedback provided eventually [13]. In terms of user interaction, this concept could, for instance, extend a rec- ommender system such that it allows the artist to query the machine based on existing musical material, but instead of returning just similar sound files, it facilitates setting the level of “strangeness” the artist would like for the results. Using the same or a comparable algorithm as one would have traditionally used for a recommendation of a similar file, the system hence lets the artist set an explicit level of otherness, allowing to preserve artistic control between variation and opposition. By re-introducing the art strategy of defamiliarisation we hope to propose an interface for working creatively with music without creating a sys- tem of sameness and consensus. We are aware that this leads to new questions regarding (semantic) descriptions of similarity spaces and personalised and con- textual definitions of “the other”, which need further investigation to become applicable. Another consideration that needs to be made, when factoring in the findings of this paper, is that the degree of opposition and strangeness desired depends on the preferences and the working style of the music maker. Taking personality traits into account is a growing topic in recommendation [14] and the proposed concept, being highly user-specific in a domain where user individ- uality is a key factor, would benefit from being defined around user properties and personality aspects from the very beginning. Beyond our use case, we also see controlled “strangeness” as a potential extension for other, non-creative domains. As commercial recommenders have become a ubiquitous (and seldom inspirational) commodity, opposition to the user might be a way to make them exciting again and provoke interaction. 5 Acknowledgments This work is supported by the European Union 7th Framework Programme FP7/2007-2013 through the GiantSteps project (grant agreement no. 610591). References 1. Adamopoulos, P., Tuzhilin, A.: On unexpectedness in recommender systems: Or how to better expect the unexpected. ACM Transactions on Intelligent Systems and Technology 5(4), 54:1–54:32 (Dec 2014), http://doi.acm.org/10.1145/2559952 6 P. Knees, K. Andersen, and M. Tkalčič 2. Casey, M.A., Veltkamp, R., Goto, M., Leman, M., Rhodes, C., Slaney, M.: Content- Based Music Information Retrieval: Current Directions and Future Challenges. Proceedings of the IEEE 96, 668–696 (April 2008) 3. Castells, P., Hurley, N., Vargas, S.: Novelty and diversity in recommender systems. In: Ricci, F., Rokach, L., Shapira, B. (eds.) Recommender Systems Handbook. Springer, 2nd edn. (2015) 4. Celma, O., Herrera, P.: A new approach to evaluating novel recommendations. In: Proceedings of the 2008 ACM Conference on Recommender Systems. pp. 179–186. RecSys ’08, ACM, New York, NY, USA (2008), http://doi.acm.org/10.1145/ 1454008.1454038 5. Collins, N.: Contrary motion : An oppositional interactive music system. In: Beil- harz, K., Bongers, B., Johnston, A., Ferguson, S. (eds.) Proceedings of the Interna- tional Conference on New Interfaces for Musical Expression. pp. 125–129. Sydney, Australia (2010), http://www.nime.org/proceedings/2010/nime2010_125.pdf 6. Di Noia, T., Ostuni, V.C., Rosati, J., Tomeo, P., Di Sciascio, E.: An analysis of users’ propensity toward diversity in recommendations. In: Proceedings of the 8th ACM Conference on Recommender Systems. pp. 285–288. RecSys ’14, ACM, New York, NY, USA (2014), http://doi.acm.org/10.1145/2645710.2645774 7. Li, L., Wang, D., Li, T., Knox, D., Padmanabhan, B.: Scene: A scalable two- stage personalized news recommendation system. In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval. pp. 125–134. SIGIR ’11, ACM, New York, NY, USA (2011), http://doi.acm.org/10.1145/2009916.2009937 8. Radu, A.L., Ionescu, B., Menéndez, M., Stöttinger, J., Giunchiglia, F., De An- geli, A.: A hybrid machine-crowd approach to photo retrieval result diversification. In: Gurrin, C., Hopfgartner, F., Hurst, W., Johansen, H., Lee, H., O’Connor, N. (eds.) MultiMedia Modeling, Lecture Notes in Computer Science, vol. 8325, pp. 25–36. Springer International Publishing (2014), http://dx.doi.org/10.1007/ 978-3-319-04114-8_3 9. Said, A., Fields, B., Jain, B.J., Albayrak, S.: User-centric evaluation of a k-furthest neighbor collaborative filtering recommender algorithm. In: Proceedings of the 2013 Conference on Computer Supported Cooperative Work. pp. 1399–1408. CSCW ’13, ACM, New York, NY, USA (2013), http://doi.acm.org/10.1145/2441776. 2441933 10. Shklovsky, V.B.: Art as technique (1917). In: Lemon, L.T., Reis, M.J. (eds.) Rus- sian Formalist Criticism: Four Essays. University of Nebraska Press, Lincoln, USA (1965) 11. Smyth, B., McClave, P.: Similarity vs. diversity. In: Proceedings of the 4th Interna- tional Conference on Case-Based Reasoning: Case-Based Reasoning Research and Development. pp. 347–361. ICCBR ’01, Springer-Verlag, London, UK, UK (2001), http://dl.acm.org/citation.cfm?id=646268.758890 12. Vargas, S., Castells, P.: Rank and relevance in novelty and diversity metrics for recommender systems. In: Proceedings of the Fifth ACM Conference on Recom- mender Systems. pp. 109–116. RecSys ’11, ACM, New York, NY, USA (2011), http://doi.acm.org/10.1145/2043932.2043955 13. Viappiani, P., Boutilier, C.: Regret-based optimal recommendation sets in con- versational recommender systems. In: Proceedings of the Third ACM Conference on Recommender Systems. pp. 101–108. RecSys ’09, ACM, New York, NY, USA (2009), http://doi.acm.org/10.1145/1639714.1639732 Music-Making Between Recommendation and Obstruction 7 14. Wu, W., Chen, L., He, L.: Using personality to adjust diversity in recommender systems. In: Proceedings of the 24th ACM Conference on Hypertext and Social Media. pp. 225–229. HT ’13, ACM, New York, NY, USA (2013), http://doi.acm. org/10.1145/2481492.2481521 15. Zhang, Y.C., Séaghdha, D.O., Quercia, D., Jambor, T.: Auralist: Introducing serendipity into music recommendation. In: Proceedings of the Fifth ACM Interna- tional Conference on Web Search and Data Mining. pp. 13–22. WSDM ’12, ACM, New York, NY, USA (2012), http://doi.acm.org/10.1145/2124295.2124300