Music streaming companies collectively serve billions of songs per day. Radio-based music services may intersperse audio advertisements among the songs as a means to generate revenue, much like traditional FM radio. Regardless of the monetization approach, the recommender system should decide when to play content that the listener is known to enjoy (exploit) and content that is novel to the listener (explore). Recommender systems that rely on this explore/exploit type framework have been deployed in a wide variety of applications such as movies, books, music, shopping and more. In this work, we investigate the impact of dierent ad/song sequences on listener behavior. In particular, we focus on the impact of exploring new song content for the listener given the previous sequence of ads and songs in the listener's session. Our results show that the prior sequence maers when considering song exploration and that this prior sequence has an impact on the listener's tendency to interrupt their current session.
Recommender systems (RS) have been deployed in numerous domains including music, movies, e-commerce and books. In music recommendation, one of the overarching goals of the RS is to nd the best song to play for each listener, personalized to their specic taste(s) in music. In general, companies oering music recommendation services provide two dierent types of subscriptions: (1) Ad-supported membership where the music is free, but the listener is subject to advertisements and (2) premium membership where listener pays a monthly membership fee in exchange for ad-free listening. is paper focuses on the former, ad-supported listening. Unsurprisingly, listeners prefer hearing songs over ads. However, the business depends on the revenue that it makes from the ads and cannot operate without serving them. erefore, playing ads is crucial to keep the business alive and should be considered as a content served to the listener along with music.
One of the fundamental concepts in RS is the idea of exploration
and exploitation [
We have compiled data from a large-scale music recommendation service for our analysis. To nd the eect of dierent sequences of songs and ads on the probability of a user switching the station aer listening to an exploratory song, we looked at one million sessions on mobile devices where the ad placement had been made completely at random. Note that the randomness of ad placement is important in order to make sure our analysis is not biased toward any particular ad placement algorithm. We compare the impact of explore songs versus exploit songs in the context of the previous three events. For example, given the prior three events Ad, Song, Song, where each song is an exploit, what is the probability of the listener changing the station if the next song spun for them is an explore song versus the probability of station change given an exploit song? Station change is used as a proxy for discontent with the current stream of music.
We calculated the probabilities of users changing the station when they are exposed to dierent sequences of ads and songs as 208 64 104
SAA ASS Prior Event Sequence
ASA
AAS
AAA follows: there are a total of 8 possible event combinations for a set of three items as shown in gure 1. We denote explore song by S0 and exploit song by S. Station change is represented by, C.
Percentage Dierence = P (C j S 0) P (C j S) 100 (1) P (C j S) P (C j S 0) is the probability of a user changing the station given the last played content is an explore song. P (C j S) is the probability of a user changing the station when the last played content is an exploit song. e lower and upper condence bounds for the computed percentage increases, shown in gure 1 as vertical blue lines on top of the bars, are computed as follows,
P (C j S 0) P (C j S) 1:96 SE 100 (2)
P (C j S) where SE (i.e. the standard error) is calculated using equation 3, s P (C j S 0) (1
NS 0
P (C j S 0)) + P (C j S) (1
P (C j S))
(3) NS where NS 0 is the total number of times an explore song has been played. e total number of times an exploit song has been played is denoted by NS . Figure 1 shows the percent increase of station changes aer playing an explore versus an exploit song when a user has observed the respective prior sequence of exploit songs and ads. Due to the company’s data privacy policy, we have not included the individual probabilities of switching the station for explore and exploit songs, but have provided the probability dierence of change.
An exploit song is denoted by S and an ad is shown by A. As you can see, depending upon the previous sequence of songs and ads, the probability of a user switching the station when we show them an explore song is higher than the probability when we show an exploit song. is is true for all 8 dierent combinations of songs and ads. Moreover, some sequences are riskier than the others for placing an explore song. For example, the ASA sequence (which means playing an ad, then a song and then another ad) has the highest probability increase (+531.13%) of a user switching the station when given an explore song aer that sequence. Clearly, this is not the best opportunity to explore new content. On the other hand, the SAA sequence has the lowest probability increase (+64.42%), but is still positive. While playing an explore song is still riskier than an exploit song in all cases, it is beer to explore aer particular sequences over others. Certainly, dierent sequences of songs and ads have dierent eects on station switching behavior and a recommender system should try to take these sequences into account when doing exploration and exploitation, as in our sequential music recommendation system. Overarching, instead of a blind explore-exploit platform, we advise taking an intelligent approach that accounts for a listener’s state of listening (whether they are happy with the past couple of songs/ads or not) into account when deciding to exploit or when to explore.
e idea of explore-exploit has been studied in recommender
systems by some researchers [
In this work, we investigated the impact of dierent ad/song
sequences on listener behavior. In particular, we focused on the
impact of exploring new song content for the listener given the
previous set of ads and songs in the listener’s session. Our
experimental results show that the previous sequence of ads/songs
maers in deciding what the right time is for exploration versus
exploitation. For our future work, we will launch an A/B experiment
controlling for the placement of explore songs and see how
dierent users behave when they observe dierent sequences of songs
and ads. We will also investigate more sophisticated oine models,
such as HMMs and RNNs in a reinforcement learning seing that
could learn superior personalized playlist sequencing. is work is
a starting point for a larger project in which we aim to optimize the
stream of recommendations of mixed types of content (i.e. contents
from dierent stakeholders) [
We would like to thank Pandora Media, Inc. for access to their vastly rich dataset.