In this short paper we present work in progress that tries to leverage crowdsourced music metadata and crowdsourced afective word norms to create a comprehensive dataset of music emotions, which can be used for sentiment analyses in the music domain. We combine a mixture of diferent data sources to create a new dataset of 90,408 songs with their associated embeddings in Russell's model of afect, with the dimensions valence, dominance and arousal. In addition, we provide a Spotify ID for the songs, which can be used to add more metadata to the dataset via the Spotify API.
Related work As for related work on that topic, Delbouys et al. (2018), Yang & Chen (2011)
as well as Turnbull (2010) provide a concise overview of the diferent approaches to emotion
detection and classification in music. To that end, a multitude of datasets have been put
together over the years, with diferent machine learning approaches to try and classify them
based on numerous audio and non-audio features. This multitude of datasets is the result of a
lack of consensus as to what categories ought to be used, infamous copyright issues that hinder
the field, and the diversity of machine learning approaches that require diferent sorts of data.
Those studies that do not rely on small datasets of public domain music, such as Lu, Liu &
Zhang (2006) and Laurier, Grivolla & Herrera (2008), either use the Million Song Dataset
(MSD)
Focus of this work While we see the value of a lot of these existing approaches, we try
and adopt a method that does not rely on the inherent assumptions made by the respective
methods. Using already compiled datasets, like the MSD, forces us to deal with whatever bias
the original gathering method forces on the mood distribution of the ensuing dataset. Indeed,
some have noted a bias toward positively charged music when collecting the social tags
In this paper we present the MuSe (Music Sentiment) dataset, a collection of 90,408 songs, sentiment information and various metadata. We use the Allmusic mood taxonomy of music, the creation of which involves human experts, to seed a scraping of social tags-based Last.fm data which we then enrich with all available tags2. We then filter the additional tags based on a corpus of afect-related terms. We also provide the Spotify ID, which can be used to obtain all kinds of metadata via the Spotify API.
Conceptualizing the understanding of emotions in general and music in particular is not a new
problem. Diferent approaches to the subject involve diferent representations. Some are based
on a categorical approach that considers emotions as categories or classes which one experiences
separately (Yang and Chen 2011), or as a cluster of discrete categories
2A similar approach was used by Delbouys et al. (2018), who used a Last.fm-based filtering approach on the MDS to create a corpus of about 18k songs with afective information. Others use a dimensional approach, which can be achieved by asking subjects to rate musicrelated concepts on a continuous scale for several dimensions (Warriner, Kuperman & Brysbaert 2013). While a dimensional approach may include various factors of scale, such as tensionenergy, gaiety-gloom, solemnity-triviality, intensity-softness, pleasantness-unpleasantness, etc. (Yang and Chen 2011), Scherer (2004) shows that these various factors can mostly be reduced to three fundamental dimensions of emotion: 1. valence (pleasantness and positivity) 2. arousal (energy and stimulation) 3. dominance (potency and control)
This approach can be traced back to Russell’s seminal “circumplex model”
This section describes the diferent stages of creating a comprehensive dataset that contains songs along with values for the dimensions valence, arousal and dominance. Seed stage: Collecting mood labels from Allmusic.com We start building our dataset by collecting mood labels that are available from Allmusic.com3. There is a total of 279 labels that were created manually by Allmusic.com editors. These will serve as the starting point of our analysis and provide an overarching set of mood descriptors which we will use to guide our collecting of further tags. If we had let other factors guide our collection eforts, we might have ended up with emotional biases where certain moods are over-represented. This approach will ensure that we collect a balanced dataset that encompasses a large variety of moods.
which can be used to address diferent categories of musical emotions, we can use each of these terms to collect song objects from the Last.fm API. For every one of the 279 seed moods, we collect a maximum of 1,000 songs, which currently is the official limit of the Last.fm API.
For every single song, we then collect the top-100 tags4 assigned to it by the users of Last.fm5 as well as other metadata such as artist name, album name and number of listeners. Using this approach, we collected a total of 131,883 songs. We did not collect the theoretical amount of 279 ∗ 1, 000 = 279, 000 songs, because we could not find 1,000 songs on Last.fm for each of the 279 seed labels. The total number is also reduced, as some songs were collected for multiple tags (but obviously kept only once in the dataset). After removing duplicate songs, the corpus 3https://www.allmusic.com/moods; It is worth noting that the MIREX emotion categories were originally also derived by clustering the co-ocurrence of moods in songs from the Allmusic.com moods taxonomy. 4https://www.last.fm/api/show/track.getTopTags 5For some songs, the initial seed mood tags were not among the Last.fm top-100 tags. As we did not want to lose these seed moods, we included them for every song, regardless of their ranking on Last.fm. contains a total of 96,499 songs. These songs altogether have more than 3 million tags, from which about 261k tags are unique. At this stage, songs in our collection have a mean of 33.83 Last.fm tags (std: 31.01; min: 1, max: 100). The most frequent tags are typically genre-related tags, such as the top-3 tags rock (29,810 times), alternative (24,763 times) or indie (23,006). However, there are also afective tags among the highly frequent tags, for instance chill (11,841 times), sad (8,350 times) and melancholy (8,344 times).
the collected tags correspond to mood categories, we apply the pre-processing step described
by Hu, Downie & Ehmann (2009) and employed by Delbouys et al. (2018), which is to compare
the list of tags of a song to the lemmas of WordNet-Afect
From the 261k unique tags in our corpus, only 873 tags (for some example tags and their frequencies see Table 1) could be matched to the WordNet-Afect list, which altogether contains a collection of 1,606 afective words 6.
Mapping stage: Embedding mood labels into Russell space Having a way to filter out
tags based on whether or not they refer to mood leaves us with the task of estimating the
valence, arousal and dominance of each one of these tags. Warriner, Kuperman & Brysbaert
(2013) mention the hegemony of the ANEW norms dataset
We make use of the Warriner et al. (2013) wordlist – which we will refer to as the V-A-D list – to map the mood labels of our song collection to the dimensions valence, arousal and dominance. Each of the mood tags from each song is looked up in the V-A-D list and is assigned a 3-dimensional coordinates triple for a word. If the word is not present, it will return a value of 0 for each dimension. From the overall 1,827 unique mood labels in our dataset, only 765 (41.9%) are matched with the lemmas in the V-A-D list. Taking a closer look at the mood tags collected from Last.fm and the V-A-D list, it becomes clear that a basic lemmatization procedure might further improve the match7. We will look into this optimization as we will further develop the dataset. For now we were content to observe that, although many of the Last.fm tags did not match with the V-A-D lemmas, these were typically tags with very low frequencies. In total, 88.1% of the mood tags used to characterize the songs in our dataset are actually matched with the V-A-D list, i.e. Last.fm tags that occur frequently are typically already in lemmatized form and thus have a higher chance of matching the V-A-D list.
6As we believe our initial seed moods collected from Allmusic.com are important afect words in any case, we add those that are not already included on top of the WordNet-Afect list, resulting in a total of 1,827 afect tags.
7Some examples: fevered (Last.fm)/fever (V-A-D), horrify/horrific, sickish/sick, peacefulness/peaceful
From the overall 96,499 songs, 90,408 (93,7%) are matched with a least one tag that is also present in the V-A-D list, with a mean of 3.36 mood tags (std: 3.71; min: 1, max: 42). As songs typically will have multiple mood tags, we calculate the weighted average for every dimension of every word separately, whereas each word is weighted according to scores we retrieved for each tag via the Last.fm API (see Figure 1 for an example). Higher Last.fm weights indicate a higher relevance of a tag.
afective metadata we now add the Spotify ID whenever we were able to retrieve one. In order to find the appropriate ID for each of the songs, we performed an API lookup for a track based on its title and artist. Some of the problems we encountered here include untranslated artist names for languages such as Japanese or Korean, as well as search queries that are too wordy. In the end, we were able to track down a Spotify ID for a total of 61,484 songs. This allows us to append any additional information to our dataset that is available via the Spotify API8, for instance: • metadata: release date, popularity, available markets, explicit lyrics (boolean), etc.
Title Moop Bears Await The King’s Justice Et Tu Magnolia The Angels Eternity Cupid De Locke Breaking News Reflections I Love A Man In A Uniform Mosquito Brain Surgery ...
Artist Momus Ramin Djawadi Siddharta The Hush Sound Melissa Etheridge Robbie Williams 000PUfi7X3otImjyjJXvFS ... [naive] 001VMKfkHZrlyj7JlQbQFL ... [dramatic, joy,
score] 002SF61pDJexw3oSRcUgnE ... [monumental,
easy] 004VU4cWTkRqVMrlv8KW3D ... [introspective, optimism, love, cool] 004ddQGTS8w7sDEKuwXZhi ... [earthy] 7zw9OxtVotLfxlfavSADXQ The Smashing 7zwwvrJAWGjfc9wFD3bVzZ Pumpkins Anuj Rastogi
7zxG55mLhPAx2EIMj6loEg Gang of Four 7zy6jG8RIUI8qNYYVuLGbY
... [cynical, cynical] Spastic Ink 7zzbwY4h8Q6kI1ZX3gB1B3
... [complex] ...
... • low-level audio features: bars, beats, sections, duration, etc. • mid-level audio features: acousticness, danceability, tempo, energy, valence9, etc.
The final song dataset (for a snippet see Table 2) contains basic metadata, such as artist, title and genre, as well as the mood tags, the three afective dimensions as well as the Spotify ID that allows us to further extend the dataset with additional information via the Spotify API. Figures 2 and 3 plot the overall distribution of diferent artists and genres in the dataset.
Figure 4 shows a Pearson correlation matrix for the three afective dimensions and various Spotify parameters. While the matrix reveals a strong correlation between Spotify features such as loudness and energy (0.78) or acousticness and energy (-0.74), there seems to be no positive or negative correlation between valence / arousal / dominance and other acoustic features. However, this might well be diferent, if we calculate correlations for diferent genres. Another interesting insight is that Spotify’s valence value – which very generally describes a song’s acoustic sentiment – is only very weakly correlated with our valence / arousal / dominance dimensions, indicating that sentiment based on user generated tags is diferent to sentiment that is exclusively derived from audio features. Another interesting observation in the correlation plot is the strong correlation of the valence and dominance values (0.87) we calculated. This suggests that the emotion data might be plotted also in a 2-dimensional way, without loosing to much information on the general distribution of songs. Figure 5 shows an example that plots all songs in our collection on the axes arousal and valence, indicating a slightly positive trend for most of the songs, which aligns with the notion of a bias toward 9Note: Energy and valence obviously point toward a similar direction as our calculated arousal and valence dimensions. We want to stress that Spotify uses audio features exlusively while we make use of user-generated tags, which might be considered a more holistic description of a song. positively charged music by Cano & Morisio (2017).
Moretti’s (2000) notion of distant reading – as an antipode to the more traditional close reading – by now has become a dictum in the Digital Humanities, summarizing any kind of computational, empirical approach to analyzing text. Tilton & Arnold (2019) even adopted the concept for the field of image and video analysis, proclaiming a distant viewing paradigm. Along these lines, it feels natural to coin distant hearing as a corresponding analogy for the ifeld of music.
In this paper we have described the reasonings and practical steps for the creation of a music sentiment dataset that is not solely based on an analysis of lyrics or audio features, but rather takes into account actual human judgement of a song’s emotional characteristics by mining user-generated mood tags from the social music platform Last.fm. With our current MuSe dataset we provide a resource that enables diferent kinds of research questions that may be subsumed as afective distant hearing . These research questions may extend existing studies, such as Elvers’ (2018) sentiment analysis of European chart music or Argstatter’s (2015) study of emotions in music across-cultures. Further empirical research questions that come to mind might investigate the relation of music emotion and genres, gender (either of the artist or the predominant audience), chart placement, artist collaborations or audio features available via the Spotify API, e.g. danceability or acousticness.
The current dataset is available upon request. Although it can already be used to empirically investigate research questions in music emotions, we consider this to be work in progress and plan to further enhance the dataset in the near future. More concretely, we think about extending the scope of songs we collect from Last.fm, by enhancing the number of seed tags by the 873 mood tags we identified in our current dataset. We also plan to enhance the match between Last.fm tags and the V-A-D list by experimenting with lemmatization procedures. In addition, we plan to integrate further metadata, for instance from Discogs10 and MusicBrainz11, into the dataset. The latter seems particularly interesting, as Last.fm readily provides a MusicBrainz ID for many songs. All in all, we hope the MuSe dataset will help to advance the field of computational musicology and thus provide an incentive for more quantitative studies on the role of emotions in music.