=Paper=
{{Paper
|id=None
|storemode=property
|title=Gaining Musical Insights: Visualizing Multiple Listening Histories
|pdfUrl=https://ceur-ws.org/Vol-565/paper7.pdf
|volume=Vol-565
}}
==Gaining Musical Insights: Visualizing Multiple Listening Histories==
https://ceur-ws.org/Vol-565/paper7.pdf
Gaining Musical Insights: Visualizing Multiple
Listening Histories
Ya-Xi Chen Dominikus Baur Andreas Butz
Media Informatics Media Informatics Media Informatics
University of Munich University of Munich University of Munich
yaxi.chen@ifi.lmu.de dominikus.baur@ifi.lmu.de andreas.butz@ifi.lmu.de
ABSTRACT ists/tracks in different time periods provides an overview of
Listening histories are rich sources of implicit information. one’s taste in music. Last.fm's large collection of data also
Combining listening histories of multiple users allows allows automation of social music tasks: Using collabora-
comparisons in musical taste and discovery of new music, tive filtering, recommendations of similar music can be
though few existing work specifically addresses this issue. generated from the user’s profile and users with similar
In this paper we present two interactive visualizations taste (so-called "neighbors") can be determined. Still,
which give users a deeper insight into consent and dissent Last.fm's web-bound services are mostly textual and limited
in their listening behaviors, and help them to compare their (for neighbors, overlaps in artists is all that is given as in-
musical tastes. HisFlocks shows overlaps in genre and artist formation on similarity). To grant users access to this vast
in certain time periods and LoomFM illustrates sequential data-set, graphical visualizations are needed to let them
listening patterns. Our first feedback on these systems was understand personal listening behavior, re-discover forgot-
very promising and we plan to extend our concepts to ten music and understand others’ taste in music.
broader scenarios with a greater number of listening histo-
ries. In this paper, we explore different interactive visualization
concepts for multiple users’ listening histories and propose
Author Keywords two prototypes which facilitate the comparison of multiple
Listening history, visualization, multiple users, Last.fm, users’ musical tastes.
time-based visualization. There are some existing visualization tools regarding time-
based data and specifically listening history. Time-based
ACM Classification Keywords data is mostly visualized as a temporal flow which indicates
H.5.2. User Interfaces: Graphical User Interfaces (GUI). hidden trends over time. ThemeRiver [6] introduces a river
metaphor to represent topical changes within a large collec-
INTRODUCTION tion of documents. Similarly, most of the existing visualiza-
"What kind of music do you like?" is a common question tions for personal listening histories use the flow metaphor.
while getting to know someone. Learning about musical Extra Stats 1 , LastGraph 2 and Last.fm Spiral 3 all extract
taste is appealing and there are various effects of social artist or genre trends from consumed songs and show their
bonding surrounding music [2, 4]. Still, describing one's impact as waves along a time-line. In our former work [1],
musical taste in a short and precise way is hard, so mostly we deviate from this trend by visualizing listening histories
the above question is answered vaguely, for example by "I either as chains of songs based on self-contained listening
like all kinds of music". Online services like Last.fm can sessions or based on listening proximity. Last.fm Explorer
help here: The accompanying Audioscrobbler plugin auto- [7] is one of the few works that allows showing two users'
matically logs all music that the user listened to and pub- listening histories as two respective graphs (see Figure 1)
lishes it in his/her profile page, thus making music con- which can be displayed on 3 different levels based on
sumption social without additional effort. This personal Last.fm tags, artists or tracks.
profile with lists of tracks recently listened to and top art-
In order to gain more insight into the effective use of meta-
data contributed by the users, we discussed with 13 Last.fm
users about the issue of musical taste and listening history.
During the interview, the participants were shown the
1
http://build.last.fm/item/34
Workshop on Visual Interfaces to the Social and Semantic Web
2
(VISSW2010), IUI2010, Feb 7, 2010, Hong Kong, China. http://lastgraph3.aeracode.org/
Copyright is held by the author/owner(s). 3
http://build.last.fm/item/377
1
Last.fm Explorer visualization result of their own listening Time period consistency: Since a listening history is time-
history and one of another participant. The concept of visu- based, the visualization should be built with a global time-
alization of multiple users’ listening histories received posi- line. Each listening history should be aligned to facilitate
tive feedback. It was appreciated that the history can be comparisons within the same time period.
displayed in three different levels. Participants appreciated
Good overview: The visualization should offer a good
that they could control the visualization with interactions
overview especially for long listening histories, so the user
such as filtering, stacking and brushing. But their comments
can get a general impression at first glance without continu-
revealed that Last.fm Explorer's focus on a single user's
ously switching back and forth.
listening history leads to some intrinsic issues in need of
improvement for visualizing multiple histories: As shown Managing complexity: Music-related information is com-
in Figure 1, only a recent part of history can be shown in plex and has metadata on various levels. The most promi-
the limited space, so the two timelines might be misaligned nent ones are genre, artist and track. Combined with infor-
especially when one user did not listen to music at the be- mation about the consumption behavior, such as listening
ginning or end of the dataset. Although information can be histories, the complexity steadily increases. Therefore it is
displayed on three different levels, the lack of zooming essential to support the user's task specifically comparison
might cause users to lose the current context. It is not possi- by focusing on key data elements and controls.
ble, for example, to learn an artist's genre by switching from
artist to genre view, because the size and placement of Easy to compare: Multiple listening histories should be
represented in one interface to facilitate convenient com-
items change. The main issue of Last.fm Explorer claimed
by the participants, however, is that it is difficult to actually parison between different users.
compare user's taste in music: The stacked graphs make the Based on these design principles, we developed two differ-
impact of certain artists or tracks hard to read and finding ent visualizations, Last.fm HisFlocks and LoomFM, aiming
correlations in two graphs can only be performed by a at different aspects: HisFlocks gives users an overview of
lengthy visual search process. relations in genre and artist in certain time periods, while
LoomFM allows a comparison of consumption patterns
over time. We will present these two applications here.
Last.fm HisFlocks
Since the genre and artist level provide a better overview
than the track level, we chose genre and artist as the main
categories and visualized this general information in our
first interface Last.fm HisFlocks (see Figure 2).
In HisFlocks, the entire history is represented as a series of
time frames with all the artists listened to in each time pe-
riod being displayed on a genre map, where similar artists
are grouped into clusters, labeled with genre names. By
default, the time interval between each frame is one week,
but this can be adjusted freely. Within a frame, each artist’s
name is color-coded (1 color per compared user) and its
size represents the play count. Shared artists are further
grouped into highlighted sub-clusters. The user can pan and
Figure 1. Last.fm Explorer: visualizations for two users. zoom to navigate within the map. In different frames, the
http://alex.turnlav.net/last_fm_explorer/ same artist always appears at the same position. As Figure 2
Although existing visualization tools receive positive feed- shows, an overview for all the artists listened to is provided
back, most of them target single users and present static, in the first frame and by dragging the slider, the user can
non-interactive images. There is a lack of tools for brows- browse all frames sequentially in which the artists being
ing and comparing multiple users’ listening histories di- listened to in this time period are highlighted and others
rectly instead of tediously clicking through web pages. fade out. The user can easily see the change of an artist, for
example, by appearance, disappearance and growth of size,
VISUALIZATION OF MULTIPLE USERS’ LISTENING while she or he adjusts the time slider, and thus have an
HISTORIES understanding of how the musical taste changes over time.
Based on the analysis of existing tools and results of the In Figure 1, the two users share a lot in metal, which is scat-
interview, we gathered the following design principles for tered in some sub-genres such as heavy metal, gothic metal
visualizing multiple users’ listening histories within an in- and melodic death metal, and thus cannot be derived at first
teractive interface: glance. Therefore, some aggregation of these tags could
improve the understandability of musical similarity on the
genre level. In HisFlocks, all the artists in one frame are
Figure 2. Last.fm HisFlocks.
grouped into genre-clusters based on their most representa- same time period. For example, in one time period as Fig-
tive genre-tags. This organization is based on a semantic ure 3 shows, two users both listened to 3 common artists
analysis which determines the structure and position for (Killswitch Engage, Children of Bodom, Dream Theater),
each genre and artist. We applied text analysis to create a but only one (Dream Theater) was exactly listened to by
semantic clustering of similar tags while excluding redun- both users in this time period. Besides, only the “red” user
dancy: After removal of different separators, such as “_” listened to “Symphony X”.
and “&”, the Porter algorithm [8] is applied to detect the
stem for each tag. Tags with the same stemmed words will
be clustered in the same group: The aforementioned metal-
related ones are grouped into one metal cluster, thus provid-
ing an abstract overview for general users. The locations of
each genre and artist are determined by the semantic simi-
larity. This semantic similarity (Sim) between tag A and B
equals the ratio between the number of resources in which
tags co-occur and the number of resources in which any of Figure 3. Detailed view in one frame.
the two tags appears (see Equation 1). LoomFM
Sim ( A, B ) =| A I B | / | A U B | (1) HisFlocks focuses on displaying artists on an abstract genre
map while hiding the “sequential”, track-based aspects of
With the semantic analysis all the tags will be well organ- listening behavior: How many songs are consumed in one
ized: the initial location of each tag is assigned by means of session? How often are certain songs listened to? Are there
a 2D projection based on a multidimensional scaling of co- repeating patterns within this list (such as albums or pre-
occurrences (see [3] for more detail). defined playlists) and large gaps (due to vacations etc.)?
The overview shown in Figure 2 indicates that both users To address these issues we designed the complementary
listen to rock and metal a lot, and they do share many artists visualization LoomFM (see Figures 4 and 5). A horizontal
in these two genres. Other genres they also listen to are timeline displays the temporal scope of two listening histo-
punk, industrial and electro. Only the user with the blue ries. Based on that, songs of one user are displayed above
color listens to folk, pop and jazz. Although they have many and of the other below this timeline and symbolized as
artists in common, they listen to few of them at the exact small circles (hovering with the mouse above an icon
gives more detailed information about what song it is and
3
Figure 4. LoomFM global view. Repetitions and main musical trends become visible (the white box shows the borders of Figure 5).
when and by whom it was listened to). The distance of a defined playlists or albums). In Figure 4 the release of a
song icon to the middle timeline represents the overlap with new album by a shared artist leads to repeated listening by
the other history: similar songs will be shown with their both users (two yellow arc bundles to the left of indie rock).
titles and displayed very close to the center, while less simi-
Artist name and genre of each song are displayed beyond
lar songs that are only represented by their artists or even
the user timelines. The higher the number of consecutive
their genres are further away (with a completely unrelated
songs sharing the same genre or artist, the larger the label,
song at the farthest). The up and down of the song circles
which makes general trends appear at the maximum zoom
thus represents the general similarity between the two users.
level (see Figure 4: indie and electronic as genres for user 2
and indie rock as a common genre).
The user can again navigate the displayed song plane by
panning and zooming. Zooming out gives an overview of
the complete listening histories with the most influential
genres and artists still visible.
SUMMARY AND FUTURE WORK
Music listening histories and their accompanying metadata
are complex information that can bring rich insights into
one’s own and other’s behavior. We explore the possibili-
ties to facilitate browsing and comparing multiple users’
listening histories, and present two interactive visualization
approaches: To illustrate how the musical taste changes
over time, both of the visualizations are time-based. They
emphasize certain aspects of the music-related data in order
to reduce the complexity and support visual comparison by
merging the users' histories into one representative graph.
Last.fm HisFlocks focuses on time periods and aggregates
Figure 5. LoomFM detailed view. a) Timeline, b) User 1 musical taste based on genres and artists within these peri-
history (red) , c) User 2 history (purple), d) Artist names. ods. LoomFM is more timeline-centric and highlights se-
quential aspects of histories, such as repetitions and the
Instances of songs that appear more than once are con- differences in taste over time. These visualizations provide
nected by yellow arcs within one history as well as between users with means to get a deeper understanding for
histories. Similar to Arc Diagrams [10], several aspects can neighboring listeners and also discover new songs.
be visually derived from these arcs: For one, the higher an Given that both systems are implemented as first prototypes,
arc is, the further two instances of a song are apart, which some issues need to be improved in upcoming versions. In
means that in general the height of the arcs represents the HisFlocks, although the same artist always appears at the
consumption intervals of the same song (each instance is same position, the overall similarity-based placing of clus-
connected to all others). No arcs or very low ones show that ters and that of artists in each cluster is rather arbitrary. Us-
a user has a fluctuating listening behavior with a low ten- ers cannot anticipate where exactly new artists will appear.
dency to listen to the same songs in the long term (see A more comprehensible placement would be appealing.
lower user in Figure 4). Additionally, adjacent bundles of LoomFM suffers from a low performance when displaying
lines show repeated sequences of songs (for example, pre-
more than a thousand songs, an issue that comes up with 3. Chen Y-X, Santamaria R., Butz, A., Theron R. Tag-
realistically sized collections. Clusters: Semantic Aggregation of Collaborative Tags
beyond TagClouds. Proc. of SG 2009, 56-67.
We discussed our prototypes with Last.fm users and re-
ceived rather positive feedback and valuable suggestions. 4. Cunningham S., Reeves N., Britland M. An ethno-
We plan to add some extra functionalities, such as filters. In graphic study of music information seeking: implica-
the current state only two histories are compared and we tions for the design of a music digital library. Proc. of
plan to examine our concepts with a far greater number and JCDL 2003, 5-17.
size of listening histories. We hope to gain additional ideas 5. Dubinko M., Kumar R., Magnani J., Novak J. Ragha-
regarding understandability, data reliability and the human van P., Tomkins A. Visualizing tags over time. ACM
perception of musical taste. In our future work, we will ex- Transactions on the Web, 2007, Vol1, No. 7.
tend our concepts to broader scenarios with consumption
6. Havre, S., Hetzler, B., Nowell, L. ThemeRiver: Visual-
record, for example with books or photos.
izing Theme Changes over Time. Proc. of InfoVis 2000,
115-123.
ACKNOWLEDGMENTS
This research was funded by the Chinese Scholarship 7. Maxwell A. Pretzlav. Last.fm Explorer: An Interactive
Council (CSC) and the German state of Bavaria. We would Visualization of Hierarchical Time-Series Data.
like to thank the participants of our study. http://vis.berkeley.edu/courses/cs294-10-
fa08/wiki/images/9/9f/PretzlavFinalPaper.pdf.
REFERENCES 8. Porter M. F. An algorithm for suffix stripping. Program,
1. Baur D., Butz A. Pulling Strings from a Tangle: Visual- vol. 14, no. 3, pp 130-137, 1980.
izing a Personal Music Listening History. Proc. of IUI
2009, 439-444. 9. Viégas, F. B., Golder, S., Donath, J. Visualizing Email
Content: Portraying Relationships from Conversational
2. Bentley F., Metcalf C., Harboe G. Personal vs. commer- Histories. Proc. of CHI 2006, 979-988.
cial content: the similarities between consumer use of
photos and music. Proc. of CHI 2006, 667-676. 10. Wattenberg, M. Arc Diagrams: Visualizing structure in
strings. Proc. of InfoVis 2002, 110-116.
5