A single `odd' interaction can cause two user interaction sessions to diverge in similarity, and stand in the way of generalization. The sensitivity of session-based recommenders to session similarity motivates us to explicitly identify and remove such `similarity blockers'. Speci cally, we leverage huge amounts of data, which allow us to identify blockers in the form of non-co-occurring items. Other blockers can be identi ed using content-based similarity. Our experiments reveal that explicitly eliminating relatively few blockers improves performance.
Session-based recommender systems leverage information about the current user session to predict how it will continue. Session-based methods often rely heavily on similarity between sessions. A single aberrant user interaction in a session has the power to cause two otherwise highly similar sessions to suddenly become dissimilar. We refer to such an interaction as a `session blocker', since it causes two sessions to be di erent without re ecting a real underlying di erence of user preference.
Session blockers can arise in two ways. First, a user may interacts with an item only once, incidentally and unrelated to preference. Second, the user may click on a preferencerelated item, but it is a near-duplicate, and has an unexpected item ID.
The current era of big data opens the ability to search
for individual interactions that may have been incidental
clicks. In the past, non-occurrence, or non-co-occurrence of
items could be attributed to lack of data. Now, however, it
makes sense to investigate whether this non-co-occurrence is
truly incidental or whether it could be considered the source
of additional information that can be exploited for further
improve recommendations. For completeness we also look at
conventional near-duplicates identi ed on the basis of their
metadata. We are inspired by our previous work on
nearduplicates in large collections [
In this paper, we show that considering items similar due to non-co-occurrence, and, more conventionally, on the basis of metadata will improve recommendation e ectiveness. The e ect may be subtle, but a larger implication is that we should not assume that items occurring in similar sessions contribute positively to recommendations. 2.
We identify items (pairs or small clusters) that have different item IDs, but should be treated as the same item, since they will otherwise introduce a non-negligible `similarity blocking' e ect. We demonstrate two points: (1) these items should not be recommended together (since they are redundant) and (2) collapsing them before training a model could enhance performance (by eliminating `blocking').
We adopted two di erent approaches to nd similarityblocker items. The rst and more straightforward approach, called technical duplicates, is based on the idea of detecting near-duplicates in the collection of songs. In our case, the available metadata is a Last.fm1 URL, consisting of a string containing artist and title. We apply some basic NLP techniques to compute the similarity between two items: after properly parsing the URL, we compute the Jaccard similarity among the resulting words. Since we are looking for near-duplicates, we apply a threshold to decide which items should be considered equal.
The second and more sophisticated approach, called collaborative duplicates, leverages the fact that two items fail to ever co-occur despite their occurrence in similar (but separate) contexts. The idea nds support in our observation that, for a given user, the similarity between sessions containing technical duplicates is signi cantly higher than the average pairwise session similarity: this suggests that we can expect near-duplicates to appear in similar sessions from the same user. Moreover, we expect two near-duplicates never to co-occur in the same session, based on the rationale that the user would not listen to both of them together.
Next, we describe our technique. During training, we consider the set of sessions from each user in turn, and compute the pairwise Jaccard similarity on this set. We retain the pairs above a certain threshold. From these pairs of sessions, we extract pairs of items, such that one belongs to one session and one to the other, with the constraint that they must not co-occur in a session the training set for any user. The score given to the pairs of items is calculated as the average similarity between the pairs of similar sessions from which they come from, with a shrinkage factor. Finally, we keep the pairs of items with a score above the average, thus creating a cluster for each of these items. In this way, many cluster overlap each other, and creating the transitive closure of the clusters will result in a unique single big cluster containing all the clustered songs. Since we are instead interested in micro-clusters we kept only the disjoint clusters, which usually contain 2 or 3 items.
We carried out our experiments with an already existing
large scale music recommender system: the implicit playlist
recommender (IPR) [
To compute technical duplicates, we applied some heuristic optimization, such as the assumption that two nearduplicates should have at least one bigram in common. In this way, we avoid comparing each item with all the others. By leveraging the map-reduce paradigm, our algorithm took about 4 hours to identify near-duplicates. As one could expect, using this technique we nd redundant items as: 'Chris James feat. Ria Moran - Song For Her' and 'Chris James Song for Her (feat. Ria Moran)'. In this way, we reduced the number of items by 6.09%.
As for the collaborative duplicates, the number of items we were able to nd was much lower (around 600). All of them belong to the long tail, but come from the most active users, who have a higher probability of contributing.
We adopted the original experimental settings of the IPR2
used in [
20 40 60 80
100 N
The di erence in performance can be explained by the fact that merging redundant items allows the creation of more playlists. In all the cases, our techniques improve the performance for big N . This is because the original algorithm poses an hard threshold on a soft similarity (shrunk Jaccard) and, by merging duplicates, a larger number of useful implicit playlists are above the similarity threshold. In the future, we plan to analyze the e ects with di erent recommendation algorithms and also to evaluate online.
Finally, we double-checked our assumptions by mapping our new recommendations into the original space, which we did by attening the clusters. As one would expect, this degrades the performance. This result con rms our conclusion that the groups of items we identi ed as redundant should not be recommended together.
The ndings in this paper are interesting in light of the currently growing importance of session-based recommendation. E ectively, we have shown that not all similarity is good: instead, items can potentially be too similar. The approach of eliminating such similarity-blocker items explicitly, is greater than might be otherwise assumed, given their relative limited numbers. Merging items before training a model can boost performance, and, moving forward, may also have important implications for diversity: by removing too-similar items, more slots open for other items.