Collaborative tagging systems allow users to describe and organize items using labels in a free-shared vocabulary (tags), improving their browsing experience in large collections of items. At present, the most accurate collaborative filtering techniques build user profiles in latent factor spaces that are not interpretable by users. In this paper, we propose a general method to build linear-interpretable user profiles that can be used for user interaction in a recommender system, using the well-known simple additive weighting model (SAW) for multi-attribute decision making. In experiments, two kinds of user profiles where tested: one from free contributed tags and other from keywords automatically extracted from textual item descriptions. We compare them for their ability to predict ratings and their potential for user interaction. As a test bed, we used a subset of the database of the University of Minnesota's movie review systemMovielens, the social tags proposed by Vig et al. (2012) in their work “The Tag Genome”, and movie synopses extracted from the Netflix's API. We found that, in “warm” scenarios, the proposed tag and keyword-based user profiles produce equal or better recommendations that those based on latent-factors obtained using matrix factorization. Particularly, the keyword-based approach obtained 5.63% of improvement. In cold-start conditions-movies without rating information, both approaches perform close to average. Moreover, a user profile visualization is proposed arising an accuracy vs. interpretability tradeoff between tag and keywordbased profiles. While keyword-based profiles produce more accurate recommendations, tag-based profiles seems to be more readable, meaningful and convenient for creating profile-based user interfaces.
1.
An approach for improving the exploration of large collections of items such as books (librarything.com), films (netflix.com), pictures (flickr.com), research papers (citeulike.com) and web bookmarks (del.icio.us) is the leveraging of collaborative information from the users. This approach allows the knowledge of certain individuals on certain items in the collection propagates towards other users. In this way, a self-generated collaborative intelligence guides users in their exploration by recommendations tailored to their preferences and away from dislikes.
Currently, collaborative filtering approaches derive user profiles and produce recommendations based primarily on user feedback whether explicit (e.g. ratings, “likes”, tagging, reviews) or implicit (e.g. web logs). As the time goes by, user profiles grow while their preferences evolve. Generally, users are allowed to update their explicitly given information with the aim of adjusting their profiles to get better recommendations. In this scenario, when a user wants to update his (her) profile, it depends—for instance— on a large number of ratings making of this a difficult and even overwhelming task. The users should make a significant number of targeted edits in their profiles to obtain the desired effect. The situation worsens in systems based on implicit feedback where user profiles are not interpretable nor accessible by users. Most of the state-of-the-art methods for collaborative filtering build user profiles projected in latent factor spaces. These latent factors reduce considerably the dimensionality of the user profiles providing more accurate recommendations at the expense of interpretability. Unfortunately, users cannot make modifications on these low-dimensional and highly informative profiles. A first step to tackle this issue could be the design of interfaces based on interpretable user profiles. For instance Lops et al. [16] proposed a system where the user profiles are defined in a space indexed by keywords automatically extracted from textual item descriptions —keyword-based user profiles. However, in many cases the number of extracted keywords is similar or even larger than the number of items in the collection making it difficult the interaction of users with their profiles.
Alternatively, user profiles can also be built using tags
[
In this paper, we propose a method based on matrices for building
linear user profiles based either on social tags or on automatically
extracted keywords. From the users’ point of view, these profiles
behave as a linear simple aggregative weighting model SAW [28],
that is one of the most comprehensive method for multi-attribute
decision making [
To evaluate the performance of SAW user the profiles, they were
compared against user profiles based on latent factors obtained
using matrix factorization techniques [15], [
In addition, a visualization of user profiles is provided with the aim of analyzing the potential of SAW user profiles for the construction of user interfaces for recommender systems. In that visualization the profile of a single user is shown as a list of tags, or keywords, ranked by preference. We argue that the hypothetical user interaction with the top and the bottom of that list would provide a mechanism for updating his user profile with little effort. Simultaneously, the profiles of the nearest users are also shown as a collaborative resource for suggesting updates.
There have been several works that let users directly interact with
keyword-based user profiles or tag-based user profiles. For
example, the work of Pazzani and Billsus (1997) [
Another example is the work of Diederich & Iofciu (2006) [
The main limitation of the above mentioned approaches is that
only first order relations between user and resource are considered
to build these profiles. Consequently these approaches are
incapable to find new tags or keywords relevant to the profile.
Other approaches integrate collaborative tagging information, and
keywords found in textual descriptions of resources, in
algorithms that outperform classic collaborative filtering
approaches, but they sacrifice interpretability for accuracy [
NER 1
Semantic web 2
Probably, the most popular and accurate method used for product
recommendation is matrix factorization [
, then the dot product and vectors are found , which is calculated using the ̂ =∑ ( ∙
) = ! − #( ∙ )$ To avoid overfitting, it is common to introduce a regularization coefficient % that penalizes the norm of the user and item vectors. Thus, the regularized prediction error & is defined as: & = + % () →ℛ ) + ) →ℛ ) * Finally, user and item vectors are found minimizing the regularized prediction error over the set of known ratings. min # . /,0 / 1/2 ∈ℝ ∧ 1/2 67 ! − #( ∙
)$
+ % ()
→ℛ ) + )
→ℛ ) *
In this expression, we organize the known ratings in the matrix
ℝ.×0 , of size × , where is the number of users and is
the number of items. In this matrix, unknown ratings are
assigned to 0, and known ratings are in the interval [
In spite of the fact that it could be considered incorrect3, we will
use the canonical form of matrix factorization to express the
matrix of estimated ratings ℝ9.×0 as an affinity measure between
the user profile matrix :.× and the item profile matrix ;0× ,
both characterized in the same latent factor space. Thus:
ℝ9.×0
= :.×
∙ ;0×
Now, we can generalize this affinity measure to any space of
dimension < —denoted by ℛ=— using the expression:
3 It is important to keep in mind that, in order to calculate the
approximation of :.× and ;0× matrices, ratings = 0 must
be ignored in the expression to minimize. This is why in the
recommendation study area, instead of using already implemented
matrix decomposition methods, it is preferable to use optimization
methods such us LBFGSB [
F .→ℛ E Where G represent the affinity coefficient between the user and the HIJ dimension in the space ℛ=, for values of in K1, . . , N and values of H in K1, . . , <N. In that notation, the vector →ℛE is the X-based user profile of user in the space ℛ=. Similarly, the ℛ=-based user profile matrix ;0×= can be denoted as: ;0×= = ? ⋮ 0 = ⋮ C = D 0= →ℛE ⋮
F 0→ℛ E in K1, . . , <N. space ℛ=.
Where G denotes the relevance coefficient of the item to the HIJ dimension in the space ℛ=, for values of in K1, . . , N and H →ℛE represents the profile of the item m in the Now, if we choose an interpretable space ℛ= in which the item profile matrix ;0×= can be directly calculated, then all the user profiles in :.×= can be obtained by the following expression: :.×= = ℝ.×0 ∙ ((;0×= ) )O Where ((;0×= ) )O denotes the pseudo-inverse [18] of the transposed item profile matrix characterized in ℛ=, and ℝ.×0 is the matrix of known ratings.
Once the user profiles are obtained the estimated ratings ̂ be calculated with the expression: can ̂ = #
G ∙
G
G∈K ,…,|=|N
Therefore, from the point of view of decision making, it has the
well-known canonical form of the simple additive weighting
method (SAW) for multi-attribute decision making [
Thus, our proposed model, behave as a SAW model for decision making where: i) the appraisal of the resource is the rating of the resource ̂ ; ii) ratings are expressed as a weighted linear combination of the resource features in the interpretable space ℛ=; and iii) weights or the affinity coefficients G are discovered by the proposed model.
In the following subsections 3.2.3 and 3.2.4, we will explain how this generic model can be applied in two different interpretable spaces, namely keywords and tags. Besides, we will also show how the proposed user profiles : can be used in combination with the matrix factorization model to obtain rating predictions (see subsection 3.2.5). To clarify the notation used in the following sections, we will replace < for the specific size (dimensionality) of the space in which we will focus the discussion. Thus, ℛQ will be used instead of ℛ=, to denote he space of keywords. Similarity, in subsection 3.2.4, the space defined by the tags will be denoted by ℛ .
As mentioned before, the proposed model that automatizes the process of construction of user profiles relies (in turn) in the construction of the item profiles. Therefore, the matrix :.×Q (keyword-based user profiles) is calculated using the matrices ;0×Q (keyword-based item profiles) and ℝ.×0 (known ratings) using the following expression: :.×Q
= ℝ.×0 ∙ ((;0×Q ) )O Most of the content-based approaches that build keyword-based item profiles [16] use the vector space model [20] for representing the textual descriptions of the items as vectors →ℛR. Components of this vector, denoted by S, are values that quantify the relevance of the word w to the item m. Thus, a value close to 0 indicates that the word is not relevant to the item. Negative values can also be used if polarized relevance scores are available.
These relevance scores can be inferred from the occurrences of the words in the collection of textual descriptions of the items. The common practice to obtain relevance scores is to use the popular tf-idf term weighting scheme [14] or weights derived from the Okapi BM-25 retrieval formula [19]. These techniques prevent that common words get high relevance scores and promote less frequent words that occur systematically in particular textual descriptions.
Analogously to the keyword-based profiles, the :.× matrix with the tag-based user profiles is calculated in the same way: :.×
= ℝ.×0 ∙ ((;0× ) )O is the matrix with tag-based item profile vectors
0→ℛ T, in which the individual relevance of the tag t to the item m.
The tag-based item profiles →ℛT can be obtained using several
techniques [16], [29]. The simplest approach consists in an item
profile based on Boolean occurrences. That is, set I = 1 when
the tag U has been applied to the item and I = 0 otherwise.
It is important to note that the proposed method to obtain the
tagbased user profiles, using the pseudo-inverse, is equivalent to a
linear regression. Therefore, the tags should be independent
among them. That independence can be promoted grouping tags
that are morphologically related using stemmers and lemmatizers.
Lops et al. [17] went beyond grouping tags semantically related
using WordNet synsets [
Item profiles with graded, instead of Boolean relevance scores can be obtained with more sophisticated methods. For instance, Vig et al. (2012) [26] obtained the tag genome—a tag-based item profile for movies—by training a support vector regressor [24]. The training data came from a survey applied to users from the MovieLens system. The users where asked to estimate the relevance of the tags applied on selected movies. With these answers and a set of features extracted from movie reviews, textual descriptions, metadata and tag applications, among others, they trained a regressor whose predictions were used as relevance scores.
The proposed method for generating the rating predictions is a combination of matrix factorization (subsection 3.1) and the user profiles proposed in subsections 3.2.3 and 3.2.4. The aim of the method is three fold. First, we look for rating predictions as good as the ones produced by matrix factorization. Second, the method should be hybrid, that is, a combination of the collaborative filtering approach of matrix factorization and the content information from keywords or tags. Third, the users should be able to edit their keyword-based (or tag-based) user profiles and the rating predictions must be updated with little computational cost. The method comprises four steps: 1. 2. 3. 4.
An initial matrix of rating estimations is obtained using matrix factorization: ℝ9.7×0 = :.× ∙ ;0× .
An initial matrix of keyword-based user profiles is obtained: :.7×Q = ℝ9.7×0 ∙ ((;0×Q ) )O .
The matrix V.×Q , containing users edition operations to their profiles (positive of negative differences) is added to obtain updated user profiles: :.×Q = :.7×Q + V.×Q .
Estimations are obtain by: ℝ9.×0 = :.×Q ∙ (;0×Q )
These four steps can be expressed in a single expression:
Note that ((;0×Q ) )O ∙ (;0×Q ) ≅ X0×0 (the identity
matrix) only when the item profiles are linearly independent
among them. The contrary is the common case. Thus, this matrix
multiplication infers the affinities among the items induced by the
keywords content information. In a final post-processing step, the
values on each row in the output matrix ℝ9.×0 are standardized in
the interval [
The experiments aim to evaluate the accuracy of the recommendations produced by the proposed methods. This section contains a comprehensive description of the data and the evaluation measure used to compare the proposed models against baselines. 4.1 Data This subsection is intended to provide insight about how the used dataset was obtained and preprocessed. Besides we provide information about its content, size and distribution.
The dataset of users, movies and ratings was obtained from a
production database dump of the MovieLens system in April
2012. From this dataset, we extracted a subset filtering by the
users and movies with more than 1,000 ratings. This filtering
produced a subset of 200 users, 1,462 movies and 150,915 ratings.
The rating scale in MovieLens is in the usual interval [
Textual descriptions were obtained from the synopsis field in the movie records from the Netflix public API4 during the year 2012. These texts were assigned to movies in the MovieLens dataset by a mapping obtained through a research collaboration with the GroupLens5 research group.
These textual descriptions were represented in a vectorial bag-ofwords model. The dimensionality of that representation was reduced with the aim of obtaining a vocabulary based on popularity and informativeness. Thus, a vocabulary of 5,848 words was obtained using the following series of preprocessing ad hoc actions: (1) all characters were converted to lowercase equivalents; (2) people first and last names were concatenated with the underscore character; (3) numeric tokens were removed; (4) 334 stop words taken from the source code of the gensim6 framework were removed; (5) words occurring in less than 10 synopses and in more than the 95% of the synopses, were removed; and finally (6) all punctuation marks were cleaned. The term weights used to register the relevance of a word in a synopsis vector were obtained with the Okapi BM25 retrieval formula [19] using the method proposed by Vanegas et al. [25]. Thus, the weight `(a, b) of a word a in a document (synopsis) b is given by: `(a, b) = cde f − bg(a) (i + 1)Ug(a, b)
h j = i k(1 − l) + l j + Ug(a, b) bc(b)
o mnbc Where, bg(a) is the number of documents where a occurs, = 1,462 is the number of movies, Ug(a, b) the number of occurrences of word a in the document b, and mnbc = 33is the average document length. The additional used parameters were i = 1.2 and l = 0.75 (see [e]). A pair of examples of the resulting keyword vectors using the proposed method is shown in Table 2. The aggregation of vectors obtained from synopses produce the items profile matrix ;0×Q , whose dimensions are = 1,462 movies (rows) by s = 5,848 words (columns). This matrix is sparse, having only 0.518% of non-zero entries.
The tag set used to characterize the movies is the selection of tags proposed by Vig et al. in “The Tag Genome” [26]. This tag set is a subset of 1,128 tags out of nearly 30,000 unique tags freely applied by 416 users in the MovieLens system. This subset was obtained by removing tags with less than 10 applications, misspellings, people names and near duplicates. Thereafter, they selected the top 5% ranked tags with and entropy-based quality measure proposed by Sen et al. [22]. Only 1,081 tags from the tag genome’s set occurred in the 1,462 movies in the item-profile matrix ;0× .
There are 13,332 tag associations to the movies considered in this study. 1,370 movies have at least one tag associated with an average of 9.7 tags per movie (σ = 8.5). Besides, all tags were assigned at least to one movie. The distribution of the tag applications is considerably more uniform than the Zipf distribution. Thus, the 108 more frequent tags (10%) represent only the 42% of the tag associations. This can be roughly seen in Table 3, which shows tag samples selected from uniformly separated rank ranges. The association of movies and tags produce the items profile matrix ;0× (1,462 movies by 1,082 tags) with binary entries and a density of 0.844% (also very sparse). 12,989 43,068 55,025 27,193 10,229 To evaluate the performance of the proposed methods we provided two scenarios of validation in 10 folds: cross validation and product-cold-start [24]. In the cross validation scenario, the ratings were divided in ten randomized folds. In each fold 90% of ratings were used for training and the remaining 10% was used for testing. In the product-cold-start scenario, the procedure for extracting the training and test datasets is the same, but all the ratings from the movies in the test set are removed.
The evaluation measure to assess the accuracy of the recommendations is root-mean-square error (RMSE) defined as: { ∑K1/2 N∈IxyI( ̂ t uv = w −
)
|U zU|
Where U zU is the test set of the ratings and |U zU| its cardinality.
Given that the methods proposed in section 3 provide rating
estimations standardized in [
Note that the matrix factorization method cannot be applied in the cold-start scenario because movies without ratings cannot be represented in the latent factors space. Consequently, for this scenario, the method proposed in subsection 3.2.5 uses ℝ.×0 in the second step and the first step must be
The results of our experiments are presented in Table 4. The first two rows show the results for the proposed baseline methods for each one of our test settings. The remaining two rows show the results obtained by the proposed methods presented in subsection 3.2.4. For each system, the “RMSE” columns present the average for the 10 folds and the columns labeled with "σ" reports the standard deviation.
METHOD System average+user’s bias Matrix factorization Keyword-based user prof.
Tag-based user profiles Regarding the “warm” scenario (i.e. cross validation), the obtained results show that the two proposed methods based on user profiles outperformed the baseline matrix factorization method. Particularly, the margin obtained by the keyword-based user profile system was clearly significant, being more than 3 standard deviations apart. Clearly, the proposed methods reached a performance level in the state of the art for the rating prediction task. Unlike matrix factorization, our recommendations were produced by a fully interpretable model suitable for better user interaction and better explanations.
The cold-start evaluation setting was clearly more challenging. Our systems barely overcame the proposed average-based baseline. However, the proposed tag and keyword-based systems have the potential to provide to the user mechanisms to get the system “warmer” with little effort. Accurate methods such as matrix factorization require a considerable number of initial ratings before starting to produce good predictions. In contrast, our methods provide a completely customizable user profile with just a small number of initial ratings.
Comparing the tag-based and keyword-based models, the results show that keyword-based user profiling performs better in “warm” conditions and slightly better in “cold” conditions
In order to visualize the profiles, we selected the User 156 from the fold 1 in our dataset. We must say that users in our data are completely anonymous. This user was manually chosen based on the user-to-user pairwise Pearson correlation matrix obtained from the keyword-based user profiles :.×Q . Comparing these correlations we observed that the User 156 had high negative and positive correlations against the other users. So, we considered that the preferences and dislikes of this user was being shared by several users and rejected by others. Consequently, we considered him as an interesting candidate to be visualized. In Figure 2, the keyword-based user profile of the User 156 is showed jointly with his 10-nearest users according to the user-to-user correlation matrix. The ranked list of keywords that this user prefers the most is shown on the left side. The right side shows the list of his most disliked keywords. The user profile is represented by the thick black line. In its turn Figure 3, shows the same plots but using tagbased user profiles instead of keywords.
Now it is possible to qualitatively compare a user keyword-based versus a tag-based profile. From this comparison we observe that User 156’s tag-based profile is more cohesive in comparison with the word-based profile. This cohesiveness can be observed by the semantic relatedness of the tag set. In this profile, 20 out of 40 tags preferred by User 156 are related to action and teens movies. These tags are: Dark hero, Effects, Explosions, Indiana jones, German, Drug addiction, Arms dealer, Weapons, Life & death, Videogame, First contact, Comic book adapt, Bond, 007 series, Stop motion, Fantasy world, Dreamworks, Video games, Harry potter, Emma Watson. Regarding the keyword-based profile, the keyword set doesn’t exhibit a clear pattern. Although we know that these particular observations cannot be generalized, we think that this observation opens an interesting research direction about the necessity of measuring the semantic cohesiveness of the produced profiles.
Concerning the potential of interaction we have not yet conducted any experiments with users, but it seems reasonable that users will understand the general interaction idea. It is expected that the users will be prone to experiment modifying their own profiles varying the level of preference or dislike for the more relevant tags or keywords in their profiles. Also, it seems that the feature of seeing the profile of similar users could motivate the desire to interact with the interface. That is because, showing other people behaviors and allows a kind of warm start with the system. New concerns arise from the observations of these profiles. For instance, what should be done with “negative” tags that appear in the list of preferred tags of users? This situation is illustrated by the tag “boring!” in the User 156’s “likes” list.
Probably, this tag can be reasonable and predictive for some users,
so, maybe it shouldn’t be removed from the tag set. But trenchant
criticisms of user tastes should be prevented. A possible
alternative to this problem would be the use of a linear regression
algorithm, similar to the one used in a previous work [
We proposed a generic method to extract user profiles, in interpretable spaces, in which it is possibly to directly characterize items from the collection. The proposed user-profiling methods were indexed in two different spaces: keywords and tags. Besides the proposed models are suitable for user interaction in the user profile component.
The proposed user-profiling methods were evaluated in a subset of the MovieLens dataset and compared against strong baselines. It was concluded that in “warm” scenarios both methods produce recommendations with the same accuracy than those produced by matrix factorization methods. In a cold-start scenario, both methods performed slightly better than a recommender system based on average ratings. Regarding the proposed visualization of the keyword-based and the tag-based user profiles, we could observe that cohesion of the profile is an important measure to have into account when two profiles methods are compared. Non-cohesive profiles might be misunderstood by users leading them to avoid the interaction with those profiles. An interesting research question could be how to discriminate cohesive profiles, from non-cohesive profiles. The proposed approach also contributed to a better classification of the content-based recommendation techniques, separating the user-profiling task from the item-profiling task, suggesting a uniform framework to share and compare the contributions made on each one of the tasks.
Our especial thanks to Prof. John Riedl and Daniel Kluver from GroupLens, the University of Minnesota; Prof. Shilad Sen of the Macalester College; and Prof. Fabio Gonzalez of the Universidad Nacional de Colombia. The work was partially funded by the Colombian Department for Science, Technology and Innovation (Colciencias) via the grant 1101-521-28465 from “El Patrimonio Autónomo Fondo Nacional de Financiamiento para la Ciencia, la Tecnología y la Innovación, Francisco José de Caldas” and by the Universidad Nacional de Colombia via the grant DIB QUIPU: 201010016956. The third author recognizes the support from Mexican Government (SNI, COFAA-IPN, SIP 20131702, CONACYT 50206-H) and CONACYT–DST India (grant 122030 “Answer Validation through Textual Entailment”).