=Paper=
{{Paper
|id=None
|storemode=property
|title=Recommending
Items in Social Tagging Systems Using Tag and Time Informations
|pdfUrl=https://ceur-ws.org/Vol-1210/SP2014_01.pdf
|volume=Vol-1210
|dblpUrl=https://dblp.org/rec/conf/ht/LacicKSTP14
}}
==Recommending
Items in Social Tagging Systems Using Tag and Time Informations ==
https://ceur-ws.org/Vol-1210/SP2014_01.pdf
Recommending Items in Social Tagging Systems Using
Tag and Time Information
Emanuel Lacic∗ Dominik Kowald∗
Knowledge Technology Know-Center
Institute Graz University of Technology
Graz University of Technology Graz, Austria
Graz, Austria dkowald@know-center.at
elacic@know-center.at
Paul Seitlinger Christoph Trattner Denis Parra
Knowledge Technology Know-Center CS Department
Institute Graz University of Technology Pontificia Universidad Católica
Graz University of Technology Graz, Austria de Chile
Graz, Austria ctrattner@know-center.at Santiago, Chile
paul.seitlinger@tugraz.at dparra@ing.puc.cl
ABSTRACT Categories and Subject Descriptors
In this work we present a novel item recommendation ap- H.2.8 [Database Management]: Database Applications—
proach that aims at improving Collaborative Filtering (CF) Data mining; H.3.3 [Information Storage and Retrieval]:
in social tagging systems using the information about tags Information Search and Retrieval—Information filtering
and time. Our algorithm follows a two-step approach, where
in the first step a potentially interesting candidate item-set
is found using user-based CF and in the second step this can-
Keywords
recommender systems; social tagging; collaborative filtering;
didate item-set is ranked using item-based CF. Within this
item ranking; base-level learning equation
ranking step we integrate the information of tag usage and
time using the Base-Level Learning (BLL) equation com-
ing from human memory theory that is used to determine 1. INTRODUCTION
the reuse-probability of words and tags using a power-law Over the past few years social tagging gained tremendously
forgetting function. in popularity, helping people for instance to categorize or de-
scribe resources on the Web for better information retrieval
As the results of our extensive evaluation conducted on data- (e.g., BibSonomy or CiteULike) [13, 23]. Although the pro-
sets gathered from three social tagging systems (BibSonomy, cess of tagging has been well explored in the past and in
CiteULike and MovieLens) show, the usage of tag-based and particular the task of predicting the right tags to the user in
time information via the BLL equation also helps to improve a personalized manner [12, 20], studies on predictive models
the ranking and recommendation process of items and thus, to recommend items to users based on social tags are still
can be used to realize an effective item recommender that rare. To contribute to this sparse field of research, in this
outperforms two alternative algorithms which also exploit paper we present preliminary results of a study that aims at
time and tag-based information. addressing this issue. In particular, we provide first results
of a novel attempt to improve item recommendations by tak-
ing into account peoples’ social tags and the information of
the time the tags have been applied by the users. As shown
in related work, recommending items to users in a collabo-
rative manner relying on social tagging information is not
an easy task in general (e.g., [24] or [17]). However, other
related work has also proved that the information of time is
∗ an important factor to make the models more accurate in
Both authors contributed equally to this work. the end (e.g., [26] or [10]).
Contrary to the previous work mentioned above, we suggest
a less data-driven approach that is inspired by principles of
human memory theory about remembering things over time.
As shown in our previous work on tag recommender systems
[15], the base-level learning (BLL) equation introduced by
Anderson and Schooler [16] (see also Anderson et al. [1]),
which integrates tag frequency and recency (i.e., the time
since the last tag usage), can be used to implement an effec-
tive tag recommendation and ranking algorithm. In partic-
ular, the BLL equation models the time-depended drift of Dataset |B| |U | |R| |T | |T AS|
forgetting of words and tags using a power-law distribution BibSonomy 82,539 2,437 28,000 30,919 339,337
in order to determine a probability value that a specific tag CiteULike 36,471 3,202 15,400 20,937 99,635
will be reused by a target user. MovieLens 53,607 3,983 5,724 14,883 92,387
In this work, we apply this equation for ranking and recom-
mending items to users. To this end, we present a novel rec- Table 1: Properties of the datasets, where |B| is the
ommender approach called Collaborative Item Ranking Us- number of bookmarks, |U | the number of users, |R|
ing Tag and Time Information (CIRTT) that firstly identi- the number of resources, |T | the number of tags and
fies a potentially interesting candidate item set and secondly, |T AS| the number of tag assignments.
ranks this candidate set in a personalized manner (similar
to [10]). In this second step of personalization, we integrate
, where items(u) is the set of items the target user u has
the BLL equation to include this information about tags
bookmarked in the past. This item-based CF step helps us
and time. To investigate the question as to whether tag and
to give a higher ranking to candidate items that are more
time information can improve the ranking and recommen-
similar to the items the target user has bookmarked in the
dation process, we conducted an extensive evaluation using
past (see also [10]).
folksonomy datasets gathered from three social tagging sys-
tems (BibSonomy, CiteULike and MovieLens). Within this
To finally realize CIRTT in order to integrate tag and time
study we compared our approach to two alternative tag and
information we make use of the base-level learning (BLL)
time based recommender algorithms [26, 10] amongst others.
equation proposed by Anderson et al. [1]. As described
The results show that integrating tag and time information
in our previous work [15], the BLL equation can be used to
using the BLL equation helps to improve item recommenda-
determine a relevance value for a tag t in the tag assignments
tions and to outperform state-of-the-art baselines in terms
of a target user u based on tag frequency and recency:
of recommender accuracy.
n
X
The remainder of this paper is organized as follows. We be- BLL(u, t) = ln( t−d
i ) (2)
i=1
gin with explaining our tag and time based approach CIRTT
in Section 2. Then we describe the experimental setup of our , where n is the number of times t has been used by u and ti
evaluation in Section 3 and summarize the results of this is the recency, i.e., the time since the ith occurrence of t in
study in Section 4. Finally, in Section 5, we close the paper the tag assignments of u. The exponent d is used to model
with a short conclusion and an outlook into the future. the power law of forgetting memory items and is usually set
to .5 (see [1]). In order to map these BLL values on a range
of 0 - 1, we used the same normalization method as used in
2. APPROACH our previous work [15].
In this section we provide a detailed description of our item
recommendation approach called Collaborative Item Rank- We adopt this equation for the ranking of items in social
ing Using Tag and Time Information (CIRTT). In general, tagging systems using a similar method as proposed in [26]
our CIRTT algorithm uses a similar strategy as the approach and [10]. Thus, a user is assumed to prefer an item if it
proposed by Huang et al. [10] and thus, consists of two steps has been tagged with tags of high relevance for the user,
relying on a combination of user- and item-based CF: in the that is, with tags exhibiting a high BLL value. Given this
first step, a potentially interesting candidate item set for the assumption, the BLL value of a given item i for the target
target user u is determined and in the second step, this can- user u is determined using the following formula:
didate item set gets ranked using item similarities and tag X
and time information. BLL(u, i) = BLL(u, t) (3)
t∪tags(u,i)
Step one (i.e., determining candidate items) is conducted , where tags(u, i) is the set of tags u has used to tag i.
using a simple user-based CF approach. Hence, we first
find the most similar users for the target user u (i.e., the Taken together, the prediction value pred(u, i) of a candi-
neighborhood) based on the binary user-item matrix Bu,i date item i using our CIRTT approach is given by:
(see also [26]) and then, use the bookmarked items of these X
neighbours as our candidate item set. We use a neighbour- pred(u, i) = sim(i, j) ×BLL(u, i) (4)
hood of k = 20 users and the Cosine similarity measure [7] j∈items(u)
(see also Section 3.3). | {z }
sim(u,i)
In the second step (i.e., ranking candidate items) we use an This approach enables us to weight higher the items within
item-based CF approach in order to determine the relevance the candidate set that are more important to the target user
of each candidate item for the target user based on the items (i.e., items associated with tags exhibiting a high BLL value
she has bookmarked in the past. Hence, for each candidate that integrates tag frequency and recency). CIRTT and the
item i in the candidate item set we calculate this combined baseline algorithms presented in this work are implemented
similarity value sim(u, i) by the item-based CF formula: in the Java programming language, are open-source software
X and can be downloaded online from our Github Repository1
sim(u, i) = sim(i, j) (1) [14].
j∈items(u) 1
https://github.com/learning-layers/TagRec/
3. EXPERIMENTAL SETUP by their frequency in all bookmarks, meaning that the most
In this section we describe in detail the datasets, the evalu- frequently occurring items in the dataset are recommended.
ation methodology and metrics as well as the baseline algo-
rithms used for our experiments. User-based Collaborative Filtering (CF): Another ap-
proach we benchmarked against is the well-known User-
based Collaborative Filtering (CF) recommendation algorithm
3.1 Datasets [19]. The main idea of CF is that users that are more similar
In order to evaluate our approach and for reasons of re- to each other (i.e., have similar taste), will probably also like
producibility we used freely-available folksonomies gathered the same items. Thus, the CF approach first finds the k most
from three well-known social-tagging systems. We used data- similar users for the target user and afterwards recommends
sets of the social bookmark and publication sharing system their items that are new to her (i.e., have not been book-
BibSonomy2 , the reference management system CiteULike3 marked before). We calculated the user-similarities based
and the movie recommendation site MovieLens4 . As sug- on both, the binary user-item matrix as proposed in [26]
gested by related work in the field (e.g. [11, 9]), we excluded (hereinafter referred to as CFB ) and the tag-based user pro-
all automatically imported and generated tags (e.g., bibtex- files as proposed in [10] (hereinafter referred to as CFT ).
import). In the case of CiteULike we randomly selected 10% Although we also considered using Item-based CF [18], we
of the user profiles for reasons of computational effort (see dismissed this method based on the tag-based recommender
also [7]). experiments of Bogers et al. [2] showing that user-based
CF always beats item-based CF. They explain the result
We did not use a full p-core pruning technique, since this given that the number of items in their dataset is larger
would negatively influence the recommender evaluation re- than the number of users, and this is also the case in our
sults in social tagging system as shown by Doerfel and Jäschke three datasets (Table 1).
[6], but excluded all unique resources (i.e., resources that
have been bookmarked only by a single user). The final Collaborative Filtering Using Tag and Time Infor-
dataset statistics can be found in Table 1. mation (Z / H): We also compared our approach to two
alternative algorithms that focus on improving Collabora-
3.2 Evaluation Methodology tive Filtering for social tagging systems using tag and time
To evaluate our item recommender approach we used a train- information. The first one has been proposed by Zheng et
ing and test-set split method as proposed by popular and al. [26] (hereinafter referred to as Z ) and improves the tradi-
related work in this area [10, 26]. Hence, for each user tional CF approach based on the binary user-resource matrix
we sorted her bookmarks in chronological order and used using tag and time information. As in our CIRTT approach
the 20% most recent bookmarks for testing and the rest for this is done using information about tag frequency and re-
training. With the training set we examined then whether cency but in contrast to our solution the authors model the
a recommender approach could predict the bookmarked re- forgetting process using an exponential distribution rather
sources of a target user in the test set. This procedure also than a power-law distribution. Moreover, this information
simulates well a real environment where the bookmarking is already used in the user similarity calculation step and
behavior of a user in the future is tried to be predicted based not in the item ranking step as it is done in our approach.
on the bookmarking behavior in the past [3].
The second tag and time-based mechanism we tried to bench-
To finally quantify the recommendation accuracy of our ap- mark against was proposed by Huang et al. [10] (hereinafter
proaches, we used a set of well-known information retrieval referred to as H ). As in our approach, this algorithm uses
metrics. In particular, we report Normalized Discounted Cu- a 2-step recommendation process, where in the first step a
mulative Gain (nDCG@20), Mean Average Precision (MAP potentially interesting candidate item-set for the target user
@20), Recall (R@20), Diversity (D) and User Coverage (UC) is determined using user-based CF and in the second step
[21, 8]. All performance metrics are calculated and reported this candidate item-set is ranked using item-based CF. In
based on the top-20 recommended items. Moreover we also contrast to CIRTT, the authors calculate the user and item
show the performance of the algorithms in the plots of all similarities based on user tag-profiles rather than based on
three accuracy metrics (nDCG, MAP and Recall) for 1 - 20 the binary user-item matrix. Furthermore, in this algorithm
recommended items (see also [4]). the forgetting process is modeled using a simple linear func-
tion rather than a power-law distribution.
3.3 Baseline Algorithms All CF-based approaches mentioned in this section use a
In order to evaluate our tag and time based approach, we neighborhood of 20 users and make use of the Cosine simi-
compared CIRTT to several baseline methods in terms of larity measure as it is also done in CIRTT (see also [7]).
recommender accuracy. The algorithms have been selected
with respect to their popularity, performance and novelty.
MostPopular (MP): The most basic approach we utilized
is the simple Most Popular (MP) approach that recommends 4. RESULTS
for any user the same set of items. These items are weighted In this section, we present the results of the evaluation com-
paring our CIRTT approach to the baseline algorithms de-
2
http://www.kde.cs.uni-kassel.de/bibsonomy/dumps scribed in Section 3.3 with respect to recommender accuracy
3
http://www.citeulike.org/faq/data.adp on three different folksonomy datasets (BibSonomy, CiteU-
4
http://grouplens.org/datasets/movielens/ Like and MovieLens).
Dataset Metric MP CFT CFB Z H CIRT T
nDCG@20 .0143 .0448 .0610 .0621 .0564 .0638
MAP@20 .0057 .0319 .0440 .0447 .0394 .0464
BibSonomy R@20 .0204 .0618 .0820 .0834 .0816 .0907
D .8307 .8275 .8852 .8528 .6209 .8811
UC 100% 99.76% 99.52% 99.52% 99.76% 99.76%
nDCG@20 .0062 .0407 .0717 .0762 .0706 .0912
MAP@20 .0036 .0241 .0453 .0484 .0459 .0629
CiteULike R@20 .0077 .0630 .1033 .1077 .0928 .1225
D .8936 .7969 .8642 .8145 .6318 .8640
UC 100% 98.38% 96.44% 97.32% 98.38% 97.61%
nDCG@20 .0198 .0361 .0602 .0614 .0484 .0650
MAP@20 .0075 .0201 .0347 .0367 .0263 .0413
MovieLens R@20 .0366 .0561 .1031 .1013 .0763 .1058
D .9326 .8861 .9267 .9119 .7789 .9176
UC 100% 97.82% 95.90% 98.43% 97.82% 95.90%
Table 2: nDCG@20, MAP@20, R@20, D and UC values for BibSonomy, CiteULike and MovieLens showing
that CIRTT, that integrates tag and time information using the BLL-equation, outperforms state-of-the-art
baseline algorithms (highest accuracy values are highlighted in bold).
In an extensive empirical study, Cremonesi et al. [5] have When looking at the other two not accuracy-based metrics,
shown that standard Information Retrieval accuracy metrics interestingly, the approach of Huang et al. (H) always results
(e.g., Recall or nDCG) are well suited to evaluate recom- in the lowest Diversity (D) of recommended items. This re-
mender systems, at least in case of top-N recommendation sult might appear because this approach is based on the user
tasks. Therefore, Table 2 provides measures of accuracy tag-profiles and the Diversity metric is calculated based on
(nDCG@20, MAP@20, R@20) and - additionally - measures tags. Finally, as all personalized approaches utilize a user-
of Diversity (D) and User Coverage (UC) for each approach based CF approach for finding similar users, the measure of
and for each of the three datasets. User Coverage (UC) does not appear to deviate between the
different algorithms. We observed the maximum deviation
As expected, the MP baseline, which is not personalized at of 2.53% within the MovieLens dataset.
all, resulted in the lowest accuracy estimates. Regarding the
two traditional CF algorithms, CFB , which constructs a bi- 5. CONCLUSIONS & FUTURE WORK
nary user-item matrix based on bookmarks, performs better In this work we have presented preliminary results of a novel
than CFT , which is based solely on the user tag-profiles. Re- recommendation approach called Collaborative Item Rank-
garding the two alternative tag- and time-based approaches, ing Using Tag and Time Information (CIRTT) that aims at
a same phenomenon can be observed as the algorithm of improving Collaborative Filtering in social tagging systems.
Zheng et al. (Z) [26], that is also based on the binary user- Our algorithm follows a two-step approach as also done in
item matrix, performs better than the method of Huang et [10], where in the first step a potentially interesting can-
al. (H) [10], that is based on the user tag-profiles. didate item set is found performing user-based CF and in
the second step this candidate item set is ranked perform-
With respect to all accuracy metrics (nDCG@20, MAP@20, ing item-based CF. Within this ranking step we integrate
R@20), our CIRTT approach, that integrates tag and time the information of frequency and recency of tag use apply-
information using the BLL-equation, performs best in all ing the Base-Level Learning (BLL) equation [1]. Thus, in
three datasets (BibSonomy, CiteULike and MovieLens). This contrast to existing approaches that also consider informa-
may suggest that applying a power-law function as it is done tion about tags and time (e.g., [26, 10]), CIRTT draws on
via the BLL-equation is more appropriate to account for ef- an empirically well established formalism modeling the reuse
fects of recency than an exponential function (Zheng et al. probability of memory items (tags) in form of a power-law
[26]) or a linear function (Huang et al. [10]). A same pat- forgetting function. In recent work, the same formalism has
tern of results can be observed when looking at Figure 1 that turned out to substantially improve the ranking and recom-
reveals estimates of the nDCG, MAP and Recall measures mendation of tags [15].
for different sizes of the recommended item set. These plots
show that only in the case of BibSonomy the approach of The current evaluation conducted on datasets gathered from
Zheng et al. reaches slightly higher accuracy estimates than three social tagging systems (BibSonomy, CiteULike and
our method for the first 7 recommended items. However, MovieLens) reveals that applying the BLL equation also
this changes when increasing the number of recommended helps to improve the ranking and recommendation process
items where our approach again produces the best recom- of items. Most important, the results speak in favor of an
mender quality. Furthermore, we have also tried to integrate integrative research endeavor that places a data-driven ap-
an exponential recency function [26] in our approach which proach on a theoretical foundation provided by research on
resulted in lower accuracy estimates than the BLL power- human cognition and semiotics.
law forgetting function.
Our future work will aim at improving the approach pre-
sented in this paper. For example, we will examine as to
0.07 0.10 0.07
MP CFB H MP CFB H MP CFB H
CFT Z CIRTT CFT Z CIRTT CFT Z CIRTT
0.06 0.06
0.08
0.05 0.05
0.06
0.04 0.04
nDCG
nDCG
nDCG
0.03 0.03
0.04
0.02 0.02
0.02
0.01 0.01
0.001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0.001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0.001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Number of recommended items Number of recommended items Number of recommended items
(a) nDCG (b) nDCG (c) nDCG
BibSonomy CiteULike MovieLens
0.05 0.07 0.045
MP CFB H MP CFB H MP CFB H
CFT Z CIRTT CFT Z CIRTT CFT Z CIRTT
0.06 0.040
0.04
0.035
0.05
0.030
0.03
0.04
0.025
MAP
MAP
MAP
0.03 0.020
0.02
0.015
0.02
0.01 0.010
0.01
0.005
0.001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0.001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0.0001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Number of recommended items Number of recommended items Number of recommended items
(d) MAP (e) MAP (f) MAP
BibSonomy CiteULike MovieLens
0.10 0.14 0.12
MP CFB H MP CFB H MP CFB H
CFT Z CIRTT CFT Z CIRTT CFT Z CIRTT
0.12
0.10
0.08
0.10
0.08
0.06
0.08
Recall
Recall
Recall
0.06
0.06
0.04
0.04
0.04
0.02
0.02
0.02
0.001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0.001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 0.001 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Number of recommended items Number of recommended items Number of recommended items
(g) Recall (h) Recall (i) Recall
BibSonomy CiteULike MovieLens
Figure 1: nDCG, MAP and Recall plots for BibSonomy, CiteULike and MovieLens showing the recommenda-
tion accuracy of our tag and time based CIRTT approach along with state-of-the-art baseline algorithms for
1 - 20 recommended items (k ). We can see that CIRTT reaches the highest levels of recommender accuracy
over all three metrics and on all datasets.
whether the BLL equation can also help to improve the cal- [10] C.-L. Huang, P.-H. Yeh, C.-W. Lin, and D.-C. Wu.
culation of user similarities and thus, to find more suitable Utilizing user tag-based interests in recommender
user neighborhoods and candidate items. Additionally, we systems for social resource sharing websites.
will put more emphasis on dynamics that have been found Knowledge-Based Systems, 2014.
to play out in tagging systems (e.g., [22]) and how individ- [11] R. Jäschke, L. Marinho, A. Hotho,
ual learning and forgetting processes are influenced by other L. Schmidt-Thieme, and G. Stumme. Tag
individuals’ behavior in the system. Moreover, we also plan recommendations in folksonomies. In Knowledge
to further improve the item ranking process using insights Discovery in Databases: PKDD 2007, pages 506–514.
of relevant research dealing with recommender novelty and Springer, 2007.
diversity (e.g., [25]) in order to increase the user acceptance. [12] R. Jäschke, L. Marinho, A. Hotho,
Finally, it would also be interesting to evaluate our proposed L. Schmidt-Thieme, and G. Stumme. Tag
approach against state-of-the-art matrix factorization item recommendations in social bookmarking systems. Ai
recommender methods (e.g., SLIM or CLiMF). Communications, 21(4):231–247, 2008.
[13] C. Körner, D. Benz, A. Hotho, M. Strohmaier, and
Acknowledgments: This work is supported by the Know- G. Stumme. Stop thinking, start tagging: tag
Center, the EU funded project Learning Layers (Grant Nr. semantics emerge from collaborative verbosity. In
318209) and the Austrian Science Fund (FWF): P 25593- Proceedings of the 19th international conference on
G22. The Know-Center is funded within the Austrian CO- World wide web, pages 521–530. ACM, 2010.
MET Program - Competence Centers for Excellent Tech- [14] D. Kowald, E. Lacic, and C. Trattner. Tagrec:
nologies - under the auspices of the Austrian Ministry of Towards a standardized tag recommender
Transport, Innovation and Technology, the Austrian Min- benchmarking framework. In Proc., HT ’14, New
istry of Economics and Labor and by the State of Styria. York, NY, USA, 2014. ACM.
COMET is managed by the Austrian Research Promotion
[15] D. Kowald, P. Seitlinger, C. Trattner, and T. Ley.
Agency (FFG).
Long time no see: The probability of reusing tags as a
function of frequency and recency. In Proc. WWW
6. REFERENCES ’14. ACM.
[1] J. R. Anderson, M. D. Byrne, S. Douglass, C. Lebiere, [16] J. R. A. Lael J. Schooler. Reflections of the
and Y. Qin. An integrated theory of the mind. environment in memory. Psychological Science, 1991.
Psychological Review, 111(4):1036–1050, 2004. [17] D. Parra-Santander and P. Brusilovsky. Improving
[2] T. Bogers and A. van den Bosch. Recommending collaborative filtering in social tagging systems for the
scientific articles using citeulike. In Proc., RecSys ’08, recommendation of scientific articles. In WI-IAT, 2010
pages 287–290, New York, NY, USA, 2008. ACM. IEEE/WIC/ACM.
[3] P. G. Campos, F. Dı́ez, and I. Cantador. Time-aware [18] B. Sarwar, G. Karypis, J. Konstan, and J. Riedl.
recommender systems: a comprehensive survey and Item-based collaborative filtering recommendation
analysis of existing evaluation protocols. User algorithms. In Proc., WWW ’01, pages 285–295, New
Modeling and User-Adapted Interaction, pages 1–53, York, NY, USA, 2001. ACM.
2013. [19] J. B. Schafer, D. Frankowski, J. Herlocker, and S. Sen.
[4] P. Cremonesi, P. Garza, E. Quintarelli, and R. Turrin. Collaborative filtering recommender systems. In The
Top-n recommendations on unpopular items with adaptive web, pages 291–324. Springer, 2007.
contextual knowledge. In Workshop on Context-aware [20] P. Seitlinger, D. Kowald, C. Trattner, and T. Ley.
Recommender Systems ’11. Recommending tags with a model of human
[5] P. Cremonesi, Y. Koren, and R. Turrin. Performance categorization. In Proc., CIKM ’13, pages 2381–2386,
of recommender algorithms on top-n recommendation New York, NY, USA, 2013. ACM.
tasks. In Proc., RecSys ’10, New York, NY, USA. [21] B. Smyth and P. McClave. Similarity vs. diversity. In
ACM. D. Aha and I. Watson, editors, Case-Based Reasoning
[6] S. Doerfel and R. Jäschke. An analysis of Research and Development, LNCS. Springer, 2001.
tag-recommender evaluation procedures. In [22] L. Steels. Semiotic dynamics for embodied agents.
Proceedings of the 7th ACM conference on Intelligent Systems, IEEE, 21(3):32–38, 2006.
Recommender systems, pages 343–346. ACM, 2013.
[23] C. Trattner, Y.-l. Lin, D. Parra, Z. Yue, W. Real, and
[7] J. Gemmell, T. Schimoler, M. Ramezani, P. Brusilovsky. Evaluating tag-based information
L. Christiansen, and B. Mobasher. Improving folkrank access in image collections. In Proc., HT ’12, pages
with item-based collaborative filtering. Recommender 113–122, New York, NY, USA, 2012. ACM.
Systems & the Social Web, 2009.
[24] K. H. Tso-Sutter, L. B. Marinho, and
[8] J. L. Herlocker, J. A. Konstan, L. G. Terveen, and L. Schmidt-Thieme. Tag-aware recommender systems
J. T. Riedl. Evaluating collaborative filtering by fusion of collaborative filtering algorithms. In Proc.
recommender systems. ACM Transactions on of SAC ’08. ACM.
Information Systems (TOIS), 22(1):5–53, 2004.
[25] S. Vargas and P. Castells. Rank and relevance in
[9] A. Hotho, R. Jäschke, C. Schmitz, and G. Stumme. novelty and diversity metrics for recommender
Information retrieval in folksonomies: Search and systems. In Proc., RecSys ’11. ACM.
ranking. In The semantic web: research and
[26] N. Zheng and Q. Li. A recommender system based on
applications. Springer, 2006.
tag and time information for social tagging systems.
Expert Syst. Appl., 2011.