<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Leveraging User-Interactions for Time-Aware Tag Recommendations</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Daniel Zoller</string-name>
          <email>zoller@informatik.uni-wuerzburg.de</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Christian Pölitz</string-name>
          <email>poelitz@informatik.uni-wuerzburg.de</email>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Stephan Doerfel</string-name>
          <email>s.doerfel@micromata.de</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Andreas Hotho</string-name>
          <email>hotho@informatik.uni-wuerzburg.de</email>
          <xref ref-type="aff" rid="aff3">3</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Micromata GmbH</institution>
          ,
          <addr-line>Kassel</addr-line>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>University of Würzburg, Data Mining and Information Retrieval (DMIR) Group</institution>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>University of Würzburg, Data Mining and Information Retrieval (DMIR) Group</institution>
        </aff>
        <aff id="aff3">
          <label>3</label>
          <institution>University of Würzburg, Data Mining and Information Retrieval (DMIR) Group, L3S Research Center</institution>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2017</year>
      </pub-date>
      <abstract>
        <p>For the popular task of tag recommendation, various (complex) approaches have been proposed. Recently however, research has focused on heuristics with low computational e ort and particularly, a time-aware heuristic, called BLL, has been shown to compare well to various state-of-the-art methods. Here, we follow up on these results by presenting another time-aware approach leveraging userinteraction data in an easily interpretable, on-the- y computable approach that can successfully be combined with BLL. We investigate the in uence of time as a parameter in that approach, and we demonstrate the e ectiveness of the proposed method using two datasets from the popular public social tagging system BibSonomy.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>INTRODUCTION</title>
      <p>Tagging allows users to easily organize and share resources online.
Users add keywords, called “tags”, to resources and store them
together, thus enabling themselves and others to retrieve these
resources using those tags as cues. Tagging is the basis of social
bookmarking systems, such as Delicious,1 BibSonomy,2 or Flickr.3
However, it has also long found its way as a secondary feature into
many applications, like web shops, wikis, blogs, or libraries.</p>
      <p>
        The task of recommending tags has proven to be a fruitful line of
research. Many approaches (see Section 3) utilize complex or
computationally costly models, often relying on the full data collected
in the system. However, [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] already showed that simple and fast
popularity-based heuristics can achieve comparably good results
as more di cult and more expensive state-of-the-art methods.
      </p>
      <p>
        Recently, [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] con rmed this tendency in a large-scale
experiment on multiple datasets, using a time-aware heuristic that
transfers the base-level learning (BLL) equation to social tagging. BLL
scores a tag based on the time that has passed since it was last used,
1https://del.icio.us (for storing web links)
2https://www.bibsonomy.org (for storing web links and publications)
3https://www. ickr.com (for storing images)
(1) We describe a heuristic to harvest tag recommendation
candidates from user interactions with the system before
posting a resource.
(2) We test two variants of a tag recommender, that utilize the
extracted recommendation candidates, one using a x
number of the most recent interactions and one using speci c
time-windows over which interactions are considered.
(3) We evaluate the overall gain in performance for
recommendations of the interaction-based heuristics, as well as
a hybridized version in combination with BLL.
      </p>
      <p>Our results demonstrate that the time-aware approach provides
better suggestions than the variant using a xed number of
previous interactions. Furthermore, a hybrid combining the time-aware,
interaction-based recommender with BLL outperforms plain BLL.
Moreover (like plain BLL), our proposed recommender is
independent of the tagged resources (making no use of their contents) and
respects the requirement of easy, online computability. We expect
our approach to be relevant for the tag recommender community,
as well as for operators of web systems who want to support their
users with tag recommendations without spending e ort and
resources on costly optimization procedures.
The remainder of this paper is structured as follows: First, we
introduce our recommender (Section 2) that leverages user
interaction in a tagging system and the incorporation of the aspect of
time. Then, we discuss related work in Section 3. We describe the
dataset and experimental setup in Section 4. Next, we present our
evaluation results for time-based and user interaction-based
recommenders in Section 5. Finally, we conclude the paper in Section 7,
after discussing our ndings in Section 6.
2</p>
    </sec>
    <sec id="sec-2">
      <title>RECOMMENDERS</title>
      <p>
        Tag recommenders provide recommendations during the process
of posting a resource. When a user u is posting a resource r , the
recommender will provide a list of tags that u might want to assign
to r . For that task, we propose and describe a new time-aware,
interaction-driven recommender which exploits the previous
interactions of the active user (the one to provide recommendations for)
with the tagging system in this section. Lastly, we describe a hybrid
approach, combining our approach with BLL (Trattner et al. [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]).
2.1
In tagging systems, and more so in systems where tagging is a
secondary feature, a user’s interactions with the system are much
more diverse than just adding and tagging new resources. Still, tag
recommendation approaches commonly focus only on the result
of these interactions (the tagged resources) to compute candidates
to recommend. In this paper, we propose going beyond only the
tagging activities and consider the context of all interactions that a
user has with the system. To generate meaningful recommendations
for a resource r , we focus especially on the most recent interactions,
as they are likely to belong to the same context as r . The rationale
behind the approach is, that by browsing and searching resources,
users reveal their current interests, which are useful to select tting
tags. A user’s interactions occur in the requests she sends to the
system. Therefore, we call our recommender approach Last Requests
(LR). To select requests from which candidate tags are computed
we distinguish two options: First, a time-window based variant
LRt: When user u stores a new resource r , recommendations are
computed using all previous requests that u made during the time
frame of length d immediately preceding the current posting of r .
In the second variant, called LRn, only the last n interactions of u
are considered. While the rst variant is explicitly time-aware and
thus more likely to capture interactions relevant in the context of
the active post, the second variant is more broadly applicable, as
users do not necessarily interact with the system prior to posting a
new resource (e.g., when using a posting tool, like a bookmarklet
or browser add-on). In such a case, older but perhaps still relevant
information could be used.
      </p>
      <p>In both variants, recommendation candidates are derived from
each of the considered interactions. To yield a ranked list of
recommendations, the resulting tags are ordered by their frequency
among the interactions.4 We describe methods for deriving
recommendable tags from interactions in Section 4.3. Both recommenders
can compute the candidates more e ciently than other
recommenders that leverage all previously used tags of a user, because
they only compute frequencies on a set of tags which is (usually
4 In the case of ties, we return the tags in lexicographic order.
much) smaller than the complete list of a user’s previously used
tags, let alone the complete historical posting data in the system.
2.2</p>
    </sec>
    <sec id="sec-3">
      <title>Hybrid with BLL</title>
      <p>
        Both proposed variants of the tag recommender cannot provide
results for new users that post resources immediately after
registration without interacting with the system. Also, it is not unusual
that users do not interact with the system before they store a new
resource. This fact also impedes the time-aware variant LRt. To
compensate, we combine each of the interaction-based methods
with BLL into a hybrid recommender. BLL has been shown to
outperform other (time-dependent) approaches (see Section 3). It uses
all tags previously used by the active user and computes a
recencybased ranking on them: Let Tu be the set of tags previously used
by user u and timep¹pº the timestamp when user u stored post p.
Further, let Yt;u be the set of tag assignments for tag t of the user u
(i.e., we add a tuple ¹u; r ; t º every time a user u annotates a resource
r with tag t to the corresponding set), and time¹yº be the timestamp
of the tag assignment y 2 Yt;u . The BLL-score of each tag in Tu
is calculated as ln¹Íy 2Yt;u ¹timep¹pº time¹yºº d º and normalized
by the softmax function over all scores. We set d = 0:5 for our
evaluation – the setting which obtained the best results in [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ].
      </p>
      <p>Our interaction-based recommender approaches are combined
with BLL in the following way: First, the interaction-based method
is used to compute a ranked list L1 of candidates (possibly of length
0). Similarly, BLL also yields a ranked list L2 of candidate tags. From
L2, we remove all tags that also occur in L1. Then the two lists are
concatenated, such that the nal ranked list of recommendations
contains the suggestions from L1 followed by those from L2. We
denote the two resulting hybrids by LRt + BLL and LRn + BLL.
3</p>
    </sec>
    <sec id="sec-4">
      <title>RELATED WORK</title>
      <p>
        Recommending tags can serve various purposes, such as increasing
the chance of getting a resource annotated, reminding a user what
a resource is about, and consolidating the vocabulary across users.
Furthermore, as Sood et al. [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ] pointed out, tag recommenders
lower the e ort of annotation by changing the process from a
generation to a recognition task: rather than “inventing” tags, the
user only needs to select some of the recommended tags.
      </p>
      <p>
        Since the emergence of social bookmarking, the topic of tag
recommendations has raised considerable interest among researchers.
As for all recommender domains, tag recommender algorithms can
roughly be classi ed into three classes: Content-based algorithms
use the content of resources, for instance to compute similarities
between items and to present items that are similar to the ones the
active user previously liked. Collaborative algorithms make use of
the relations between the users and the items, for instance by
identifying similar users and suggesting items similar users liked. The
third class are algorithms that exploit both data sources, sometimes
called hybrid recommenders. A major drawback of content-based
approaches is that the usability of a resource’s content depends
on the type of resource. For example, when the tagged resources
are textual, using words from those resources (e.g., from the title,
like Lipczak et al. [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] did) can be successful. However, the same
is much harder when the resources are images. Moreover, even
for textual resources, the implementation of the recommendation
(e.g., the word selection strategy) is resource dependent. Thus, since
our goal is to create a simple and highly versatile recommender,
we focus only on the second class of recommender algorithms –
those that exploit the folksonomy structure between users, tags,
and resources, which exists in any tagging system.
      </p>
      <p>
        An evaluation of collaborative algorithms, such as collaborative
ltering, the FolkRank algorithm [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ], and simpler, popularity-based
methods was performed in [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] on various datasets. FolkRank
outperformed the other methods. However, the hybrid heuristic based
recommender, that combined users’ frequently used tags with tags
that were frequently used to annotate the resource, was second.
Rendle and Schmidt-Thieme [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] produced recommendations with
a statistical method based on factor models. They factorized the
folksonomy structure to nd latent interactions between users,
resources and tags. Using a variant of the stochastic gradient descent
algorithm, the authors optimized an adaptation of the Bayesian
Personal Ranking criterion [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]. Seitlinger et al. [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ] proposed an
approach that simulates human category learning in a three-layer
connectionist network. In the input layer, Latent Dirichlet
Allocation is used to characterize the resource (and user). [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]
introduced a slightly di erent folksonomy graph model in which edges
are weighted and directed. On the resulting graph, PageRank is
used to produce a ranking of tags. [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] proposed ‘TagRank’, a
variant of topic-sensitive PageRank upon a tag-tag correlation graph
which they integrate into a hybrid with collaborative ltering and
popularity-based algorithms. The selection of the algorithms for
the hybrid is guided by a greedy algorithm. A drawback of all the
presented algorithms is their reliance on complex methodology that
uses the full corpus of folksonomy data to learn a recommendation
model. While they are suitable approaches to boost performance,
they also require a lot of e ort in terms of additional computation
time, hardware, implementation (e.g., additional data structures,
methods to update the trained models), and expertise. Due to the
fact that a folksonomy changes over time, the learned models must
be updated regularly to t the current data.
      </p>
      <p>
        In [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ], the authors introduce GIRP, a temporal tag usage pattern
model. It uses an exponential function that considers the rst- and
last-time usage of a tag. A short-term interests model is proposed
in [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], recommending the most popular tags of users based on
recent data, that is, data from a time-window of xed length (one day
or higher). It is found that a window of 30 days works best on the
overall BibSonomy dataset. Recently, Trattner et al. [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] presented
a comprehensive study of various tag recommender strategies,
including their own development based on a model of human memory
(BLL). In contrast to GIRP [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ], BLL models the temporal tag usage
using a power function rather than an exponential function (see
Section 2.2). They compare BLL with other methods (including several
of those mentioned above) and nd that BLL performs better than
the time-dependent algorithm GIRP and other methods based on
matrix factorization. Only more computationally expensive models
achieve a higher F-score on the evaluated datasets. Most of these
models extend basic models by re-ranking the tag candidates by
the semantic context of the resource.
      </p>
      <p>
        In this work, we assume the perspective of a tagging system
operator or, respectively, the operator of a system that includes
tagging as a secondary feature. We aim at supporting the tagging
process with as little cost as possible while still delivering good
results. Following the strong results of BLL in [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], and given its
low computational e ort and the convenient fact that it requires
no extra data structures nor precomputed values (see Section 2.2),
we use this approach as our baseline. Our method is similar to that
of Yin et al. [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ], however, instead of using tags of the previous
posts, we use tags extracted from previous user interactions with
the tagging system. We will show that the time-frame for collecting
such interactions is critical and that time-frames of less than a day
are worth considering.
4
      </p>
    </sec>
    <sec id="sec-5">
      <title>EXPERIMENTAL SETUP</title>
      <p>In this section, we introduce the tagging system BibSonomy, of
which we use data for our study. We further describe the datasets
with all preprocessing steps, the experiments and their evaluation.
4.1</p>
    </sec>
    <sec id="sec-6">
      <title>BibSonomy</title>
      <p>
        BibSonomy [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] allows users to collect references to (scienti c)
publications and bookmarks to websites, and to annotate these with
arbitrary keywords, so called tags. While entering the metadata
of a resource in a form, the user can also enter tags, which she
can later use for retrieval. The system assists the user with a tag
autocompletion relying on her previously used tags. Next to the
possibility of ltering resources by tag and/or user, users can nd
new interesting resources through a full text search. Additionally,
users can form groups in which they can share posts and
literature. On group pages, all group members’ resources are displayed.
Overview pages for websites and publications enable users to see
who else bookmarked a speci c resource and the tags they used
to describe the resource. Detail pages for publications show the
metadata that the user who saved the resource had entered for the
publication. While browsing the system, a user can copy resources
of other users into her own collection. Another feature allows users
to group tags to concepts (e.g., the tags "time" and "tag" to the
concept "recsys"). These concepts can be used for retrieving resources.
BibSonomy is a popular target for spammers, that is, for users who
store links to advertisements to promote their visibility. For that
reason, users are classi ed by a learning algorithm and manually
by the system’s administrators. For our analysis, we only used data
generated by users that were not marked as spammers.
4.2
      </p>
    </sec>
    <sec id="sec-7">
      <title>Datasets</title>
      <p>Our experiments rely on two types of data gathered from the
realworld tagging system BibSonomy: posts and user interactions. The
latter type of data is rarely published – due to privacy concerns.
However, BibSonomy makes such data available to researchers
in the form of collected HTTP-request server logs.5 Thus, at the
moment, BibSonomy is the only source enabling the analyses
presented here. The methodology, however, is transferable to other
tagging systems.</p>
      <p>
        Request Log Data: The request log data contains every web
request any user made to the tagging system. We removed all
nonhuman requests like redirects to other pages, or requests by bots or
other applications (using the user agent information). Also, we only
considered requests to HTML sites and excluded system pages (e.g.,
the login page). We used two di erent time frames for the
evaluation: (i) from 2006-01-01 (the start date of BibSonomy) through
2011-12-31 (a dataset already used in behavioral analyses in
previous work [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]) and (ii) a more recent share, ranging from 2014-07-01
5http://www.kde.cs.uni-kassel.de/bibsonomy/dumps
through 2016-06-30. For the remainder of this paper, we refer to the
older dataset as Bib1 and to the newer dataset as Bib2. The older
dataset contains 2,074,182 and the newer 246,472 requests.
Content Data: The second type of data is the data generated by
the users of the system by annotating resources with tags. We split
the content dataset of BibSonomy into the same two time frames
as the interaction data. We applied several cleaning steps to each
dataset. First, we removed system tags (like myown or imported).
To lter imports, we deleted all posts of a user that shared a posting
date with another post.6 The remaining tags were normalized by
decapitalizing and removing all non-alphanumeric characters from
the tag string. We did not prune our datasets using a p-core, to avoid
biases to the results (see Section 4.4). Furthermore, the goal of this
research is to produce broadly applicable recommender strategies,
but restricting the dataset to only its dense part would neglect new
users, rare tags and rare resources, thus providing an incomplete
impression on the overall performance. The statistics for the two
cleaned content datasets can be found in Table 1.
4.3
      </p>
    </sec>
    <sec id="sec-8">
      <title>Extraction of Tag Recommendation</title>
    </sec>
    <sec id="sec-9">
      <title>Candidates from User Interaction</title>
      <p>While browsing in a tagging system, the user queries di erent page
types. For extracting tag recommendation candidates from requests
to BibSonomy, we are using the following methods for the di erent
page types. After the extraction, we also normalized the extracted
tags as described in the previous section.</p>
      <p>
        Tag Pages: In BibSonomy, users can restrict the global collection of
resources or the collections of other users, groups and search results
by tags on a separate tag page for the corresponding entity. All
pages also support to lter with more than one tag. We represented
a request to one of the tag pages with the speci ed tag(s).
User Pages: For user pages, we extracted the user’s tags that she
used before the request was made for her own posts. We only
considered user pages where the logged-in user requested a user
page of another user. This is the same representation that [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ] used
for their analysis.
      </p>
      <p>
        Resource Pages: Also, like [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], we represent a publication or
website overview page with the tags that any user used to describe
the requested resource. We restrict the tags extracted from a details
page to the page owner’s tags.
      </p>
      <p>Concept Pages: Users can request concept pages for users, groups,
or globally, showing resources tagged with the keywords the user,
group or all users de ned as subtags of the concept. We added all
subtags of the concept to the set of considered tag candidates.
Search Pages: We represent a search request to BibSonomy with
the terms of the search query after removing stop words using a
multi-language stop word list.
6Furthermore, we removed user accounts that are used by libraries, like DBLP, from
the two BibSonomy datasets.</p>
      <p>External Referer: When users use external engines for searching
and click on results linking to BibSonomy, the external source is
logged as referer in the request logs. For candidate extraction, we
tokenize the value of the ’q’ url parameter (because it is the most
commonly used parameter by search engines) of these requests and
remove stop words.</p>
      <p>Tags of a copied post: While browsing by tags or searching, the
system presents the user with the resources that match her entered
query. Next to every resource, the user can click on a copy button.
This click is recorded in the request logs. We extract the tags of the
copied resource to represent this type of request.</p>
      <p>While some of the request types (e.g., concepts) are speci c to
BibSonomy, most of them are usually present in a tagging
system, representing search options as well as the typical navigation
paradigm in a folksonomy.
4.4</p>
    </sec>
    <sec id="sec-10">
      <title>Evaluation</title>
      <p>For tuning the parameters t and n of the two recommender
heuristics, LRt and LRn, we split each of the two datasets into a validation
and a test set: each part contains 50 % of the posts. We use the
validation set to determine the best parameters t and n. Then we
use the test to evaluate a hybrid with BLL (cf. Section 2.2).</p>
      <p>
        The choice of the evaluation setup often has a strong in uence
on the experiments. For example, Cremonesi et al. [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] observed
that di erent sampling strategies yield di erent outcomes, while
Doerfel et al. [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ] showed that di erent restrictions on the dataset can
also lead to di erent (contradictory) results. Both suggest that the
scenario should be selected such that it resembles reality as much
as possible and that choices should be based on the use case rather
than on issues like sparsity. Therefore, we adapt the rating-based
temporal leave-one-out method introduced in [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] to our scenario. In
our experiments, we consider each post as a test post. Moreover, we
ensure that the algorithms only use data from before the creation of
the test posts. More speci cally, for each post p, we do the following:
We select all posts (and requests) that have been created before p
and use it to compute recommendations for p based on the user
and resource of p. Then we compare the recommended tags to the
actual tags of p and evaluate the number of correctly predicted tags.
This scenario is the most realistic o ine evaluation scenario as it
considers each occasion for recommendations and uses only data
resembling exactly the state of the system at the time the test post
was actually created.
      </p>
      <p>
        Metric: We use the standard information performance metric
Fscore for measuring the quality of the recommendations [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Online
systems usually present users with only a limited number of tag
recommendations while saving a new resource (often ve tags). For
that reason, we report F@5, that is, the F-score computed for the
set of the rst ve suggested tags from the ranked list of
recommendations (cf. Section 2.2). The parameters t and n are selected based
on the best F@5 score found in experiments on the validation sets.
Signi cance: To test the obtained results for statistically signi
cant di erences, we use the Wilcoxon signed-rank test. Since we
consider a large number of posts for the evaluation, we compute
two versions – considering either the posts or the users as the
population. For the latter case, we averaged the obtained F@5 scores over
all posts per user in the test set, and we conduct the signi cance
test based on these averages. To indicate statistical signi cance, we
use the symbol * after a reported F-score when the user-based test
BLL
LRn=N
LRn=N + BLL
      </p>
      <p>BLL
LRt =T
LRt =T + BLL
0.13
0.16
Bib1 (T=1m)
recommender</p>
      <p>BLL
hybrid
indicates signi cance, and we use + for the post-based test, in both
cases using the α -level of 0:01.
5</p>
    </sec>
    <sec id="sec-11">
      <title>RESULTS</title>
      <p>In this section, we present the results of our experiments. In the rst
set of experiments, we use the validation datasets to tune our two
approaches LRt and LRn, using various settings for t and n, thus
including di erent sets of user interactions. In all these experiments,
we consider only those posts where the situation was suitable for
the respective approach, that is, where interactions with the system
had been recorded. Thus, the number of considered posts for which
an approach is tested varies from recommender type and setting.
On the same subsets of posts, we also evaluate BLL and the hybrid
with BLL for comparison. Eventually, in Section 5.2, we use the
test datasets to evaluate the impact of our approach on the overall
performance, that is, we evaluate the recommender strategies using
all posts without any restriction.
5.1
First, we report the results for the recommenders LRt and LRn (see
Section 2.1). We report results averaged over all posts where the
respective heuristic was applicable (i.e., where there were observable
interactions), and we compare to BLL on the same set of posts.</p>
      <p>In our rst evaluation, we vary the parameter n – the number of
included previous interactions – of the LRn recommender from 1 to
10 on the validation set and report the results of the best parameter
in Table 2a. We nd that on Bib1 and Bib2 similar con gurations
(n = 6 for Bib1 and n = 7 for Bib2) worked best for LRn. Further,
we can observe that LRn alone is clearly inferior to BLL in terms of
F@5. Combining results by concatenating the lists of recommended
tags (LRn + BLL), as described in Section 2.2, can improve the F@5
score, but still cannot reach that of plain BLL. Our hypothesis is that
the last requests are too far in the past and thus have no relevance
for the current post. Thus, the time of the considered interactions
is a critical factor.</p>
      <p>Therefore, in Table 2b, we switch the mode of selecting requests
to time-windows, using LRt. We vary the considered time-window
for including requests from one minute to 30 days.7 Although the
number of posts for which the heuristic is applicable grows with the
selected time-window length, we observe decreasing scores for LRt
on both datasets – more evidence for the above hypothesis. We nd
that using a time-window t of one minute yields the highest
performance on (the respective validation sets of) both datasets, Bib1 and
Bib2. Other than before with LRn, the time-aware interaction-based
heuristic LRt yields F@5 scores comparable to BLL on both Bib1
and Bib2. When combining the two recommenders (LRt =T + BLL),
the scores improve signi cantly, by ten percentage points over
plain BLL on both sets. The improvements obtained for Bib1 are
signi cant according to the Wilcoxon signed-rank test, considering
both the users and the posts as entities. On Bib2, the signi cance
of the hybrid’s improvement is con rmed when the posts are used
as entities in the test.</p>
      <p>
        Figure 1 shows the trend of the F@5 scores calculated on BLL
compared to the hybrid LRt + BLL for the time-windows ranging
730 days is the setting for which Yin et al. [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ] report the best results on the overall
dataset for their approach based in previous posts.
      </p>
      <p>Bib2 (N=7)
0.10
0.17
Bib2 (T=1m)
910 163 0.33
0.35
0.45+
from one minute to ve minutes. We can observe, that BLL
remains roughly constant when t increases. On the other hand, the
combination of LRt and BLL decreases from 0.42 to 0.35 when we
increase the time-window t from one minute to ve minutes, but
outperforms BLL for every considered time-window. For the dataset
Bib2, we nd similar results, except that LRt decreases faster ( gure
omitted due to space limitations).
5.2</p>
    </sec>
    <sec id="sec-12">
      <title>Overall Performance</title>
      <p>In the previous section, we saw that the time-window based LRt
recommender achieved better results than BLL on those subsets
of the data where the respective heuristic was applicable. In this
section, we evaluate the hybrid of LRt with BLL on the complete
test sets to get an impression of its overall impact as an
improvement over plain BLL. In the following, we combine LRt with those
parameters that produced the best results on the validation sets.
Results are given in Table 3. The combination with the request-based
LRt recommender improves the recommendation result by about
three per cent on the Bib1 test set and one per cent on the Bib2 test
set. A Wilcoxon signed-rank test, conducted on the average F-score
of each user, indicates that the di erence is signi cant for Bib1.
Testing with all posts as entities, signi cance is con rmed for both
Bib1 and Bib2. Thus, we can conclude that exploiting user
interactions in very short time-windows immediately before posting a
resource can boost the performance of the already well-performing
recommender algorithm BLL.
6</p>
    </sec>
    <sec id="sec-13">
      <title>DISCUSSION AND LIMITATIONS</title>
      <p>In our studies, we found that the time-window based recommender
LRt provides better recommendations than the recommender LRn,
which uses the last requests. We also saw that in those situations
where it is applicable its results are comparable to the more complex
algorithm BLL. Combining the time-window variant with BLL into
a hybrid signi cantly improves the performance. In the following
we discuss several aspects and limitations of our study.
Timing: Overall, we could demonstrate the applicability of our
time-aware interaction-driven heuristic. We saw particularly that
short time spans immediately before the posting of a resource yield
good recommendations. It seems that users store posts in bursts,
each representing di erent aspects of their (shifting) interests. Thus,
relying on very recent tags is a reasonable approach.</p>
      <p>Applicability: The number of posts and users for which the
timewindow based recommender LRt can provide tag recommendations
is only a relatively small subset of the data. One reason for this
phenomenon may be the fact that BibSonomy o ers browser
extensions for posting resources, thus a useful means of posting to
BibSonomy without visiting the system rst.</p>
      <p>
        Generalizability of the Results: Since we could evaluate our
recommendation approach only on two BibSonomy datasets (due
to the unavailability of suitable data from other systems), it remains
an open research question to see how it would perform in other
tagging systems. Heckner et al. [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ] found that users of tagging
systems with di erent resources tend to tag for di erent reasons,
for example, Flickr (images) is used mostly for sharing, Delicious
(websites) mainly for retrieving resources from one’s own collection.
Since the resource types in BibSonomy and Delicious are similar
(references to documents; websites in both systems, publications
only in BibSonomy), we hypothesize that they are often used for
similar purposes. Also, most page types (e.g., a user page) exist
in both systems. Thus, we would expect results of recommender
algorithm experiments conducted on Delicious to be qualitatively
similar to those found for BibSonomy. Another in uence on the
usage of tagging systems is the user interface which varies from
system to system. For example, BibSonomy always links the entities
of the folksonomy, while other systems may exclude some links or
place links on di erent positions within a page.
      </p>
      <p>Transferability of the Approach: While the above mentioned
issues are limitations to the generalizability of the results in our study,
our methods of extracting tag recommendations from interactions
can easily be adapted to other tagging systems.</p>
      <p>Computational Cost: In contrast to many other methods (cf.
Section 3), our heuristic does not require large user pro les, as it draws
the recommendations only from the interactions in the tagging
system directly preceding a new post. Moreover, the heuristic is easy
to implement into arbitrary tagging systems, and requires only data
collected from browsing activities. While, in our experiments, we
made use of the request logs to exploit interactions, in a production
environment tags can be derived directly from the interactions and
can be stored into a temporary cache. Thus, the recommendations
can be computed directly without accessing additional data sources.
Relying solely on counting occurrences and restrictions on small
subsets of interactions, recommendations can be computed online
without previous training. Consequently, they require only little
e ort, making them ideal candidates for systems where tagging is
included as a secondary feature and for quickly prototyping a new
tagging system.</p>
      <p>Explainability: Finally, it is worth pointing out that, in our
approach, the choice of the recommended tags can be easily explained.
Explanations are suitable for increasing users’ acceptance of the
recommendations, particularly as the explanations reveals that no
pro ling of the user is necessary as only the few recent activities
are exploited.
7</p>
    </sec>
    <sec id="sec-14">
      <title>CONCLUSION</title>
      <p>In this work, we have proposed a time-aware, interaction-driven
tag recommendation heuristic, for the rst time leveraging user
interaction in a tagging system beyond the publicly visible posting.
We have evaluated our approaches on data of the real-world tagging
system BibSonomy. We have shown that time is a critical factor
and that particularly interactions immediately before a new post
are a good source for recommendable tags.</p>
      <p>Finally, combining the time-aware approach with BLL leads to
a hybrid recommender that outperforms both individual
components. The result is an e ective recommender system that is easy
to implement and independent of the tagged resources and that
requires no o ine training. The approach is thus not only suitable
for dedicated tagging systems, but also for broader web systems in
which tagging is merely a secondary feature.</p>
      <p>Future work: In this study, we have evaluated LRt and LRn with
xed parameters (time-window length or number of considered
interactions) for all users. A chance for improving the results
further would be an analysis of personalized parameters for each
user or di erent user groups. It would also be conceivable to use
more re ned methods to detect the user’s current context when
posting a new resource. For example, by looking at the requests,
it might be possible to distinguish situations where users do
research regarding one speci c topic from situations where users just
“stumble” through the system changing their focus based on what
they nd on each page. Another topic for future work is the tag
candidate extraction. For this study, we extracted only the directly
requested tags or query terms from a request. In tagging systems,
one could leverage the semantic context of such tags (arising from
the co-occurrence with other tags on the same resources). Finally,
instead of exploiting retrieval interaction on the level of individual
requests, one could attempt to identify sessions. These might yield
a more comprehensive understanding of the current user context
than considering individual requests independently.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>Dominik</given-names>
            <surname>Benz</surname>
          </string-name>
          , Andreas Hotho, Robert Jäschke, Beate Krause, Folke Mitzla , Christoph Schmitz, and
          <string-name>
            <given-names>Gerd</given-names>
            <surname>Stumme</surname>
          </string-name>
          .
          <year>2010</year>
          .
          <article-title>The social bookmark and publication management system BibSonomy</article-title>
          .
          <source>The VLDB Journal 19</source>
          ,
          <issue>6</issue>
          (
          <year>2010</year>
          ),
          <fpage>849</fpage>
          -
          <lpage>875</lpage>
          . DOI: https://doi.org/10.1007/s00778-010-0208-4
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>Robin</given-names>
            <surname>Burke</surname>
          </string-name>
          .
          <year>2010</year>
          .
          <article-title>Evaluating the Dynamic Properties of Recommendation Algorithms</article-title>
          .
          <source>In Proceedings of the Fourth ACM Conference on Recommender Systems. ACM</source>
          , New York, NY, USA,
          <fpage>225</fpage>
          -
          <lpage>228</lpage>
          . DOI:https://doi.org/10.1145/1864708. 1864753
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Paolo</given-names>
            <surname>Cremonesi</surname>
          </string-name>
          , Yehuda Koren, and
          <string-name>
            <given-names>Roberto</given-names>
            <surname>Turrin</surname>
          </string-name>
          .
          <year>2010</year>
          .
          <article-title>Performance of Recommender Algorithms on Top-n Recommendation Tasks</article-title>
          .
          <source>In Proceedings of the Fourth ACM Conference on Recommender Systems. ACM</source>
          , New York, NY, USA,
          <fpage>39</fpage>
          -
          <lpage>46</lpage>
          . DOI:https://doi.org/10.1145/1864708.1864721
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>Stephan</given-names>
            <surname>Doerfel</surname>
          </string-name>
          , Robert Jäschke, and
          <string-name>
            <given-names>Gerd</given-names>
            <surname>Stumme</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>The Role of Cores in Recommender Benchmarking for Social Bookmarking Systems</article-title>
          .
          <source>ACM Transactions on Intelligent Systems and Technology 7</source>
          ,
          <issue>3</issue>
          (
          <year>February 2016</year>
          ),
          <volume>40</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>40</lpage>
          :
          <fpage>33</fpage>
          . DOI: https://doi.org/10.1145/2700485
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>Stephan</given-names>
            <surname>Doerfel</surname>
          </string-name>
          , Daniel Zoller, Philipp Singer, Thomas Niebler, Andreas Hotho, and
          <string-name>
            <given-names>Markus</given-names>
            <surname>Strohmaier</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>What Users Actually do in a Social Tagging System: A Study of User Behavior in BibSonomy</article-title>
          .
          <source>ACM Transactions on the Web</source>
          <volume>10</volume>
          ,
          <issue>2</issue>
          (
          <year>2016</year>
          ),
          <volume>14</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>14</lpage>
          :
          <fpage>32</fpage>
          . DOI:https://doi.org/10.1145/2896821
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>Folke</given-names>
            <surname>Eisterlehner</surname>
          </string-name>
          , Andreas Hotho, and Robert Jäschke (Eds.).
          <year>2009</year>
          .
          <article-title>ECML PKDD Discovery Challenge 2009 (DC09). CEUR-WS.org</article-title>
          , Vol.
          <volume>497</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>Asela</given-names>
            <surname>Gunawardana</surname>
          </string-name>
          and
          <string-name>
            <given-names>Guy</given-names>
            <surname>Shani</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>A Survey of Accuracy Evaluation Metrics of Recommendation Tasks</article-title>
          .
          <source>Journal of Machine Learning Research</source>
          <volume>10</volume>
          (
          <year>2009</year>
          ),
          <fpage>2935</fpage>
          -
          <lpage>2962</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>M.</given-names>
            <surname>Heckner</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Heilemann</surname>
          </string-name>
          , and
          <string-name>
            <given-names>C.</given-names>
            <surname>Wol</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>Personal Information Management vs</article-title>
          . Resource Sharing:
          <article-title>Towards a Model of Information Behaviour in Social Tagging Systems</article-title>
          .
          <source>In Proceedings of the thrid AAAI Conference on Weblogs and Social Media</source>
          . San Jose, CA, USA.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>Andreas</given-names>
            <surname>Hotho</surname>
          </string-name>
          , Robert Jäschke, Christoph Schmitz, and
          <string-name>
            <given-names>Gerd</given-names>
            <surname>Stumme</surname>
          </string-name>
          .
          <year>2006</year>
          .
          <article-title>Information Retrieval in Folksonomies: Search and Ranking</article-title>
          .
          <source>In The Semantic Web: Research and Applications: Third European Semantic Web Conference</source>
          , ESWC 2006 Budva, Montenegro, June 11-14,
          <year>2006</year>
          Proceedings (LNCS),
          <source>York Sure and John Domingue (Eds.)</source>
          , Vol.
          <volume>4011</volume>
          . Springer Berlin Heidelberg, Berlin/Heidelberg, 411-
          <fpage>426</fpage>
          . DOI:https://doi.org/10.1007/11762256_
          <fpage>31</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Robert</surname>
            <given-names>Jäschke</given-names>
          </string-name>
          , Leandro Marinho, Andreas Hotho, Lars Schmidt-Thieme, and
          <string-name>
            <given-names>Gerd</given-names>
            <surname>Stumme</surname>
          </string-name>
          .
          <year>2008</year>
          .
          <article-title>Tag Recommendations in Social Bookmarking Systems</article-title>
          .
          <source>AI Communications</source>
          <volume>21</volume>
          ,
          <issue>4</issue>
          (
          <year>2008</year>
          ),
          <fpage>231</fpage>
          -
          <lpage>247</lpage>
          . DOI:https://doi.org/10.3233/ AIC-2008-0438
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Marek</surname>
            <given-names>Lipczak</given-names>
          </string-name>
          , Yeming Hu, Yael Kollet, and
          <string-name>
            <given-names>Evangelos</given-names>
            <surname>Milios</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>Tag Sources for Recommendation in Collaborative Tagging Systems</article-title>
          ,
          <source>See [6]</source>
          ,
          <fpage>157</fpage>
          -
          <lpage>172</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Feichao</surname>
            <given-names>Ma</given-names>
          </string-name>
          , Wenqing Wang, and
          <string-name>
            <given-names>Zhihong</given-names>
            <surname>Deng</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>TagRank: A new tag recommendation algorithm and recommender enhancement with data fusion techniques</article-title>
          .
          <source>In Social Media Retrieval and Mining</source>
          ,
          <source>Shuigeng Zhou and Zhiang Wu (Eds.)</source>
          .
          <source>Communications in Computer and Information Science</source>
          , Vol.
          <volume>387</volume>
          . Springer, Berlin/Heidelberg, 80-
          <fpage>91</fpage>
          . DOI:https://doi.org/10.1007/978-3-
          <fpage>642</fpage>
          -41629-
          <issue>3</issue>
          _
          <fpage>7</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Thomas</surname>
            <given-names>Niebler</given-names>
          </string-name>
          , Martin Becker, Daniel Zoller, Stephan Doerfel, and
          <string-name>
            <given-names>Andreas</given-names>
            <surname>Hotho</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>FolkTrails: Interpreting Navigation Behavior in a Social Tagging System</article-title>
          .
          <source>In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management. ACM</source>
          , New York, NY, USA. DOI:https://doi.org/10. 1145/2983323.2983686
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>Maryam</given-names>
            <surname>Ramezani</surname>
          </string-name>
          .
          <year>2011</year>
          .
          <article-title>Improving graph-based approaches for personalized tag recommendation</article-title>
          .
          <source>Journal of Emerging Technologies in Web Intelligence</source>
          <volume>3</volume>
          ,
          <issue>2</issue>
          (
          <year>2011</year>
          ),
          <fpage>168</fpage>
          -
          <lpage>176</lpage>
          . DOI:https://doi.org/10.4304/jetwi.3.2.
          <fpage>168</fpage>
          -
          <lpage>176</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Ste</surname>
            <given-names>en Rendle</given-names>
          </string-name>
          , Christoph Freudenthaler, Zeno Gantner, and
          <string-name>
            <surname>Schmidt-Thieme Lars</surname>
          </string-name>
          .
          <year>2009</year>
          .
          <article-title>BPR: Bayesian personalized ranking from implicit feedback</article-title>
          .
          <source>In Proceedings of the 25th Conference on Uncertainty in Arti cial Intelligence</source>
          . AUAI Press, Arlington, Virginia, United States,
          <fpage>452</fpage>
          -
          <lpage>461</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          <source>[16] Ste en Rendle and Lars Schmidt-Thieme</source>
          .
          <year>2009</year>
          .
          <article-title>Factor Models for Tag Recommendation in BibSonomy</article-title>
          ,
          <source>See [6]</source>
          ,
          <fpage>235</fpage>
          -
          <lpage>242</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <given-names>Paul</given-names>
            <surname>Seitlinger</surname>
          </string-name>
          , Dominik Kowald, Christoph Trattner, and
          <string-name>
            <given-names>Tobias</given-names>
            <surname>Ley</surname>
          </string-name>
          .
          <year>2013</year>
          .
          <article-title>Recommending tags with a model of human categorization</article-title>
          .
          <source>In Proceedings of the 22nd International Conference on Conference on Information and Knowledge Management. ACM</source>
          , New York, NY, USA,
          <fpage>2381</fpage>
          -
          <lpage>2386</lpage>
          . DOI:https://doi.org/10.1145/ 2505515.2505625
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Sanjay</surname>
            <given-names>Sood</given-names>
          </string-name>
          , Sara Owsley, Kristian Hammond, and
          <string-name>
            <given-names>Larry</given-names>
            <surname>Birnbaum</surname>
          </string-name>
          .
          <year>2007</year>
          .
          <article-title>TagAssist: Automatic Tag Suggestion for Blog Posts</article-title>
          .
          <source>In Proceedings of the 1st International Conference on Weblogs and Social Media</source>
          . Boulder, Colorado, USA.
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Christoph</surname>
            <given-names>Trattner</given-names>
          </string-name>
          , Dominik Kowald, Paul Seitlinger, Tobias Ley, and
          <string-name>
            <given-names>Simone</given-names>
            <surname>Kopeinik</surname>
          </string-name>
          .
          <year>2016</year>
          .
          <article-title>Modeling Activation Processes in Human Memory to Predict the Use of Tags in Social Bookmarking Systems</article-title>
          .
          <source>Journal of Web Science</source>
          <volume>2</volume>
          ,
          <issue>1</issue>
          (
          <year>2016</year>
          ),
          <fpage>1</fpage>
          -
          <lpage>16</lpage>
          . DOI:https://doi.org/10.1561/106.00000004
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Dawei</surname>
            <given-names>Yin</given-names>
          </string-name>
          , Liangjie Hong, Zhenzhen Xue, and
          <string-name>
            <given-names>Brian D.</given-names>
            <surname>Davison</surname>
          </string-name>
          .
          <year>2011</year>
          .
          <article-title>Temporal Dynamics of User Interests in Tagging Systems</article-title>
          .
          <source>In Proceedings of the 25th AAAI Conference on Arti cial Intelligence</source>
          . AAAI.
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <surname>Lei</surname>
            <given-names>Zhang</given-names>
          </string-name>
          , Jian Tang, and
          <string-name>
            <given-names>Ming</given-names>
            <surname>Zhang</surname>
          </string-name>
          .
          <year>2012</year>
          .
          <article-title>Integrating Temporal Usage Pattern into Personalized Tag Prediction</article-title>
          .
          <source>In Proceedings of the 14th AsiaPaci c International Conference on Web Technologies and Applications (APWeb'12)</source>
          . Springer-Verlag, Berlin, Heidelberg,
          <fpage>354</fpage>
          -
          <lpage>365</lpage>
          . DOI:https://doi.org/10. 1007/978-3-
          <fpage>642</fpage>
          -29253-8_
          <fpage>30</fpage>
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>