<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>TCNSVD: A Temporal and Community-Aware Recommender Approach</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Mohsen Shahriari</string-name>
          <email>shahriari@dbis.rwth-aachen.de</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Ralf Klamma</string-name>
          <email>klamma@dbis.rwth-aachen.de</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Collaborative Filtering, Community Detection, Community Dri,</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Martin Barth</string-name>
          <email>barth@dbis.rwth-aachen.de</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Christoph Traner</string-name>
          <email>christoph.trattner@modul.ac.at</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>MODUL University Vienna</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>RWTH Aachen University</institution>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Time-Aware Recommender Models</institution>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2017</year>
      </pub-date>
      <abstract>
        <p>Recommender systems support users in nding relevant items in overloaded information spaces. Researchers and practitioners have proposed many dierent collaborative ltering algorithms for different information scenarios, domains and contexts. One of the laer, are time-aware recommender methods that consider temporal dynamics in the users' interests in certain items, topics, etc. While there is extensive research on time-aware recommender systems, surprisingly, researchers have paid lile aention to model temporal community structure dynamics (community dri). In consequence, recommender systems seldom exploit explicit and implicit community structures that are present in online systems, where one can see what others have been watching, sharing and or tagging. In this paper, we propose a recommender method that not only considers temporal interest dynamics in online communities, but also exploits the social structure by the means of community detection algorithms. We conducted oine experiments on the Netix dataset and the latest MovieLens dataset with tag information. Our method outperformed the current state-of-the-art in rating and item-ranking prediction. is work contributes to the connection of two separate recommender research directions, in which exploits community structure and temporal eects together in recommender systems.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>INTRODUCTION</title>
      <p>
        Since many years, recommender systems based on collaborative
ltering techniques provide recommendations for us by applying
specic approaches on a huge rating matrix. However, it is
expensive to create and maintain such huge rating matrices for online
shops and rating websites. From our own experience, we know
that we do not update ratings when our judgment has changed
due to changes in our taste, nor do we reect that our ratings
are based on the inuence of somebody we know. But, research
has shown that we can increase the recommendation accuracy by
taking into account temporal eects with computational methods
e.g. changes in the user preferences or the item popularity over
time. is has lead to the development of time-aware recommender
systems [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. In parallel, the social network research community
has investigated the detection and analysis of community
structures as implicit inuence among members of a social network
[
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. Members of a community are supposed to possess similar
properties so that they form dense connections inside
communities and sparse connections among them. Correspondingly, more
and more recommender systems consider community structures
to e.g. improve accuracy [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]. However, one important property of
social network research is still missing in recommender research,
namely the temporal dynamics of online community structures
[
        <xref ref-type="bibr" rid="ref1 ref22">1, 22</xref>
        ]. Temporal community structures - detected from explicit
and implicit users’ interactions and item-item networks - provide
dynamics of collective information carried by groups of people.
ese communities are dynamic similar to the network, in which
this needs to be reected in recommender systems. To the best of
our knowledge, there is still no other work that explains to what
extent temporal dynamics of online communities can be eective
in the proposal of a recommender system.
      </p>
      <p>
        Objective. In this line of research, we propose two recommender
models named CNSVD and TCNSVD. CNSVD considers the
collective user preferences and item receptions at the same time.
TCNSVD, on the other hand, includes temporal dynamics of
(overlapping) community structures, which is not yet addressed by the
research community. ese models are extensions to the NSVD and
TNSVD models proposed by Koren [
        <xref ref-type="bibr" rid="ref20 ref21">20, 21</xref>
        ] - leaning upon
neighborhood and factor models of recommendation. Using user-user and
item-item networks contributes to the evaluation of (overlapping)
community detection algorithms as well as the graph
construction methods. Furthermore, we perform extensive studies of the
proposed models on two large-scale and popular datasets -
MovieLens and Netix - and compare them with the state-of-the-art
approaches.
2
      </p>
    </sec>
    <sec id="sec-2">
      <title>RELATED WORK</title>
      <p>In this paper, the proposed algorithms are related to both
timeaware and community-aware recommender models. As such the
related work in the area is shortly reviewed.</p>
      <p>
        Time-Aware Recommendation. Some research has been done in
the proposal of recommender systems dealing with temporal eects
in recent years. Campos et al. [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] presented a survey and analysis
of time-aware recommender systems. ey claimed that time is
one of the most useful contextual dimensions in recommender
systems. Koren [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] applied a model-based collaborative ltering
approach using a combination of neighborhood and factor models.
He used models that track temporal shis over several relevant
characteristics e.g. user and item biases, covering both long-term
and short-term temporal eects. Daneshmand et al. [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ] assumed
a hidden item network structure that can be inferred from users’
sequences of selecting items. e basic idea is that if two items in
the hidden network are related, then a user selecting one of the two
items is likely to also select the other item. Baltrunas and Amatriain
[
        <xref ref-type="bibr" rid="ref2">2</xref>
        ] proposed a slightly dierent approach to time dependency in
recommendation, and assumed time-changing but repetitive user
preferences. ey recommended music, which depends on the
time of day, week or year using a collaborative ltering approach.
Charlin et al. proposed a dynamic matrix factorization model based
on Poisson factorization for recommendation, which considers
temporal users’ interests and item popularity [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ].
      </p>
      <p>
        Community-Aware Recommendation. ere has also been work
on using community detection for recommendation tasks.
Dolgikh and Jelinek [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ] proposed an approach to recommend music
using community detection. ey constructed an artist-tag
network for each user from the user’s favorite artists and the tags
assigned to these artists. Community detection was applied on
these networks to determine each user’s interest subeld, which
were then used for recommending artists to the user. Cao et al.
[
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] applied a neighborhood-based collaborative ltering approach
for recommending movies to users. ey reduced computation
time and improved recommendation precision by using
community structures. ey applied a community detection algorithm
on the network constructed from similarity among users which
was derived from the user-item ratings. Choo et al. [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ] proposed
a neighborhood-based collaborative ltering approach that uses
user communities. e user network was derived from
reviewreply paerns, i.e. if one user reviews an item and another user
replies to that review then there may be a relation between the two
users. User communities were derived from this network and used
as a basis for the recommendation process. Bellogin and Parapar
[
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] constructed a user graph using Pearson correlation similarity
and applied normalized graph cuts to nd clusters of users. ese
clusters were then used for neighbor selection in user-based
collaborative ltering. Other approaches using community structures
alleviated the cold-start problem in collaborative ltering. [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ] and
[
        <xref ref-type="bibr" rid="ref30">30</xref>
        ] both addressed the cold-start problem for new users by taking
into account additional user information.
      </p>
      <p>Summary. To the best of our knowledge, there are no approaches
that take into account temporal dynamics of community structures
to support recommendation. us, the literature manifests that 1)
our knowledge regarding performance of (overlapping) community
structures on recommendation systems is imperceptible. 2) we
are not aware about the goodness of graph construction similarity
metrics, e.g. Jaccard, Cosine, etc, in time-evolving recommender
models. 3) we also know very lile regarding the eect of metadata
information on graph construction in temporal community-aware
recommender systems. 4) we need models to connect temporal
dynamics with (overlapping) community structures to improve item
ranking and precision accuracy metrics in recommender systems.
3</p>
    </sec>
    <sec id="sec-3">
      <title>PROPOSED RECOMMENDER MODELS</title>
      <p>
        In this section, we introduce the proposed recommender models.
Koren [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ] employed the neighborhood and factor models to
compute the rating of a specic user on a particular item. In this model,
weights of user or item similarities are interpreted as osets that
need to be added to a baseline estimation. In other words, this
approach combines three components including a baseline estimation,
a neighborhood and a factor model, in which can be wrien as
follows:
      </p>
      <p>
        1 X ((ru j
μ + bu + bi + jIu j 2
where
the rst block of terms refers to the baseline estimation, in which
μ is the average rating over all users and items and bu is the user
bias, i.e. the deviation of the average rating given by user u from
μ. Besides, bi is the deviation of the average rating given to item
i from μ (item bias).
the second block of terms refers to the neighborhood model
contribution, in which Iu is the set of items rated by user u, wi j
relates to explicit rating feedback, which is multiplied by ru j bu j ,
and ci j is related to implicit feedback and is added whenever user
u has given a rating to item j. In addition, to avoid overestimation
of the rating of users who provide much feedback, i.e. jIu j is
1
high, the estimation is scaled down by multiplying with jIu j 2 .
nally the last block of terms indicates the factor model
contribution, in which qi and pu are vectors characterizing item i and
user u, respectively. Moreover, the user preference vector u is
complemented by a sum of vectors yj , that represent implicit
feedback from each item j 2 Iu .
is model is named Neighborhood-Integrated SVD (NSVD), in
which parameters, i.e. bu , bi , wi j , ci j , yi j , qi and pu are learned by
minimizing a squared error function. Koren extended the NSVD
model by using temporal information to improve recommendation
accuracy. e temporal information allows the modeling of user
preferences and item characteristics that change over time [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ].
Temporal information was included into each of the three
components i.e., baseline estimation, neighborhood model and factor
(2)
(3)
(4)
(5)
(6)
where we sum over all communities that user u belongs to, in
other words, we add the corresponding biases bC weighted by
the user’s membership level muC regarding each community.
b Ci refers to the community bias for item i with Ci as the set of
item communities that i belongs to. Similarly, we dene the item
community bias as follows:
b Ci =
      </p>
      <p>X
C 2 Ci
bC
miC ;
where bC shows the community bias of community C, in which
miC represents the membership level of item i belonging to
community C.</p>
      <p>Neighborhood Model. To use user community information for
the neighborhood model, we extend the original neighborhood
model to capture additional implicit feedback from each item that
has been rated by a member of one of the user’s communities. e
extension is as follows:</p>
      <p>1 X ((ru j
jIu j 2
j 2ICu
X</p>
      <p>C 2 Ci
where the block of terms shows the community contribution in the
neighborhood model. Here, I Cu represents the set of items that any
user belonging to one of user u’s communities has rated. en, di j
shows the oset that such an item j contributes to the rating for
item i.</p>
      <p>Latent Factor Model. For the latent factor model, we again
consider the community information as implicit feedback, in which
each item rated by a user’s community contributes to the user’s
preference vector pu . Using the original factor model, we can extend it
as follows:
(qi +
oC
miC +)T (pu +
oC</p>
      <p>muC + +
X</p>
      <p>C 2 Cu
jIu j 12 X yj + jI Cu j 21 X zj );
j 2ICu
where the vector zj represents the contribution from item j. Also,
to model the user’s communities, we introduce an additional vector
oC that represents the preferences or characteristics of community
C. Since a user can belong to multiple communities, we dene o Cu
as the combined community preferences of all communities that
user u belongs to. is is achieved by summing the community
factors oC weighted by the user’s membership level muC to each
community. Likewise, to model the item community characteristics,
we dene o Ci as the combined characteristics of the communities
that item i belongs to. Combining baseline estimation,
neighborhood model and factor model from equations 5, 8 and 9, we compute
the predicted rating rˆui from a user u to an item i.
3.2</p>
      <p>Time and Community-Aware NSVD
(TCNSVD) Model
e TCNSVD model is an extension of the TNSVD model that uses
temporal dynamics of community structures. To capture user and
item community dri, i.e. time-changing community structures,
models as follows:</p>
      <p>1 X e ϕu jt tu j j ((ru j
+ jIu j 2
j 2Iu
μ + bu (t ) + bi (t )
where
the rst block of terms refers to the time-aware baseline
estimation. bu (t ) and bi (t ) indicate the time-dependent user and
item bias at time t , in which bu (t ) can be computed as with
bu + αu devu (t ) + bu;t . Here, bu , αu devu (t ) and bu;t describe
time-independent user bias, the linear dri of the user bias and
a time-specic parameter capturing short-lived eects.
Moreover, item characteristics are less complex to be described and
thus bi (t ) can be simply computed by bi + bi;Bin(t ) , in which
short-lived eects are captured through time by bi;Bin(t ) .
the second block of terms describes the time-aware neighborhood
model contribution where the function e ϕu jt tj j decays the
item contributions wi j and ci j such that ratings that are more
distant to time t have less impact on the estimation.
the last block of terms indicates the contribution of the
timeaware factor model. pu (t ) represents the time-dependent user
preferences that can be captured through parameters including
time-independent preference, gradual and short-lived eects.
is model is known as Time-aware Neighborhood-Integrated
SVD (TNSVD), in which parameters, i.e. bu , bu;t , bi , bi;Bin(t ) , wi j ,
ci j , yi j , αu , αuk , ϕu , qi , puk and puk;t are again learned by
minimizing a squared error function. In the following subsections, we
introduce two models based on NSVD and TNSVD models.
3.1</p>
    </sec>
    <sec id="sec-4">
      <title>Community-Aware NSVD Model</title>
      <p>e community-aware neighborhood-integrated SVD (CNSVD)
model is an extension of the NSVD model that use community
information to improve rating prediction accuracy. Similar to the
NSVD model, it consists of baseline estimation, neighborhood and
factor model contributions.</p>
      <p>Baseline Estimation. For the baseline estimation, we presume that
user and item communities have their own bias. e average rating
of a user community tends to deviate from the overall average
rating and each user’s average rating tends to deviate from the
community’s rating, and likewise with item communities. is is
based on the assumption that users or items in a community have
similar preferences or characteristics, which may imply a common
bias and thus the baseline estimation bui , is extended as follows:
μ + bu + b Cu
+ bi + b Ci ;
, in which
b Cu shows the community bias for user u, in which Cu represents
the set of user communities that user u belongs to. We model
the user community bias as follows:
b Cu =</p>
      <p>X
bC
muC ;
(7)
(9)
we compute user and item graphs and their community structures
for several time ranges. Here, available time range is divided into
community time bins. e set of communities of a user u at time
t is represented by Cu;CBin(t ) and the corresponding set of item
communities is shown by Ci;CBin(t ) , where CBin(t ) indicates a
function that returns the community time bin for a given time t .
TCNSVD model has three components that are explained in the
following.</p>
      <p>Baseline Estimation. For the baseline estimation, we extend the
Equation 5 to capture time-dependent community biases and a
time-dependent linear dri in community biases and thus we write
the baseline estimation for TCNSVD model as follows:
μ + bu + αu devu (t ) + bu;t
+ bu;Period(t) + bi + bi;Bin(t ) + bi;Period(t)
(bC + bC;t ) muC +
αC</p>
      <p>
        devC (t ) muC +
X
C 2 Cu
+
vector function, oC (t ) is made up of multiple functions, each
representing a component of the vector, i.e. oC (t )T = (oC1 (t ); oC2 (t ); : : : ;
oCn (t )). Each component is dened as:
oCk (t ) = oCk + αCk devC (t ) + oCk;t
k = 1; : : : ; n;
(13)
where oCk is the time-independent part of the community
preferences, oCk;t captures short-term eects and αCk devC (t ) models
a linear shi of community preferences. Using the user community
preferences, we extend the formula for the factor model as follows:
rˆui (t ) = (qi +o Ci )T (pu (t )+o Cu (t )+jIu j 21 X yj +jI Cu j 12 X zj ):
(14)
Koren [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] does not make the item vector qi time-dependent. He
claims that item characteristics are inherent and do not change with
time. We expect that this also applies to the item community vector,
so we also leave it to be time-independent. Finally, we combine the
baseline estimation, the neighborhood model and the factor model
by summing their predictions.
      </p>
      <p>
        For all the models, a squared error function is minimized to learn
the parameters of the model e.g. regularization parameters, using
stochastic gradient descent. A regularization factor λ is used to
penalize high parameter values to avoid overing the training data.
An implementation is included in LibRec 1 and we adapt it to our
NSVD-based models [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]. We performed some validations on
holdout data and found the optimum learning rate and regularization
parameters for the NSVD, TNSVD, CNSVD and TCNSVD models
using RMSE error. We selected the learning rate decay strategy
from LibRec and used the same combination of learning rate and
regularization for parameters of each model. e best learning rates
and regularization parameters for each model are given in Table 1.
j 2ICu
4
      </p>
    </sec>
    <sec id="sec-5">
      <title>EVALUATION</title>
      <p>
        Regarding experiments, we use the popular Netix (NF) and the
latest MovieLens (ML) dataset to evaluate the proposed
recommendation algorithms. e basic statistics of the datasets are shown
Table 2. In MovieLens both ratings and tags information are
available. As for tags-based graph construction, an edge is created
between any two users that have used the same tag on any item.
Similarly, an edge is created between any two items that have
received the same tag from any user [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. Regarding MovieLens and
Netix graph construction based on rating information, we apply
k-NN proposed by Park et al. as an approach mainly suitable for
information retrieval and recommender algorithms [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]. As such,
to compute the similarities between users and items from the rating
information, we use similarity measures such as Pearson
Correlation, Cosine Similarity and Jaccard Mean Squared Distance (JMSD)
[
        <xref ref-type="bibr" rid="ref17 ref19 ref28 ref4 ref6">4, 6, 17, 19, 28</xref>
        ].
1http://www.librec.net/
in which the block of terms shows the community contributions.
In this formula, the time-independent bias of community C is
denoted as bC , and community bias based on short-lived eects at
time t is denoted as bC;t for user communities and bC;CBin(t ) for
item communities. Linear dri in community biases is captured by
αC devC (t ). All community biases are summed over the respective
set of communities and weighted by the respective user’s or item’s
community membership level. For the time-independent biases and
the short-lived temporal eects, we use the dynamic community
structures. However, for the linear dri we use static community
structures so that longer-term temporal eects on each community
can be captured. In addition, the periodic user and item biases are
also reected by bu;Period(t) and bi;Period(t) , where Period(t)
represents a function that returns an index showing the week day of time
t . To keep our model from geing overly complex, we capture only
one of these potential recurring temporal eects, namely weekly
recurring eects.
      </p>
      <p>Neighborhood Model. For the neighborhood model, we use a
decay function e ψu jt tj j on the additional implicit feedback di j ,
which was added to Equation 8. is leads to the following formula:
1 X e ϕu jt tj j ((ru j
jIu j 2</p>
      <p>bu j )wi j + ci j )+
j 2Iu</p>
      <p>Latent Factor Model. For the latent factor model, we change the
vector modeling the user community preferences o Cu , which was
introduced in Equation 8 to being a function o Cu (t ):
o Cu (t ) =</p>
      <p>X oC (t ) ; muC ;</p>
      <p>C 2 Cu
with the community preference vector oC being replaced by the
time-dependent vector function oC (t ). As with the user preferences
E 1:05
SM 1
R 0:95</p>
      <p>
        For the evaluation of our proposed recommender model, we
compute both rating prediction and the item ranking quality as
well as accuracy. For measuring the accuracy of rating predictions,
we used the Root Mean Squared Error (RMSE). For measuring the
accuracy of item rankings, we used the measures Precision at k
(Prec@k), Recall at k (Rec@k) and Normalized Discounted
Cumulative Gain (NDCG) [
        <xref ref-type="bibr" rid="ref18 ref27">18, 27</xref>
        ]. Finally, we use Wilcoxon Rank Sum
tests to identify statistical dierences of the results generated by the
models [
        <xref ref-type="bibr" rid="ref29">29</xref>
        ]. Evaluation of time-aware recommender algorithms
is challenging since the time ordering of the ratings needs to be
considered and thus the use of cross validation approaches are not
suitable. Campos et al. [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ] describe in detail the issues
regarding time-aware recommender systems in their 2014 UMUAI paper.
Instead of k-fold cross-validation, we applied a sliding-window
approach taking snapshots along the timeline [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ]. Taking a snapshot
at time t means using the ratings within a certain number of days
before t for training and the ratings within a certain number of
days aer t for testing. In total we took ve of these snapshots
over the timeline in each dataset and report the overall means in
the results section. Regarding complexity, the running times of
the proposed methods are more than the baselines, in which we
used a subsampled version of datasets on a compute cluster. We
considered a maximum running time of ve days, in which
TCNSVD and CNSVD models had higher running times compared to
NSVD and TNSVD models. As for future works, we plan to
perform time complexity analysis of the models, and ignore individual
learning parameters to nd a compromise between accuracy and
time complexity.
      </p>
    </sec>
    <sec id="sec-6">
      <title>5 RESULTS</title>
      <p>
        In the following section, we present the results of our oine
simulations. We did preliminary experiments on the current
state-of-theart community detection algorithms regarding time complexity and
number of found communities. We chose DMID [
        <xref ref-type="bibr" rid="ref26">26</xref>
        ] and Walktrap
[
        <xref ref-type="bibr" rid="ref24">24</xref>
        ] as two alternatives that can identify overlapping and disjoint
communities. ey not only scale well on large amount of data
but can also handle weighed and directed networks. To use
Walktrap, a step size input parameter needs to be set that was 2 and 5
in our case. Figure 1 presents the results with respect to RMSE,
Prec@10 and Rec@10 for the NSVD, TNSVD, CNSVD and TCNSVD
models on the MovieLens dataset (for space reasons we omied
the Netix results here which show similar tendencies). As shown,
the RMSE values are quite close to each other for both of the two
community detection algorithms investigated. However, as also
shown, Walktrap performs slightly beer compared to the DMID
algorithm. Moreover regarding RMSE, algorithms based on the
TCNSVD model - using tags and ratings constructions - achieve
higher values than the CNSVD model. is trend also holds for
precision as well as recall. In general though, the results show that
Walktrap (WT5) yields the beer performance regarding RMSE,
CNSVD-tags TCNSVD-tags CNSVD-ratings TCNSVD-ratings
CNSVD-tags TCNSVD-tags CNSVD-ratings TCNSVD-ratings
      </p>
      <p>CNSVD-tags TCNSVD-tags CNSVD-ratings TCNSVD-ratings
recall as well as precision. e only exception is TCNSVD recall
results based on tags and ratings, in which DMID slightly
outperforms Walktrap versions. To verify the overall performance of the
proposed algorithms, online user studies need to be done.</p>
      <p>Figure 2 illustrates how the models perform when using
dierent similarity metrics using the Netix dataset (we omit for space
reasons the results of the MovieLens dataset, which though are
comparable). k was set to 10 in this case. As shown, there are
observable dierences with respect to the measures chosen for both
models. In general we can observe the following: Pearson achieves
in four cases the best results for RMSE, Prec@10 and Rec@10,
followed by Cosine and JMSD. is paern is also emerging when
testing with dierent ks and the dierent similarity metrics at the
same time. Table 3 shows the best performing parameters for the
ratings-based graph construction regarding RMSE, precision and
recall and the best performing parameters for k. To gure out the
practical performance of the models and similarity metrics, we need
to deploy them online and study users’ feedback.</p>
      <p>To illustrate how the models perform compared to some
baselines, a series of experiments have been performed. First, the models
were compared with the baseline methods NSVD and TNSVD on
the MovieLens (ML) and Netix (NF) datasets. ereaer, we
compared them to current state-of-the-art item-ranking models such as
WRMF and ItemKNN as present in the LibRec library.</p>
      <p>Table 4 shows the rst set of results in this respect. In general we
can observe that the TCNSVD model achieves the best results here.
For instance in the MovieLens dataset, compared to its baseline
(TNSVD), the method increases NDCG, Precision and Recall with
45.45 %, 675 % and 443 % in the best case relying on a ratings-based
graph. Similar trends are also observed for the Netix dataset. Here,
TCNSVD can also improve NDCG, Prec@10 and Rec@10 with 15.67
%, 126.73 % and 115.90 %, compared to it’s baseline method. e
result “paerns” on the other hand for the CNSVD model are not
that clear as RMSE and NDCG values are sometimes decreased,
while Prec@10 and Rec@10 are not. is is actually an interesting
behaviour which we need study further in future work.</p>
      <p>
        Figure 3 provides an overview of the computed item ranking and
precision metrics. Regarding the MovieLens dataset and the NDCG
metric, the TCNSVD model could achieve the best results, that is
statistically beer than the baselines WRMF, ItemKNN, CNSVD,
NSVD and TNSVD. TCNSVD surpasses the other baselines also
for Prec@10 and Rec@10 with even higher dierences. As for
Prec@10, TCNSVD achieves 0.28359, which is higher than 0.18842
and 0.17433 as obtained by WRMF and ItemKNN. e results and
the ranking of the methods are consistent with other benchmarks
run on the MovieLens dataset, although our evaluation protocol is
time-based [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ]. As for the Netix dataset, again TCNSVD gives us
the best results with notable statistically signicant dierences to
the other models.
      </p>
    </sec>
    <sec id="sec-7">
      <title>6 SUMMARY &amp; FUTURE WORK</title>
      <p>e main ndings of the paper can be summarized as follows:
We proposed a community-aware model named CNSVD based
on neighborhood and factor models of recommendation.
Furthermore, we introduced TCNSVD as a model that considers
temporal community structure and dynamics.</p>
      <p>We showed the eect of community detection algorithms on the
recommendation performance and found that Walktrap is the
beer option.
10 00::0034
ceR@ 00::0012
0
0:55
CGD 00:4:55
N 0:4
0:35</p>
      <p>ML
ML
ML</p>
      <p>Also it was shown that Pearson correlation as the similarity
metric for graph construction achieves the best performance
when considering temporal dynamics of community structures
in the recommendation task.</p>
      <p>Finally, we show that TCNSVD as a temporal and
communityaware recommender model performs sign beer than CNSVD
and compared state-of-the-art baseline recommendation approaches
on the MovieLens and Netix datasets.</p>
      <p>One limitation of the proposed CNSVD and TCNSVD models are
their high dimensionality. As such, it is planned for future work
to improve the models in such a way that less parameters need
to be set, e.g. by omiing individual user and item parameters.
In our experiments we selected the optimum learning rate and
regularization based on RMSE. Future work needs to assess whether
further improvements can be found when optimizing the models
e.g. with respect to NDCG.</p>
      <p>Although temporal dynamics of overlapping community
structures are considered by TCNSVD, modelling the eect of
community life cycles, i.e. birth, death, atrophy, grow, split and merge, on
recommendation systems still need to be studied. Moreover, we
plan to study the impact of time bins as well as speed of community
changes in the proposal of a recommender system. In addition,
more community detection algorithms need to be investigated with
our models, to identify best performing ones. Finally, we plan to
investigate the eect of explicit community structures as the current
ones are based on implicit ones and already achieving remarkable
results.</p>
    </sec>
    <sec id="sec-8">
      <title>ACKNOWLEDGMENTS</title>
      <p>is work is in part supported by BIT research school.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Backstrom</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Huttenlocher</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kleinberg</surname>
            ,
            <given-names>J. M.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Lan</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          <article-title>Group formation in large social networks: Membership, growth, and evolution</article-title>
          .
          <source>In Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '06</source>
          (
          <year>2006</year>
          ), ACM Press, pp.
          <fpage>44</fpage>
          -
          <lpage>54</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Baltrunas</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Amatriain</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          <article-title>Towards time-dependant recommendation based on implicit feedback</article-title>
          .
          <source>In Workshop on context-aware recommender systems</source>
          (
          <year>2009</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Bellogin</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Parapar</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <article-title>Using graph partitioning techniques for neighbour selection in user-based collaborative ltering</article-title>
          .
          <source>In Proceedings of the 6th ACM conference on recommender systems</source>
          (
          <year>2012</year>
          ), ACM, pp.
          <fpage>213</fpage>
          -
          <lpage>216</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Bobadilla</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ortega</surname>
            ,
            <given-names>F.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hernando</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <article-title>and Gutie´rrez, A. Recommender systems survey</article-title>
          .
          <source>Knowledge-Based Systems</source>
          <volume>46</volume>
          (
          <year>2013</year>
          ),
          <fpage>109</fpage>
          -
          <lpage>132</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Campos</surname>
            ,
            <given-names>P. G.</given-names>
          </string-name>
          , D´ez, F., and
          <string-name>
            <surname>Cantador</surname>
            ,
            <given-names>I.</given-names>
          </string-name>
          <article-title>Time-aware recommender systems: A comprehensive survey and analysis of existing evaluation protocols</article-title>
          .
          <source>User Modeling and User-Adapted Interaction 24</source>
          ,
          <fpage>1</fpage>
          -
          <lpage>2</lpage>
          (
          <year>2014</year>
          ),
          <fpage>67</fpage>
          -
          <lpage>119</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Candillier</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Meyer</surname>
            , F., and Boulle´,
            <given-names>M.</given-names>
          </string-name>
          <article-title>Comparing state-of-the-art collaborative ltering systems</article-title>
          .
          <source>In Proceedings of the 5th international conference on machine learning and data mining in paern recognition</source>
          (
          <year>2007</year>
          ), Springer, pp.
          <fpage>548</fpage>
          -
          <lpage>562</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Cao</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ni</surname>
            ,
            <given-names>Q.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Zhai</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          <article-title>An improved collaborative ltering recommendation algorithm based on community detection in social networks</article-title>
          .
          <source>In Proceedings of the 2015 annual conference on genetic and evolutionary computation</source>
          (
          <year>2015</year>
          ), ACM, pp.
          <fpage>1</fpage>
          -
          <lpage>8</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Charlin</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ranganath</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>McInerney</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Blei</surname>
            ,
            <given-names>D. M.</given-names>
          </string-name>
          <article-title>Dynamic poisson factorization</article-title>
          .
          <source>In Proceedings of the 9th ACM Conference on Recommender Systems</source>
          (New York, NY, USA,
          <year>2015</year>
          ),
          <source>RecSys '15</source>
          , ACM, pp.
          <fpage>155</fpage>
          -
          <lpage>162</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Choo</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Yu</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chi</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Sun</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          <article-title>Revealing and incorporating implicit communities to improve recommender systems</article-title>
          .
          <source>In Proceedings of the 15th ACM conference on economics and computation</source>
          (
          <year>2014</year>
          ), ACM, pp.
          <fpage>489</fpage>
          -
          <lpage>506</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Daneshmand</surname>
            ,
            <given-names>S. M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Javari</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Abtahi</surname>
            ,
            <given-names>S. E.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Jalili</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <article-title>A time-aware recommender system based on dependency network of items</article-title>
          . e
          <source>Computer Journal</source>
          <volume>58</volume>
          ,
          <issue>9</issue>
          (
          <year>2015</year>
          ),
          <fpage>1955</fpage>
          -
          <lpage>1966</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Dolgikh</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          , and
          <article-title>Jel´nek, I. Graph-based music recommendation approach using social network analysis and community detection method</article-title>
          .
          <source>In Proceedings of the 16th international conference on computer systems and technologies (</source>
          <year>2015</year>
          ), ACM, pp.
          <fpage>221</fpage>
          -
          <lpage>227</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Eberhard</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Trattner</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <article-title>Recommending sellers to buyers in virtual marketplaces leveraging social information</article-title>
          .
          <source>In Companion of the 25th international conference on World Wide Web</source>
          (
          <year>2016</year>
          ), ACM, pp.
          <fpage>559</fpage>
          -
          <lpage>564</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Gantner</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rendle</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Freudenthaler</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Schmidt-Thieme</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          <article-title>MyMediaLite: A free recommender system library</article-title>
          .
          <source>In Proceedings of the 5th ACM Conference on Recommender Systems (RecSys</source>
          <year>2011</year>
          ) (
          <year>2011</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Gauvin</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Panisson</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Cattuto</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <article-title>Detecting the community structure and activity paerns of temporal networks: A non-negative tensor factorization approach</article-title>
          .
          <source>PLoS ONE 9</source>
          ,
          <issue>1</issue>
          (
          <year>2014</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Guo</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          , Zhang, J.,
          <string-name>
            <surname>Sun</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Yorke-Smith</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          <article-title>Librec: A java library for recommender systems</article-title>
          . In Posters, demos, late
          <article-title>-breaking results and workshop proceedings of the 23rd conference on user modeling, adaptation and personalization (</article-title>
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Helic</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Strohmaier</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Trattner</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Muhr</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Lerman</surname>
            ,
            <given-names>K.</given-names>
          </string-name>
          <article-title>Pragmatic evaluation of folksonomies</article-title>
          .
          <source>In 20th International World Wide Web Conference (WWW2011)</source>
          , Hyderabad, India, March 28 - April 1,
          <string-name>
            <surname>ACM</surname>
          </string-name>
          (
          <year>2011</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Herlocker</surname>
            ,
            <given-names>J. L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Konstan</surname>
            ,
            <given-names>J. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Borchers</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Riedl</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <article-title>An algorithmic framework for performing collaborative ltering</article-title>
          .
          <source>In Proceedings of the 22nd annual international ACM conference on research and development in information retrieval</source>
          (
          <year>1999</year>
          ), ACM, pp.
          <fpage>230</fpage>
          -
          <lpage>237</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Herlocker</surname>
            ,
            <given-names>J. L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Konstan</surname>
            ,
            <given-names>J. A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Terveen</surname>
            ,
            <given-names>L. G.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Riedl</surname>
            ,
            <given-names>J. T.</given-names>
          </string-name>
          <article-title>Evaluating collaborative ltering recommender systems</article-title>
          .
          <source>Transactions on Information Systems 22</source>
          ,
          <issue>1</issue>
          (
          <year>2004</year>
          ),
          <fpage>5</fpage>
          -
          <lpage>53</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19] Jesu´ s Bobadilla,
          <string-name>
            <given-names>J.</given-names>
            ,
            <surname>Serradilla</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            , and
            <surname>Bernal</surname>
          </string-name>
          ,
          <string-name>
            <surname>J.</surname>
          </string-name>
          <article-title>A new collaborative ltering metric that improves the behavior of recommender systems</article-title>
          .
          <source>Knowledge-Based Systems 23</source>
          ,
          <issue>6</issue>
          (
          <year>2010</year>
          ),
          <fpage>520</fpage>
          -
          <lpage>528</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Koren</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          <article-title>Factorization meets the neighborhood: A multifaceted collaborative ltering model</article-title>
          .
          <source>In Proceedings of the 14th ACM international conference on knowledge discovery and data mining</source>
          (
          <year>2008</year>
          ), ACM, pp.
          <fpage>426</fpage>
          -
          <lpage>434</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <surname>Koren</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          <article-title>Collaborative ltering with temporal dynamics</article-title>
          .
          <source>In Proceedings of the 15th ACM international conference on knowledge discovery and data mining</source>
          (
          <year>2009</year>
          ), ACM, pp.
          <fpage>447</fpage>
          -
          <lpage>456</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Leskovec</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kleinberg</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Faloutsos</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <article-title>Graphs over time: Densication laws, shrinking diameters and possible explanations</article-title>
          .
          <source>In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery in Data Mining</source>
          (New York, NY, USA,
          <year>2005</year>
          ),
          <source>KDD '05</source>
          , ACM, pp.
          <fpage>177</fpage>
          -
          <lpage>187</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>Park</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Park</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lee</surname>
          </string-name>
          , S.-g., and
          <string-name>
            <surname>Jung</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          <article-title>Greedy ltering: A scalable algorithm for k-nearest neighbor graph construction</article-title>
          .
          <source>In Proceedings of the 19th international conference on database systems for advanced applications</source>
          (
          <year>2014</year>
          ), Springer, pp.
          <fpage>327</fpage>
          -
          <lpage>341</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <surname>Pons</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Latapy</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          <article-title>Computing communities in large networks using random walks</article-title>
          .
          <source>In Proceedings of the 20th international symposium on computer and information sciences</source>
          (
          <year>2005</year>
          ), Springer, pp.
          <fpage>284</fpage>
          -
          <lpage>293</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <surname>Sahebi</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Cohen</surname>
            ,
            <given-names>W. W.</given-names>
          </string-name>
          <article-title>Community-based recommendations: A solution to the cold start problem</article-title>
          .
          <source>In Proceedings of the 3rd workshop on recommender systems and the social Web</source>
          (
          <year>2011</year>
          ), ACM, pp.
          <fpage>40</fpage>
          -
          <lpage>44</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <surname>Shahriari</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Krott</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Klamma</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          <article-title>Disassortative degree mixing and information diusion for overlapping community detection in social networks (DMID)</article-title>
          .
          <source>In Proceedings of the 24th international conference on World Wide Web</source>
          (
          <year>2015</year>
          ), ACM, pp.
          <fpage>1369</fpage>
          -
          <lpage>1374</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <surname>Shani</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Gunawardana</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          <article-title>Evaluating recommendation systems</article-title>
          .
          <source>In Recommender Systems Handbook</source>
          . Springer,
          <year>2011</year>
          , pp.
          <fpage>257</fpage>
          -
          <lpage>297</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <surname>Su</surname>
            ,
            <given-names>X.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Khoshgoftaar</surname>
            ,
            <given-names>T. M.</given-names>
          </string-name>
          <article-title>A survey of collaborative ltering techniques</article-title>
          .
          <source>Advances in Articial Intelligence</source>
          (
          <year>2009</year>
          ),
          <volume>421425</volume>
          :
          <fpage>1</fpage>
          -
          <lpage>421425</lpage>
          :
          <fpage>19</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <surname>Trattner</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kowald</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Seitlinger</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Ley</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Kopeinik</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          <article-title>Modeling activation processes in human memory to predict the use of tags in social bookmarking systems</article-title>
          . 
          <source>e Journal of Web Science</source>
          <volume>2</volume>
          ,
          <issue>1</issue>
          (
          <year>2016</year>
          ),
          <fpage>1</fpage>
          -
          <lpage>16</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <surname>Yanxiang</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Deke</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fei</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          , and
          <string-name>
            <surname>Honghui</surname>
            ,
            <given-names>C.</given-names>
          </string-name>
          <article-title>User-based clustering with topn recommendation on cold-start problem</article-title>
          .
          <source>In Proceedings of the 3rd international conference on intelligent system design and engineering applications</source>
          (
          <year>2013</year>
          ), IEEE, pp.
          <fpage>1585</fpage>
          -
          <lpage>1589</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>