The large array of recommendation algorithms proposed over the years brings a challenge in reproducing and comparing their performance. This paper introduces an open-source Java library that implements a suite of state-of-the-art algorithms as well as a series of evaluation metrics. We empirically nd that LibRec performs faster than other such libraries, while achieving competitive evaluative performance.
Recommender systems have been developed for decades, and a large number of algorithms have been proposed by the community. As more and more algorithms are being designed, the concern of reproducibility of algorithm performance grows [1]. Although multiple open-source frameworks exist for the purpose of algorithm reproduction and comparison, many of them only implement a set of classic algorithms which are now outdated [2]. Hence, we posit that more e orts are warranted in comparing the current state-of-the-art algorithms with new algorithms that are rapidly emerging.
This paper proposes an open-source Java library for recommender systems, called LibRec1. The LibRec library implements a suite of state-of-the-art recommendation algorithms as well as the traditional methods. In addition, a series of evaluation metrics are implemented including diversity-based metrics which are rarely enabled in other libraries. LibRec provides a platform for fair comparisons among di erent algorithms in multiple aspects, given the fact that the evaluative performance depends on data characteristic. It also provides a high exibility for expansion with new algorithms. LibRec is GPL-licensed Java software2 (version 1.7 or higher required), which can be easily deployed and executed in platforms including MS Windows, Linux, 1 http://www.librec.net. Version 1.3 of LibRec is described in this paper 2 Source code hosted in GitHub: https://github.com/guoguibing/librec
Generic Interfaces
Recommender
IterativeRecommender GraphicRecommender
SocialRecommender ContextRecommender Data Structures SparseMatrix SparseVector DenseMatrix DenseVector SymmMatrix DiagMatrix DataDAO DataSplitter DataConverter and Mac OS. It facilitates the study of the two classic problems of recommender systems, namely rating prediction and item recommendation. The LibRec framework consists of three major components, namely generic interfaces, data structures and recommendation algorithms, as illustrated in Figure 1. 2.1
Generic interfaces de ne a set of abstract recommenders that can be extended and implemented by speci c recommendation algorithms. In LibRec version 1.3, ve generic recommenders are implemented. Recommender de nes a general recommender which is extended by baselines and some other algorithms such as SlopeOne. IterativeRecommender de nes a recommender usually based on iterative learning techniques. Matrix factorization-based approaches (e.g., SVD++, BiasedMF) are often derived from it. GraphicRecommender suits for the algorithms based on probabilistic graphic models (e.g., LDA, BUCM). SocialRecommender de nes a recommender that incorporates social information, such as SocialMF and TrustSVD. Lastly, ContextRecommender de nes a recommender that integrates additional contextual information into recommendations, for example, temporal information by TimeSVD. Although social connections are indeed a form of contextual information, we determine to expand from IterativeRecommender rather than from ContexRecommender. This is because many social recommenders have been proposed and the format of social connections is relatively simple and consistent across di erent data sets. Additional generic interfaces can be de ned by further development. By de ning those interfaces, new algorithms can be easily implemented by focusing on their own logics. 2.2
At least four data structures are widely and heavily used to implement
recommender systems, namely sparse matrices and vectors, and dense matrices and
vectors. Other structures include symmetric and diagonal matrices. The data
structures have a great in uence on the execution time of recommenders. An
LibRec: A Java Library for Recommender Systems
important characteristic of LibRec is that it runs much faster than other
counterparts3. Our implementations are mainly inspired by a Java matrix library
MTJ4, one of the most powerful matrix libraries in Java. MTJ implements
several sparse matrix structures in a compressed row (column) storage for e ective
row (column) operations. Since it is often necessary to operate both in rows and
columns, we choose to implement a sparse matrix class by keeping both
compressed row and column storages, whereby additional utility functions can be
easily added. Similarly, our sparse vector is an enhanced version of that in MTJ
by incorporating a number of functionalities. For dense matrix and vector, we
discard the implementations of MTJ which only stores data in a single array
and hence not suitable for large-scale data storage. In contrast, we implement
our own versions using two-dimensional Java arrays. Java caching techniques are
also adopted to further boost algorithm executions. In addition, LibRec includes
(
We identify and implement three kinds of recommendation algorithms: (
Recommendation algorithms can be evaluated in di erent settings: Given
N (ratio) ratings, K-fold cross validation and cold start, to name a few. Three
kinds of evaluation metrics are implemented: (
Many open-source libraries are available including Mahout5, Duine6, Co 7, LensKit8
MyMediaLite9 and PREA10. Lee et al. [2] provide a detailed comparison among
3 The comparison is elaborated at http://www.librec.net/example.html
4 Matrix Toolkits Java (MTJ): https://github.com/fommil/matrix-toolkits-java
5 Mahout: https://mahout.apache.org
6 Duine: http://www.duineframework.org
7 Co : http://www.nongnu.org/cofi/
8 LensKit: http://lenskit.org/
9 MyMediaLite: http://www.mymedialite.net
10 PREA: http://prea.gatech.edu
these di erent frameworks, and report that Mahout, Duine and Co focus only
on memory-based algorithms and hence are outdated. LensKit provides only a
few classic recommendation algorithms. We give a comparison with more
advanced packages, i.e., PREA and MyMediaLite. MyMediaLite is a well-known
recommendation library written in C#. Some toolkits, e.g., Rival11 and WrapRec12
are recently designed as a wrapper of MyMediaLite for better use (e.g., data split
and evaluation). PREA is a more recently released framework implemented in
Java. However, the two libraries become less active for further development.
The last update of MyMediaLite was September 2013 and that of PREA was
June 2014. Consequently, some newly proposed algorithms are not supported by
these libraries, e.g., TrustMF, FISM and TrustSVD as we do. We also note that
graphic recommenders are rarely provided by other libraries. Besides, LibRec
provides more baseline and extension algorithms than PREA and MyMediaLite,
such as UserCluster and NMF. To evaluate recommendation performance, PREA
only provides predictive error-based metrics while MyMediaLite does not provide
novel measures beyond accuracy. In contrast, our library provides novel measures
as well as the traditional accuracy-based measures. Lastly, we empirically
demonstrate that LibRec runs much faster than PREA and MyMediaLite while
achieving competitive recommendation performance. The amount of performance gain
di ers from algorithm to algorithm. A detailed comparison in training and
evaluation time can be found at: http://www.librec.net/example.html. Another
important characteristic of our library is that LibRec con gures recommenders
using a con guration le, while PREA and MyMydiaLite use command lines.
The advantages of a con guration le are (
LibRec contributes to the community of recommender systems by providing (