The Reading[&]Machine project exploits the support of digitalization to increase the attractiveness of libraries and improve the users' experience. The project implements an application that helps the users in their decision-making process, providing recommendation system (RecSys)-generated lists of books the users might be interested in, and showing them through an interactive Virtual Reality (VR)-based Graphical User Interface (GUI). In this paper, we focus on the design and testing of the recommendation system, employing data about all users' loans over the past 9 years from the network of libraries located in Turin, Italy. In addition, we use data collected by the Anobii online social community of readers, who share their feedback and additional information about books they read. Armed with this heterogeneous data, we build and evaluate Content Based (CB) and Collaborative Filtering (CF) approaches. Our results show that the CF outperforms the CB approach, improving by up to 47% the relevant recommendations provided to a reader. However, the performance of the CB approach is heavily dependent on the number of books the reader has already read, and it can work even better than CF for users with a large history. Finally, our evaluations highlight that the performances of both approaches are significantly improved if the system integrates and leverages the information from the Anobii dataset, which allows us to include more user readings (for CF) and richer book metadata (for CB).
Published in the Workshop Proceedings of the EDBT/ICDT 2023 Joint Conference (March 28-March 31, 2023, Ioannina, Greece) * Corresponding author. $ ale.speciale@studenti.polito.it (A. Speciale); greta.vallero@polito.it (G. Vallero); luca.vassio@polito.it (L. Vassio); marco.mellia@polito.it (M. Mellia)
0000-0002-6420-231X (G. Vallero); 0000-0002-2920-1856 (L. Vassio); 0000-0003-1859-6693 (M. Mellia)
© 2023 Copyright for this paper by its authors. Use permitted under Creative Commons License and the conclusions are drawn in Section 7. 1 hCPWrEooUrctkReshtdoinpgpssIhStpN:/c1e:6u1r3-w/-0s.o7r3g/smACtatErirbUtudtRioantW4a.0.opInroteklrnistahtoioo.nipatl/(PrCeCraoBdYcie4n.0e)g.d-imngasch(CinEeU/R-WS.org) 2https://www.anobii.com/
RecSys have applications in disparate domains of our In this work, we leverage data provided by the public daily-life. To weigh the users’ preferences, they need to libraries located in Turin, Italy, called Biblioteche Civiche have feedback from users. As claimed in [3, 4], this in- di Torino (BCT). Then, we integrate them with data from formation can be acquired explicitly, by collecting users’ the online social network for reading enthusiasts Anobii. ratings; or implicitly, by observing users’ actions [5, 6].
Authors in [7] categorise the RecSys into two main fami- BCT Dataset lies: CB and CF systems. CB systems focus on analysing user or item metadata only. As explained in [8, 9], they BCT is a network of 49 public libraries which hosts a make recommendations based on users’ choices made large collection of books. Users can access these books in the past: given properties of items that the user likes, through loans. The anonymized data useful for our goal the system will suggest other items with similar proper- are reported in the following tables: ties. According to [10, 11], CF systems make recommen- 1. Books Table contains the information of each disdations to each user based on information provided by tinct book that is present in the collection. In those users we consider to have the most in common with total there are 290 125 distinct books. Each has them. In [12, 13, 14], authors propose the employment of a unique identifier, called book ID, followed by RecSys for entertainment applications, to recommend TV the author(s), title, type of the item (monograph, programs, movies, and travels, respectively. In [15, 16], manuscript, DVD, etc.), and language of the ediauthors use RecSys to improve the efectiveness of e- tion. government applications, while works presented in [17] 2. Loans Table contains details of the 5 484 078 use them for commercial services. loans that occurred between 2012 and 2020. For
To the best of our knowledge, in the literature there are each loan, i.e., for each row, the anonymized user few investigations on recommendation systems for books ID is reported, as well as the date of the loan. In and libraries, likely due to the dificulties in getting ex- total, we observe 163 321 users subscribed to the tensive and reliable data. One of the few examples is the BCT that borrowed at least one book. The averwork discussed in [18], where a CF approach is proposed age number of loans per book is 5 (median is 4). for libraries used by college and university students. The In addition, even if users borrow on average 33 students’ selected courses and learning trajectories are items, 75% of them has less than 24 items. taken into consideration to provide a timely recommendation. Diferently, we focus on general-purpose libraries For this work, we restrict our analysis to monographies and compare multiple approaches. In [19] the authors and manuscripts written in Italian language (Books table), classify the textual comments given to books in Amazon, keeping 228 059 books for the analysis. to determine whether the opinion is positive, negative, or neutral. However, the obtained classifier is not used Anobii dataset wiaaaanmbcnhclTcpeioAochlrahenredntjoiHiiudnficmfneigpiapclnuslttlnibioenIoolmniknwvctieeahwlCniltabliiteivttrsgnhaeteatrhrnaayaeRrcrlymceeeLhccroi(iSeobAdtyaemreIdascr)m,rtn-auybtienosraOneadss,tdeouitiadodhtgtniepgi,bo.iierornBnostpetv,Fssribyicdiendosafelteleoaselsrnkemueiddstns.,steciosrseOussuasbwns[toa2eieti0rttnxhs]i--,,. vItcaBtuiinoaaeadlrwlylinz.OsleiiC.bvndAuro2iarcln0rarqe0ybubn6,ioitr,nsloeAyhdk2,ans0ibrto,1eybwh9Iioa)tih,sapieAlsioirnanveainenoourbcnsooi1senim,rmilsinapninccalelardnienotihaeencssenreum(tedMesaaeritotrteirssncatdwpthpaioolendaprigotrlufdsrooliwarawirmninidntd2eys0v,pr1iioeern4f--Ictoinstebnast,etdhoatncsoixllecchtautbseortss’, ienatcehrersetcsobmymcheantdtiinngg. sIptsecRieficc- wofhAicnhoabrioiudantdas4e0t0a0re00realerevainntItfaolryt.hTishewfoorlklo: wing tables Sys addresses the challenge of choice overload for library visitors, presented in [21, 18]. Indeed, while choosing, users face an enormous quantity of possibilities, which makes them confused. Diferently from Obotti, in this paper we enrich library data with a social network dataset (Anobii), and we share with the research community the comparison of diferent recommendation systems. 1. Items Table contains the information of 8 021 517 items, mostly books, which are discussed on the social network. In particular, for each book (identified by Item ID), we have author(s), title, language, plot, and keywords. Moreover, each item is associated with genres, provided by users. As a result, each book has multiple genres, with the number of users who did this association reported.
There are 41 possible genres and each book is associated with 4 genres on average. 1.0 0.8 F0.6 D C 0.4 0.2 0.0 101
Readings Per Book 102
103 Comics Thrillers Fantasy
History Relationships
Crime
Humor Romance
Young Biography Philosophy
Travel Technical
Political Free Time Wellness
Horror SociNaloSn-cfiiecntiocen
Science 0 10 20 30 Genre distribution [%] 40
we focus on items that are books written in Italian. Moreover, in the Rating Table, we remove rows with ratings lower than 3, since we assume that those are negative feedback. The rationale is to recommend items that have received only positive feedback.
As mentioned above, multiple genres are associated with a book. We neglect genres associated with almost all books or with very few books (e.g, Fiction and Literature, Textbooks, References, and Self Help). To have the distribution of genres among books as balanced as possible, we 4. Recommendation Systems aggregate some genres by considering the entropy value calculated using their occurrences, and the aggregation Design is performed if it leads to the entropy reduction. Finally, In the case of explicit feedback, users quantify their apin order to make reliable the association of a genre to preciation through ratings, e.g., by giving between 1 and a book, we keep only the top 4 genres associations per 5 stars. In this case, RecSys predicts the rating the user book, according to the number of votes. Each of the 4 would give to an unread book and recommends the genres has a probability proportional to the number of books with the highest ratings. Unfortunately, user in association occurrences (the sum of the genre probability libraries do not express any degree of appreciation for is equal to one for each book). books, and we have only the list of their readings (implicit feedback). Therefore, we assume that if a user read Merging BCT and Anobii datasets a book, it is appreciated. Note that this is not always the case in practice: we leave for future work a study of possible features to reduce the limitations of this assumption, e.g., using the duration of the loan. The RecSys provides a ranking to each user, which is a sorted list of books representing the possible level of appreciation, and recommends the top ranked books. The value of We generate a final dataset by joining the information from BCT and Anobii. For each book present in both the BCT and Anobii datasets, we keep all the attributes from both datasets that might be useful for the recommendation systems. We also create a Readings Table that contains the Loans Table of the BCT dataset and the is set in order to have a good trade-of between the Authors of [23] trained SBERT using a labelled dataset of quality of recommendations and the prevention of users’ text pairs, for which the semantic distance is known. The choice overload. semantic distance quantifies how much the two texts are
In this work, we consider diferent implicit feedback- similar (i.e., close in the vector space). The loss function based RecSys, which we detail in the following. aims at minimizing the error of the semantic distance prediction. Given the numerical representations of words, Random Items we compute , as the cosine similarity of the numerical representation of the metadata summary of the book and , respectively.
RecSys is properly learning. Given a user , it randomly recommends a set of books, which have not been read yet by that user.
Most Read Items
are read in the training set, and then recommends the top most read books to all users. Therefore, the same recommendations apply to all users.
Closest Items This is a content-based RecSys. Its main idea is that users will read books similar to books they have already read in the past. In order to do this, given , the books of the catalogue, we define as the set of books that user has already read, and , given by ∖ , the set of the books which the user dis not have read yet. For each book ∈ we compute , the average similarity to the books in : = ∑︀∈ , || (1)
Bayesian Personalised Ranking (BPR) As presented in [6], the BPR is a CF RecSys for implicit feedback. It adapts the Matrix Factorization (MF), introduced in [25], to the implicit feedback case. Applying [6] to our scenario, given and the number of users and books, respectively, the user-item interactions matrix ∈ R is the matrix where , is 1 if user has read book , 0 otherwise. Then BPR decomposes the matrix into the product of two lower-dimensional matrices, and . The first has a row for each user, while the second has a column for each book. The row associated with a specific user and the column associated with a specific book are called latent factors. The predicted useritem interactions matrix is calculated as ˜ = , where ∈ R, given the number of users and latent factors , and ∈ R , where is the number of latent factors and is the number of books.
We train the model, to learn the book’s and user’s latent factors which provide the rank of books for each user.
The books read by a user are assumed to be preferred over unread books. Ranks are given according to a score (, |, ) for a user and a book , where V and P are the latent factor matrices we want to find. Therefore, we define the function: where , is the similarity between book and book and || is the cardinality of . Once is computed for each in , we recommend the books with the highest to the user .
For the computation of similarity , we first extract ( > |, ) = ( (, |, ) − (, |, )) (2) the metadata of the books, to create a metadata summary.
It is a string given by the concatenation of the book’s where (· ) is the sigmoid function. Considering metadata. In this work we use all the possible combina- the set of read books for user and the set of untions of (i) the book title, (ii) the author(s), (iii) the book read books for , we want to maximize this likelihood plot, (iv) the genres, and (v) the book keywords. function ( > |, ) when ∈ and ∈ , i.e.,
To compute the similarity between the metadata sum- read books should have a score larger than unread ones. mary of two books ,, we need to derive the numerical Then, the latent factors are found through the following representation of these strings as numerical vectors, that loglikelihood maximization over all users and pairs of we use as embedding space. These vectors carry a se- read and unread books: smi maniltaicr fmalelainnitnhge swaimthe trheegmio,nsooftthhaetetmwboeidtdeminsg wsphaicceh. Waree argmax ∑︁ ∑︁ (( > |, ))− use Sentence Bidirectional Encoder Representations from , ∈ ∈,∈ Transformers (SBERT), which is a transformer for Nat- || ||2 − || ||2 (3) ural Language Processing (NLP), developed by Google, see [22]. For our implementation, we use the library de- where V and P are distributed according to a zero mean scribed in [23, 24], which provides a pre-trained model, normal distribution with variance-covariance matrices to map text given as input to the numerical embedding. obtained by multiplying the identity matrix with and , respectively. Therefore, || ||2 and || ||2 in Table 1 Equation (3) act as regularization terms. Further details Results of the diferent RecSys with =20 and mathematical passages can be found in [6]. URR NRR P R FR
To numerically perform the optimization in Equation Random Items 0.07 0.07 0.00 0.01 370 (3), we use the variant of Weighted Approximate-Rank Most Read Items 0.03 0.03 0.00 0.01 556 Pairwise (WARP) loss to learn model weights via Stochas- Closest Items 0.22 0.29 0.01 0.05 186 tic Gradient Descent, see [26]. Given a read book ∈ , BPR 0.26 0.35 0.02 0.08 130 the WARP randomly takes an unread book ∈ . If the BPR (BCT only) 0.15 0.17 0.01 0.04 298 score (, |, ) exceeds (, |, ) then the latent factors , are updated. If this is not the case, another unread book ∈ is randomly picked. The magnitude Precision (P) of the update decreases with the number of extraction of unread books before the update, since unread books This metric quantifies the average ratio of books among are extracted much more likely than read ones. Again, the recommended ones, which are also in the test set. It further details can be found in [26]. is given by:
BCT users, which are the target of the recommendation. We use 20% of the readings of each BCT user as test set. The remaining part is further split into training and validation (80% and 20% of the remaining readings for each user, respectively). All the Anobii data are used for training (80% of the readings of each user) and validation (20%), without a test set.
Consider users. Let be the books read by the user in the test set, and the set of books recommended to the user . We quantify the performance with the KPIs described below, which depend on the choice of . Number of Users with Relevant Recommendations (URR) Once the RecSys generates the recommendation for each user, we compute the fraction of users who have at least a recommended book in the test set: () = 1 ∑︁ 1∩̸=∅ (4) where 1∩̸=∅ is one if the intersection of the set of books of the test set read by the user and the recommended books is not empty.
Average Number of Relevant Recommendations per Users (NRR) It accounts for the average number of books which are recommended by the RecSys and are in the test set: () = 1 ∑︁ | ∩ | where | ∩ | is the cardinality of the intersection of the set of books of the test set read by the user and the set of books recommended to the user.
Average First Rank Position (FR)
ommendation over all the users. The rank of the first relevant recommendation for a user is the best position among the user’s recommended books obtained by the books in the user’s test set. A lower FR indicates a better performance. It does not depend on .
RecSys explained in Section 4.
First, we perform a grid search for the parameters of the BPR model. In particular, we vary the number of latent factors and the learning rate, which is the magnitude of the update of the latent factors , , at each iteration of the training phase. We choose those that maximize the URR on the validation set. The results reveal (5) that 20 latent factors and 0.2 as learning rate provide the best performance. For the Closest Items, the metadata summary is built concatenating the author(s) and the genre(s) of the book, which is the best parameters combination (see Section 6.2).
() = 1 ∑︁ | ∩ |
|| Recall (R) It is the average ratio of books in the test set which are recommended: () = 1 ∑︁ | ∩ | || (6) (7)
Table 1 details the values for the KPIs with 20 recom- Random Items mended books (i.e., =20). This is the value we choose 0.4 Closest Items in our application since it is a good trade-of between the BPR quality of recommendations and the prevention of users’ R0.3 choice overload. As expected, non-personalized RecSys RN0.2 such as Random Items and Most Read Items perform poorly. The BPR algorithm obtains the best performance, 0.1 outperforming the Closest Items by up to 47%, providing the highest URR, NRR, R, and P. 0.0 [6, 7] [8, 10] [11, 16] [17, 100]
Notice that the obtained values of the KPIs seem low Range of the number of read books in the training set because we have to choose only 20 books over 2 332 Figure 4: NRR on the test set, varying the number of books available ones, where only a small portion are actually per user in the training set, with the number of recommended read (the median of the readings per user is 18). In fact, books equal to 20. the results reveal that both Closest Items and BPR provide significantly better performance than the Random Items and Most Read Items approaches. 6.1. Impact of the Number of Read Books
In the Table, we also report results obtained when BPR is trained using users from BCT only, denoted as We analyse the performance of the trained recommenda ( ). The lower results with respect to the tion systems on diferent groups of users. In particular, BPR case highlight the importance of integrating the we analyse whether and how the results change for users Anobii dataset for obtaining good results. Given the poor that read a diferent number of books, i.e., users that have performance, for the remainder of the paper, we do not a diferent number of books that belong to the training show any more results for BPR trained only on BCT data set (recall that, for each user, we select 80% of books for and also for Most Read Books. training and validation, and test recommendation on the
Next, we compare results obtained with the diferent remaining 20% of them). Fig. 4 reports the NRR with RecSys by varying . Fig. 3a shows the URR and NRR, Random, Closest and BPR algorithms, in blue, orange, while Fig. 3b reports Recall and Precision, marked by and green, respectively. The interval bins are chosen to stars and circles, respectively. In both the figures, we have approximately the same number of users in each vary the number of recommended books between 1 group. and 50, for the Random Items (in blue), Closest Items (in We clearly notice that the growth of the number of orange), and BPR (in green). As expected, the URR, NRR, books read by users improves the NRR, with all used and R grow with the number of recommended books. In- algorithms. This is expected since the probability of recdeed, when the number of recommended books grows, ommending a book that belongs to the set of read books having more relevant recommendations is more likely. increases with the size of the latter set. The growth of However, P decreases since it considers the ratio of rele- the NRR for the Random RecSys testifies this. Moreover, vant recommendations divided by the number of recom- when many books are available in the training set, the mended books. preferences of a user can be better caught. For the users who have a number of readings in the training set that is lower than 8 and between 8 and 10, the BPR obtains 6.3. Training and Recommendation Time NRR equal to 0.32 and 0.35, respectively, while Closest Finally, we focus on the running time needed for the Items 0.18 and 0.21. Nevertheless, the growth of the training and the recommendation phase, that we report users’ readings has a small impact on BPR. Conversely, in Table 2. The Random and Closest algorithms do not the Closest Items approach benefits by a sizeable increase have a proper training phase, while, as indicated in the in the NRR, reaching 0.29 and 0.49 when the number table, the BPR algorithm needs 30.55 seconds with our of readings in the training set ranges from 11 to 16 and dataset. from 17 to 100, respectively. This last case is 35% better For the recommendation phase, we average over users than the BPR case. For BPR, the efect of the growth of the time which elapses between the reception of a rethe number of readings for a user is less evident being a quest for a recommendation and the generation of the CF, and even a few readings are suficient to exploit the recommendations. Table 2 reports it for each algorithm. user preferences of similar users. It is evident that BPR requires a little more time than the others, but it is still acceptable for the application. 6.2. Impact of the Metadata Summary
pact of the diferent metadata summary, to compute the distance between books. In Fig. 5 we show the diferent KPIs, obtained when the metadata summary is composed of diferent concatenations of metadata (we do not show all the possible combinations, for lack of space). Using only the title as metadata (blue bar) results in the worst performance, similar to the Random approach. This suggests that users do not read books with titles similar to those already read. Already using the book plot or keywords (orange and green bars) to build the metadata summary provides better performance. The author(s) alone (red bar) significantly improves the results, suggesting that many users like to read books from the same author. Results further improve by using the genres of the book (purple bar) on some KPIs, implying that users are attracted to books with genres similar to the ones already read. Among all the combinations, we obtain the best results using only authors and genres (brown bar). Not shown in the figure, we observe that adding the keywords to the best combination, i.e., concatenating them to authors and genres, slightly decreases the overall performance.
These results confirm the importance of the integration of the Anobii dataset for obtaining good results, emerged
Reading[&]Machine project, aiming at increasing the attractiveness of public libraries by designing an application capable of suggesting books tailored to the preferences of the readers. We combine data from loans in libraries and from evaluations and characteristics of books crowdsourced within a social network (Anobii). We consider a CB and a CF approach, and results show that the CF increases the number of relevant recommendations provided to a user by the CB system, whose performance is strictly dependent on the number of books that the user has already read. For our application, the integration of the Anobii dataset, which includes more users and books’ metadata, useful for the CF and CB, respectively, significantly improves the performances.
It is important to note that the metrics defined in Section 5 are objectively trying to predict the next relevant books that users read. However, it would be interesting for future work also to consider books that are not related to the ones already read, but would be liked by the users, introducing parameters and metrics for evaluating the diversity and serendipity of the recommendations, as well as the possible boredom efect [ 27]. Moreover, we do not take into consideration the user specific sequence of loans, namely the fact that a book has been chosen after another. Therefore, we could consider sequential recommendation systems algorithms (see [28]).
grant funded by Fondazione TIM for the project ‘Reading (&) Machine’, and by the ‘National Centre for HPC, Big Data and Quantum Computing’ (CN00000013, Bando M42C, D.D. n. 3138, Decreto del MUR n. 1031). ciation for Computational Linguistics, 2019. URL: [27] L. Vassio, M. Garetto, C. Chiasserini, E. Leonardi, http://arxiv.org/abs/1908.10084. User interaction with online advertisements: Tem[24] N. Reimers, I. Gurevych, Making monolingual sen- poral modeling and optimization of ads placetence embeddings multilingual using knowledge ment, ACM Trans. Model. Perform. Eval. Comput. distillation, arXiv preprint arXiv:2004.09813 (2020). Syst. 5 (2020). URL: https://doi.org/10.1145/3377144. [25] S. Rendle, Factorization machines, in: 2010 IEEE In- doi:10.1145/3377144.
ternational conference on data mining, IEEE, 2010, [28] S. Wang, L. Hu, Y. Wang, L. Cao, Q. Z. Sheng, pp. 995–1000. M. Orgun, Sequential recommender systems: Chal[26] J. Weston, S. Bengio, N. Usunier, Wsabie: Scaling up lenges, progress and prospects, in: Proceedings of to large vocabulary image annotation, in: Proceed- the Twenty-Eighth International Joint Conference ings of the Twenty-Second International Joint Con- on Artificial Intelligence, IJCAI-19, 2019, pp. 6332– ference on Artificial Intelligence, IJCAI’11, AAAI 6338. URL: https://doi.org/10.24963/ijcai.2019/883. Press, 2011, p. 2764–2770. doi:10.24963/ijcai.2019/883.