Traditional genre-based approaches for book recommendations face challenges due to the vague definition of genres. To overcome this, we propose a novel task called Book Author Prediction, where we predict the author of a book based on user-generated reviews' writing style. To this aim, we first introduce the 'Literary Voices Corpus' (LVC), a dataset of Italian book reviews, and use it to train and test machine learning models. Our study contributes valuable insights for developing user-centric systems that recommend leisure readings based on individual readers' interests and writing styles.
[
Typically integrated with online platforms, book rec- There are few attempts to exploit user reviews also for
litommender systems rely on the history of users to pre- erary genre identification. These include [
The work presented in this study falls within the con- from amateur readers. These are J.K. Rowling, Stephen
text of collective eforts to foster the habit of reading and King, J.R.R. Tolkien, Jane Austen, Sarah J. Maas, and Dan
enlarge the readership across diferent target audiences 4. Brown.6
Among these initiatives, LettERE (Letture pER TE) is a Since literary genre is not a monolithic notion [
The remainder of the paper is organised as follows. Goodreads and 800 reviews from Amazon. While we Section 2 presents LVC, the novel collection of Italian successfully obtained the desired number of reviews for book reviews referring to the books of six popular au- most authors, we encountered challenges for Austen and thors. Section 3 introduces the Book Author Prediction Maas on Amazon. Nonetheless, the number of reviews task and details the methodology and models exploited collected for these authors can still be considered reasonin this work to address it. Section 4 presents the results ably comparable to the desired amount. The statistics of of our experiments. Finally, Section 5 ofers conclusions the final LVC dataset are reported in Table 1. and outlines potential future research directions. As can be noted, the two portions of the dataset (i.e.,
the length of a single review. This diference arises in part 2. The Literary Voices Corpus from the lower number of reviews collected from Amazon, but mostly from the comparatively greater length of Goodreads reviews in terms of sentences and tokens.
thors does not correspond to an equal number of tokens.
views among the readers of certain authors, such as King,
reviews in Italian published on two leading platforms for Digital Social Reading (DSR), Amazon Books and
ifction novels. 5 This corpus is a spin-of of the ‘A Good
corpus is aimed at being representative of two diferent approaches to writing book reviews, a diversity specific to the peculiarities of the two platforms. In fact, while
3. Book Author Prediction
un-patto-per-la-lettura.
purposes.
predicting the author of a book from the readers’ reviews. We explored the performance on the task of a suite of machine learning algorithms that vary with re
included in LVC can be found in Appendix A. 7 1,100 6,224 180,680
5.65 164.25
6 800 2,695 48,275 3.36 60.34
Linear Support Vector Machine We define two LinearSVM models, referred to as ‘Profiling’ and ‘Ngrams’ models. The former takes the set of linguistic characteristics described in Sec. 3.2. Ngrams exploits lexical information since it uses as input feature a simple contiguous sequence of n words acquired from the reviews (i.e. n-grams, with n equal to 1, 2, and 3).
trained version of the BERT model (12 layers, 768
hidden units) [
Table 2 into a classifier based on LinearSVM and trained using the
Linguistic features acquired from book reviews. internal representations of the BERT model fine-tuned
on the author classification tasks. We refer to this model
as SVM (BERT). SVM (BERT+Profiling) is an additional
Linspect to the architecture and features used for training earSVM model trained using both the fine-tuned
repre(see Section 3.1). The models leverage a wide spectrum of sentations produced by BERT and Profiling-UD features.
text properties acquired from the reviews of increasing The BERT representations used as input features of the
informativeness, which range from n-grams of words SVM model were computed by averaging the embeddings
to stylistic features (Section 3.2), up to contextual sen- of all the tokens in each review.
tence representations of Neural Language Models. For all
models, we adopted a 5-fold cross-validation approach Baselines We compared the performance of the above
for training and testing. The train and test sets always models against a random uniform classifier, i.e. a model
contain reviews of diferent books, thus increasing the that uniformly generates random predictions for each
complexity of the classification tasks. Note that, consid- author.
ering the high discriminative power of proper nouns in
this classification scenario, we performed the linguistic
analysis of reviews and sanitized the text [
To model the linguistic properties of the reviews, we
relied on a set of 150 linguistic features. These features
correspond to specific aspects of the document structure
and were derived using Profiling-UD [
hibit on average higher accuracy scores overall. This is possibly due to a typical trait of commercial platforms like
beyond the book’s content, such as parcel delivery or the edition’s book cover. These topics cause the reviews to be quite standardised, thus more dificult to discriminate.
book’s content possibly containing a larger amount of stylistic elements which help the automatic classification.
except Rowling for the Profiling and Ngrams models.
When looking at the results obtained for individual authors, Sara J. Maas turned out to be the most accurately predicted author on both platforms, considering the average scores across all models. However, upon closer inspection of the results obtained with the topperforming model (BERT ), we observe that while Maas remains the most accurately identified author in Amazon reviews, the reviews of Jane Austen’s books exhibit the highest level of distinctiveness on Goodreads.
Table 3 presents the classification accuracies for the task of Book Author Prediction. Notably, all models outperformed the random uniform baseline on both Amazon 4.1. Discussion and Goodreads. Upon closer examination of the models, we notice that lexical information has more discrimina- To take a closer look at the classification results, Fig. 1 tive power than linguistic properties in the task. As proof, reports the confusion matrices with the percentage of consider the global and author-level scores obtained by the predictions made by all models in the Book Author the Profiling model compared to the Ngram and, most Prediction task. This complements the classification renotably, the BERT models. Interestingly, using the fine- sults by showing which authors are more confusing and tuned BERT representations as input features for the SVM which are the most wrongly classified ones. classifier ( SVM (BERT)) yielded lower results than simply In general, we observe that as the model performance using pre-trained BERT, and the results are comparable – improves, the matrices become less sparse, regardless of or lower – when combining contextualized representa- the platform. This means that when the correct author tions with linguistic features (SVM (BERT+Profiling) ). is predicted most of the time, the erroneous predictions
Comparing the two platforms, Goodreads reviews ex- are distributed quite evenly among all possible authors. Consider, for instance, the matrices obtained from the to Goodreads reviews, we observe that Maas is the most analysis of BERT and compare them with the matrices frequently predicted author, leading to other authors’ referring to the Profiling and Ngrams models, which yield books being frequently misclassified as Maas’s works. the most sparse matrices. Notably, the reviews of It by King and of the fourth book
Notable diferences arise in the distribution of pre- from the Harry Potter saga by Rowling are often incordicted authors across the two platforms. For in- rectly assigned to Maas. The content of these books, at stance, when considering the Profiling model applied the crossroads between the fantasy and horror genres, may contribute to the model confusion. However, the BERT model. Both authors, despite their diferences, are most influencing factor to the Profiling model predictions known for building suspense and tension in their narraappears to be the review length. On Goodreads, reviews tives and incorporating detailed historical settings and of King’s and Rowling’s books that are longer than 150 psychological aspects into their work. tokens are wrongly classified as referring to Maas in over The classification of Goodreads review performed by 40% of cases. On Amazon, we observe an opposite ten- the SVM (BERT) and SVM (BERT + Profiling) models dency, but for a diferent author: when a review has less highlight author commonalities that did not emerge so than 10 tokens, the model assigns the review to Rowling strongly with other models. The reviews of Rowling’s in around 60% of cases. books, for instance, are frequently wrongly classified
The analysis of the feature rankings8 produced by the as referring to Maas’s work. Both authors are known classifiers trained on both Amazon and Goodreads re- for their contributions to popular literature, particularly views confirms the importance of review length for the in the genres of fantasy and young adult fiction, which Profiling model. Indeed, features that capture structural attract a readership interested in exploring themes of perproperties are particularly relevant for the model: the sonal growth and self-discovery through the characters’ use of subordination (subordinate_dist) is crucial for clas- coming-of-age journeys. sifying Rowling’s and King’s reviews on Goodreads, as Overall, no particular author appears to be systematithey exhibit respectively the lowest and highest use of cally confused by all models. This finding is particularly subordinate clauses. Conversely, on Amazon, the average interesting from our perspective since it shows that using number of verb dependents (verb_edges) and the distribu- user-generated reviews as an information source allows tion of function words (namely, conjunctions, auxiliary to successfully address the Book Author Prediction task. verbs and determiners) are discriminative for Rowling, It suggests that books authored by diferent novelists atTolkien, and Maas. tract readers who are interested in similar topics and also
For what concerns the Ngram model, the feature rank- adopt similar communication strategies in their writing. ing consists of the n-grams employed by the model or- It also implies that the proposed methodology could have dered by relevance for book author classification pur- a positive impact on the development of user-centric poses on Amazon and on Goodreads. Quite expectedly, book recommender systems. the analysis of the top 100 most relevant n-grams reveals that, on Amazon, parcel delivery is a highly referenced topic (e.g. ‘tempi previsti’, expected timing, and ‘ben con- 5. Conclusions fezionato’, well packaged), especially among the readers of Tolkien and Rowling, which have the most similar This paper has explored an innovative approach that n-gram rankings (Spearman correlation score = 0.235, leverages user reviews as a source of information for < 0.05). The two authors are the most frequently Book Author Prediction. Building upon our prior work, confused by the model, especially for what concerns the we introduced a novel dataset of Amazon and Goodreads reviews of Tolkien’s ‘The Hobbit’ and ‘The Silmarillion’, book reviews, LVC, which has been used for training wrongly classified as referring to Rowling’s books. In- and evaluating machine learning models addressing the deed, it is possible that the two authors attract a sim- novel book author prediction task. ilar readership interested in books involving intricate Our findings highlight the challenging nature of premythologies, and that feature multi-dimensional charac- dicting the author of a novel from a reader’s review. Howters with strengths, flaws, and internal struggles. Such ever, the analysis of erroneous predictions pointed us to closeness between the Amazon reviews of these authors cases of books sharing a similar readership. This observais captured also by the BERT model which, although per- tion supports the intuition that user-generated reviews forming better than other models on the task, seems quite can efectively serve as a basis for personalized book recconfused by the reviews of the same Tolkien books. ommendations. By analyzing reviews, we gained insights
On Goodreads reviews, where parcel delivery is not rel- into readers’ preferences beyond the writing style of the evant, the most impactful n-grams tend to revolve around book’s author, opening up new avenues for more tailored and user-centric recommendations. book appreciation (e.g., ‘ho apprezzato’, I appreciated; ‘letMoving forward, this research could be expanded by tura piacevole’, pleasant reading; ‘non mi aspettavo’, I did investigating the impact of exploiting user judgments as not expect) or plot (‘il maghetto’, the little wizard; ‘signore an additional feature for classification. Furthermore, the di’, lord of; ‘chiesa’, church; ‘di epoca’, historical; ‘drago’, dragon; ‘di vampiri’, of vampires). Therefore, it is not sentiment expressed by readers about a book, whether surprising to see that King’s reviews are most frequently positive or negative, could be leveraged to validate and misclassified as referring to Brown’s work, also by the ifne-tune personalized recommendations.
2024) funded by Regione Toscana (Progetti Congiunti di
Author
B. Feature ranking Profiling Model (Goodreads) Feature ttr_lemma_chunks_100 ttr_form_chunks_100 aux_tense_dist_Pres ttr_form_chunks_200 ttr_lemma_chunks_200 n_prepositional_chains n_tokens upos_dist_AUX upos_dist_ADP dep_dist_orphan upos_dist_DET aux_mood_dist_Ind dep_dist_aux aux_tense_dist_Imp dep_dist_case dep_dist_cop dep_dist_mark verbs_form_dist_Part dep_dist_flat:name aux_num_pers_dist_Sing+3
Stephen King Feature upos_dist_CCONJ dep_dist_cc avg_prepositional_chain_len ttr_form_chunks_200 ttr_lemma_chunks_200 prep_dist_2 prep_dist_1 subordinate_post dep_dist_orphan prep_dist_3 subordinate_dist_1 tokens_per_sent n_tokens aux_tense_dist_Pres ttr_lemma_chunks_100 avg_verb_edges subordinate_pre verbal_head_per_sent dep_dist_case upos_dist_ADP
C. Feature ranking Profiling Model (Amazon) Feature upos_dist_AUX dep_dist_det dep_dist_aux upos_dist_ADV ttr_lemma_chunks_100 ttr_form_chunks_100 dep_dist_cop upos_dist_DET dep_dist_root dep_dist_advmod verb_edges_dist_2 verb_edges_dist_3 ttr_lemma_chunks_200 aux_tense_dist_Pres verb_edges_dist_4 avg_verb_edges verbs_form_dist_Part dep_dist_case ttr_form_chunks_200 verb_edges_dist_1
Stephen King Feature dep_dist_det dep_dist_cc upos_dist_CCONJ upos_dist_DET ttr_form_chunks_200 ttr_lemma_chunks_200 avg_verb_edges verbs_form_dist_Part aux_tense_dist_Pres verbs_form_dist_Fin verbs_form_dist_Inf lexical_density ttr_lemma_chunks_100 verb_edges_dist_1 dep_dist_root upos_dist_AUX dep_dist_aux principal_proposition_dist dep_dist_det:poss dep_dist_flat:foreign