In this article we describe the OAUC's participation in the CLEF 2015 SBS Search Suggestion track. We are trying to represent appeal elements, used in readers' advisory theory and practice, to see if they can be used in an automatic retrieval and recommendation context. We are starting out with the pace appeal element, used on ction to representing how quickly a buildup of the story is. The results so far indicate that much tuning is needed when building models that can represent pace.
There are many qualities to books besides their formal characteristics, such as title, author and subject (the latter being examples of metadata). Books, particularly ction, also evoke the readers' emotions, which is arguably their major mission. This article explores how emotions and other subtle qualities can be discovered in user generated data and subsequently used in a system for automatic classi cation of books, as a part of an automatic recommender system. For this year's task we try to measure the performance of book suggestion based on a number of emotion evoking characteristics. The challenge is twofold: try to identify certain emotion waking characteristics of books, and measure whether identi cation of such characteristics helps us match readers' wishes based on similar characterization of their recommendation requests. We see it as a start of a process where we try to operationalize diverse subtle properties (appeal elements) known from literature promotion, so that they can be used as extra evidence in recommender systems. 2.1
[
Frame The frame is about the tone of a book (melancholic, positive), its feeling (funny or romantic), and its atmosphere (menacing or elevating). though di cult to de ne, this element is often decisive for the reader's choice.The book can be amusing, bleak, bittersweet4 Storyline The storyline is of course dependent on the previously discussed elements. But typical values5 will be Issue-oriented, Nonlinear or Open-ended. 2.2
The appeal elements are not directly manifest in the book text, let alone its metadata, and we need to nd some representation so that a recommender system can take them into account. To this end we need to nd some manifest indicators that can automatically match a recommendation request and a book using the appeal elements as evidence (in addition to other evidence), when recommending a book based on this recommendation request.
Finding and using such indicators is a challenge, which character di ers
among the elements. Being metadata of di erent kinds rather than full
content, the texts we have are sparse, but on the other hand (for a portion of the
books) include reader reviews, which should be a condense summarization of the
book done by readers, the target group of a recommender system.
3 [
One way of representing an appeal element (element-name element-value) is, using occurrences of sentences that are characteristic to some value of an appeal element. Feeding these to an NLP-system, The NLP system may identify functionally similar sentences in any analyzed book-review to use in the classication of the books. In our implementation, a model of an appeal element is a summary of sentences that are likely to appear in a review of a book that has this value (or valence) of this element. This method has a potential for accuracy, but needs quite a large set of reader reviews given to books with known values (and valences) of the appeal element, and is extremely prone to over tting. A simpler but less exact method will be identifying single words or word combinations used by readers when reviewing books of di erent values / valences of appeal elements. Such words need somehow to be classi ed, so that a system looking for appeal elements in reviews has a broader repertoire of words to look for than the one occurring in the training set.
Matching can thereafter be done by attempted applying the same, or a slightly di erent model to the recommendation request, assuming that a recommendation request and a review belong to the same genere. Here we have several options: { Retrieving books by a traditional retrieval model (using text-based metadata elements for matching) and then reranking so that books with matching appeal element rank ahead of other books { Weighing up books with matching appeal at retrieval time { Traditional retrieval accompanied by pseudo relevance feedback based on the appeal models.
As our current main experimentation line is around pace, we will be more detailed when suggesting a model for pace, than for the other elements.
We will use Wordnet to expand our model with di erent speach forms / styles / synonyms that may appear in the reviews. A similar procedure will be applied to the topic queries, and matching will consider match in appeal. 2.3
Pace can be seen as a binary variable, either "low" or "high", making it the easiest element to model and represent, but at the same time less controllable. Saricks poses some questions the answers to which may provide clues as to the pacing: { Is the book densely written? { Are there short sentences / short paragraphs, short chapters? { Is there a straight line plot
Our work belongs in the realm of content based recommender systems, like for
example [
Saricks' framework is reportedly being extensively used in libraries, and in recent years it is starting to gain more systematic use, prominently in a Reader's Advisory resource like NovelistTM. NovelistTM6 is a paid service by EbscoTM, marketed towards Reader Advisory (RA) services of libraries active since the late 90's. Among other book characteristics used for recommendation, They have, since 2010 also been recording and utilizing Saricks appeal elements.
In a more research related context, [
Interestingly, [
requests (not having other books to relate our recommendations to), and we therefore feel that we need to take a slightly more general approach, which is based on a broader classi cation of Parts of Speech, particularly adjectives.
Resembling [
As we were preparing this year's experiments, based on the approach we have taken, we have seen that the crunching of the data is extremely time-consuming and at the moment of writing the data are still in the preparation stage. 4.1
The SBS Suggestion Track's (SST) data consist of metadata drawn from LibraryThing and Amazon, describing about 2,8 million books, keyed by their ISBN (meaning the number of distinct works is somewhat lower, as ISBN keys manifestations of works). About half of these, (over 1.3 millions) have non-vacuous reader reviews as a part of their metadata. It is these reviews (free texts) that constitute the data of this paper.
In order to analyse this data, we have so far been taking the following steps: { POS-tagging of all free texts of the reviews using the Apache OpenNlp7 { Collecting all adjectives, basic (< J J >), comparative (< J J R >) and superlative (< J J S >) { normalizing the adjective-forms captured by the POS-tagger, and linking each review to the normalized forms of the adjectives.
We have also extended a request to Ebsco to obtain the basic data of the NoveList appeal terms so that we can analyse our methods' ability to extract appeal terms against their data. This will hopefully give us a better possibility to analyse the net-contribution of identifying appeal elements to the ranking of recommendations. 4.2
As an overall, guiding design principle when approaching this issue, we intend to assign values or valences of appeal elements to unseen books (represented by their respective review texts), based on intellectually assigning such values 7 https://opennlp.apache.org/ to chosen books, and building models based on reviews of the latter ones. The most straight-forward way of doing so based on existing NLP-tools, is using reviews of books with known values to build document-categorization models that can use reviews of unseen books to classify those into appropriate categories (low vs. high valence, di erent intervals of element values a.s.o). Classifying the recommendation requests (topics) in the same manner, can provide us with an additional piece of evidence when matching requests to books.
The problem is that such models, done the ideal way are bound to be week
and (as already apparent in the current results, see below) tend to over t. To
combat this problem we need to look at auxiliary procedures mostly based on
occurrences of di erent types of words or word categories. Such procedures are,
in their nature simpler than the former, but, as we see it, have the potential
of complementing these, hopefully with better results. Here we can hopefully
utilize work done on di erent Parts of Speech around Wordnet8, and possibly
other semantic and lexical resources ([
We start out experimenting with pace as the simplest and (according to [
the topics (recommendation requests), trying to see if matching those can give better recommendations. We use two strategies, one based on weighing up relevant terms at retrieval time, and the other one re-ranking results of traditional retrieval based on
The current results seem to su er from an insu cient pace model, that we obviously need to work more on, tuning particularly the wordnet expansion. The weighing up strategy seems to perform much worse than the re-ranking strategy.