In this paper, we describe our participation in the Social Book Search(SBS) Suggestion Task. We developed some new re-ranking models, based on what we applied in last year. We used Galago search from the index which was built with active books. The queries input to search engine were composed by key words from topics, and then we performed re-ranking models(popularity related) on Galago searching results on enriched XML index by 14 di erent elds. Experiments on these approaches shown that an enriched index and key query model improves the e ectiveness. As our approaches in INEX2014 [1],SBS2015 [2] and combined those re-ranking models according to a speci c order shown the best performance.
In this paper, we describe our participation in the Social Book Search 2016
suggestion task. Our goals for this task were (
The structure of this paper is as follows. We start Section 2 by describing our methodology: pre-processing on the XML formatted documents, indexing, searching by Galago, introduction of key query and active books set. In the section 3,we describe the re-ranking models and the re-ranking models experiments. In section 4, we describe the results of our enriched index, key query model, active books set and re-ranking models. Section 5 describes about the runs which we submitted,with the results of those runs presented. We discuss our results and conclude in Section 6.
We perform a process similar to [1], such as expand and enrich the documents XML with replacing the numeric information with textual information. For instance: We change it to
<tag count="3"> ction</tag> <tag> ction ction ction</tag>.
In addition, this time we expand to 14 elds when we clean the XML documents, it is di erent from last year. They are title; isbn; tags; review content; review summary; dewey; f irstwords; lastwords; chracters;places; subjects; browsenodes; abstract; addcontent.
Galago 1 is an open-source search engine. In order to improve the search effectiveness, we study two strategies to build the index. One indexing strategy is the normal indexing approach described as follows. Experimentally, we nd that the elds (etc. the title, tag, content and summary) are more relevant and meaningful than others in the XML formatted documents. So we build our basic index by removing the other elds which were not useful. Another strategy is to enrich the basic index. Observing the book information from the Library Thing, we nd out that a large proportion of books lack the content and summary elds. Therefore, documents expansion technology is expected to utilized to enrich the basic index. Firstly, we select two web sites which contain a large amount of more useful metadata of books. The books we use are the literatures written in English in douban.com 2 and all books in lookupbyisbn.com. Then we crawl the brief introduction of douban.com and the book description eld of lookupbyisbn.com. Both web sites are available by ISBN. With the content from both web sites, we enrich six hundred thousand of books (see the examples of book document which is used for index in XML 1 and XML 2). The enriched index is based on the enriched information.
XML 1: Book document <book> <title>Mister Monday</title> <summary>So good, you can't put it down!</summary> <content>Now, I had...</content> <tag count="9">children's literature</tag> </book> 1 http://www.galagosearch.org/ 2 http://book.douban.com/
XML 2: Enriched book document <book> <title>Mister Monday</title> <summary>So good, you can't put it down!</summary> <content>Now, I had...</content> <tag count="9">children's literature</tag> <brief introduction>the content is from the douban.com</brief introduction> <description>the content is from the lookupbyisbn.com</description> </book> 2.3
Generate Keywords from queries
We collect all the topics from 2011 to 2015. Our purpose is to get the best evaluation results after a great quantity of attempts. So that we can get the best queries for these topics, then we summed up a word list for topic queries. In Social Book Search 2016, we used the word list to lter the request eld, then combine it with the title eld compose the queries for Galago. 2.4
Active subset in book collections
Through the analysis of the pro le, we select a subset of high frequent books which is called active book set. In this set, all the books are considered much more should be recommended to the topic author. We lter out all the books which are not in the active set. The operation is similar to lter the catalog and example books of the topic. 3
Some of the re-ranking approaches were proposed and used by USTB at INEX2014 [1] and proposed by Toine Bogers in 2012 [4], which proved to be e ective. This year our re-rank approach can be roughly divided into two categories 1) Tag-Rerank (T), similar product re-rank (S), Read by one re-rank (R) 2) popularity re-rank (P ), example recommended re-rank (E), Reader number re-rank (R), Browsnodes re-ranking (B). The rst category was based on computing the similarity of two books that from the original result from Galago. The second category was based on attributes (number of readers, popularity ) of the books or the example books of the topics.
We use these models to re-rank by the following stages:
1)Similarity Calculation. Models like T , N focus on the eld <tag> and
<BrowseNode>. We can build a feature matrix for features for each eld.
Equation (
Features like I focus on the eld <similar-product>, the similarities of two
documents based on the feature I is calculated by the Equation (
score0(i) =
score(i) + (
Pr2Ri r
jreviews(i)j
score0(i) =
score(i) + (
score0(i) =
score(i) + (
(
In addition, Tag re-rank and read by one re-rank (we hold the opinion that if two books are read by same reader, they are related) are very similar to similar product re-rank.
We got a data set from Library Thing like this:
W ork idi;
popularityi
When we input initial search result to this model, we will change the works
score by equation (
Just as what we see, popularity is the number from the set, score is from Galago search result, score is the nal output score from this re-rank modes.
Example recommended reranking approaches
Through summarizing the information from the pro le, we got a example recommended list for some topics like this format:
T opic idi recommended1; recommended2; recommended3; etc:
When using the re-ranking approach, if a work of the result is contained in
the topics list. We would change it rate by the equation (
We extracted information from the pro le, then we got a set which showing the workss reader number.Like this:
word idi; reader numberi Through this approach, we optimize the search result by the below equation(8) 3) Combining. We applied these approaches on the key query search result according to a speci c order, then got the nal result.
As shown in Table 1, the best performance is obtained from Initial+keywords+ allReRank + f iltercatalogandexampl ,and active; readernumber make greater contributions to the improvements. 4
We selected six automatic runs for submission to SBS2016 based on our Key Query and re-ranking Models. They are:
run1. This run was made by a searching-re-ranking process where the initial retrieval result was based on the selection of query keywords and a small index of active books, the re-ranking results based on a combination of several strategies (number of people who read the book from pro le, similar-product from amazon.com, popularity from LT forum, etc.).
run2. This run was made by a searching-reranking process where the initial retrieval result was based on the selection of query keywords and a small index of active books, the re-ranking results based on number of people who read the book from pro le.
run3. This run was made by a searching-reranking process where the initial retrieval result was based on the selection of query keywords and a small index of active books, the re-ranking results based on the books in a same user's pro le.
Run4. This run was made by a searching-reranking process where the initial retrieval result was based on the selection of query keywords and a small index of active books, the re-ranking results based on similar products provided by amazon.
run5. This run was made by a searching-reranking process where the initial retrieval result was based on the selection of query keywords and the full index ltered by active books, the re-ranking results based on the books in a same user's pro le.
run6. This run was made by a searching-reranking process where the initial retrieval result was based on the selection of query keywords and the full index ltered by active books, the re-ranking results based on a combination of several strategies (number of people who read the book from pro le, similar-product from amazon.com, popularity from LT forum, etc.). 5
The runs submitted to the Social Book Search 2016 were evaluated using graded relevance judgments. The relevance value were labeled manually according to the behaviors of topic creators, for example, if creator adds book to catalogue after it's suggested, the book is treated as highly relevant. A decision tree was built to help the labeling 3. All runs were evaluated using NDCG@10, MRR, MAP, R@1000 with NDCG@10 as the main metric. Table 2 shows the o cial evaluation results. Results of the six submitted runs on Social Book Search 2016, evaluated using all 120 topics with relevance value calculated from the decision tree. The best run scores are printed in bold, we got the rst place in the competition. In addition, all the results we submitted are in the top six.
So we got the rst place in the Social Book Search suggestion task 2016.
It necessary to state that run 7, run 8 and run 9 are our additional experiments, obvious we can nd that key query and active books set have a great increase in results .Key query improve Initial+stopwords run by about 25 percent, then active books set improves the Initial+keyQuery run by about 24 percent.We see that the best-performing run on all 120 topics was run1 with an NDCG@10 of 0.2157. Run 1 used Key Query, small index built by active books and all re-ranking models combine. Also we see that re-ranking model does improve over the initial results by Galago searching engine.
All the runs from 1 to 6 were ltered by the topics catalog books set and example books set. 6
All of the re-ranking approaches can improve the evaluation results, the best results are from combined all re-ranking approaches. Of course, the use of key queries and active book set make the greatest contributions to the e ectiveness of our systems.
This year, we used much information from pro le, and we got a better performance, but we failed to make use of the random forest to combine the re-ranking to improve the result. We keep the opinion that machine learning can get a better results. So, it is worth discussing how to combining the re-ranking results with machine learning algorithms.