Conceptual Impact-Based Recommender System for x CiteSeer

Abstract

CiteSeerx is a digital library for scienti c publications written by Computer Science researchers. Users are able to retrieve relevant documents from the database by searching by author name and/or keyword queries. Users may also receive recommendations of papers they might want to read provided by an existing conceptual recommender system. This system recommends documents based on an automaticallyconstructed user pro le. Unlike traditional content-based recommender systems, the documents and the user pro le are represented as concepts vectors rather than keyword vectors and papers are recommended based on conceptual matches rather than keyword matches between the pro le and the documents. Although the current system provides recommendations that are on-topic, they are not necessarily high quality papers. In this work, we introduce the Conceptual Impact-Based Recommender (CIBR), a hybrid recommender system that extends the existing conceptual recommender system in CiteSeerx by including an explicit quality factor as part of the recommendation criteria. To measure quality, our system considers the impact factor of each paper's authors as measured by the authors' h-index. Experiments to evaluate the e ectiveness of our hybrid system show that the CIBR system recommends more relevant papers as compared to the conceptual recommender system.

Keywords

• eol>Recommender System
• h-index
• Content-based Recommender System
• CiteSeerx
• Information Retrieval

Categories and Subject Descriptors

Information Systems [Information retrieval]: Retrieval tasks and goals:Recommender systems CBRecSys 2015, September 20, 2015, Vienna, Austria.

Copyright remains with the authors and/or original copyright holders

1. INTRODUCTION

In recent years, recommender systems have become ubiquitous, recommending movies, restaurants, and books etc. The recommendations ease information overload for users by pro-actively suggesting relevant items to the users, moving the burden of discovery from the user to the system. The number and type of applications that use recommender systems keeps growing [ 1 ]; one practical application that is of interest to researchers in any domain is the ability of recommender systems to suggest relevant scienti c literature. These systems can expedite scienti c innovation by helping researchers keep abreast of new publications in their elds and also help new researchers learn about the most important literature in an area new to them. Digital libraries can employ recommender systems that suggest papers to their users based on each user's research interests. However, an e ective recommender system should not only consider the subject of a paper, it should also take into account the paper's quality when making recommendations. To this end, we present a recommender system that recommends scienti c papers based on user preferences as well as paper quality as measured by the authors' impact factors to provide recommendations of high-quality papers that are relevant to the user's research area. To help CiteSeerx users locate scienti c papers related to their work, a citation-based recommender system was developed by Chandrasekaran et al. in 2008 [ 4 ] . Although citations are e ective at identifying papers that have relevant content and are also high quality, this approach is only e ective in recommending papers with many citations. These unfortunately tend to be older papers that have been published long enough ago to generate many citations. Especially in a fast-moving domain like computer science, researchers need to know about recent contributions to their eld, yet recent papers have few citations. To solve this problem, a content-based recommender system for CiteSeerx was developed by Pudhiyaveetil et al.[ 8 ]. This conceptual recommender system automatically builds conceptual pro les for users based on their interactions with the system. It also builds conceptual pro les for each document and recommends papers based on conceptual matches between document and user pro les. Even though the recommendations were shown to be more relevant than those produced by a keyword-based recommender system, they are not always high quality papers that the researcher wanted to read. Our objective is to improve upon the conceptual recommender system by providing better quality recommendations to the users. To do so, we developed a recommender system that recommends papers based on the paper authors' impact factors. We combined the impact-factor based recommendations with the concept-based recommendations in varying proportions to create a hybrid recommender system. We evaluated the e ectiveness of the conceptual recommender system, the impact-factor recommender system, and the hybrid recommender system and found that the hybrid recommender system provides the most accurate recommendations. The rest of this paper is organized as follows: In section 2 we review related work. Section 3 describes the Conceptual Impact-Based Recommender (CBIR) system in detail. In section 4, we present our experimental evaluation to analyze the e ectiveness of our recommender system. Finally, we present our conclusions and discuss future work in section 5.

2. RELATED WORK

The design of a recommender system can vary based on the nature of user feedback or the availability of data. There are three main approaches: collaborative ltering, content based recommender systems, and recommender systems that are a hybrid of the two [ 1 ]. The rst approach generates recommendations based on similarities between the users' behavior or/and preferences. In contrast, content-based approaches recommend items to the users based on similarities between the attributes of the items themselves [ 10 ]. Collaborative approaches are typically used when semantic features cannot easily be extracted from the items, so indirect evidence based on user's likes or ratings must be compared. To be e ective, collaborative ltering requires a large active user community to avoid the well-known "cold-start" problem in which there are many more items to be recommended than there are users with likes or ratings upon which recommendations can be based. On the other hand, pure contentbased recommender systems do not consider external information that might be available from the users, e.g., popularity. For these reasons, many recommender systems employ a hybrid approach combines both of the previously-described approaches.

Content-based recommender systems match the users' preferences to each items' features to recommend new objects [ 10 ]. Many share the approach of building a user pro le from a set of features extracted from previously liked items. This user pro le is then compared to the features of all items in the collection and the most similar items are recommended to the user [ 12 ]. This type of recommender system can be used in domain for which semantically relevant features can be extracted and it is particularly well-suited for domains that include textual items as scienti c literature or domains with annotations such as movies or music [ 12 ]. Kompan et al. used this approach to recommend news articles on a web site [ 9 ]. In this domain, the volume of articles and the dynamic nature of news make collaborative ltering infeasible so they implemented a content-based recommender system based on cosine similarity that suggested articles that best matched an implicitly constructed user model [ 9 ]. Our work is a hybrid approach that enhances a contentbased recommender system with a quality measure to recommend scienti c literature. According to Beel et al., recommender systems for research papers are ourishing with more than 80 approaches existing today that have been discussed in over 170 articles and patents [ 2 ]. Such recommender systems are useful for researchers to be up to date in their research area. Many content-based recommender systems represent the user interests and the documents as weighted keyword vectors. One example is [ 13 ] in which tf idf weights are calculated for keywords and the cosine similarity measure is used to determine the relevancy of a paper to a user's pro le. An approach similar to ours is used in [ 5 ]. In their work, each paper's features are represented as concepts created by automatically extracting keyphrases. User pro les are constructed from the concepts in previously viewed papers and the recommender system matches the user pro le concepts to each papers' concepts to suggest new papers in a scienti c library. In [ 8 ], a conceptual recommender system was presented that recommends research papers for CiteSeerx users. Unlike the previous work, the concepts for each paper are assigned by automatically classifying papers into a set of concepts de ned by a pre-existing ontology. A conceptual user pro le is implicitly built as users view papers in the collection and this user pro le is used to recommend conceptually similar papers.

The content-based recommender systems can recommend literature that is similar in topic to the user's pro le, but it does not necessarily recommend high-quality papers. Although there is no perfect way to measure the quality of articles, the Impact Factor (IF) introduced in 1955 is still considered the best way to evaluate a paper's scienti c merit [ 6 ]. There are several types of IFs, including the widely used h-index that evaluates a researcher's impact [ 7 ]. It has been recently used is several elds such as health services research [ 3 ], business and management [ 11 ] or even academic psychiatry [ 14 ] . Although the work in [ 5 ], [ 8 ], and [ 13 ] are similar to ours, our recommender system expands upon their work by incorporating a quality factor as measured by the authors' h-indexes.

APPROACH

The architecture of the Conceptual Impact-Based Recommender System (CIBR) is shown in Figure 1. The Pro le Subsystem classi es all documents in the CiteSeerx database into the 369 prede ned categories in the ACM Computing Classi cation System (CCS). Documents manually tagged with ACM categories by their authors are used as the training set for a k-nearest neighbor classi er. As users interact with the system, the documents that they examine are input to the Pro le Subsystem. The categories associated with each examined document are combined to create a weighted conceptual user pro le. This user pro le is used by both the Conceptual Recommender and the Impact-Based Recommender described in the following sections. The outputs of these two Recommenders are combined to produce the recommendations from the CBIR. 3.1

Concept-Based Recommender System

As a user views documents in CiteSeerx, the Pro le Subsystem builds a conceptual user pro le for them by accumulating the concept weights associated with the documents that the user examines. The Conceptual Recommender System then recommends documents to the user based on the similarity between each document's conceptual pro le and the user's conceptual pro le [ 8 ]. The weight of the conceptual match between document i and user j is calculated using the cosine similarity function over all M=369 concepts in the ACM taxonomy:

ConceptualW eightij = PM

K=1 (cwtik cwtjk)

Where

cwtik = weight of concept k in document pro le i and cwtjk = weight of concept k in user pro le j as explained and detailed in [ 8 ]. 3.2

Impact-Based Recommender System

The Impact Factor Generator precalculates an impact factor for each document in the collection as measured by its authors' h-indices. As described by Hirsch, an author has an h-index of m based on his/her N published articles if m articles have at least m citations each, and the other N-m articles have no more than m citations each [ 7 ]. The impact factor for a document is calculated by nding the h-index value of each of the authors of the document and then selecting the highest h-index value. Thus, document i's h-index is equal that of its most impactful author:

ImpactW eighti = max(hindexil) l2Ail (1)

Where

Ail = list of the authors l of document i Since the impact factor is independent of users, the ImpactBased recommendations would be the same for all users, i.e., the most impactful documents in the entire collection. We do, however, use the user pro le to lter out documents from categories in which the user has shown no previous input. Thus, Impact-Based Recommender returns highimpact documents from categories of some interest to the user. We tried other approaches to calculate the impact factor among which we consider the sum of each authors' h-indices. This particular method is limited since the highest weighted papers would usually be the ones with many authors.

3.3 Conceptual Impact-Based Recommender

System

The Conceptual Impact-Based Recommender System (CIBR) combines the Conceptual Weights and the Impact Weights to produce its recommendations. The two sub-component weights are normalized to fall between 0 to 1 using linear scaling and then combined based on a tunable parameter, . The weight of the conceptual impact match between document i and user j, ij , is calculated using: ij =

Ci0j + (1 ) I0 i (2)

Where

Ci0j = normalized ConceptualW eightij =

ConceptualW eightij minj(ConceptualW eight) maxj(ConceptualW eight) minj(ConceptualW eight)

Ii0 = normalized ImpactW eighti =

ImpactW eighti minj(ImpactW eight) maxj(ImpactW eight) minj(ImpactW eight) = controls the relative contributions of two sub-weights By varying from 0 to 1, we can adjust the relative contributions of two underlying recommender systems. When = 0, the CBIR is a pure impact-based recommender system whilst when = 1, the CBIR is a purely Conceptual recommender system.

4. EXPERIMENTAL EVALUATION

4.1 Subjects and Dataset

We conducted several experiments to measure the e ectiveness of our hybrid recommender system. Experiments were done with 30 subjects, undergraduate and graduate computer science and computer engineering students from the university of Arkansas. We use the 2190179 documents in our snapshot of the CiteSeerx, a digital library and a search engine for computer and information sciences literature. Because previous experiments have shown that pro les become stable after viewing 20 papers, users we asked to search for and view at least that many papers related to their own research area. Based on those documents, user pro les were automatically constructed for each user

4.2 Evaluation Method

The goal of this experiment was rst to determine what combination the conceptual match and the paper quality is most e ective in our hybrid recommender system. The relative combinations of the two is given by the equation in Section 3. By changing the value of we are able to control the relative contributions of the two recommender systems with = 0.0 being a pure impact-based recommender system and = 1.0 being a pure conceptual recommender system and = 0.5 using even contributions from both. We varied the value of from 0.0 to 1.0 with an increment of 0.1 for each of the subjects in the experiment and for each value of we collected the top ten recommended documents. For each user, we presented them with the set of all documents recommended by any of the versions of the system (removing duplicates) in random order. They provided explicit relevance feedback by rating the papers as very relevant (2), relevant (1), or irrelevant (0). We then used the Mean Average Weighted Precision (MAWP) of each user for each as a metric. The MAWP is essentially the Mean Average Precision modi ed to handle weights from 0..2 rather than just Boolean relevance judgments. The mean of every MAWP for each is calculated and summarized in Figure 4. As shown on Figure 4, an of 0.9 gives the best results, 0.6355, meaning that a 90% contribution from the conceptual recommender system and a 10% contribution from the impact-based recommender performed the best. For the second part of our analysis, we compared the e ectiveness of the three recommender systems head-to-head. The hybrid recommender system with = 0.9 outperformed the conceptual recommender system's MWAP of 0.6083 ( = 1.0) by 4.5% relative (or 2.72% absolute) and the impact-based recommender system's MWAP of 0.2867 ( = 0.0) by 121.67% relative or 34.88% absolute. Both of these results are statistically signi cant (p < 0.05), based on the paired two-tailed student t-test.

5. CONCLUSION AND FUTURE WORK

In this paper, a hybrid recommender system was introduced that recommends high quality papers to CiteSeerx users. The new recommender combines a conceptual recommender system along with an impact-factor-based recommender system. The former incorporates the user's preferences represented as a concept vector whilst the latter incorporates paper quality using the authors' impact factors as measured by their h-indexes. User experiments were conducted to compare the concept-based recommender system and the impact-based recommender system with our hybrid system. The results con rm that our hybrid recommender generates relevant documents as compared to the conceptual or the impact-factor-based recommender. Future work could consider using social networks of co-authors or di erential weighting of the papers. Another direction would be to investigate the e ectiveness of our hybrid recommender system by considering the g-index that gives a stronger weight to highly-cited papers as compared to the h-index. Alternatively, we could use the e-index that complements the hindex by distinguishing authors having the same h-index but di erent numbers of citations.

References

[1] G. Adomavicius and A. Tuzhilin . Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. Knowledge and Data Engineering , IEEE Transactions on, 17 ( 6 ): 734 { 749 , 2005 .

[2] J. Beel , S. Langer , M. Genzmehr , B. Gipp , C. Breitinger , and A. Nu rnberger. Research paper recommender system evaluation: A quantitative literature survey . In Proceedings of the International Workshop on Reproducibility and Replication in Recommender Systems Evaluation , pages 15 { 22 . ACM, 2013 .

[3] Y. Birks , C. Fairhurst , K. Bloor , M. Campbell, W. Baird , and D. Torgerson . Use of the h-index to measure the quality of the output of health services researchers . Journal of health services research & policy , 19 ( 2 ): 102 { 109 , 2014 .

[4] K. Chandrasekaran , S. Gauch , P. Lakkaraju , and H. P. Luong . Concept-based document recommendations for citeseer authors . In Adaptive Hypermedia and Adaptive Web-Based Systems , pages 83 { 92 . Springer, 2008 .

[5] D. De Nart and C. Tasso . A personalized concept-driven recommender system for scienti c libraries . Procedia Computer Science , 38 : 84 { 91 , 2014 .

[6] E. Gar eld . Journal impact factor: a brief review . Canadian Medical Association Journal , 161 ( 8 ): 979 { 980 , 1999 .

[7] J. E. Hirsch. An index to quantify an individual's scienti c research output . Proceedings of the National academy of Sciences of the United States of America , 102 ( 46 ): 16569 { 16572 , 2005 .

[8] A. Kodakateri Pudhiyaveetil , S. Gauch , H. Luong , and J. Eno . Conceptual recommender system for citeseerx . In Proceedings of the third ACM conference on Recommender systems , pages 241 { 244 . ACM, 2009 .

[9] M. Kompan and M. Bielikova . Content-based news recommendation . In E-commerce and web technologies , pages 61 { 72 . Springer, 2010 .

[10] P. Lops , M. De Gemmis , and G. Semeraro . Content-based recommender systems: State of the art and trends . In Recommender systems handbook , pages 73 { 105 . Springer, 2011 .

[11] J. Mingers , F. Macri , and D. Petrovici . Using the h-index to measure the quality of journals in the eld of business and management . Information Processing & Management , 48 ( 2 ): 234 { 241 , 2012 .

[12] M. J. Pazzani and D. Billsus . Content-based recommendation systems . In The adaptive web , pages 325 { 341 . Springer, 2007 .

[13] S. Philip and A. O. John. Application of content-based approach in research paper recommendation system for a digital library . International Journal of Advanced Computer Science & Applications , 5 ( 10 ), 2014 .

[14] S. Selek and A. Saleh . Use of h index and g index for american academic psychiatry . Scientometrics , 99 ( 2 ): 541 { 548 , 2014 .