Keyphrase candidate compilation is a crucial step for both supervised and unsupervised keyphrase extractors. The traditional methods are usually based on the lexical or frequency properties of the phrases to come up the list. However, terms collected based on these properties do not always semantically meaningful. We show that Wikipedia can be a great auxiliary resource to compile meaningful keyphrase candidates for scientific literature. We conducted empirical experiments on digital libraries of two disciplines, namely Computer Science and Chemistry. The results suggest that Wikipedia has a good coverage of the two disciplines and has the potential to be applied to other scientific disciplines.
Extracting keyphrases from articles is essential for natural language processing and digital libraries. The extracted keyphrases can also be the foundation of other services, such as expert search [ ], collaborator search [ ], venue search, and algorithm search [ ]. Although the problem has been investigated for decades, recent research suggested that automatic keyphrase identification is still challenging [ , ].
Keyphrase extraction can be supervised or unsupervised. Supervised keyphrase extraction typically formulates the task as a binary classification problem in which a model is trained to determine a phrase to be a keyphrase or not. Such method is highly dependent on the training data. As a result, the model could be biased toward a certain domain and less effective in others. In addition, it is not easy to obtain numerous articles with keyphrases of high quality for training. On the other hand, unsupervised keyphrase extractors rely on the characteristics of the words or the phrases to infer their likelihood of being keyphrases. Common techniques include TF-IDF and its variations, graph based ranking, cluster based ranking, etc. [ ]
Both supervised and unsupervised keyphrase extractors usually require generating a list of potential keyphrases, called keyphrase candidates, before performing keyphrase extraction. Since the final set of extracted keyphrases is a subset of the keyphrase candidates, the candidate list should include as many potential keyphrases as possible to achieve a higher recall. However, naïvely adding terms to the list may hurt the analysis efficiency and lower the precision. Several heuristics are commonly applied to compile the list. We list three possible methods below. First, allowing only terms of certain part-of-speech (POS), such as a noun or a noun phrase, to be included in the list [ ]. Second, only -grams conforming to certain conditions are collected [ ]. Third, removing the stop words and treat the single-word terms as the candidates [ ]. Although these approaches are widely used, they analyze only the lexical properties, not the semantic properties, of the terms in the article. As a result, it is very likely to include trivial terms, such as “experimental results” and “difficult problem”, in the candidate list.
We propose to utilize Wikipedia as an auxiliary resource to compile the list of keyphrase candidates for scientific literature. Since Wikipedia is manually edited, the titles, the links, and the category structure are typically non-trivial terms. Experiments were performed on two scientific domains, namely Computer Science and Chemistry. The results suggested that Wikipedia is a promising resource for keyphrase candidate compilation and has a good coverage of the two disciplines.
We collected the titles and the anchor texts (i.e., the visible and clickable text in a hyperlink) of Wikipedia pages to compile keyphrase candidates. Compared to the POS tagger and -gram based approaches, using Wikipedia has three advantages, as described below.
First, the title or the anchor text of a Wikipedia page typically represents one concept, such as a person, an algorithm, a molecule, etc. Thus, it is usually appropriate to assume the entire title or the entire anchor text as exactly one keyphrase candidate, no matter how long or how short the phrase is. On the other hand, when using only lexical properties, it is sometimes challenging to automatically decide which terms should be joined together to represent one concept. For example, the term “Barnes & Noble” should be one phrase to represent the giant book corporation, but it is very likely to be treated as two separated terms “Barnes” and “Nobel” by a lexical-based analyzer; the term “Markov chain Monte Carlo” should be one term, although both “Markov chain” and “Monte Carlo” are valid concepts by themselves. Several languages, such as Thai, Chinese, and Japanese, can be even more challenging in determining a set of characters as a meaningful concept, because these languages exhibit no space boundaries between words and therefore difficult to tokenize and identify a valid term.
Second, the title or the anchor text of a Wikipedia page is usually written as a commonly represented format. Therefore, we do not need to worry about converting a term into its normally used type, such as converting a plural noun into a singular noun. Traditionally, format conversion is accomplished by stemming. However, not every term should be expressed in the stemmed format. , randomly selected documents (b) , randomly selected documents with at least words in title and words in abstract
For example, we mostly say “social media” rather than “social medium”, and we use “data analysis” instead of “datum analysis”. In addition, a stemmer may make mistakes, such as over-stemming or under-stemming, because natural languages are not always regularly constructed. The stemming problem can be more severe in other languages, such as Hebrew and Arabic, which have much more complex rules than English.
Third, Wikipedia can be helpful in identifying the ambiguous terms or the acronym of many possible candidate terms. Given the targeted documents are within a certain domain, say Computer science, we could crawl only the pages related to the topic. In practice, we utilize the category structure of Wikipedia to perform focused crawling. A disambiguated term, such as SVM, may refer to Saskatchewan Volunteer Medal, a civil decoration for volunteers in Canada, Schuylkill Valley Metro, a proposal for a railway system linking Philadelphia and Reading in Pennsylvania, or Support Vector Machine, a powerful machine learning technique. When crawling Wikipedia pages of Computer Science domain, SVM would naturally be determined as Support Vector Machine, since the other alternatives do not of fall in the Computer Science category.
To identify the keyphrases from a document, we compared the context with the candidate list and claimed a phrase to be one keyphrase if it is in the candidate list. To efficiently search the candidate list and perform the longest prefix matching lookup, we created a trie (a prefix tree) for the keyphrase candidates, as suggested in [ ]. 0 1 . 0
Wikipedia is edited manually and therefore the title or the anchor text typically represents a meaningful topic. However, the coverage of Wikipedia in scientific domain, such as Computer Science or Chemistry, is unknown. To answer the question, we conducted empirical study on two digital libraries of different discipline: ( ) CiteSeerX, a digital library currently focused on Computer Science and several related fields, and ( ) the publicly available metadata of documents from Royal Society of Chemistry (RSC), a professional chemistry society in UK.
We randomly selected 10, 000 documents from CiteSeerX as Set A and 10, 000 documents from RSC as Set B. Using the title and the abstract, we counted the number of terms appeared in the keyphrase candidate. .
Results
We have utilized the discovered keyphrase candidates to support several systems. Here we introduce some of them. CSSeer is an expert recommender system built on top of four million academic documents in the fields related to Computer Science and Information Science [ , ]. To efficiently return a list of experts of the specified sub-domain (e.g., information retrieval), CSSeer preprocesses the texts in the title and the abstract of each document to extract the keyphrase candidates as the input texts for more complex algorithms. Since most interesting keyphrases are preprocessed and indexed, CSSeer can effectively return a list of experts within seconds. On the other hand, if a user submits a query term which is not included in the preprocessed keyphrase list, calculating the expert score of a user to the query term in real time is impractical [ ]. Alternatively, we probably need to approximate the expert score by considering only the top related important documents (instead of the full four million documents). However, the approximation considers at most hundreds of documents, which inevitably ignores most of the available information. As a result, the keyphrase candidate extracting method forms an essential component in the CSSeer recommendation service.
Figure shows two snapshots of the CSSeer system. On the left (i.e., Figure (a)), the list of expertise of Dr. W. Bruce Croft is compiled based on the http://csseer.ist.psu.edu/ (a) W. Bruce Croft’s expertise list and (b) Related keyphrases and experts of “inpublication list formation retrieval” keyphrase candidates extracted from his publications. On the right (i.e., Figure (b)), the phrases that are most relevant to the query phrase “information retrieval” is also generated based on the keyphrase candidates compiled by our introduced method.
CollabSeer is another system that was leveraged on the keyphrase candidate compiled based on the introduced method. Essentially, CollabSeer recommends potential collaborators to a researcher’s interested area within her academic social circle. Like CSSeer, we identify each user’s research interest and expertise based on the keyphrase candidates discovered from her previous publications. Figure shows a snapshot of the expertise list of an author. CiteSeerX is an autonomous digital library for scientific literature. For each document, CiteSeerX provides a summary tab that shows the abstract and the keyphrases extracted from the abstract, as shown in Figure .
The current online version of the keyphrase list is compiled based on an unsupervised method which tags the nouns and the noun phrases by the Stanford POS Tagger and noun phrase rules [ , , ] and naïvely treats these noun phrases as the keyphrase candidates. However, we found that the recall of such a method is only about 70%. Since the final extracted keyphrases are only a subset of the keyphrase candidates, we would like the keyphrase candidates to include many potential keyphrases to achieve a higher recall. We plan to update this keyphrases candidate generating process by a mixture of the original method (POS-tagging-based) and the method introduced in this paper (Wikipedia-based) to increase the recall.
As an initial study, we randomly selected 1, 000 papers whose abstract contains at least words, and compile the keyphrase candidates by a mixture of the original and the new method (i.e., we merge the keyphrase candidates returned by the two methods). We found that, on average, the mixture approach increases the number of keyphrase candidates per document from the original 14.49 to 23.29. The increase ratio is (23.29 − 14.49)/14.49 = 60.73% on average. Table shows the summary of the increase ratio of the 1, 000 sampled documents, and Figure displays the empirical cumulative density function (ECDF) of the increase ratio of these documents.
In the meanwhile, we manually labeled the keyphrases of these 100 documents. We computed the recall of the keyphrase candidates generated from the following methods: ( ) generating keyphrases based on the POS tagging; ( ) generating keyphrases based on the Wikipedia terms; ( ) a combination of ( ) and ( ). The average recall from this test dataset is shown in Table . By combining these two methods, we can achieve an average recall rate to 91.67% (increasing the number of keyphrase candidates by 9.01 on average).
In this paper, we empirically validated that Wikipedia titles and the anchor texts are valuable resources to generate keyphrase candidates for scientific articles. We found that, based only on the abstract texts of the scientific documents, such a simple method can generate . keyphrase candidates for a typical paper in the field of Computer Science and Information Systems and . keyphrase candidates for a typical Chemistry paper. If we combine the Wikipedia resource 1.00 0.75 F D C iilrca0.50 p m E 0.25 0.00 0 2 4
6 increase ratio and simple POS-tagging technique, the generated keyphrase candidates yield a very high recall rate (over 90% on average).
We built several systems partially based on the concept. Specifically, we generated each author’s research expertise based on the keyphrase candidates of her previous publications and integrated the function into CSSeer (an expert recommender system for computer scientists) and CollabSeer (a collaborator recommender system for computer scientists). We generated the keyphrases for the documents collected by CiteSeerX and plan to update the current keyphrase list shown online.
For future work, we plan to apply similar concept to different domains. Finally, we are also in the process of releasing the title, abstract, and the extracted keyphrases of the million academic documents collected by CiteSeerX. We hope that such a large dataset can benefit the research community in the digital library and information retrieval. . Chen, H.H., Gou, L., Zhang, X., Giles, C.L.: CollabSeer: a search engine for collaboration discovery. In: Proceedings of the th annual international ACM/IEEE joint conference on Digital libraries. pp. – . ACM ( ) . Chen, H.H., Ororbia, I., Alexander, G., Giles, C.L.: ExpertSeer: a Keyphrase Based
Expert Recommender for Digital Libraries. arXiv preprint arXiv: . ( ) . Chen, H.H., Treeratpituk, P., Mitra, P., Giles, C.L.: CSSeer: an expert recommendation system based on CiteseerX. In: Proceedings of the th ACM/IEEE-CS joint conference on Digital libraries. pp. – . ACM ( ) . Hasan, K.S., Ng, V.: Conundrums in unsupervised keyphrase extraction: making sense of the state-of-the-art. In: Proceedings of the rd International Conference on Computational Linguistics: Posters. pp. – . Association for Computational Linguistics ( ) . Hasan, K.S., Ng, V.: Automatic keyphrase extraction: A survey of the state of the art. In: ACL ( ). pp. – ( ) . Liu, Z., Li, P., Zheng, Y., Sun, M.: Clustering to find exemplar terms for keyphrase extraction. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing: Volume -Volume . pp. – . Association for Computational Linguistics ( ) . Mihalcea, R., Tarau, P.: Textrank: bringing order into texts. In: Proceedings of
EMNLP. vol. . Barcelona, Spain ( ) . Nguyen, T.D., Kan, M.Y.: Keyphrase extraction in scientific publications. In: Asian Digital Libraries. Looking Back Years and Forging New Frontiers, pp. – .
Springer ( ) . Treeratpituk, P., Teregowda, P., Huang, J., Giles, C.: SEERLAB: a system for extracting keyphrases from scholarly documents. In: Proceedings of the th International Workshop on Semantic Evaluation. Association for Computational Linguistics ( ) . Tuarob, S., Mitra, P., Giles, C.: Building a search engine for algorithms. ACM
SIGWEB Newsletter p. ( ) . Williams, K., Chen, H.H., Choudhury, S.R., Giles, C.L.: Unsupervised ranking for plagiarism source retrieval. Notebook for PAN at CLEF ( ) . Williams, K., Chen, H.H., Giles, C.L.: Classifying and ranking search engine results as potential sources of plagiarism. In: Proceedings of the ACM symposium on Document engineering. pp. – . ACM ( )