In this paper we present a large-scale approach for the extraction of verbs in reference contexts. We analyze citation contexts in relation with the IMRaD structure of scienti c articles and use rank correlation analysis to characterize the distances between the section types. The results show strong di erences in the verb frequencies around citations between the sections in the IMRaD structure. This study is a "one-more-step" towards the lexical and semantic analysis of citation contexts.
Citation analysis has been the subject of numerous studies during the last
decades and there has been a constant interest in producing a theory of citations.
The works of Cronin [5{7], Small [
Scienti c articles typically follow the standardized IMRaD (Introduction,
Method, Result and Discussion) structure. It gives a rhetorical outline for
scienti c writing that began to predominate in 1965 and, as Sollaci [
We have processed a corpus of scienti c articles to produce ordered lists of verbs according to their occurrence frequencies in the di erent section types. Our method is based on the following steps: (i) the XML documents are parsed, sections are extracted and categorized according to the four section types of the IMRaD structure; (ii) we segment the sections into sentences and extract the sentences containing references; (iii) we use POS-tagging and lemmatization tools to identify verbs in citation contexts and construct the ranked verb lists according to their frequencies in each section type. 2.1
Dataset For this study, we have used a corpus of ve scienti c journals: PLoS Biology, PLoS Computational Biology, PLoS Genetics, PLoS Neglected Tropical Diseases and PLoS Pathogens, published by PLoS1 and available in Open Access in the XML format. The articles are structured using the Journal Article Tag Suite (JATS)2, where the sections in the text are represented as separate elements. We have processed the entire set of research articles of these journals up to September/October 2012. Table 1 shows the number of articles and citation contexts extracted from each journal.
PloS Bio.
PloS Comp. Bio.
PloS Gen.
PloS Negl. Trop. Dis.
PloS Path.
Total
Nb of articles Nb of sentences Nb of citations Citation contexts 1,587 356,816 150,429 79,703 1,976 487,045 177,742 92,437 2,435 544,569 227,121 126,230 1,240 200,920 83,402 45,714 2,208 496,371 209,685 115,750 9,446 2,085,721 848,379 459,834
2 This Standard is an application of NISO Z39.96-2012 and JATS is a continuation of the NLM Archiving and Interchange DTD (http://jats.nlm.nih.gov)
Section Categorization and Part-Of-Speech-Tagging Each section is presented in an XML element containing a title and some text content. Our rst task was to categorize the sections according to the four types of the IMRaD structure: Introduction, Method, Result and Discussion. To do this, we analysed the section titles and used a set of regular expressions related to each section type in order to account for the possible variations in section titles. For example, the Method section type can be expressed by several di erent titles such as "Method", "Methods", "Method and Model", etc.
The articles in our corpus often contain other section types such as additional information, acknowledgement, etc. that were not taken into consideration. A small number of articles in the corpus do not follow the IMRaD structure and use domain-speci c section titles. They were excluded from the study.
The basic unit in our study is the sentence and the quality of the sentence segmentation is important to reliably determine the citation contexts. In fact, rather than de ne the contexts in terms of number of words around citations, we prefer to use the sentences boudaries as natural delimiters of citation contexts. Sentences are basic linguistic units of texts that we consider as most suitable to model text progression. Table 2 shows the number of citation contexts extracted for each section type.
PloS Bio.
PloS Comp. Bio.
PloS Gen.
PloS Negl. Trop. Dis.
PloS Path.
Total
Introduction Method Result Discussion 19,769 13,911 25,263 20,760 25,721 18,964 27,907 19,845 31,476 23,239 39,781 31,734 14,103 9,611 5,533 16,467 31,107 21,202 29,676 33,765 122,176 86,927 128,160 122,571
The extracted citation contexts were processed using TreeTagger, a
part-ofspeech tagger3 [
Taking into consideration the set of verbs that appear in all four sections, we have obtained a set of 1807 verbs. Then we produced the ranked list of verbs for each section, ordered by the verb frequencies.
A classical phenomenon is the fact that most of the verb occurrences in citation contexts belong to only a small set of verbs. Table 3 shows that, for example, in the Introduction section, 70 verbs account for 50% of all verb occurrences, and 486 verbs account for 90% of the occurrences.
Table 4 shows the ranked lists of the top 10 most frequent verbs for each section type. It is interesting to observe some of the di erences. For example, we can see that the verb show does not appear in the Method section while it is on the rst or second position in all the other sections. This means that the verb show is used very often in citation contexts except in the Method section where it is quite rare. Similarly, we can observe that the Method section contains some speci c verbs (perform, follow, obtain, generate) that do not appear among the top 10 verbs of any other section.
Rank Introd. Method Result Discussion 1 show use use show 2 use perform show suggest 3 include follow nd use 4 suggest obtain report report 5 identify generate observe nd 6 nd base suggest include 7 require determine identify observe 8 associate contain express require 9 involve calculate see associate 10 lead carry include involve Figure 1 gives the heatmaps for some selected verbs along the text progression of each section. The horizontal axis corresponds to the progression of the text in each section, from 0% to 100%. Most of these verbs express citation acts. This representation shows that the densities of some verbs vary considerably, especially in the beginnings and ends of the sections. Certain verbs, such as perform, obtain, include, describe, have rather important variations. This result is compatible with the hypothesis that certain citation functions are more likely to be present at some speci c positions in texts. From in Information Retrieval point of view, it can be interesting to take this into account for the de nition of new term weights related to text positions.
To compare and observe the correlations between the di erent ranked lists,
we have used the Kendall tau rank correlation coe cient [
The Kendall measure is de ned as:
(C) = 12 n=(n (D) 1) ; (1) where C is the number of concordant pairs and D is the number of discordant pairs. 2 [ 1; 1], = 1 if the ranks are identical and = 1 if the ranks are inverse.
Figure 2 shows the values of Kendall and the scatterplots for the di erent section pairs. The scatterplots were obtained by comparing the ranked lists of verbs for each section pairs. On the horizontal and vertical axes we have the 1807 verbs that appear in all sections. The verbs are arranged according to their rank in the Introduction section.3
The biggest similarity is between the Introduction and the Discussion
sections ( = 0:76), which means that for these two sections the majority of the
verbs are ranked on similar positions. On the corresponding scatterplot, this is
expressed by the density around the main diagonal. These two sections use most
often the same verbs in citation contexts. The similarity between the Method
and the Result is the smallest ( = 0:39) which means that most of the verbs in
the Result are rarely employed in the Method and vice versa. On the scatterplot
this is expressed by a larger dispersion which accounts for the fact that these two
sections tend to make use of di erent sets of verbs around citations. A similar
case is the pair Method and Discussion which also shows large dispersion.
These results show clearly that the section structure of research papers is an
important element to consider as classi ers for citation context analysis.
Furthermore, we are able to propose corpora of verb classes related to sections in
the IMRaD structure of research papers. These corpora can serve as a reference
data for other works, for example construction of large-scale corpora dedicated
to machine learning (see Athar and Teufel [
This work con rms the hypothesis that citations play di erent roles according to their position in the rhetorical structure of scienti c articles. The study of citation act verbs is the rst step for the categorization of citations and network structures, such as co-citations and bibliographic coupling. Our results show that citation acts are expressed by a relatively small number of verbs that appear in citation contexts. The next step will be automatic semantic reference categorization based on the verbs in the citation contexts as well as other contextual elements.