In this paper, we analyze citation contexts and characterize the di erent sections of scienti c articles in terms of the verbs that appear in citation contexts. We have performed Factorial Correspondence Analysis (CA) using the four sections of the IMRaD (Introduction, Methods, Results and Discussion) structure as categories. Our dataset contains about 80,000 research articles published in the six PLOS journals. The results of this approach show that the sections in the rhetorical structure of research articles have very di erent characteristics when we take into consideration the occurrences of verbs, and more generally, their lexical content. Our results demonstrate a strong relation between verbs used around citations and the positions in the rhetorical structure.
The study of in-text citations is a very old subject, but it still remains an
active area of research. This problem interests many researchers and covers several
di erent disciplines and research areas: Bibliometrics, Information science,
Sociology of science, Computer science. Despite numerous studies, there do not yet
exist automated approaches for the analysis of in-text citations, mainly because
of the di culties in constructing a complete model of the function of citations.
As Cronin [
In this paper, we address this problem from the point of view of textual
statistics. Recent works have shown that the distribution of in-text citations in
scienti c articles are strongly correlated to their IMRaD structure (see [
In this paper, we report on a set of experiments to classify sentences containing in-text citations, according to their position in the rhetorical structure. We believe that the study of citation contexts implies observing the use of citations in a certain amount of textual data. In this exploratory study, we will focus on this perspective that seems relevant for the understanding of citation acts.
For this purpose, our study uses a multivariate statistical method, namely
Correspondence Factorial Analysis (CFA) (see [
In this study, we investigate the relationship that exists between the rhetorical structure of papers and the text structure and more speci cally the lexicon. By analysing occurrence frequencies of di erent lexical items, the main objective of this method is to achieve an optimal projection of the multidimensional system on a factorial plot. The hypothesis that we want to verify can be formulated as follows: the lexical forms present in the contexts of in-text citations are not randomly distributed. They are strongly dependent on the particular positions of the rhetorical structure. 2.1
Our dataset consists of six peer-reviewed academic journals published in Open Access by the Public Library of Science (PLOS): six domain-speci c journals (PLOS Biology, PLOS Computational Biology, PLOS Genetics, PLOS Medicine, PLOS Neglected Tropical Diseases ) and PLOS ONE, a general journal that covers all elds of science and social sciences. We have processed the entire dataset of about 80,000 research articles published up to September 2013.
We have identi ed the section structure in each article by analyzing the section titles. All six journals use similar publication models, where authors are explicitly encouraged to follow the IMRaD (Introduction, Methods, Results, and Discussion) structure. As a result, more than 97% of all research articles contain these four section types, although not always in the same order.
We have identi ed and extracted all textual segments that contain in-text citations. To do this, the text was segmented into sentences and for each of the four sections we have considered the set of sentences containing in-text citations. As a result, we have obtained a total of 3,314,884 sentences, 31.52% out of which belong to Introduction sections, 19.50% to Methods, 14.66% to Results and 34.33% to Discussion sections.
Next, we propose to use these sets to examine the characteristics of citations in the di erent sections and the ways citations are used according to their position in the rhetorical structure of articles. 2.2
We study the sets of sentences from sections that are identi ed according to the
IMRaD structure and the presence of in-text citations. For this purpose, we use
two text analysis tools. The rst one is an R Commander plugin (see temis [
The set of sentences have been split into words and lemmatized: all di erent forms of a lexical item are identi ed and associated with the same lexical item. IRaMuTeQ performs stemming from dictionaries, without disambiguation, also called endogenous lemmatization. After lemmatization, we have ltered all verb forms and ranked them by occurrence frequency for each section. This allowed us to produce a map displaying proximity among variables (Lexical vs Rhetorical Structure).
To perform the analysis, we have created a subset of sentences that consists of about 2,000 randomly extracted sentences for each section of the rhetorical structure and for each journal. This amounts to a total of 48,000 sentences for this analysis. As shown in Table 1, this corpus contains 47,714 unique terms, that have 1,569,201 occurrences.
Total 327,506 21,389
6.5 8,809
2.7 327,506 124,618
38.1 43,499 13.3 5.7 295,798 337,379 22,848 22,316
7.7 6.6 10,539 9,326
3.6 2.8 295,798 337,379 102,422 114,858
34.6 34.0 34,032 38,668 11.5 11.5 5.5 5.4 608,518 1,569,201 28,694 47,714
4.7 3.0 11,689 18,082
1.9 1.2 608,518 1,569,201 222,373 564,271
36.5 36.0 76,932 193,131 12.6 12.3 5.6 5.5 We have performed Factorial Correspondence Analysis (CA) using the four sections of the IMRaD structure as categories. The interpretation of this analysis is a graph representation of associations between rows and columns. Columns express the sections while the rows correspond to all forms of occurrences. As word meanings strongly depend on their contexts, we consider only sentences containing in-text citations, thus limiting the possible ambiguities.
The interpretation of an axis in CA in a linguistic context is de ned by the opposition between the extreme points. Figure 1 presents the projection of the four sections. This gure shows that, for example, the Methods section is in opposition to all other sections. Similarly, the Results and the Introduction sections are opposed to each other on the vertical axis.
Figure 2 presents the projection of a sample of the most frequent verbs. For the same verbs, table 2 presents the values for the relative frequencies in the di erent sections: Introduction - Methods - Results - Discussion. If certain verbs are more or less homogeneous among the sections, some of the verbs, as observed, are only predominantly found in the Discussion and Results sections.
For example, we can see on gure 2 that the verbs performs and calculate (see i) are mainly present in the Methods section. For the same verbs, table 2 indicates the values [perform - 52.16] and [calculate - 22.47]. The position of the verbs cause and review (see ii) on gure 2 show that they are characteristic of the Introduction section. This is con rmed by the values given in table 2, respectively [case - 16.79] and [review - 14.06]. Another example are the verbs analyse approach assume calculate (i) cause (ii) carry characterize compare con rm (iii) consider contrast de ne demonstrate detect determine develop encode establish examine expect (iii) identify include indicate involve mean measure note observe obtain perform (i) predict present propose provide regulate remain represent reveal review (ii) study suggest 6.38 9.18 3.7 1.32 9.35 2.98 4.38 11.22 5.74 5.87 9.18 4.93 19.3 10.2 6.04 9.18 7.91 5.61 4.04 5.57 18.02 25.67 8.33 14.71 4.72 7.86 7.18 27.03 4.76 3.87 8.8 15.6 12.45 11.01 12.37 5.65 6.8 6.89 7.95 91.72 40.16 6.33 11.47 3.09 1.18 16.79 4.87 7.96 8.19 2.96 5.6 6.19 7.19 13.52 7.69 7.46 15.61 12.29 6.51 3.6 2.96 23.16 32.86 5.55 17.93 3.23 7.24
2 11.29 4.32 3.82 7.46 11.83 10.06 11.92 12.42 9.01 5.64 7.15 14.06 70.54 25.67 con rm and expect (see iii) that mainly belong to the Results section. Their values for this section in table 2 are [expect - 7.96] and [con rm - 9.01].
The results of this approach show that the sections in the rhetorical structure of research articles have very di erent characteristics when we take into consideration the occurrences of verbs, and more generally, their lexical content. Our results demonstrate a strong relation between verbs used around citations and the positions in the rhetorical structure. In addition, gure 1 shows some proximity between the Results and the Discussion sections, as well as between the Discussion and the Introduction sections. 4
This study con rms the results of previous work (see [
The results can also be considered in the perspective of other studies on the
distribution of references [
The results of our study have numerous applications, especially in
Information Retrieval ([
We thank Benoit Macaluso of the Observatoire des Sciences et des Technologies (OST), Montreal, Canada, for harvesting and providing the PLOS data set.