Weak Links and Strong Meaning: The Complex Phenomenon of Negational Citations

Abstract

The latest advances in research in the eld of bibliometrics take into consideration in-text references. The purpose of this paper is to focus on citation contexts that express negational citations and which are relatively rare in articles. Our goal is to automate the extraction of negational citation contexts and to put them in relation to positions in the text progression of articles. After extracting sentences from the full text body of articles, we construct linguistic resources in order to identify some of the negational citation discursive patterns. We show the distribution of negational citations in the IMRaD structure. The identi cation of negational citations has numerous potential applications and might be used to improve information retrieval of scienti c papers.

Keywords

• Negative Citations
• Citation Context Analysis In-text References
• Bibliometrics
• Semantic Annotation
• IMRaD

Introduction

Citations can perform a wide variety of functions. One of the most interesting functions of citations is the expression of a negation when citing authors, papers, work or object of studies. In this paper we examine the nature of negational citations and show some of the complexity of the relation between citing papers and cited papers.

The aim of this work is to focus on the negative discursive forms located in sentences with in-text references. We already know that citations for which the relation between citing paper and cited paper has a speci c meaning expressed by the author represent only small fraction of all citations (see [ 1 ]). In fact, most citations are perfunctory and their contexts cannot be exploited [ 2 ] by Natural Language Processing tools.

To study Negational References, we must de ne the notion of negational citations. According to Gar eld, the notion of Negative credit refers to the fact of criticizing, correcting, disclaiming and disputing other works using negative references. Gilbert, in Referencing as Persuasion (see [ 3 ]), suggests that some references in a rmative contexts can be in fact negative. Furthermore, the article The Negative reference by MacRoberts (see [ 4 ], p.92) arrives at the conclusion that there are few negational references in the scienti c literature. The essence of negational citations is related to the very nature of the discourse and of the scienti c debate. Hyland [ 5 ] argues that it is the relationship with the scienti c truth, "the idea being that the facts must be allowed to speak for themselves with no human intrusion". One of the consequences is that: "Hedges (underlined) indicate interpretations and allow writers to convey their attitude to the truth of the statements they accompany, thereby presenting unproven claims with caution and softening categorical assertions." (see [ 5 ], p.33)

If the phenomenon of negational citations is complex, the rst attempts for their identi cation in texts were made using machine learning. By having access to the author of the article or an expert in the eld, the categorization of citations can be done for a training dataset. For example, Catalini et al. proposed to study negative citations in the Journal of Immunology [ 6 ]. After manually annotating the training set of 15.000 citations, they used NLTK Naive Bayes algorithm and cosine similarity to identify other negative citations.

Citation contexts have to be explored in order to determine these relations. This eld of study is part of Content Citation Analysis. In the eld of context citation analysis, the relations between citing papers and cited papers are determined by exploring citation contexts. The main approach is to focus e orts on the categorization of citations by studying the linguistic context, generally located near the in-text reference. The context window could be de ned by the number of words on the right/left side of the reference, or by sentence boundaries because sentences are textual units that can express meaning which is relatively independent from their context. This last method is more technical but more accurate for this purpose.

In fact, the question of the de nition of the context windows around in-text references is all the more di cult as, in the words of Athar and Teufel (see [ 7 ], p.597), we face a "non-local expression of sentiment". Their study based on the annotation of 20 papers extracted from the corpus ACL Anthology shows that we need to increase the context window for the task of sentiment citation analysis. However, linking sentiments to contexts still remains a di cult task. This work is in the continuity of the works of Teufel et al. [ 8 ]. Their protocol mentions the fact that they use "the last sentiment mentioned in the context windows as this is pragmatically most likely to be the real intention (MacRoberts and MacRoberts, 1984)" (see [ 8 ], p.598). They demonstrate that for negative sentiments, this approach triples the recall (see [ 7 ], p.598, table 2). This is an essential point, which is at the heart of our experimental protocol. 2

Method

We propose an exploratory study to de ne a conceptual framework around the identi cation of negational references. For this purpose, we set experimental benchmarks around the location of negational references.

Firstly, we have constructed a dataset of scienti c articles, including metadata, full-text, information of IMRaD (Introduction, Methods, Results and Discussion) structure, segmentation into sentences and identi cation of in-text references. Secondly, we have built linguistic resources allowing the identi cation of negational references. These resources do not cover the full spectrum of negational references. From a methodological point of view, the linguistic resources were not built to be exhaustive, but to limit the false positives. Thirdly, after building a set of Finite State Automata (FSA), we have extracted sentences with in-text references. The results obtained allow us to propose a prototypical framework to put into relation negational references to both the IMRaD structure (see [ 9 ]) of articles and enhanced context windows. 2.1

Protocol

Our experimental protocol for the exploration of citation contexts in view to identify negational citation considers the following limitations. The only use of contexts including in-text references is insu cient. This implies that new types of processing should be taken into consideration such as anaphoric resolution to enhance contexts. Furthermore, the negational references are often hidden, di use or disassembled (see [ 4 ]). Moreover, there are few negational references in scienti c literature. Having considered these limitations, our study is based on two elements : 1. The expression of negation : We consider the discursive forms present in the text that can be identi ed using linguistic markers, and that express negational relations. 2. Context windows: For this study, we work at the sentence level.

Our objective is to study discursive forms that express negational relations in the contexts of in-text references and also outside these contexts. To do this, we consider two types of sentences: those containing in-text references and those without in-text references. For the moment, we cannot identify negational references exhaustively ; we focus on some of the known forms and look into the relationship between their occurrences in sentences with in-text references and in sentences without in-text references. 2.2

Dataset

Our dataset consists of seven peer-reviewed academic journals published by the Public Library of Science (PLOS)3. We have processed the entire dataset of about 80,000 research articles published up to September 2013. 3 Six domain-speci c journals (PLOS Biology, PLOS Computational Biology, PLOS Genetics, PLOS Medicine, PLOS Neglected Tropical Diseases, and PLOS Pathogens) and PLOS ONE, a general journal that covers all elds of science and social sciences. (http://www.plos.org)

We identi ed the section structure in each article by analysing the section titles, in order to identify the four main section types in the IMRaD structure (Introduction, Methods, Results, and Discussion). More than 97% of all research articles in the corpus contain these four section types. We segmented all sections into sentences and extracted sentences containing in-text references. The dataset contains a total of 12,556,466 sentences without in-text references and 3,601,842 sentences with one or more in-text references. 2.3

Linguistic Resources

From a linguistic point of view, Hyland [ 10 ] presents 400 di erent verbs used in citations. We agree with the fact that verbs are an important feature to understand the acts of citations at the semantic and rhetorical level [ 11 ].

The main di culty is to create linguistic resources for our rule-based approach. Our objective is to obtain a set of rules that have a high level of precision. The recall is not taken into consideration for this study because we want, rst of all, to determine whether the same resources could be applied sentences with and without in-text references. This is not a traditional methodological perspective ; however this approach allows to foreground the role of sentences without in-text references in the phenomenon of negational citations.

For the constitution of the linguistic resources expressing negational relations in the act of citations, we focus on verbs and adjectives. The extracted citation contexts were processed using TreeTagger [ 12, 13 ] which performs both partof-speech tagging and lemmatization. In the output verb forms are tagged by labels such as VB, VBD, VBG, VBN, VBP, VBZ that stands for base form, past tense, present participle, . . . To identify the relevant verbs we analyse and extract all verbs from citation contexts. Then we focus on author disagreement using counter-factive verbs like : fail, ignore, exaggerate, etc. One particularity of the negation is that it can be expressed in di erent ways: e.g. not agree with | disagree with. For example : { 'We do not agree with the methodology of using a training set without MCI to select biomarkers to di erentiate between AD and MCI [ 1 ].' { 'Results disagree with Bequet and Przeworski [ 3 ] who reported a split time of 1.4 million years and an ancestral e ective size of (C.I. ).'

We have also identi ed some sentences containing quali ers that we call non-consensual dubious and their synonymous (absurd, doubtful, farcical, farfetched, far-out, imsy, frivolous, grotesque, implausible, impractical, improbable, inconceivable, inoperable, ludicrous, non-viable, outrageous, questionable, remote, ridiculous, strained, uncredible, unenforceable, unfeasible, unlikely, unrealistic, unreliable, etc.)

For this study, we have a set of regular expressions for identifying discursive forms. The constituted resources represent a core around the discursive expression of negative citation forms. 3

Results

This work brings one more element in response to the existence of negational citations situated outside the context window located around in-text reference. 3.1

Prototypical Model of Negational Citations

A sample of sentences with in-text references are presented in the tables 2 and 3. Here are two examples:

However, we and other authors (Table S2) disagree with the low percentage of anxious-type OCD in patients with GTS as observed by Shapiro and Shapiro [30].

Note also that I disagree with Livezey and Zusi ([64]: character 1142) regarding the condition in Pelecanus, which I interpreted as possessing the plesiomorphic state (Figure 12).

The tables 4 and 5 present sentences with negational contexts but without in-text references. Some examples are:

Most biologists nowadays disagree with Darwin's view of species, largely because of Mayr's biological species concept.

After 1986, some authors persisted in the confusion.

These sample gives us precise information on the nature of sentences containing negational contexts without in-text references. This allows us to construct the following categorization of the object of negation: { model or theory (Darwin's view of species) { actor of science (proper names of people) { object as a material product of research (work, table, result, report) { object as an abstract product of research (idea, ndings, concept, notion, opinion) { framework (classi cation, norm)

We have also detected speci c cases which are more complex and need further investigation. For example, if an author cites an author who gives negative credit. As a last point, we con rm that anaphora play an important role and we agree with the conclusions of Athar and Teufel [ 7 ] on the fact that the context window must be enhanced. In some cases, forms such as "some authors", "most biologists", "them" , "him", etc. are found in sentences before or after citation contexts. The few examples we have given show that if the study of anaphora is necessary, it is not enough and the task will require the implementation of other methods. 3.2

Ratio of Negational References

The largest number of negational references are found in the Discussion section, then in the Results and Introduction sections. The Methods section contains relatively small number of negational references. It is interesting to note that this number remains small also when we consider sentences without in-text references. In the Introduction and Results sections, we nd that the number of negational contexts doubles if we consider contexts without in-text references. In the Discussion section, we nd more than 56% of negational contexts without in-text references. 4

Discussion

The low relative frequency of the negational citations is not expressed in this study because the linguistic coverage of the resources and the method used is not optimal. But the most signi cant result is that the Methods section contains almost no negational citations and that the Discussion section contains the largest number. Regarding the Methods section, this result is consistent with the work of Bertin et al. [ 9 ], who show the atypical character of this section in relation to the other sections. The fact that the Discussion section contains the highest percentage of negational citations gives it a new property that has not yet been described in scienti c literature.

We show that the ratios obtained in table 1 corroborate the work of Athar and Teufel [ 7 ]. It is necessary to extend the context window of study in order to carry out citation analysis with better recall. If this phenomenon can be understood through the action of anaphora, it remains counter-intuitive. 5

Conclusion and Perspectives

This work is not su cient to provide a comprehensive method for the study of negational references. However, the fact that the context window must be extended for negational references analysis is an important phenomenon which is also counter-intuitive. This study raises a number of questions. For example, we can consider the fact that an author may be cited several times in a paper and study the di erent categories of the references. Another point is the veri cation that the last reference near the negational context is the one that will be retained in the analysis. This assumption was proposed by MacRoberts [ 4 ] and was used by Athar and Teufel [ 7 ].

In our future work, we will focus on the distance between a sentence with an in-text reference and the sentence that carries the negational context. This metric is important to determine the size of the context windows that must be taken into consideration.

If the frequency of negational citations is low, they carry a strong meaning of the nature of the relation between citing papers and cited papers. Indeed, a relationship expressed by a negation induces polarity that can be exploited from a theoretical point of view in the analysis of networks, such as co-citation networks and bibliographic coupling, and also in applications for scienti c monitoring.

More precisely, these works are of special interest in the CBRC system construction (Context-Based Recommendation System Citation) [ 6, 14 ] as a tool to propose a recommendation system of scienti c articles based on the analysis of citation contexts. Another prospect is a better understanding of the motivations for citations in the articles. Furthermore, the negational references can play a predicative role in the detection of articles that may or must be retracted. Naturally, this work will nd an impact on networks of co-citations [ 15 ], the generation of surveys [ 16 ] or in the eld of Information Retrieval [ 17, 18 ], but also for plagiarism detection [ 19 ]. 6

Acknowledgments

We thank Benoit Macaluso of the Observatoire des Sciences et des Technologies (OST), Montreal, Canada, for harvesting and providing the PLOS data set. 7

Annex

DOI

Buckwalter JA believes that the lumbar spine

degeneration initially occurs within then central NP of the IVD [ 18 ].

Thus, the full model proposed by Nietzsche [ 3 ] has remained empirically unproven. Our results appear to contradict those of [35]. As these observations contradict those of a similar published study [ 13 ] we considered whether some e ect of natural T reg might be being masked by the rapidly ascending parasitaemia and early mortality associated with infection with 104 PyL parasites.

Our results are in striking contrast with previous ndings that co-expression of CGG and CCG expansions in ies leads to mitigated toxicity in a ago2-dependent manner [31], suggesting that toxicity derived from interactions between sense and anti-sense repeat transcripts may be speci c to CTG/CAG situations.

Our observations in humans also contrast with the molecular responses observed during acute synergist ablation induced hypertrophy [66], raising further doubts over the relevance of such pre-clinical models to inform about physiological muscle growth in humans.

These results di er from previously published agglutination data for a frzS insertion mutant (DZ4219) [ 8 ].

They also di er from previous ndings of overactivation in occipital regions [35], [38]. DOI

Section

result pone.0022117 discussion pone.0044996 discussion pone.0065040 discussion pcbi.0010067

result pgen.1000160

method pone.0070707 introduction pgen.0020052 discussion pntd.0001986 discussion pone.0050952 discussion pntd.0001001 discussion 4 7 4 9 1 1 4 9 1 4 2 4 4

Our results do di er from previously published vibration reports that have subjected laryngeal broblasts to strain and vibration [10], [11].

However, we and other authors (Table S2) disagree with the low percentage of anxious-type OCD in patients with GTS as observed by Shapiro and Shapiro [30].

Note also that I disagree with Livezey and Zusi ([64]: character 1142) regarding the condition in Pelecanus, which I interpreted as possessing the plesiomorphic state (Figure 12).

However, our ndings disagree with a recent report showing that megakaryocytes/platelets speci c deletion of Cdc42 had no e ect on

lopodia formation on immobilized brinogen or CRP [ 13 ].

Our results disagree with a prior study which showed a mutant form of AICD (Y682A/Y687A) that was suggested not to bind Fe65, was still active in promoting cell death [53].

However, our results disagree with the results in a recent report that DDX5 silencing strongly increased p24 release [59].

We thus believe that the motif reported in [27] is dubious.

This may suggest that a signi cant fraction of the variants reported in [ 16 ] are dubious.

Krings et al. [ 5 ] reviewed the fossil record of the Peronosporomycetes and concluded that all the reported occurrences of this group older than Devonian are dubious or inconclusive.

Furthermore, this study has ignored upstream ORFs, which may contribute many short proteins [39].

Some authors believe, these animals pose a major threat to man [ 7 ].

Surprisingly, only four studies [37], [24], [27], [25] performed some form of validation of their models.

A study of rabies in Tanzania also suggested dog rabies control was feasible, but was hampered by perceived problems that were largely unfounded [ 7 ]. DOI

Section Pos. in sect. Sentence without in-text references pbio.0050068 discussion pbio.1001102 analysis pbio.0030293 discussion pcbi.1001071 discussion pgen.1001174 discussion pgen.1002739 discussion pgen.1000419 discussion pone.0014769 discussion pone.0023567 discussion pone.0023842 discussion pone.0029539 discussion pone.0029704 discussion 1 1 4 7 1 2 3 1 2 3 3 7

At rst glance, our ndings appear to contradict two recent papers that investigate the N-p50 association.

In summary, we disagree with the fundamental assertion that it is the total area of transcribed sequence that is most important.

These data may appear to contradict our results showing that the amygdala is required for the reconsolidation, but not the consolidation, of the rst-order conditioning, and not needed for the formation of the second-order response.

On its own, this information would appear to contradict the knowledge that proliferation rates are highest distally, however Map6 was also the one with a strong distal fanning-out movement along the AP axis (central row in Figure 4B).

The conclusions that derived from our observations contrast with the widely held belief that RNAi initiates heterochromatin assembly at ssion yeast centromeres.

Our results appear to contradict this wellestablished dogma.

One is that all wild strains described from the Midwestern United States (TR388, TR389, and TR403) are dubious.

These results contradict those obtained by previous analyses where Neanderthals have been traditionally viewed as a species feeding mostly on animal proteins and more speci cally large game animals.

This result may initially appear to contradict the ndings in MDCK cells, but the di erence may be explained in several ways.

Our results disagree with this possibility.

Our results disagree with this assertion as we did not nd that parthenolide inhibited caspase-1 in response to AIM2 stimulation.

This result is similar to the clustering algorithm's results and we disagree with Chang's classi cation. DOI

Section

Pos. in sect. Sentence without in-text references pone.0057887

discussion pone.0058442 pone.0070922 pone.0037786 pone.0038998 discussion discussion discussion discussion pcbi.1000912

method pcbi.0020099

undef pcbi.0030002 introduction pmed.0050110

undef pone.0012983

result pone.0016409

result pone.0045897 introduction pbio.0030152 pbio.0030382 undef undef

At rst glance, our results appear to contra

dict this nding.

However, we disagree with this opinion. These results appear to contradict the known e ects of attention on P300 and N2pc. Our results disagree with either of them. Some of our observations appear to directly contrast with those previously described (Table 4).

Whereas some authors consider standard logical functions for all components, we do not impose such a restriction.

Some authors can be unresponsive or uncooperative, thus impeding completion of their entry.

Some authors argue that increasing the number of characters sampled per taxon improves the accuracy, while others state that accuracy is better improved by subdividing long branches by including more taxa, resulting in fewer characters overall.

Some authors have speculated about UTR sequences making direct RNA:RNA interactions with target transcripts and thereby inuencing their stability or translation. Some authors think that the initial experiment that leads to hypothesis generation is most important, while others consider the core experiments or gures that lead to the main conclusion of the article to be most important.

After 1986, some authors persisted in the confusion.

Some authors believe that it may be due to a defective extracellular receptor-associated kinase (ERK) pathway.

Most biologists nowadays disagree with Darwin's view of species, largely because of Mayr's biological species concept.

Historians still disagree whether or notor the degree to whichDarwin had tumbled to the idea of evolution while still on the Beagle.

References

1. Teufel , S. , Siddharthan , A. , Tidhar , D. : Automatic classi cation of citation function, Association for Computational Linguistics ( 2006 ) 103 { 110

2. Moravcsik , M.J. , Murugesan , P.: Some Results on the Function and Quality of Citations . Source: Social Studies of Science 5 ( 1 ) ( 1975 ) 86 { 92

3. Gilbert , G.N. : Referencing as persuasion . Social Studies of Science 7 ( 1 ) ( 1977 ) 113 { 122

4. MacRoberts , M.H., MacRoberts , B.R.: The Negational Reference:or the Art of Dissembling . Social Studies of Science 14 ( 1984 ) 91 { 94

5. Hyland , K. : The author in the text: Hedging scienti c writing . Hong Kong papers in linguistics and language teaching 18 ( 1995 ) 33 { 42

6. Catalini , C. , Lacetera , N. , Oettl , A. : The incidence and role of negative citations in science . Proceedings of the National Academy of Sciences ( 2015 )

7. Athar , A. , Teufel , S. : Context-enhanced citation sentiment detection . In: Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics ( 2012 ) 597 { 601

8. Teufel , S. , Siddharthan , A. , Tidhar , D.: An annotation scheme for citation function . In: Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue , Association for Computational Linguistics ( 2009 ) 80 { 87

9. Bertin , M. , Atanassova , I. , Larivire , V. , Gingras , Y. : The invariant distribution of references in scienti c articles . Journal of the Association for Information Science and Technology ( 2015 )

10. Hyland , K. : Academic attribution: citation and the construction of disciplinary knowledge . Applied Linguistics 20 ( 3 ) ( 1999 ) 341 { 367

11. Bertin , M. , Atanassova , I. : A study of lexical distribution in citation contexts through the IMRaD standard . In: Proceedings of the First Workshop on Bibliometric-enhanced Information Retrieval co-located with 36th European Conference on Information Retrieval . Volume 1143 ., Amsterdam, The Netherlands (April 13 2014 ) 5 { 12

12. Schmid , H.: Probabilistic Part-of-speech Tagging Using Decision Trees . In: Proceedings of international conference on new methods in language processing . Volume 12 ., Manchester, UK ( 1994 ) 44 { 49

13. Schmid , H.: Improvements in Part-of-speech Tagging with an Application to German. In: In Proceedings of the Association for Computational Linguistics SIGDATWorkshop . ( 1995 )

14. Duma , D. , Klein , E.: Citation Resolution: A method for evaluating context-based citation recommendation systems . In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics , Baltimore, Maryland, USA (June 23-25 2014 ) 358 { 363

15. Boyack , K.W. , Small , H. , Klavans , R.: Improving the accuracy of co-citation clustering using full text . Journal of the American Society for Information Science and Technology 64 ( 9 ) (sep 2013 ) 1759 { 1767

16. Mohammad , S. , Dorr , B. , Egan , M. , Hassan , A. , Muthukrishan , P. , Qazvinian , V. , Radev , D. , Zajic , D. : Using citations to generate surveys of scienti c paradigms . In: Proceedings of Human Language Technologies : The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics . NAACL ' 09 , Stroudsburg , PA, USA, Association for Computational Linguistics ( 2009 ) 584 { 592

17. Ritchie , A. , Teufel , S. , Robertson , S. : Using Terms from Citations for IR: Some First Results . In Macdonald, C. , Ounis , I. , Plachouras , V. , Ruthven , I. , White , R., eds. : Advances in Information Retrieval. Volume 4956 of Lecture Notes in Computer Science . Springer Berlin Heidelberg ( 2008 ) 211 { 221

18. Mayr , P. , Scharnhorst , A. : Scientometrics and information retrieval: weak-links revitalized . Scientometrics ( 2014 ) 1{ 7

19. Gipp , B. : Citation-based Plagiarism Detection - Detecting Disguised and Crosslanguage Plagiarism using Citation Pattern Analysis . Springer Vieweg Research ( 2014 )