Reverse Bibliography Mikhail M. Gorbunov-Posadov Keldysh Institute of Applied Mathematics, Miusskaya sq., 4 Moscow, 125047, Russia Abstract A reverse bibliography, i.e. a list of referring publications – is a dynamically compiled list of works published after the publication in question and contained a reference to it. Leading publishers of scientific journals, as a rule, post lists of referring publications on the web pages of their articles. In Russia, the placement of lists of citing publications has not yet become the norm, but it is gradually becoming widespread. The reverse bibliography is extremely interesting to the reader since it allows you to find out in which direction the research presented in the original edition is developing. A traditional bibliography, i.e. an ordinary bibliographic list of references, can successfully exist both in a printed publication and online. In contrast, the list of referring publications is essentially an exclusively online, dynamic structure formed “on the fly” on the screen of an online reader. The list of referring publications always brings the reader a fresh current state of accumulated bibliographic data. Publications appearing on the Internet and indexed in bibliographic databases containing a link to the scientific work in question in their bibliography are reflected here almost immediately. The implementation of the list of referring publications in a number of characteristic projects is considered. The terminology used is discussed. Keywords 1 List of referring publications, citing publications, prospective bibliography 1. Introduction A traditional attribute of any scientific publication is a bibliography demonstrating which works this edition is based on. Thus, the bibliography carries an obvious ethical burden. The bibliography is interesting not only to the historian of science but also, of course, to the ordinary reader, allowing him to delve into the study of the subject, find out important circumstances, detailed information about for one reason or another was not included directly in the text of this publication. However, the list of referencing publications is of much greater interest to the reader, i.e., a list of works that followed this publication and contain a reference to it. The referencing publications allow you to determine in which direction the research presented in the source publication is developing. In other words, if an ordinary (retrospective) bibliographic list looks, as it were, back into the past, then a prospective list, i.e., the list of referencing publications, looks forward to the future, covering the latest work in the field of science under consideration. The mechanisms for forming the usual bibliographic list and the list of referencing publications are different, although they are closely related. An ordinary, traditional bibliographic list is compiled by the author, and he does it now in almost the same way as they did before him hundreds of years ago. Conversely, publication referencing lists generate bibliographic databases using modern information technology. The publication of a scientific work currently implies the posting of information about it in open access on the Internet, as well as the transfer of meta-attributes of the work to bibliographic databases. In this case, it is essential that there is a bibliographic list of the work among the posted and transferred SSI-2021: Scientific Services & Internet, September 20–23, 2021, Moscow (online) EMAIL: gorbunov@keldysh.ru (M.M. Gorbunov-Posadov) ORCID: 0000-0002-7044-8287 (M.M. Gorbunov-Posadov) © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) data. The bibliographic records that make up the list fall into databases maintained by Google Scholar and into bibliographic databases. Based on such databases, not only are numerous bibliometric estimates of interest primarily of interest to bureaucrats from science are calculated, but also lists of referencing publications are generated, which are extremely useful for the general reader. If the bibliography, i.e., a usual bibliographic list, can successfully exist both in print and online, the list of referencing publications is essentially an exclusively online structure, dynamic, formed on the fly on the screen of an online reader. The list of referencing publications always brings the reader a fresh current state of the accumulating bibliographic data. It almost immediately reflects publications appearing on the Internet and indexed in bibliographic databases, containing a link to the scientific work in question in their bibliography. The terminology in this area has not yet been established. Along with the term “list of referencing publications”, there are also “reverse bibliography” and “list of citing publications”. However, both of these phrases have certain weaknesses. The “reverse” list is obviously aptly contrasted with the “direct” one – the usual bibliography but in the word “reverse”, in this case sometimes one can hear the directly opposite connotation: “turned back, into the past”. A “citing publication” can be perceived too literally – as a publication that includes quotations from a specified source, while such explicit mechanical copying of parts of the text is relatively rare in scientific works. Nevertheless, the terms “cited-by” and “citing” are often used to designate the list of interest to us. The traditional bibliography and referenced publications are more precisely and clearly contrasted when using the corresponding terms “retrospective” and “prospective” (bibliographic list). Unfortunately, in the Russian-language scientific vocabulary, the word “prospective”, that is, directed to the future, referring to moving forward, is rarely found, and in the context of bibliography, it is simply not used yet. There is no doubt that the online reader is interested in the apparatus of lists of referencing publications. Most of the leading publishers of scientific journals already post lists of published articles to their web pages in one form or another. In Russia, reverse bibliographies are still rare. In this paper, we consider well-known lists of referencing publications in some large projects and the implementation of such lists in the online version of the Keldysh Institute Preprints. 2. Google Scholar The most widely known and popular implementation of the list of referencing publications offered by Google Scholar. Google Scholar robots systematically surf the Internet in search of scientific papers. Each found scientific publication is analyzed in detail; in particular, the bibliographic list is isolated and analyzed in it. The parsed bibliographic records are saved in Google Scholar, and then the database built from them is used, in particular, to build lists of referencing publications. To get a list of referencing publications, in the Google Scholar search field, you must either specify the address (URL) of the publication of interest to us, or write in this field the authors and the title of the publication. In both cases, Google Scholar produces a result similar to that shown in Figure 1. Now, to see the list of publications referring to the given one, you need to click on the “Cited by 4” field located in the penultimate line. The first two records of the result of this call are shown in Figure 2. The reader usually wants to see the reverse bibliography when working with an online publication. Therefore, developers of the modern online publishing environment are striving to reduce the number of manipulations required in that environment to retrieve a list of referencing publications. The most natural and convenient solution, in this case, is the placement of a previously formed hyperlink to the generated list directly in the publication card. Fortunately, Google Scholar allows you to generate such a hyperlink statically. Among Russian online repositories of scientific publications, this useful opportunity was used, in particular, by the MathNet.ru portal, the eLibrary.ru library, and Keldysh Institute Preprints (Figure 3). 157 Figure 1: Search result in scholar.google.com for the text “Gorbunov-Posadov M.M. Scientific Publication in Russia: Why and How?” Figure 2: Search result in scholar.google.com for the text “Gorbunov-Posadov M.M. Scientific Publication in Russia: Why and How?” 158 Figure 3: A hyperlink leading to the formation of a list of referencing publications on the Keldysh Institute Preprints website 3. eLibrary.ru In addition to the above-mentioned hyperlink to the reverse bibliography generated by Google Scholar, posted on the publication page, the scientific electronic library eLibrary also forms its own reverse bibliography based on the data stored in this library. The Russian scientist is especially interested in this reverse bibliography since the eLibrary funds contain the richest collection of information about Russian scientific publications. It is not difficult to see the eLibrary-generated reverse bibliography (Figure 4): first, you need to use the search to get to the page of the publication of interest generated by the eLibrary, and then click on the link “List of articles in the eLibrary citing this” link located on the right of this page. Figure 4: List of referencing publications in elibrary.ru At the same time, as already mentioned, the desire to see the reverse bibliography usually arises from the online reader at the moment of reading the online publication. If, at this moment, the reverse bibliography is “at hand,” i.e., the hyperlink to the list of referencing publications is placed directly on the publication's web page, then the reader may well become interested in this hyperlink and follow it. If to access the list, it is required to perform very laborious manipulations – to log in to the bibliographic system, find the publication of interest in it, etc. – then, in this case, only a rare reader will be ready to spend their time and energy here. 159 Here eLibrary is hopelessly losing out to Google Scholar. While it is easy to place a direct hyperlink to the Google Scholar-generated reverse bibliography on the publication's web page, building a similar link for the eLibrary-generated reverse bibliography fails. The point is that the reverse bibliography in eLibrary does not have a constant (static) address (URL). The hopes of online libraries of scientific publications were at one time associated with a recently built API for accessing individual data stored in the eLibrary, but, unfortunately, access to the list of referencing publications was not included in this API. 4. Scopus Unlike eLibrary, the largest international bibliographic databases still, as a rule, one way or another provide an opportunity to place a reverse bibliography or a hyperlink to it directly on the publication's web page. In Figure 5 presents a list of referencing publications for article [1] from the journal "Information Processing & Management", compiled on the basis of data from the Scopus bibliographic database and posted on the web page of this article. Figure 5: List of referencing publications in elibrary.ru In Figure 5, the third position appears somewhat unexpectedly: it mentions an article taken not from the principal Scopus repository but from the arXiv.org preprint server. The fact is that in 2021, when compiling a list of referencing publications, Scopus began [2] to include, along with its basic data, articles posted on the popular preprint servers arXiv, bioRxiv, ChemRxiv, and medRxiv. Scopus rightly notes that the most recent publications, which are now available primarily on these servers, are particularly interesting to the reader. Increased interest in preprint servers has arisen in connection with the massive placement of publications there on the Covid-19 pandemic. For such publications, the key role was played by the speed of their appearance in the public online, and here, as you know, preprints simply have no equal. During the pandemic, about half of all publications devoted to the coronavirus were published in the form of a preprint. 5. Crossref Crossref's reverse bibliography based on publication data receiving DOIs is among the most popular and authoritative. In particular, Springer publishes prospective bibliographic lists on the web pages of its publications (Figure 6). 160 Figure 6: List of Crossref referencing publications on Springer website On the Crossref website, the list of referencing publications can only be obtained by calling the API, i.e. here you need to write the program code. This code is implemented, in particular, on the Keldysh Institute Preprints website. The result is shown in Figure 7. 161 Figure 7: List of Crossref referencing publications on the Keldysh Institute Preprints website However, an ordinary reader who has the DOI of the publication that interested him can get the corresponding reverse bibliography without programming. To do this, he will have to refer to third- party sites, for example, to [3]. There, to get a list of referring publications, it is enough to enter the existing DOI value in the corresponding search field. 6. Other dynamic bibliographic lists Along with the reverse bibliography discussed above, there are other dynamically compiled bibliographic lists used in bibliographic systems. The most popular of them are lists of “consonant”, “related” publications, which collect publications related to the topic of the work in question. Leading the way today is the Google Scholar “related” list. For example, in the bibliographic system mathnet.ru this list can be obtained by clicking on the link “Related articles on Google Scholar”. The number of such works offered by Google Scholar is incredibly large, but the quality of their selection is still poor. In particular, Google Scholar apparently uses the names of the authors of the publication in question as one of the search attributes, so a noticeable part of the results is made up of works that are not related to the topic under consideration, but written by the authors' namesakes. Elibrary.ru also collects a list of related publications. This list accompanies each publication and is linked to by a link titled “Find related publications”. And here the reader is offered tens of thousands 162 of answers, the quality of these answers is also not always pleasing, although it surpasses the quality of the results of Google Scholar. Lists of related publications are widely distributed and use a variety of mechanisms. For example, interesting results are obtained in the project Connected papers [4], where, when searching for related publications, the proximity of publications is understood not as the presence of direct bibliographic links between articles, but the proximity of their bibliographic lists. And yet, the reader is usually most interested in the main subject of this article - the reverse bibliography: the mechanism of its construction is clear to the reader, and at the same time, the results are of undoubted benefit. Note that dynamic bibliographic lists with obvious construction algorithms are currently the most in-demand. Among them, along with the reverse bibliography, it is necessary to note the ORCID apparatus [5], which allows the reader, in one click to see the list of publications of the author he is interested in. 7. Conclusion The reverse bibliography, generally speaking, lies in wait for dangers similar to those who played a fatal role in the fate of the interesting project COinS [6]. An unscrupulous author can significantly spoil the attitude towards this dynamic construction by including in the bibliography of his work a link to an outstanding scientific publication not in essence, but only in the expectation that his work will be noticed due to its appearing in the widely sought-after list of those citing this outstanding publication. Fortunately, such a development of events now seems unlikely. The reverse bibliography is slowly becoming a must-have in today's online publishing environment. We can only be glad for the online reader, who has acquired such a useful tool for working with scientific literature. 8. References [1] Y. Gao, Q. Wu, L. Zhu, Merging the citations received by arXiv-deposited e-prints and their corresponding published journal articles: Problems and perspectives, Information Processing & Management 57 5 (2020). https://doi.org/10.1016/j.ipm.2020.102267. [2] R. McCullough. Preprints are now in Scopus!, blog.scopus. URL: https://blog.scopus.com/posts/preprints-are-now-in-scopus. [3] Open Ukrainian Citation Index (OUCI). URL: https://ouci.dntb.gov.ua. [4] Connected papers. URL: https://www.connectedpapers.com. [5] ORCID connecting research and researchers. URL: https://orcid.org. [6] COinS (ContextObjects in Spans). URL: https://en.wikipedia.org/wiki/COinS. 163