Abstract

This paper presents a study of bibliographical references cited in articles published by MathNet.Ru journals. Based on data obtained from mathematical portal Math-Net.Ru, we built a journals citation graph, with its vertices denoting journals, and edges representing bibliographical references (citations) between papers published in these journals. To increase the reliability of the constructed graph, we chose a 2010-2021 citation time interval, when distribution of citing papers (papers that have cited other works) had stabilized at 3500-4500 citations per year. The structure of citation ageing is investigated; it is shown that the half-life of these citations is 8 years. So, the publication date of cited papers (papers that have been cited by other works) was limited to the year 2002. The constructed citation graph was found to have a small diameter and high density, indicating that there is a high level of research collaboration in Math-Net.Ru. It is shown that there is no Matthew effect as a pronounced advantage in the citations of leading journals in relation to less well-known ones. The adequacy of the Math-Net.Ru journal citation graph as a scientific collaboration model is confirmed by comparing the ranking of journals included the citation graph with their Science Index ranking in scientific electronic library eLIBRARY.RU. The two rankings were found to have a direct moderate relationship between themselves. A number of substantive conclusions are drawn from analysis of the citation graph.

Keywords

• 1 bibliographic reference
• journal citation networks
• citation ageing
• Matthew index
• MathNet
• Ru

1. Introduction

The idea of determining the significance of scientific journals by measuring their citation rate emerged as early as 1927 [ 1 ], when terms like “centrality in graph theory and network analysis” were not used. Note that a graph is more commonly referred to as a mathematical object (as a set of vertices and a set of pairs of vertices to which the theoretical apparatus of graph theory is applicable), and a network is the same structures with some meaningful content (as journal citation networks or coauthorship networks). In the context of this paper, these two notions are practically identical. In [ 2 ], it is stated that “... the study of citation networks as a window to science is a time-honored tradition”, and a good overview of publications in English is given. The terms “graph” and “network” which are used in [ 2 ] are also understood in a similar sense.

In studies devoted to analysis of scientific citation, large and well-known Internet search platforms, such as Web of Science (WoS) and Scopus, with tools such as Journal Citation Reports, and (no less large) specialized resources such as PubMed (https://pubmed.ncbi.nlm.nih.gov), are the basis for obtaining raw data. There are few Russian-language journals and publications on these resources. For example, the number of Russian-language publications in WoS is less than 0.5% of the total number.

In Russian studies devoted to journal citation networks, much attention is paid to the economic direction [ 3–5 ]. It seems that this is largely due to the use of the RePEc (Research Papers in Economics, http://repec.org) database, a significant part of which is freely accessible. But even here, Russian-language journals occupy an insignificant place.

The small number of publications related to studies on bibliographic citations in Russia may be due to certain difficulties in obtaining information from eLIBRARY.RU databases (https://elibrary.ru) necessary for analysis. In contrast to these cases, we have direct access to Math-Net.Ru databases (http://www.mathnet.ru), an all-Russian mathematical portal, we can make flexible SQL queries, selectively choosing information about the authors, publications, paper references and citations.

Math-Net.Ru is a well-known web resource created at the Steklov Mathematical Institute in 2006, containing a rich collection of full-text archives of leading Russian mathematical journals. At the time of the calculations discussed in this work (middle of March 2021), over 140 journals (periodicals), 135,000 authors, and more than 300,000 publications were registered on the portal.

The key component of the system is the “Journals” section, which has brought together leading Russian journals into a single information system. The section presents current issues and historical archives of periodicals and continuing editions. It contains full-text publications of which crossreferences to authors and places of their work are organized. Authors and organizations are assigned unique numeric codes, and the list of authors and organizations of all publications forms the basis of the “Persons” and “Organizations” databases. Information about authors, includes academic degrees and titles, editorial information, as well as links to their personal pages in such international bibliographic systems, as ResearcherID, Scopus, Google Scholar, ResearchGate, etc. This makes it possible to accurately identify personalities, provide the maximum possible information about scientific activities, and at the same time avoid duplication in database elements.

In addition to scientific publications, the system also indexes speeches and presentations made by Russian scientists at various scientific events (reports at conferences, seminars, scientific and popular scientific lectures, etc.), which forms the basis of the “Conferences” and “Seminars” databases. For many speeches, video recordings are attached, which form the “Video Library” section of the system. Connections are established between scientific publications of the authors and their reports at conferences.

Math-Net.Ru is a scientific space uniting several databases into one, and organizing between them a set of cross-references. Most of the information is presented both in Russian and in English, including titles, keywords and abstracts of articles, information about personalities and organizations, titles and abstracts of papers.

According to eLIBRARY.RU classification, over 85% of Math-Net.Ru journals are related to the topic “mathematics”, and the rest belong to “computer science”, “cybernetics” and “automatics and computer science” (several journals belong to several of these subjects at once). According to eLIBRARY.RU for 2018, of the first fifty journals with the highest Science Index for the “mathematics” topic, 46 journals belong to Math-Net.Ru.

One of the goals of Math-Net.Ru creators was to index and digitize not only the publications themselves, but also the lists of references. The references are stored in a database in a structured form [ 6 ]. Lists of references of all publications are combined into one database table in which author's information, title, year, volume, and pages of the cited publication are stored in separate columns.

In a somewhat simplified way, the citation record looks like this:

<citing journal><citing paper><cited journal><cited paper>.

Data about papers, in turn, have fields that allow you to accurately identify them: list of authors, title, year, pages, etc. Each individual reference corresponds to one record in the table. This approach facilitates automatic hyperlinking to bibliometric databases, solves the problem of finding backlinks, and allows to automatically export links in different formats, like PDF, XML, HTML. Among the hyperlinks from the bibliography items, there are also links to papers indexed in the Math-Net.Ru publication database. In this way, a link is made between citing and cited papers. Lists of citations are available on the page of each of the Math-Net.Ru publications. A list of the most cited papers and the most cited authors of each journal is formed on the basis of the citation data of individual papers.

A number of editions included in Math-Net.Ru are translations. For them, the Russian and English versions are published separately, which have different numbers of volumes and pages, often the issue number may not coincide. Formally the Russian and English versions of the same journal are different editions and have different ISSNs.

Authors who cite publications of Russian journals may indicate one of the following variants of references in the reference list of their publication:

- a reference to the Russian version of the journal with the name in Russian (or transliteration in publications in English), indicating the year, volume, issue, and number of pages of the Russian version of the paper;

- a reference to the English version of the journal with the title in English, indicating the year, volume, issue, and number of pages of the English version of the paper;

- references to both versions of the cited publication.

The English-language versions of some Russian journals are indexed in international bibliographic databases, like Web of Science, Scopus, etc., and when calculating the number of citations, these databases may omit references to the Russian-language version of the publication or not include them in their citation index. Besides, international databases do not take into account references from Russian-language journals due to the fact that these journals may not be included in them.

In Math-Net.Ru, citations of Russian and English versions of the same publication are considered as citations of the same research work, as the same scientific idea is cited, no matter what language it is written in. The original Russian and the translated English versions of the same paper are indexed together, considered as one record in the database of publications. The citation lists of the translated publications can include the citations of both Russian and English versions of the journal.

2. Determining the time interval

3. Citation ageing

For example, the value at point 0, which is 5,060, is calculated as the sum of all links whose age is zero. The age of a link is between 0 and 152. The maximum number of citations (14,924) falls on Papers published in the previous (with respect to the citing publication) year, i.e., with a citation age of 1, are cited the most – 14,924 times. We have almost the same number of citations for papers with a citation age of 2, and then comes a rapid decline in the number of citations. The half-life of citations for the integral age structure is 8 years, i.e., 81,000 citations out of 162,000 are no more than 8 years old.

In the same figure, the dashed line shows a graph of the age structure of outgoing citations made in publications in 2018. The value at point 0 (=498) is calculated as the number of links made in 2018 pointing to articles published in the same year. Here, the maximum is shifted one year to the right, with 1,753 citations having an age of 2 years. The half-life of citations made in 2018 is slightly less, 7.75 years. For comparison, a similar graph for 2010 looks almost the same, except that the maximum is reached at point 1, and the half-life is 9 years.

4. Citation graph properties

Taking into account the aging structure and the half-life of citations, we built a citation graph for Math-Net.Ru journals. Citations from 2010 to 2021 were used, limiting the publication year of cited papers to no earlier than 2002. In fact, we are using a computed eight-year citation half-life, and therefore links pointing to papers published prior to 2002 are not considered in our model.

This limitation brings the total number of journals under consideration down to 120, and the number of citations to 99,000, and almost 44,000 of them are self-citations. Let us remove from the set 17 journals that have only incoming or only outgoing links, and denote the generated set of journals by J103.

Using the J103 set, we construct a graph G(V, E, W), the citation graph of journals, where: V is the set of vertices (103 vertices corresponding to journals),

E is the set of edges (3873 edges linking pairs of vertices i and j, if papers in journal i have at least one reference to papers in journal j),

W is the set of edge weights (weight w(i,j) of edge e(i,j), which is equal to the number of links pointing from all papers in journal i to papers in journal j).

The sum of all weights W is the number of all citations of journals from J103, which is equal to 97,109. Of these, 43,910 are self-cites. Most of the journals have more self-citations than the most cited journal.

Let us specify the main characteristics of graph G(V, E, W). By construction, it is obvious that it is a strongly connected graph: one can pass from any vertex to any other vertex along a path with a finite number of edges. Diameter of graph (maximal number of edges in such path) equals 4. The graph density (the ratio of the number of edges to the maximal possible number of edges) is large enough and is 0.369.

The modularity structure of graph G(V,E,W) is interesting. Recall that graphs with high modularity have strong connections between vertices within modules but weak connections between vertices in different modules. If we consider edges without considering their weights, modularity is practically equal to zero. This means that the edges in the graph are distributed evenly enough.

Taking into account the edge weights, we get a slightly different picture. The modularity coefficient in this case is Q=0.356. Here, we use the definition of modularity measure Q from [ 9 ]. The value of Q falls within the interval [ –1, 1 ], and a partition is considered good if Q>0.7.

The obtained Q does not show that graph G(V,E,W) has a strong modularity, but it shows some tendencies. The graph is divided into 5 modules, each of which can be meaningfully interpreted by research areas: fundamental mathematics, mathematical modeling, experimental and theoretical physics, discrete mathematics and applied mathematics and computer science. Let us list, for example, the journals of the module – “Experimental and Theoretical Physics”: Computer Optics, International Research Journal, Journal of Experimental and Theoretical Physics Letters, Nonlinear Dynamics, Quantum Electronics, Regular and Chaotic Dynamics, High Temperature and Physics– Uspekhi.

Since the significance of scientific journals is characterized by ranking them on the basis of indicators constructed based on citation data, the citation graph can also be used for this purpose. The significance of vertices in a directed graph can be determined in various ways, and each of them requires meaningful interpretation. The Page Rank (PR) score provides an opportunity to compare the relative “significance” of the graph vertices by analogy with the significance of web pages on the Web [ 10 ]. A meaningful interpretation of the significance of vertices by PR in G(V,E,W) can be as follows: if you imagine a certain “surfing scientist” moving from one journal to another via article links an infinite number of times, he is most likely to visit the journal with the highest PR value.

For G(V,E,W), we calculated PR values for each vertex taking into account loops and weights. PR values ordered in descending order for the first five and the last five vertices are given in Table 1.

5. The Matthew effect

One of the journal reputation indicators is the so-called “Matthew effect”, introduced by Merton [ 11 ], which states that scientists who have previously been successful are more likely to succeed again, producing increasing distinction. According to the Gospel quote “... for to everyone who has will more be given, and he will have abundance; but from him who has not, even what he has will be taken away” (Matthew 25:29). With respect to scientific citation, the effect is interpreted as the citation advantage of established scholars over their lesser-known colleagues.

The

Matthew Index for identifying citation bias related to nationality of authors was first introduced in [ 12 ]. In a similar fashion, let us define MI for journals as follows: = ( − )/ where obsi is the real increase in the number of citations, and expi is the expected increase in the number of citations of papers published in journal i for the time period [t0, t1]. Here, t0 and t1 are the initial and final moments of time, which determine the interval for calculating the Matthew index. As a rule, t0 and t1 are the initial and final years of the research interval.

Let us describe how MIi is calculated as applied to our case. For each journal in the database, we determine the total number of articles published in it and the total number of their citations from the very beginning of filling the database to year t0, which allows us to calculate the average citation rate of a paper in the journal for year t0. The database then determines the increase in the number of papers the expi for the ith journal. The real increase in the number of citations obsi for time period [t0, t1] is determined from the database.

If MIi is greater than 0, then it is concluded that journal i receives more citations than can be assumed for the period [t0,t1], and vice versa. The peculiarity of this approach, used in our case, is that there is no need to determine established and less known journals through some kind of ranking, which in turn must be justified.

Here is one typical case where t0=2018 and t1=2020 were taken as an example. Some of the results are shown in Table 2.

expi 1845,2 obsi 548 15907,7

23512 15546,6

17109 1838,2 1032,9 1001,6 897,4 281,0 2220,4 1267,9 2205 1270 673 880 221 1887 1973

MIi -0,70 0,48 0,10 0,20 0,23 - 0,33 - 0,02 - 0,21 - 0,15 0,56

Here, cit18, p18 and cit20, p20 are the number of citations of the journal and the number of papers published in it for 2002–2018 and 2002–2020, respectively, while mid18 is the average number of paper citations in 2018.

The journals are sorted in descending order of mid18, i.e., Vestnik of St. Petersburg State University. Applied Mathematics. Computer Science. Control Processes for 2018 is the richest by this indicator (Quantum Electronics is the richest by the total number of citations).

As can be seen from the table, for the first ten “rich” journals in terms of average number of citations, five have a positive Matthew index, and five have a negative value.

The same picture is more or less observed overall, with the difference that the number of positive values is about 80%. Therefore, it can be said that there is no pronounced advantage in citing established journals over lesser-known ones.

6. Comparison of journal rankings in Math-Net.Ru, eLIBRARY.RU and Web of Science

Math-Net.Ru journal rankings in terms of PR were compared with those of eLIBRARY.RU and Web of Science. For this purpose, eLIBRARY.RU took this indicator as the journal's position in the Science Index (SI) ranking [ 13 ]. To calculate the SI, journals are assigned to one of 10 subject areas. The “Mathematics, computer and information sciences” area, which included 58 journals from J103, was taken for the study. Table 3 shows the first ten and the last five journals in the SI ranking; the SI column presents their SI ranks, the PR column shows the Page Rank values for these same journals, and the #PR column shows their PR ranks. The only thing that looks unexpected is the high SI ranking of Informatics and Automation.

Spearman's rank coefficient was used as a mathematical tool. The method is based on the principle of numbering the values of a statistical series. Each element of the population is assigned an ordinal number in a row, which will be ordered by the level of the attribute (for example, in descending order). Thus, the row of attribute values is ranked, and the number of each element becomes its rank. Let n be the number of observed values of a feature, xi be the rank of the ith element of the first statistical series, yi be the rank of the ith element of the second statistical series, and di be the difference between the ranks. Then the Spearman rank coefficient will be ρ = 1 − 6 ∑ =1 2 .

( 2 − 1) The relationship is considered strong if |ρ|≥0.7, medium strength if 0.5<|ρ|≤0.69.

In our case, xi was taken as the journals' positions in the SI ranking, while yi represented the positions in the PR ranking. The resulting value ρ=0.66 with a critical value of 0.01 indicates there is a moderate direct relationship between the two rankings.

WoS includes only 22 journals from J103. A sample of their impact factors [ 14 ] for 2019 was taken from the WoS and compared with a sample of their PR values. Table 4 shows the first ten and last five journals in the WoS ranking; the #WoS column presents their WoS ranks, the IF WOS column shows their WoS impact factors, the PR column shows the Page Rank values for these same journals, and the #PR column shows their PR ranks. The Spearman rank coefficient obtained in this case is –0.29 and shows no correlation.

Half of the physics-related journals stand out noticeably in the first ten. But removing the physics journals from this list also leaves the correlation almost the same.

7. Conclusion

One of Russian official documents [ 15 ] “... determines the method for calculating the value of the quality indicator characterizing the publication performance of scientific organizations” and was developed “... in order to provide methodological support for the formation of government tasks”.

According to our model, the journal “...without a quartile”, which is included in WoS, has a “quality factor” approximately eight times higher than the journal “... from the Higher Attestation Commission list”. According to our model, it turns out that the high rating obtained as a result of evaluation by Russian colleagues means practically nothing in comparison to assessments by foreign colleagues if the journal is not included in WoS. And even if it is included in WoS, its assessment does not depend on the opinion of the Russian mathematical community, and this is wrong. 8. References

References

[1] P. L. K. Gross , E. M. Gross , College Libraries and Chemical Education . Science 66 1713 ( 1927 ) 385 - 389 .

[2] M. Kas , K.M. Carley , L. R. Carley , Trends in science networks: understanding structures and statistics of scientific networks , Social Network Analysis and Mining . 2 ( 2012 ) 169 - 187 .

[3] F. T. Aleskerov , D. N. Badgaeva , V. V. Pislyakov , I. A. Sterligov , S. V. Shvydun , Znachimost' osnovnyh rossiiskih i mejdunarudnyh ekonomicheskih jurnalov: setevoi analiz , Journal of the New Economic Association . 2 ( 2016 ) 193 - 205 (in Russian).

[4] S. V. Bredikhin , V. M. Lyapunov , N. G. Shcherbakova, The structure of citation network of scientific journals , Problems of Informatics. 2 35 ( 2017 ) 38 - 52 .

[5] S. V. Bredikhin , V. M. Lyapunov , N. G. Shcherbakova , Spectral'nyi analiz seti citirovanija nauchnyh jurnalov , Problems of Informatics. 2 39 ( 2018 ) 24 - 40 (in Russian).

[6] D. Chebukov , A. Izaak , O. Misyurina , Yu. Pupyrev, A . Zhizhchenko, Math-Net. Ru as a digital archive of the Russian mathematical knowledge from the XIX century to today , Lecture Notes in Comput. Sci . 7961 ( 2013 ) 344 - 348 .

[7] V. Pislyakov , Self-Citation and its Impact on Research Evaluation: Literature Review. Part I, Preprint . URL: https://arxiv.org/ftp/arxiv/papers/2109/2109.09192.pdf.

[8] P. Heneberg , From Excessive Journal Self-Cites to Citation Stacking: Analysis of Journal SelfCitation Kinetics in Search for Journals, Which Boost Their Scientometric Indicators , PLoS ONE 11 4 ( 2016 ), e0153730 . URL: https://doi.org/10.1371/journal.pone. 0153730 .

[9] M. E. Newman , M. Girvan , Finding and evaluating community structure in networks , Physical Review E. 69 2 ( 2004 ), P 026113.

[10] S. Brin , L. Page , The Anatomy of a Large-Scale Hypertextual Web Search Engine , Computer Networks and ISDN Systems . 30 ( 1998 ) 107 - 117 .

[11] R. K. Merton , The Matthew Effect in Science. Science 159.3810 ( 1968 ) 56 - 63 .

[12] M. Bonitz , E. Bruckner , A. Scharnhorst , The Matthew Index - Concentration Patterns and Matthew Core Journals. Scientometrics 44 3 ( 1999 ) 361 - 378 .

[13] Metodika rascheta integral'nogo pokazatelja nauchnogo jurnala v reitinge Science Index . From URL: https://elibrary.ru/help_title_rating.asp (in Russian).

[14] The Clarivate Analytics Impact Factor . URL: https://clarivate.com/webofsciencegroup/essays/impact-factor.

[15] Metodika rascheta kachestvennogo pokazatelja gosudarstvennogo zadanija “Kompleksnyi ball publikacionnoi rezul'tativnosti” dlja nauchnyh organizacii, podvedomstvennyh Ministerstvu nauki I vys'shego obrazovanija Rossiiskoi Federacii, na 2020 god . URL: https://old.minobrnauki.gov.ru/common/upload/library/2020/09/main/Metodika_novaya. pdf (in Russian).