Academic rewards and honors are proven to correlate with h-index, although it was not the decision criterion for them till recent years. Once h-index becomes the rule-setting scientometric ranking measure in the zero-sum game for academic positions and research resources as suggested by its advocates, the rational behavior of competing academics is expected to converge towards its gametheoretic solution. This paper derives the game-theoretic solution, its evidence in scientometric data and discusses its consequences on the development of science. DBLP database of 07/2017 was used for mining. Additionally, the openly available scientometric datasets are introduced as a good alternative to commercial datasets of comparable size for public research in behavioral sciences.
sure competitiveness. The global expenditures on research and development lay over
1:6 trillions (1012) dollars per year, whereof USA, China, Japan, and Europe constitute
78% [
by early 2020’s. The number of scientists and engineers per inhabitant varies in strong correlation with the R&D budget up to 7h for Finland. Distributing percents of GDP over permilles of population for R&D creates an above-average income in this branch – a strong incentive for competition.
erage academic title holders are in contrast more challenging to rank for fair reward. The
academic title ‘doctor’, once introduced by catholic church in the middle-ages, made
its way through the centuries over witch hunting theologists by supervisor-student-links
into more modern and secular disciplines and became the necessary precondition. The
challenge of ranking academics is suggested to be solved by statistics of citations –
the modern-day scietometrics. The scientometric measure h-index is proven to
correlate with chances of winning the Nobel Prize, holding position at top universities and
being accepted for research fellowships [
The actual usage of h-index as a decision criterion is harshly criticized by
successful scientists from diverse disciplines [
behavior research. Its game-theoretical analysis is provided in Section 2. Section 3 lists
data-based evidences for the game-theoretical model from literature and own
experiments on Digital Bibliography & Library Project (DBLP) dataset from July 2017 [
provide a data-driven reconceptualization of the narrative for scientific process. Section
ods from game theory, behavioral economics and data science in order to understand human behavior is a direction, where such market leaders as Facebook, Microsoft and
ferences are organized for the intersection of experimental economics and machine
learning [
Equilibria of h-index ranking game
“I suggest that this index may provide a useful yardstick with which
to compare, in an unbiased way, different individuals competing for
the same resource when an important evaluation criterion is scientific
achievement.” J. E. Hirsch [
teraction. It consists of a number of participating players, their legal actions and a payoff function for every player. Let us assume for simplification that every player i 2 N produces one innovative publication pr;i 2 Pi per round r 2 N+, where Pi = N N+ Ai C.
duction speeds and qualities will be discussed in later sections. None of the players
has a publications before round 1 as the game starts. Ai Ã(N) is the set of possible
coauthors’ sets for a publication, which are subset of all players including the player
himself, i.e. 8O 2 Ai : i 2 O. The set of cited publications of participating players is
C Ã(P). C contains the cited publications from past rounds of participating players
only and hereby makes the definition of P recursive and non-circular. C includes
neither the publications written by researchers from outside nor concurrent publications
nor future publications. Citations of a publication from P by researchers from outside
are considered to be negligible or randomly and equally distributed. Every player i is
allowed to create publications with only him in the (co)authors’ set and no citations of
his competitors’ works. Since “there is no penalty to add authors to a paper” [
Only in the second round, players can get an h > 0. In the second round, it is not rational for any player to cite publications of his competitors from the first round, which do not include him as a coauthor. Surely, every player will cite outside researchers, which do not compete with him for a certain resource. Every player will achieve at least an h-index of 1 in the second round, since he will cite his publication from the first round. It is irrational to not cite own publications. If a player adds x randomly chosen players as coauthors to his publication in the first round, he will have one publication with x + 1 citations in the second round and still end up with h = 1. All x randomly chosen players will have two publications in the second round – first one x + 1 times cited and the second with only one citation. They also end up with h = 1 as well. Adding random coauthors in solo action does neither improve nor worsen ones position in the h-index ranking.
If a clique q 2 Ã(N) of x + 1 players agree to add them all as coauthors to their publications in the first round, they will achieve h = x + 1 in the second round and will rank higher than the rest N r q with h = 1. None from the clique q will improve his rank by defecting from the agreement in solo action, because this will only reduce the h-index of the whole clique by 1. Even if only one coauthor is excluded from x + 1 publications, then in the second round this will result in having x publication with x + 1 citations and one publication with x. The excluded player will not cite the publication, he was excluded from. Therefore any formed clique q is an equilibrium and the solution of the game is a set of multiple equilibria.
If every agreement for the round 1 is an equilibrium, then players will prefer to belong to a slim majority clique smq1; jsmq1j = (jNj 2) + 1. If a clique is less than majority, then the rest might form a single clique with a higher h. If a clique is much bigger than slim majority, then the members will be randomly ranked on a longer list of places on the top. The members of the slim majority clique smq1 from the first round will outperform the rest by at least 2 (jNj mod 2) 2 f1; 2g. The mechanism of making agreements is considered to depend on features of social networks and too extensive to be modeled game-theoretically in the this work. It will be referred to as collaborativeness.
majority clique smqr will add at least 1 to h more than being a member of the rest. Sets smq1 and smq2 do not need to be the same. If a player of extraordinary collaborativeness manages to be the only one player, who was a member of all slim majority cliques in all rounds, he will be the indisputable winner of h-index ranking.
This game-theoretical analysis reveals following major characteristics of rational behavior for a successful player, if the allocation of resources correlates with h-index ranking or is even based on it:
resource with you.
3
“We see here that in the real real world – when the chips are down,
the payoff is not five dollars but a successful career, and people have
time to understand the situation – the predictions of game theory fare
quite well.” R. J. Aumann [
fast dissemination of ideas and have less of extra long coauthors’ lists than conference and journal papers.
paper showed the same shape of a log-normal distribution in other disciplines too as a study on all papers indexed in the 1980-2000 annual volumes of the Science Citation
thors increased from 2 to 3 between 1980 and 1998. It converges towards the solution of h-index ranking game – the size of cliques grows. The number of citations for a paper grows close to linear with the number of its (co)authors – the slope of this relationship became steeper from 1980 to 1998.
Fig. 2 shows the development of cumulative contributions per author per year. Since the share of each coauthor in a paper is not recorded in the database, it is derived by simply dividing 1 (a paper) by the number of coauthors. If an author (co)authored more than one paper in a year, these shares are added. For instance, being a coauthor of 2 2coauthors papers will result a cumulative contribution of 1. 96% of DBLP records from 1980-2016 are used for this calculation. One can see in the graph that the approximately log-normal distribution of the recorded authors’ cumulative share drifts towards 0. The game-theoretically predicted prolongation of (co)authors’ list enables the incorporation of a growing number of scientists with far lower output into the scientific process.
A policy close to the game-theoretic solution for h-index implemented by one large scientific institute, the Collider Detector at Fermilab (CDF)3 since 1998. It enforces the addition of all its scientists and engineers as coauthors to all of its publications. Employees are added to the CDF authors’ list after one year of full-time work and removed after a year since the date they left. This list contains typically over 300 authors. A 3 www-cdf.fnal.gov s n o it a c li b u p f o r e b m u n k 0 0 5 k 0 0 4 k 0 0 3 k 0 0 2 k 0 0 1 3766094 publications from dblp.uni−trier.de, July, 2017
Papers median=3 lmean=2.67 Articles median=2 lmean=2.35 Informals median=3 lmean=2.48 Books median=1 lmean=1.07 Chapters median=2 lmean=2.21 Ref. Works median=1 lmean=1.39
Editorships median=1 lmean=1.43 1 2 3 4 5 6 10
20 number of (co)authors
eµ+s eµ+s #(Co)authors eµ+2s eµ+2s
Journal Articles Informal Publications
Book Chapters Editorships 84:13% Fig. 1. (Co)authors per paper in computer science. x-axis has a logarithmic scale. The means of the log-normal distributions are denoted as ‘lmean’. Key data for +s and +2s upper bounds is organized in the table. Corresponding types from graph and table are boldly indicated. 1980 1985 1990
1995 s r o h t u a 0 5 9 7 5 s r o h t u a 5 0 9 4 0 4 arithmethic mean median logarithmic mean Fig. 2. Drift of average cumulative share of annually (co)authored papers during last decades. In the top graph, normal distributions are fitted to logarithmicly binned histogram of chosen years - dashed grey line. Bottom graph shows the development of average activity of contributing (co)authors.
18786 consistently contributing authors from DBLP, 7/2017
18786 consistently contributing authors from DBLP, 7/2017
arithmethic mean
median
logorithmic mean
50
40
30
20
10
0
study conducted on a dataset of 189k publications [
The correlations between scientometric measures and graph measures of coauthors’
social network were calculated in a study on a dataset of 1809 authors from information
management and systems schools of 5 US universities [
h-index does not suffer from dilution of innovation into multiple papers, since the
growth of citations is strongly correlated with growth of papers per single research
project. This is showed in a study on a dataset of 96 BIF grant applicants [
anymore shortens – the quality of literature review is diminishing. Also the relationship
between impact factor and citations weakens since late 90s [
For the plot in Fig. 3, all the authors from 1980-2016 are taken, who (co)authored at least one paper in the 10 following years after their first paper. The curve shows that the average productivity of a scientist roughly doubles within ten years. This can also be interpreted as growth of dilution of invitations into multiple papers. The curve shows pattern saturation too – there might be limits to productivity or the competing individuals change their objectives. Fig.4 shows k-means centroids of productivity trajectories, where the k is set to 4. The productivity of top 1-2% of authors show a linear growth over 50% per years. Meanwhile, roughly the half of authors shows no productivity growth at all. The publication shares of the non-growing half add up to one paper a year. 4
h-index rank measures collaborativeness
The game-theoretic solution reveals that collaborative academics are preferred by the
h-index ranking, while academics writing single-author papers loose. If an academic
is several times more productive, he will still achieve the same result as those, who
achieve being added as coauthors same amount of times. An academic with a few high
quality publications like the Fields Medal nominee Grigori Perelman would also loose
in h-index ranking game. Hirsch identified as the major “short-coming” of original
hindex definition “its inability to discriminate between authors that publish alone or in
small cliques versus those authors whose papers have usually many coauthors” [
If the fast h-index growth naturally results as a by-product from certain type of
behavior, then this behavior is key to success in acquiring budget shares. According to the
critics [
cause climate change could be published in a peer-reviewed journal 71088 consistently contributing authors from DBLP, 7/2017 0 0 1 2 3 4 5 1 2 3 4 5 6 7 8 since their first paper in the subsequent 5,10,15, and 20 years. The four graphs show the k-means clustering results on maximumly available consistently contributing authors.
8 r a ye r e p sn 6 o it ca il b u p f eo 4 r a sh d e t a l u um 2 c 4 1 2 1 r a ye rep 10 sn o it ilca 8 b u p f o re 6 a sh d e t la 4 u m cu 2
A scientific community is obviously a collaborative network, which needs socially active members to exist. In the case of naturally grown h-index, it represents the degree of collaborativeness of an author inside of a scientific community. Scientific publications network is a mirror of the real social network. h-index ranking rewards its socially active members the most. Socially active members are also the rule-setter in a community and therefore will advocate the status quo of h-index ranking, since they profit from it the most. Is the core of scientific achievement to create a community around a certain topic, which urges to create innovations?
Even if the academics don’t develop own ideas for next publications, they will at
least actively adopt and develop ideas from non-scientific sources. They will become an
idea hungry community, which is eager to publish, to coauthor and to cite everything as
a consequence of rationality in h-index ranking game. The ground-truth is that the real
originator of an innovation is not always among the authors of the publication exposing
it and also not among those, who will be rewarded for this academic achievement. Like
in patent affairs [30, e.g.], where most patents are owned by non-individuals, there is a
division of labor between those, who originate ideas, and those, who promote them into
real life. Ideas might appear in different heads simultaneously – it is an honorable
scientific achievement to effectively place those ideas into the scientific community using
social skills. On the other hand, tools should provided to better honor scientific beyond
composing publications. For instance, data citation [
Scientometric datasets for behavior sciences
The bottleneck of public research in behavior mining is the limited access to large
datasets, which are mostly held by for-profit companies. Preparation and release of
large, authentic and recent datasets tend to contradict the commercial interest. Even if
several studies on commercial datasets are put into public domain, their datasets might
not be available for reproducibility of results. While the sizes of datasets from non-profit
social networks are about 100k participants [
sure – the h-index ranking established a reward system, which prefers socially active academics and therefore furthers the labor division in science. The evidences for the convergence towards the game-theoretic have been found in the data of DBLP database and in results of related work.
under grant 17-11-01294 and performed at National Research University Higher School of Economics, Russia. We thank the people behind DBLP for providing free full access to their database.