A method for the automation of the scientific peer review workflow in a mathematical journal has been developed. For this, a database of experts in various areas of mathematics has been created in the journal's information system. For each expert, using automatic search methods for citation databases, a profile has been generated that includes various classification codes, in particular, Mathematics Subject Classification 2010 (MSC 2010). A system of coefficients has been developed that takes into account the professional interests of experts. Also, indicators of their workload in the review process are taken into account. The first step in forming recommendations for the examination of an article is to determine the scope of the research conducted in it. For this, the codes of mathematical subject classification presented in this article are distinguished. Further, according to these codes, a list of possible experts is formed. The developed method is implemented as plugins of Open Journal Systems (OJS) and tested in the information system of the scientific journal Lobachevskii Journal of Mathematics (https://ljm.kpfu.ru).
Currently, there has been an increase in the number of scientific publications in
professional journals, and the growth rate is more than 5% per year (see, for example,
[
Today it is universally recognized that after acquiring new scientific knowledge,
the time needed to bring it to an interested audience should be as short as possible.
Therefore, the task of ensuring the life cycle of a scientific document is currently
being solved at new organizational and information levels (see, for example, [
This article discusses the issues of expanding the functionality of the Open Journal Systems (OJS) platform in order to automate the work processes of the scientific journal editorial staff. A method for the automated selection of reviewers, the distribution of notifications and the control of the review period is proposed. 2
Currently, almost all scientific journals use software tools in their work. The main
features of the use of ICT in the process of publishing scientific journals were studied
in [
OJS is licensed under the GNU / GPL. Today this information system is used by more than 9 thousand scientific journals around the world, in which there are at least 10 publications per year (https://pkp.sfu.ca/ojs/ojs-usage/ojs-stats/). OJS platform is used by many Russian journals (see, for example, https://neicon.ru).
OJS supports a wide range of business models for periodicals, as well as settings
for providing access from fully open access to resources to brief annotations and
commercial subscriptions. This allows you to use this system as a single platform for
managing a complex of electronic publications (for example, in a research or
educational organization) [
OJS System is configured as a cloud-based software platform that can be deployed and managed locally. All workflow processes are configured directly by the editors of each particular edition. OJS provides special tools for reading and viewing publications in PDF and HTML formats. A number of functions are available for working with bibliography and metadata. There are a number of other features.
OJS has a modular architecture, which allows you to develop your own classes and
modules if necessary. The system has an MVC structure (Model-View-Controller),
respectively, data storage, user interfaces and control functions are divided into
different levels of interaction. This architecture provides fault tolerance, performance,
flexibility and scaling of the entire system [
OJS uses a role model of users with various access rights and a multi-stage process of publishing resources. This process supports all stages of the life cycle of a scientific article submitted for publication, from its initial download in the author's version to the publication of the final version on the Internet and the formation of relevant pointers and links. The functionality of the system allows the interaction of participants in the editorial process online. OJS interface modules are implemented as sets of Smarty templates (http://smarty.php.net), which allows you to change the user interfaces of the system. 3
An important distinguishing feature of the work of the editorial board of a scientific
journal is the organization of independent peer-reviewing with the involvement of
specialists in subject areas determined by the subject of the journal. The process of
scientific review is, as a rule, the most time-consuming when considering scientific
materials in the journal [
The scientific review process support service allows you to select reviewers for
each article submitted to the journal and automatically control the review process,
including sending out notifications and reminders. One of the options for such a
system of automation of the review process is implemented in the Lobachevskii Journal
of Mathematics [
year in the period 1998–2015 up to 12 issues currently. In addition, since 2008, LJM has been indexed in the scientometric databases Scopus, Web of Science and many bibliographic and abstracts databases (Zentralblatt Math, Google Scholar, Academic OneFile, Expanded Academic, Mathematical Reviews, OCLC, SCImago, STMA-Z, Summon by ProQuest, RSCI). As a result, the number of articles submitted to the journal has significantly increased, and the previously developed algorithm for selecting experts could not always sufficiently take into account the thematic diversity of articles. At the same time, the practice of issuing specialized thematic issues of the journal has significantly expanded. Therefore, there was an additional opportunity to connect reviewers with the editors-compilers of these thematic issues, and the existing automated system for selecting and appointing reviewers was transformed into a recommendation system.
The basis for the selection of recommendations, it was decided to use the
mathematical subject classification Mathematics Subject Classification (MSC 2000, MSC
2010) [
Mathematical Subject Classification (MSC) is created and maintained by
Mathematical Reviews (MR, http://www.ams.org/publications/math-reviews/math-reviews) and
Zentralblatt MATH (zbMATH, https://zbmath.org). This classification is used by all
major mathematical journals, the archive of electronic publications of scientific
articles and their preprints arXiv.org (https://arxiv.org/), and digital mathematical
libraries [
MSC is a three-layer scheme using alphanumeric codes. The classifier code of the first layer consists of two digits, the second layer is determined by the capital letter of the Latin alphabet “A”, “B” or “C”, the third layer contains two digits. For example, the code “42C40 Wavelets and other special systems” refers to the third layer of the hierarchy, the parent code is “42C Nontrigonometric harmonic analysis”, and the code “42 Harmonic Analysis on Euclidean spaces” belongs to the top layer of the diagram.
MSC2010 contains 63 nodes of the first layer, starting with “00 General mathematics” and ending with “97 Mathematics education”. The numbering is not continuous, for example, codes “02”, “04”, “07”, “21” and some others are missing. The second layer contains 528 codes, and the third contains 5606.
In [
In [
The algorithm for generating recommendations for the selection of reviewers consists of two modules. The first module implements the creation and replenishment of the expert database in the journal information system, the second module selects experts for the specific article in question.
To form an expert profile, an automated search is conducted in the abstract and
bibliographic databases of the articles of the author, included in the composition of
experts. The MSC codes indicated in these articles (see Fig. 1) are included in the
expert profile. In case codes MSC2000 are indicated, they are converted to codes
MSC2010 [
Fig. 2. Managing Reviewer Profile in OJS
If the classifier code is already present in the profile, the weight of this code increases and it is written to the database (see Fig. 3).
If the expert, after viewing the article, selects the option “UNABLE TO DO THE REVIEW” (refusal to review) on the “Review” page of the OJS information system, then the codes in the expert profile that match the article codes have a reduced weight. Figure 4 shows a fragment of an expert profile with codes MSC2010.
When adding classifier codes to the expert profile, information retrieval methods are used. In particular, a model of the vector space of documents is created, by analogy with which the profiles of reviewers act as documents, and the classifier codes from the reviewer's profile act as the corresponding terms: ( ) ⁄∑ where is the number of occurrences of the MSC 2010 code in the general list of codes in the reviewer profile d, and the denominator contains the total number of codes in this profile.
As a rule, the first code is the main code that determines the topic of the article. For clarification, the rest of the codes given in the article are used. Therefore, in the algorithm, the first classifier in the list is assigned the highest weight (for example, twice as many as the next).
After calculating the weight of each extracted code, an entry is made in the expert profile. To perform this operation, you need to make changes to the structure of the database table. To do this, OJS provides the ability to modify the table using the database schema DTD adopted from ADOdb (see, https://github.com/pkp/pkplib/blob/master/dtd/xmlSchema.dtd, http://adodb.org/dokuwiki/doku. php?id=v5:axmls:axmls_index). So, for example, the “tf” field was added to the “user_interests” table (see Fig. 3). This field stores the value obtained by the above formula. Reviewers are selected based on a vector model. In accordance with the tf-idf method for this article, weights are calculated from three levels of MSC classifiers: Here t is the MSC code, d is the expert profile, D is the set of expert profiles, N is the total number of expert profiles, df is the number of experts who have the MSC code t.
Next, the cosine measure is calculated between the query vector generated using the previous formula and each expert profile from database of the system: (⃗ ( ) ⃗ ( )) ‖ ⃗ ( )‖‖ ⃗ ( )‖ where q is the vector of the analyzed article, and di is the vector obtained from the reviewer`s profile. Elements of these vectors are MSC codes. Fig. 5. Formed recommendation containing a list of reviewers most preferred for the selected article
In the next step, the cosine measure for each classifier layer is calculated and then summation is performed using a decreasing coefficient for each layer (0.5; 0.3 and 0.2, respectively): ( ) ∑ ( ) where δ is the decreasing coefficient for each layer.
The assignment of reviewers in the OJS system is done by a user who has the roles of “Editor” or “Section Editor”. Reviewers are selected from the list of system users who have the role of “Reviewer”. This choice can be made only in manual mode – based on the professional experience of the editor.
The OJS system provides opportunities for changing its functionality by adding modules (plug-ins) compiled according to certain rules. Since the system is written in PHP and is open, it is possible to make changes to the functionality of the system without using special OJS tools. Finally, the result is provided at Figure 5.
A method for the automated selection of possible reviewers for scientific papers submitted for publication in the information system of a scientific journal is proposed, based on the use of Mathematics Subject Classification 2010 to determine the subject area to which the research results submitted for publication belong. The method is implemented as a service using the OJS plug-in technology and can be used in any mathematical journal that runs on the OJS platform. The method was tested in the information system of the Lobachevskii Journal of Mathematics.
This work was funded by the subsidy allocated to Kazan Federal University for the state assignment in the sphere of scientific activities, grant agreement No. 1.2368.2017). The work partially contains the results of the project “Monitoring and standardization of the development and use of technologies for storing and analyzing big data in the digital economy of the Russian Federation”, carried out as part of the program of competence of the National Technological Initiative “Center for storing and analyzing big data”, supported by the Ministry of Science and Higher Education Of the Russian Federation under the Treaty of Moscow State University named after M.V. Lomonosov with the Project Support Fund of the National Technological Initiative dated 08/15/2019 No. 7/1251/2019. The work was also carried out with the partial support of the Russian Fund for Basic Researches and the Government of the Republic of Tatarstan within the framework of scientific projects 18-2903086, 18-47-160012.