=Paper=
{{Paper
|id=Vol-2919/paper26
|storemode=property
|title=The Content Creating of the Scientific Knowledge Digital Space
|pdfUrl=https://ceur-ws.org/Vol-2919/paper26.pdf
|volume=Vol-2919
|authors=Svetlana Vlasova,Nikolay Kalenov
}}
==The Content Creating of the Scientific Knowledge Digital Space==
https://ceur-ws.org/Vol-2919/paper26.pdf
The Content Creating of the Scientific Knowledge Digital
Space 1
Svetlana Vlasova [0000-0003-1533-5850], Nikolay Kalenov [0000-0001-5269-0988]
Joint Supercomputer Center of the Russian Academy of Sciences – Branch of Federal State
Institution “Scientific Research Institute for System Analysis of the Russian Academy of Sci-
ences”, 119334 Moscow, Leninsky Prospect, 32a
jscc@jscc.ru
Abstract. The paper proposes the formation problem of the Common Digital
Space of Scientific Knowledge (CDSSK) content. The CDSSK is a computer
environment, where the user should get exhaustive answers to questions con-
cerning the achievements in various fields of science. The CDSSK should con-
tain both basic and popular-scientific information and consist of a subspaces re-
lated to the different science fields. Its content should include factual data,
source link where they are from, and these sources full texts. One of the major
challenges in the formation of CDSSK is the selection of materials to be includ-
ed in its content. One of the solutions to this problem is expert assessment,
which should be carried out by a fairly wide range of highly qualified special-
ists in this field of science. This paper describes the expert WEB-based selec-
tion material system for inclusion in the CDSSK. This system was carried out
by the Joint Supercomputer Centre of the Russian Academy of Sciences in
2019. The structure of the system, its functions, metadata profiles included in
it, and the results of practical implementation are given in the paper. The system
was tested at two all-Russian competitions – for the best scientific monograph
and for the best student qualification work. Several hundred works were sub-
mitted to the competitions, and several dozen experts from various regions of
the country participated in their evaluation.
Keywords: Common Digital Space of Scientific Knowledge, Expert Assess-
ments, Automated System, WEB Technologies, Metadata, Electronic Libraries,
Content Generation.
1 Introduction
The problem of selecting materials for inclusion in databases, electronic libraries and
other information systems is one of the most important in the formation of such re-
sources. It is extremely important when forming the content of of a common digital
space of scientific knowledge (CDSSK). CDSSK is a computer environment, where
1
The research is carried out by JSCC RAS — branch of SRISA within the framework
of the State task 0580–2021–0016.
Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0
International (CC BY 4.0).
Proceedings of the of the XXIII International Conference "Enterprise Engineering and Knowledge Management"
(EEKM 2020), Moscow, Russia, December 8-9, 2020.
the user should get exhaustive answers to questions concerning the achievements in
various fields of science. This environment should contain the true basic and popular-
scientific information represented as different object types such as encyclopedic arti-
cles, factographic databases, some phenomena digital models, archival , museum and
library storage materials, presented in the form of digitized texts, images, and multi-
media resources.
The CDSSK consists of a number of subspaces complex related to separate scientific
disciplines, each of which reflects its own specifics and has its own subject ontology,
built on the same principles for all subspaces. The join ontology provides the connec-
tion between subspaces [1, 2].
Despite each subspace specifics, all of them must have a documentary component,
including full document texts that record scientific results. It is based on monographs,
reference books and textbooks that reflect generally recognized, time-tested scientific
knowledge, and are "classic" for each science field. It is clear that in any scientific
field there is a significant number of classical works, many of which duplicate each
other. In this connection there is the selection problem of the most important classical
publications for inclusion of their full texts in CDSSK.
In addition to the time-tested publications, the CDSSK should contain new
knowledge, which is also in scientific publications. The annual number of scientific
publications appearing in the world (both in print and electronic forms) is the mil-
lions. As far as, by definition, the CDSSK should contain only reliable scientific in-
formation that is relating to knowledge, there is a selection problem of the most sig-
nificant scientific publications which are the knowledge community, "worthy" for
presenting to the CDSSK.
To solve these problems, the authors suggest to use an iterative process, which pro-
vides the first material choice with using certain objective indicators (the publishing
prestige, citation, author's qualifications, etc.), followed by expert evaluation of pre-
selected materials. The fairly wide range of highly qualified specialists must imple-
ment the expert assessment in this field of science.
As already pointed out, there are not only published materials but also various prob-
lem-oriented information systems in the CDSSK. The appreciable quantity of such
systems is in many areas of both the natural and humanitarian sciences.
As [3] indicates, there are more than 1000 information systems in the history area; [4-
6] describes the monitoring of Russian information resources in the social sciences
area, the results of which are in the «NIRON» system, which contains data on more
than three thousand resources of different types and different thematic focus. The
website «Science in Runet» [7] reflects more than 10,000 information resources relat-
ed to natural and exact sciences.
The organization of an expert community in various areas within a particular science
field is a separate task that should be solved by the relevant structures that organize
the work on the formation of the CDSSK subspaces. In particular, experts from the
Russian Academy of Sciences, which is responsible for examining all scientific de-
velopments in the country, can act as the UDSSK experts.
It is obvious that the examination of materials, which are for showing in the CDSSK,
is necessary to be in networking with the experts from different regions. It is also
necessary to develop a typical automated system that can be configured for different
types of objects and for different assessment systems should be developed for con-
ducting the examination. Such a system should contain a database of objects subject
to examination and a database of experts. The result of its work is a rating list of ob-
jects, ordered in accordance with the received estimates, taking into account the pos-
sible "weights" of experts. The prototype of such a system can serve as the expert
system of acquisition [8-11], developed with the participation of authors and has been
operating successfully for many years in the Library for natural Sciences of Russian
Academy of Sciences (LNS RAS) and Central scientific library of the Ural branch of
RAS [12, 13]. The purpose of this system was to optimize acquisition - to identify the
most informative new publications appearing on the book market, for each institution
of the Russian Academy of Sciences , and then to order them for the collections of the
library serving this institution, based on available financial resources.
Taking into account the experience gained in the operation of the expert acquisition
system, the Joint Supercomputer Centre (JSCC) of RAS specialists worked out the
"Expertise" system, its structure and functional features are below.
2 «Expertise» system structure
The system includes two metadata databases (the expert database and the database of
examination objects) and the object storage.
The expert metadata profile contains the following elements:
• the expert identifier;
• surname, name and patronymic;
• organization (the expert workplace);
• academic degree, membership in academies;
• subject areas of expertise (in terms of the subject area of the object metadata profile
) with an indication of the "weight" of the assessment for each area/
• email address for correspondence with the expert;
• the individual login;
• the customized password to enter the system.
The metadata profile objects of expertise, generally speaking, depends on the specific
valued objects (e.g., the publication metadata elements and factographic information
system may be slightly different) and requirements to the object division into classes,
within which to base rankings (e.g., a separate assessment of scientific monographs,
popular scientific publications and textbooks presented in the same dataset will re-
quire inclusion in the metadata profile list on the publication type). The metadata
profile of objects of expertise must include the following elements:
• the object name;
• short description;
• thematic scope;
• the object identifier in the storage;
• data about the copyright.
The storage of digital objects to be evaluated can be centralized (in this case, objects
are uploaded to it from the outside), or it can be distributed (for example, publications
stored in various external electronic libraries are evaluated). It is important that each
object has its own unique identifier, which can be used to call it to the expert's com-
puter screen. Objects can be text files, image files, or multimedia files. When an ex-
ternal information system is evaluated, the identifier included in its metadata profile
should allow for versatile work with the system to form its multidimensional assess-
ment.
On par with metadata describing experts and assessment objects, in the «Expert» sys-
tem generates a number of overhead generic tables:
- a rating table containing verbal and numeric expressions;
- Thematic areas table;
- Academic degrees table;
- Table of organizations;
- User rights table.
The system provides for the work of various groups of users, each of which has its
own rights. These groups include administrators, editors, experts, and managers. The
administrator has the rights to configure the system, enter users and set their rights.
The editor has rights to enter, view, and correct data. The expert has the right to view
and evaluate objects. The Manager has the right to view the results of experts' work
and create rating lists.
3 System functions
The Expert system implements the following functions.
The system configuration by the administrator for specific metadata profiles, types of
ratings, and user rights.
User authorization in accordance with the settings and providing opportunities to
work with the system depending on their status.
The object and expert metadata batch loading from files of the specified structure
(performed by the editor).
Manual loading of metadata using a special user interface that provides formal logical
information control (performed by the editor).
Editing and deleting metadata (performed by the editor).
Search for objects by various metadata elements and their combinations using Boole-
an logic operators (performed by any user).
Visualization of found objects (available to any authorized user).
Input of object evaluations by experts who have the same thematic focus in their
metadata as the object being evaluated.
Changes in experts ' previous estimates.
Visualization of estimates given to each object by each of the experts (available to the
Manager).
Calculation of total and average estimates of individual objects or their groups (avail-
able to the Manager).
Creating rating lists of objects within a given group (available to the Manager).
Output of calculation results to Excel files.
Automatic e-mailing of information to experts about the receipt of new objects that
correspond to their thematic interests.
Data backup and recovery in case of failures.
In the process of operation, the system implements the "publication - expert – assess-
ment" connections. Many experts can evaluate one publication; one expert can evalu-
ate many publications; each expert can assign only one estimate to the publication.
The system is based on Microsoft technology ASP.NET 4 on the Microsoft .NET
Framework in the Microsoft Visual Studio 2017 development environment.
4 The system practical implementation example
The system feature "Expertise" is the ability of its tenability to solve different prob-
lems. One of the classes of such problems is the tendering netting process. The system
«Expert» modification, called "Competition", was of use in 2019 to conduct competi-
tions for the best scientific monograph and the best qualifying student work, orga-
nized by the publishing house "Direct-Media "(PH DM) [14] in 12 scientific areas
(see Fig. 1). The competition for the best monograph was in the categories «funda-
mental scientific book» and «popular science book». Competition of student papers -
in the categories «bachelor» and «master»
The full texts of the objects from both competitions are in the electronic library (EL)
of the PH DM, each of them has its own ID - URL. Publication metadata was loaded
at the «Competition» system in batch mode from Excel tables exported from the elec-
tronic library of the PH DM.
As far as a number of experts evaluated papers for both competitions, to avoid dupli-
cation, a single database of experts was being created, but in the profile of their
metadata, there was an indication which competition (s) this expert evaluates.
A screenshot of the page for entering expert metadata is shown in Fig. 1. Since the
expert system has a Russian-language interface, the translation of their text elements
is given below the screenshots (here and below). The expert's academic degree and
the science branches in which they specialize are from the tables of academic degrees
and subject areas.
Fig. 1. Interface for entering an expert
The «Competition» system allows to make an advanced search for experts in the fol-
lowing search fields:
• The expert full name;
• The organization's fragment name;
• Degree;
• e-mail address’ fragment;
• Science sections;
• Competition expert (Yes/no).
Search terms are being input in one or more fields. The «academic degree» values and
"science sections" fields the users select from drop-down lists. When you enter terms
in multiple fields, they will be linked by the logical operator «And». In fig. 2 there is
an example of searching for records registered in the experts’ system that have a can-
didate's degree in Economics and are experts in the «Monographs» competition.
Fig. 2. Experts’ searching example
In the list of experts obtained, their names and related organizations are active links.
When you click on the selected link, the system shows the expert's metadata, which
can be edited if necessary.
The rating tables for various contests had configurations by the system administrator
to the lists set by the contest organizers (see Fig. 3).
During the examination, the system provides the expert with a list of available the-
matic areas in accordance with the specified area of thematic interests when register-
ing. After selecting a focused area, the expert will be able to view the publication
descriptions in the specified subject area. Under each publication description is a link
to its full text, as well as a link to "Evaluate" (if the publication doesn’t have an as-
sessment by this expert). Clicking on the "Value" link allows the expert to select a
rating from the list downloaded from the rating table and enter the necessary comment
(see Fig. 3).
Fig. 3. Interface for evaluating publications
After selecting an estimate, the system returns to publication list in the corresponding
thematic area, where the evaluated publication appears with the link "the Publication
already has your estimate ". The expert can always change their estimate and com-
ment by clicking on this link.
The system Manager access to the estimates issued by competition experts is given.
After selecting a theme, the system will display descriptions of publications sorted in
descending order of the total points received. Under the description of publications
that have already been evaluated by at least one expert, there will be an average esti-
mate, as well as a list of experts who have evaluated the publication, indicating the
surname, first name and patronymic of the expert, his organization, the estimate and
comment (see Fig. 4).
Fig. 4. Example of viewing publication ratings
The evaluation of student papers submitted to the competition was conducted from
mid- July to mid -August 2019. The competition received 1,100 papers from 425
universities. The thirty two experts evaluated them. The competition jury reviewed
the final rating lists of works. The winners in the “bachelor’s” and “master’s” catego-
ries were selected and awarded for each of the 12 research areas.
There were 315 books from scientific organizations and universities for the mono-
graph competition. There was more competition time, taking into account the estimate
complexity, compared to the student papers competition - it lasted from 1.05 to 15.11
2019. The forty specialists took part in the examination, and the winners were for
each of the two categories (scientific and popular science publications).
5 Conclusion
Conducted competitions show that the developed system is quite reliable, simple and
easy to use, both from the editor viewpoint and from the expert viewpoint. There were
no questions about working with the system during the contests that lasted for several
months. The Expert system can be configured to hold online competitions of various
topics and scales, ranging from intra-University competitions to national and interna-
tional ones. It can be successfully used for the selection of literature to be included in
the National electronic library of Russia [15].
The examination work of scientific materials takes significant time and intellectual
expenses. In this regard, one of the serious issues that must have a solution when or-
ganizing an expert examination is expert motivation. The LNS RAS expert’s work the
evaluation of the book market was motivated by the fact that (a) the experts received
information on new publications and (b) when LNS RAS receives a book that was
positively evaluated by an expert the LNS automated system sends
him a special notification and he can immediately receive this book. During the com-
petitions of scientific papers mentioned above, the incentive to participate in the ex-
pert examination was to provide the expert with free access for a year to all publica-
tions available in the «Direct-media» electronic library. Currently, this library con-
tains «more than 70,000 scientific and educational publications, educational multime-
dia, interactive tests, 150,000 painting reproductions» [16].
Given the importance of forming a Common digital space of scientific knowledge and
its reliable and most significant content, leading scientists of the country should be
involved in expert selection of materials must be included in this space. They must
have the appropriate motivation to do this work. Motivation should be not only and
not so much material, but also an understanding of the necessity and social utility of
this work. One of the possible solutions to this problem is to include work on the ex-
amination of scientific materials in the state task of leading scientific organizations in
the country.
References
1. Antopol’skiy A.B., Kalenov N.E., Serebryakov V.A., Sotnikov A.N.: About the Unified
digital space of scientific knowledge. Bulletin of the Russian Academy of Sciences 89(7),
728–735 (2019).
2. Kalenov, N., Sobolevskaya, I., Sotnikov, A.: Mathematical modeling of the processes of
interdisciplinary collections formation in the digital libraries environment. CEUR Work-
shop Proceedings 2543, 391–398 (2020).
3. LNCS Homepage, https://discourse.digital/digital-humanities/katalog-istoriko-
orientirovannyh-informacionnyh-sistem/, last accessed 2020/03/26.
4. Antopol’skiy A.B.: Information resources of the historical science academic sector as a
source of Unified Russian electronic space of knowledge. Historical Informatics 1, 14–27
(2018).
5. Antopol’skiy A.B., Antonov A.M.: About information resource navigator on social Sci-
ences. Information and innovations 2, 18–24 (2018).
6. LNCS Homepage, http://nirontest.alexo.begin.tech, last accessed 2020/03/27.
7. LNCS Homepage, https://elementy.ru/catalog, last accessed 2020/03/27.
8. Vlasova S. A., Vasilchikov V. V., Kalenov N. E., Levner M. V.: Use of expert assessments
for centralized library systems acquiring. Scientific and technical information 1(5), 22–26
(2007).
9. Kalenov N.E., Kochukova E. V., Pavlova O. V.: Internet system of expert assessments in
the technology of scientific literature acquisition. Inter-industry information service 2, 63–
68 (2013).
10. Vlasova S. A.: Pluses of the expert system: active version. Library 3, 22–24 (2015).
11. Vlasova S. A., Kalenov N. E.: Informatics in the academic library. Systems and means of
informatics 26(3), 162–178 (2016).
12. Gorbich L. G., Zhivoderov L. G.: Experience in implementing the expert system of build-
ing up to prescribed strength in the CSL UB RAS. New technologies in information and
library support of scientific research: collection of scientific papers. 87–92 (2010).
13. Gorbich L. G., Pavlova A. S., Shmeleva S. G.: Integration experience of information sys-
tems of the CSL UB RAS and information flows from external sources for the problems of
expert recruitment system. Information and innovations 4, 27–31 (2018).
14. LNCS Homepage, www.directmedia.ru, last accessed 2020/03/25.
15. LNCS Homepage,
http://static.government.ru/media/files/yYAeqiBDCwITsAXr8LtXsdsdefAbLzi9.pdf, last
accessed 2020/03/25.
16. LNCS Homepage, http://pro-books.ru/company/13549, last accessed 2020/03/25.