Methods for the automatic generation of metadata for documents in digital mathematical collections in the formats of international resource aggregators in mathematics and Computer Science are proposed. Metadata normalization services for electronic collections of scientific documents in accordance with DTD rules and XML schemas Journal Archiving and Interchange Tag Suite (NISO JATS) V1.0, V1.1, V1.2 have been created. Algorithms for creating electronic collections and including them in the digital mathematical library are presented. Tools for generating metadata of collection documents in accordance with the syntactic rules of digital libraries have been developed. An algorithm for the automated preparation of metadata of electronic collections of the Lobachevskii DML library according to the rules of the dblp Computer Science Bibliography (DBLP) bibliographic database on computer sciences is given. An algorithm for converting metadata to the oai_dc format and generating the archive structure for import into DSpace digital storage has been created. Methods for integrating electronic mathematical collections of Kazan University into Russian and foreign digital mathematical libraries have been proposed and implemented.
With the development of information and communication technologies, for the first
time, the opportunity has appeared to make available the scientific knowledge created
over the entire printing period. Therefore, it is no coincidence that initiatives such as
the World Digital Mathematics Library (WDML) and Global Digital Mathematics
Library (GDML) appear. All of them are aimed at developing the basic principles of
the integration of scientific knowledge in the field of mathematics [
From the point of view of WDML program documents [
In connection with a significant increase in the volume of scientific publications, it
became necessary to create specialized methods for the automated processing of large
amounts of documents [
In line with the WDML strategy, we are developing Lobachevskii DML
(https://lobachevskii-dml.ru/) [
Lobachevskii DML is based on OntoMath digital ecosystem [
The core component of the OntoMath ecosystem is the semantic publishing
platform [
The semantic publishing platform, in turn, is based on the OntoMathPRO [
In this paper, we present a new version of the module of metadata extraction,
customized for the Lobachevskii DML library. To solve the problem of integrating the
created electronic collections into aggregating digital libraries, such as EuDML,
MathNet.Ru, DBLP, methods for converting metadata according to the schemes
adopted in these libraries are proposed. Methods for normalizing metadata of
Lobachevskii DML digital library collections are described in accordance with the
DTDrules and XML schemas of the Journal Archiving and Interchange Tag Suite (NISO
JATS, https://jats.nlm.nih.gov/archiving/) V1.0, V1 .1, V1.2 [
Representation of Digital Mathematical Libraries Metadata
Currently, many scientometric databases index articles published in leading
mathematical journals. These databases impose different requirements on the set of
metadata of these documents, as well as on the schemes of their presentation (see, for
example, [
Digital mathematical libraries use various metadata formats when forming the collections included in them. This is due to the fact that many such collections are formed from articles published in academic journals. In these cases, the relevant articles are made in accordance with the rules established in these journals and differ in the requirements for the metadata used. These differences relate primarily to the composition of the metadata and their format. Most of all, these differences are noticeable in archival collections of scientific journals. 2.1
Features of Representation Metadata The metadata content of articles of even one journal, depending on the year of its publication, differs significantly. We indicate the archive of articles in the journal “Russian Mathematics (Izvestiya VUZ. Matematika)”, https://kpfu.ru/science/ nauchnye-izdaniya/ivrm). His articles are one of the collections of the digital library Lobachevskii DML. The named journal has been published since 1957, and only articles published in this journal dating back to 2010 are accompanied by a relatively complete set of metadata. Articles published before 2008 lack keywords and abstracts (see Table 1).
With the transition of Russian journals to the international scientific space, the composition of affiliation changed. Affiliation was replenished with such new information about the authors as information about the author’s place of work, business address, and email address.
The history of expanding the set of metadata used, described in the above example,
is typical of almost all scientific journals. To replenish the set of metadata, methods
are developed for extracting metadata from documents [
English version No No No No Yes Yes Yes Yes Yes 1957 – 1959 1960 – 1965 1965 – 1969 1970 – 1974 1975 – 1994 1994 – 1997 1998 – 2007 2008 – 2009 2010 – 2019
No No No Yes No No No Yes Yes
No Yes Yes Yes Yes No No No Yes
No No No No No Yes Yes Yes Yes
Key words No No No No No No No Yes Yes
No No Yes Yes Yes Yes Yes Yes Yes
No
No
No
No
No
No
Yes
Yes
Yes
One of the stages of integration of electronic mathematical collections in EuDML is
the normalization of the metadata of these collections according to the rules for the
formation of a obligatory set of metadata. EuDML uses NISO JATS V1.0 XML
schemas to describe articles from mathematical journals, and the general metadata
schema of this digital library is described in [
The digital library Lobachevskii DML is created on the basis of the principles of WDML, according to which the leading role is given to the relationships between documents and objects in them. In this case, the documents themselves can be physically placed outside a specific digital library. A number of electronic collections of the Lobachevskii DML library are physically hosted in other digital libraries. For example, the journal collection “Russian Mathematics” is digitized, equipped with meta-descriptions and presented on the MathNet.Ru portal (http://www.mathnet.ru/php/journal.phtml?Jrnid=ivm). Our tasks are to replenish such collections with additional metadata, as well as to automatically select objects and establish semantic links between them.
When forming a fundamental set of metadata for electronic collections stored on external resources, the metadata presented on these resources is initially imported. For this purpose and using the package functions HtmlAgilityPack (https://html-agilitypack.net/) in C#, a program has been developed for extracting metadata from web pages and writing them in XML-format of the digital library Lobachevskii DML, replenishment and subsequent conversions according to EuDML schemes. For example, for the collection “Russian Mathematics” the following steps have been completed (see Algorithm 1).
It is proposed to create an article identifier as a combination of lines: a journal identifier (attribute value “jrnid =”) and an article identifier (attribute value “paperid =”) on the MathNet.Ru portal.
Algorithm 1: Extraction and normalization metadata of the journal collection “Russian Mathematics” 1: load article’s page in Russian from journal web cite 2: split article’s page, read AMSBib citation block 3: read from AMSBib citation block: author's name, article's title, begin and end pages, journal’ name, volume, urls. 4: read from article’s page: key words, annotation, affiliation, received date, UDC. 5: load article’s page in English 6: read from AMSBibcitation block: author's name, article's title, begin and end pages, journal’ name, volume, urls. 7: read from article’s page: key words, annotation, affiliation, received date, UDC. 8: form article’s identifier 9: form all metadata in EuDML xml format 10: write to file
One of the features of articles in Russian journals is that they can be translated, that is, the author creates an article in Russian, then it is published in the English version of the journal. Such articles cannot be considered as different articles, however, at present, the schemes proposed by EuDML do not allow combining an article published in Russian and its translated version in English within a single meta description. Therefore, in the fundamental set of EuDML, one has to describe the translated articles as different articles in different journals.
Note that when using Russian-language literature, the link is translated. However, as presented in Table 2, the translated and transliterated bibliographic differ in the names of articles, the name of the journal, the issue number and page of the article.
It must also be borne in mind that the same journal may have more than one name. For example, the journal “Izvestiya Vysshikh Uchebnykh Zavedenii. Mathematics” has the original (it is given), transliterated (“Izvestiya Vysshikh Uchebnykh Zavedenii. Matematika”), translated old (“Soviet Mathematics”) and new (“Russian Mathematics”) names. In the collections of the digital library Lobachevskii DML, as well as in the eLibrary.ru and MathNet.ru libraries, such articles are presented as duplicates of one document.
Note that the process of preparing metadata in the eLibrary.ru format is automated
(see [
Normalization of metadata according to DBLP schemes
One of the authoritative libraries in computer science is “dblp computer science
bibliography” [
Among the collections of Lobachevskii DML, DBLP requirements are satisfied by
the content of the “Russian Digital Libraries Journal”. Since 2015, a new model of
document submission has been used in this journal and the Open Journal Systems
(OJS) publishing system has been introduced [
One of the problems in the preparation of metadata that can be encountered when describing Russian-language scientific collections is the following question: in what language should the metadata of a Russian-language article be presented in DBLP if the journal has the title and abstract of the article in English. On the one hand, it is desirable to present the document in a form that will be understood by most users of this database, that is, in English. On the other hand, we note that in the early versions of OJS only one language was used to represent the authors, and the main language we use is Russian. Therefore, when choosing English for the presentation of article metadata, it is necessary to develop tools for the translation and transliteration of article metadata.
Normalization to DBLP format takes place in three main stages: extraction of the required metadata, addition of metadata and their normalization to this format. The corresponding algorithm is presented at Algorithm 2. It is implemented using a program developed in C#. Using the System: XML extension tools, parsing xml files is performed, and the html page is read from the NuGet functions of the HTMLAgilityPack package. As a result, an xml-file with metadata loaded into the program is generated. This file is fully compliant with DBLP rules (https://dblp.uni-trier.de/db/ journals/rdlj/).
Algorithm 2. Normalization of articles in DBLP format of the journal “Russian Digital Libraries Journal” 1: load VolCollection //Set of xml files 2: for each volume in VolCollection do 3: for each paper in volume do 4: read from paper: author’s names, title, page numbers, year of ussue, url in ojs.kpfu.ru, volume. 5: read cite page https://elbib.ru/en/year/+year \\volume page 6: split cite page, read metadata: author’s names in English, url in elbib.kpfu.ru. 7: split author’s names, 8: Answer:=Form(author’s name, Transliteration(name), title, page numbers, url, volume); 9: write Answer in file dblp.xml 10: end for 11: end for 4
DSpace-based Digital Storage of Electronic Collections
One of the important tasks when working with digital mathematical libraries is the
automated integration of repositories of mathematical documents into other
information systems. This process is based on a model of aggregation and dissemination of
metadata. Such a model (OAI Protocol for Metadata Harvesting (hereinafter
OAIPMH) [
The Open Archives Initiative (OAI) develops and promotes interoperability
standards to effectively disseminate electronic resources, as well as to increase the
availability of scientific information exchange. The corresponding OAI-PMH protocol
requires the inclusion of a Dublin Core metadata set (Dublin Core,
http://dublincore.org/) in the resource description. For this, the oai_dc format was
developed, which is based on Dublin Core and uses only a limited number of Dublin
Core tags [
In addition, to organize work with OAI-PMH it is necessary to use a digital storage
support system. An overview of various digital repositories is given in [
Uploading metadata to DSpace is as follows. A table file is generated in csv format (Comma-Separated Values). It records metadata prepared according to the Dublin Core scheme. A method for converting the archive to the Simple Archive Format is also used. In addition, it is possible to download metadata through the console or use manual input of metadata on the site. It is most rational to use archive downloads. The main advantages of this option are the ease of downloading all files in one archive and the ability to download not only metadata, but also files. Since DSpace digital storage can be used together with OJS, you can get the “storage + system" model for working with any journal or collection of articles. Data exchange occurs through the OAIPHM server, which allows you to automatically harvest metadata.
One of the important tasks is to create a service that would allow the conversion of metadata into the oai_dc format. Here, the difference from the classic Dublin Core format is that in oai_dc, Dublin Core tags are not elements of a metadata scheme, but are placed in the attributes of the <dcvalue> tag. When organizing an archive in Simple Archive Format, we need to create a dublin_core.xml file in oai_dc format for each article. This file contains all the necessary information about the article from the publication being processed. So you can prepare for downloading the data of the processed set of articles in one archive. Note that optimal file loading in DSpace requires the addition of data files whose names are written in the content file.
As a test collection for testing the method described above, we used files from a number of collections of the “Proceedings of the N.I. Lobachevskii Mathematical Center” published by Kazan Federal University. These files are a collection of articles from these collections and contain a description of each of them with the listed metadata about the relevant articles. Each article’s description contains metadata such as authors, title, start and end pages (Fig. 1). Therefore, it is necessary to add metadata such as volume, year of publication, publisher to the description of each article, as well as convert the description of each volume to a multitude of descriptions of individual articles. Information about the year and the publisher is compiled and presented as a csv-file (Fig. 2).
Thus, the input data of the program are: xml-files containing information about articles, as well as a csv-file with information about the volume, year and publishing house. File names contain the volume number. As a result, we get the description files for each article in the oai_dc format, sorted in the order accepted for uploading to DSpace.
The algorithm is implemented using the C# language using the System: XML extension (Algorithm 3).
Algorithm 3. Normalizing of metadata of the collection of “Proceedings of the Mathematical Center” in oai_dc format 1: load VolCollection\\ collection of xml files 2: for each volume from VolCollection do 3: read volume number from file’s name 4: read from info.csv file: publisher, issue year 5: Papers:=new string list 6. for each paper from volume do 7. read from paper: author’s names, title, page numbers 8. split author’s names 9. Paper:=Formoai_dc(authors’s names, title, issue year, page numbers, publisher); 10. Papers.Add(Paper) 11. end for 12. create volume folder 13. for each paper from Papers do 14. create paper’s folder 15. create file dublin_core.xml in paper’s folder 16. write paper in file dublin_core.xml 17. create content file 18. copy data files 19. end for 20. end for
A file describing an article in oai_dc format is represented at Fig. 3. In order to integrate electronic mathematical collections of Kazan University into the international scientific space, algorithms have been developed for the formation of metadata of these collections, as well as the documents included in them, in accordance with the formats of digital mathematical libraries and scientometric databases. Methods of normalizing metadata of electronic mathematical collections in accordance with the XML-schemes NISO JATS and DBLP are presented.
Acknowledgements. The work partially contains the results of the project “Monitoring and standardization of the development and use of technologies for storing and analyzing big data in the digital economy of the Russian Federation”, carried out as part of the program of competence of the National Technological Initiative “Center for storing and analyzing big data”, supported by the Ministry of Science and Higher Education of the Russian Federation under the Treaty of Moscow State University named after M.V. Lomonosov with the Project Support Fund of the National Technological Initiative dated 08/15/2019 No. 7/1251/2019. The work was also carried out with the partial support of the Russian Fund for Basic Researches (project 18-2903086) and the Russian Fund for Basic Researches and the Government of the Republic of Tatarstan within the framework of scientific project 18-47-160012.