We offer methods for the formation of digital collections from a set of documents (scientific articles, monographs, collections of reports), which are presented in various storage formats. Based on the analysis of the structure of documents and the stylistic features of their design, we have developed an algorithm for extracting the metadata of these documents. We present a software tool for dividing collections of articles into separate documents and the formation of their semantic presentation. On the example of a collection “Proceedings of N.I. Lobachevskii Mathematical Center”, which have a different format and structure, we describe the algorithm for creating a digital collection and its inclusion in the Lobachevskii-DML. Algorithms for replenishing the electronic collections of the LobachevskiiDML digital library and generating metadata of documents of these collections in selected formats are presented. Services for the normalization of the Lobachevskii-DML digital library collection metadata were developed in accordance with the DTD rules and the NISO JATS and DBLP XML schemas. Algorithms for creating a mandatory and fundamental collection of metadata collections are presented in accordance with the rules of the European digital mathematical library EuDML.
The term “Big Data”, which is currently widely used in various subject
areas [
Information and communication technologies (ICT) have completely changed
the life cycle of scientific documents both at the stages of their preparation and
dissemination, and at the stage of obtaining results. The above fully applies to
scientific activities in the field of mathematics. But when solving mathematical
problems, the expectations from the use of ICT are significantly higher. Here
we can draw an analogy with the way computers have completely eliminated
manual calculations. The complexity of manual calculations, moreover, their
routine, can be demonstrated by the example of numerous tables of function
values. Such, for example, are the four-digit tables of V.M. Bradis [
Computations have always required the use of specific methods and
nonstandard organizational solutions to cope with the volume (Volume is one of
the characteristics of big data) and to overcome the barrier of the computing
power of the individual. Riche de Prony (Gaspard Clair Franc¸ois Marie Riche
de Prony) in 1791–1802 to compile “cadastral tables” containing logarithms (8
characters), created a “factory for calculation” (“usine `a calculer”), dividing the
calculators into sections: a section of theorists from five prominent
mathematicians, including Legendre, a section of “calculators”, the number which was 7–8
people familiar with numerical and analytical calculations, and a section of 60-80
people who were engaged only in addition and subtraction [
“Manual” calculations are a typical calculation practice used almost until the middle of the 20th century. After that, the created computers saved the scientists from tedious arithmetic operations. Today, on the simplest laptop, the calculation of Pi using the same algorithm that was used for manual counting will take less than a second. Humankind is expecting the same progress now not only in calculations. In the same way, intelligent computer tools should leave in the past time-consuming routine (and not only!) operations in Mathematics. In addition to computing and document preparation, intelligent search tools are needed, including recommendation systems for finding scientific articles that are close in content; terminological annotation services; personal information assistants and information platforms for publishing automation.
This article describes approaches to managing large collections of digital mathematical documents based on semantic methods and consistent with the principles of the World Digital Mathematical Library (WDML), as well as related to the areas constituting the Big Math concept. These approaches are being developed and already partially implemented in the framework of the project for creating the Lobachevskii–DML digital math library. 2
Mathematicians, as well as scientists of other specialties, in recent decades have
faced such volumes of scientific documents that require the involvement of new
methods of working with information. These methods should be primarily based
on the use of intelligent software tools. Estimates of the growth in the volume of
scientific production made today are fairly approximate and take into account
only articles in scientific journals. As an example, we present the results of a
calculation carried out by the Center for Science and Technology Research at
the University of Leiden (SBF 2007). According to this Center (see, for example,
[
Specialized software services are being developed for working with scientific content. Currently, computer support is used at all stages of the life cycle of a scientific document. Mathematical content has features that do not always allow using general-purpose software tools to work with it. The specificity of mathematical documents is determined, first of all, by the logical structure of texts presented in the form of a strict sequence of objects – definitions, statements and proofs. They clearly indicate or are implicitly hidden links with objects from other documents that are understandable only to a specialist in mathematics.
The presence of specialized formulas is another feature of mathematical
documents, which requires the use of specialized software tools at all stages of the
document life cycle, in particular, for their input and display. Such tools, as a
rule, are developed by mathematicians themselves (for example, [
Documents that contain similar texts may differ significantly in terms of
the content laid down in the formulas contained in these documents. Moreover,
absolutely identical formulations of theorems can have qualitative differences on
the results declared in them. Examples are theorems on the improvement of
approximation estimates or reference books on special sections of mathematics
(see, for example, [
Big data in mathematics also manifests itself in studies that require
consideration and analysis of numerous cases. For example, the classification of finite
simple groups required the long-term efforts of a large group of mathematicians
and is presented on more than 10,000 journal pages. An overview of this grand
study is given in [
J. Carette, W.M. Farmer, M. Kohlhase and F. Rabe [
The main task of mathematical software systems today is to integrate the aspects that make up Big Math. 3
The system of digital mathematical libraries currently being created is intended
to consolidate and make accessible both modern mathematical knowledge and
the knowledge contained in articles and books published in the pre-digital period.
To achieve this goal, in the framework of digital libraries, methods for managing
digital information are developed that take into account the characteristics of
the presentation of mathematical content (see, for example, [
The most important tasks in the management of mathematical knowledge are
highlighted in [
An overview of digital mathematical libraries from the point of view of the
DELOS Digital Library Reference Model is given in [
In the field of integration of mathematical knowledge, the most significant is
the Global Digital Mathematics Library (GDML) initiative [
In accordance with the basic principles of WDML, a digital library Lobachevskii
Digital Mathematics Library (Lobachevskii-DML, https://lobachevskii-dml.ru/)
is being created at the Kazan University [
In the project WDML [
Note that the main purpose of the “Proceedings ...” is the publication of materials of mathematical conferences. As a result, the majority of the volumes of the “Proceedings ...” contain several dozen articles with a limited (from a modern point of view) composition of metadata. Since 1998 (since the release of the first volume), several style rules have been used to prepare materials, which influenced the choice of formats and the design of articles in the collected collections. The prerequisites for creating a digital collection from the array of files “Proceedings ...” were the division of volumes into separate articles, the selection of metadata describing each article, the generation of additional metadata containing, in particular, the bibliographic description of the article, a relation to the article file in the digital collection, as well as relations to the profiles of the authors of the article on academic portals and scientometric databases (kpfu.ru, MathNet.ru, Scopus, etc.). The main steps in creating this digital collection are as follows.
At the first stage, the processed archive was clustered: the volumes of
“Proceedings ...” were divided into classes in accordance with the similarity of their
structure and design. For each class, a set of regular expression patterns was
developed that define the rules for searching information blocks. The basis of this
algorithm is the approach proposed in [
At the next stage, the metadata that describe both the volume as a whole and the articles included in it were selected from the array of files of the“Proceedings...” volumes. In particular, for all the articles of each volume were allocated their names, as well as the page numbers of their beginning and end. For this, an algorithm was developed that uses the structural homogeneity of each volume and the style uniqueness in the design of articles in it. In addition to the listed metadata, this algorithm allowed us to also highlight lists of authors, bibliography blocks and other metadata (for example, e-mail addresses and keywords), if they are present in the text.
Further, an XML-language was proposed for describing digital mathematical collections, which consists of a set of tags and XML-schemas based on the Journal Archiving and Interchange Tag Suite (https://jats.nlm.nih.gov/1.2d2/). In the notation of this language, on the basis of the data obtained at the stage of processing the initial array of files, a description of the collection “Proceedings...” was carried out.
Using the methods of text analysis [
The next step in creating a digital collection included the procedures for dividing each volume of “Proceedings...” into separate articles. To do this, from XML-files containing meta-descriptions of volumes, we read tags, whose attributes point to the starting and ending pages of articles. Next, we divide the files into separate documents, which are assigned names in accordance with the rules adopted in the digital collection. The process of selecting articles was organized using a program developed in Python using the functions of the PyPDF2 library (http://pybrary.net/pyPdf/).
Such metadata as authors’ email addresses and their affiliation, we imported
from authors’ profiles that are presented on academic sites and in various
scientific databases, and in parallel they were refined. In this procedure, the semantic
links established in the process of forming a digital collection were applied. The
corresponding algorithm is based on the method of [
The implementation of the algorithms described above allowed us to form a digital collection of the “Proceedings of N.I. Lobachevskii Mathematical Center” and together with the specified set of metadata to include it in the digital library Lobachevskii-DML. 4.2
Metadata Formats. At present, publications on mathematics are indexed in
many scientometric databases. These databases impose different requirements
on the composition of the metadata of the documents included in them and the
schemes for their presentation. On the other hand, digital math libraries also
use various metadata formats when building their collections. This is partly due
to the fact that the articles included in such collections, being published in
journals in accordance with the rules established in them, differ in the requirements
for the metadata used. These differences can be quite significant, primarily
related to the composition of metadata and their format, and are most noticeable
in the archival collections of scientific journals. For example, in many articles
published before 2000, there are no keywords and annotations, and the
affiliation of authors appeared only in articles of recent years. At the same time,
the constantly expanding set of metadata used today testifies to their increasing
role in the improvement of modern scientific communications. Thus, there is a
need to develop both methods for extracting missing metadata from documents
and methods for converting already created metadata into the formats of
relevant scientometric databases. Note also that participation in such projects of
integration of mathematical resources as EuDML (The European Digital
Mathematics Library, https://initiative.eudml.org/) [
Note that the metadata scheme of the digital mathematical library EuDML is described in [40]: the metadata is divided into basic, fundamental, and additional [41]. To describe journal articles in the EuDML project, XML schemas (NISO JATS V1.0) [42] are used. The mandatory set of EuDML metadata is minimal in composition and contains the title of the article in the original language, the names and surnames of the authors, the list of bibliographies, the unique identifier of the article (for example, doi) and the URL of its full text. The fundamental set of metadata, in addition to the required metadata, includes annotation of the article and keywords.
A number of electronic collections of the digital library Lobachevskii-DML are physically located in other digital libraries. Our tasks are to replenish such collections with additional metadata, as well as automatically selecting objects and establishing semantic links between them.
When forming the fundamental set of metadata of electronic collections stored on external resources, the metadata presented on these resources is initially imported. For this purpose, a program for extracting metadata from web pages and writing them in the XML-format of the digital library LobachevskiiDML, as well as replenishing and subsequent conversion by EuDML schemas.
As an example, we will point out the archive of articles of the journal
“Russian Mathematics (Izvestiya VUZ. Matematika)”. This journal collection is
digitized, supplied with meta descriptions, presented on the portal MathNet.Ru
(http://www.mathnet.ru/php/journal.phtml?jrnid=ivm) (see also [
Part of the metadata was imported from the “Citation in AMSBIB format” block of the MathNet.Ru portal. Then, keywords and a hyperlink to the Springer Link portal page (https://link.springer.com/journal/11982) with the English version of the article were read from the web page. This information is included in the metadata, and a hyperlink is made.
The next step involves analyzing the web page of the English version of the
article, extracting and recording metadata. Next, a personal identifier of this
article was generated, which was proposed to be created as a string concatenation
– journal identifier (attribute value “jrnid =”) and article identifier (attribute
value “paperid =”) on the portal MathNet.Ru.
Lobachevskii-DML Metadata Factory. As a rule, the term “metadata
factory” refers to a set of software tools for managing metadata in digital libraries
(see, for example, [
On the Lobachevskii-DML digital library portal, the OntoMath digital ecosystem is presented, which is an essential part of this digital library [43]. The main components of this ecosystem are: mathematical ontologies Mocassin, OntoMathPro and OntoMathEdu, the semantic publishing platform, the semantic search service OntoMathSearch, recommender systems for the selection of scientific classifiers, search for related articles and terminological annotation.
Mathematical Ontologies. The concept of the Semantic Web assumes the semantic structuring of the Inter-net data space for its use by software agents, and the main tasks are the unification (compatibility) and binding of data from different sources. Most relevant to applying Linked Data principles is the LOD project. Its main advantage is in a standardized approach to the structuring and storage of integrated data that is loaded and presented in the form of RDF, that is, triplets of the “subject – predicate – object” type.
An important direction in the development of the Semantic Web domain was the development of ontologies of subject domains, including ontologies of the presentation of mathematical knowledge [44].
The representation and exchange of knowledge in any subject area is based
on its conceptualization (see, for example, [
Mocassin Ontology [45] is an ontology of the logical structure of mathematical documents, designed for automatic analysis of mathematical publications in the LATEX format. This ontology formally (in the OWL language) describes the semantics of the structural elements of mathematical documents (for example, theorems, lemmas, proofs, definitions, etc.) expressed in the form of classes and properties. In addition, the ontology contains the axioms of cardinality and transitivity.
The ontology of professional mathematics OntoMathPro [46, 47] is the ontology of mathematical knowledge, which is organized in the form of two hierarchies: – hierarchies of areas of mathematics: mathematical logic, set theory, algebra, geometry, topology, and so on; – hierarchies of mathematical objects: set, function, integral, elementary event, Lagrange polynomial, etc.
The OntoMathPro ontology is developed in OWL-DL/RDFS and contains 3450 classes, 6 types of object properties, 3630 instances of the IS-A property, and 1140 instances of the remaining properties. It contains five types of relationships: Class → Subclass, Defined with the help, Associative relationship, Task → Solution method and Area of Mathematics → Mathematical object. Ontology concepts contain their name in Russian and English, definition, links to external resources from the Linked Open Data cloud, and links to other concepts. Objects of semantic annotation are also formulas associated with formulas, fragments of text that specify the descriptions of variable formulas.
Ontology of educational mathematics OntoMathEdu. In the current version, this ontology is developed for the system description of the educational aspect of mathematical knowledge. The initial ontology design of OntoMathEdu is based on the OntoMathPro ontology developed by us earlier and described above. A new conceptualization has been created, reflecting the conceptual system of mathematics that corresponds to school education. Professional terminology has been adapted to educational activities, in particular, the language of school mathematics. Relationships reflecting the didactic dependence between the concepts have been added to OntoMathEdu. Ontology concepts contain their names in English, Russian, and Tatar languages, as well as basic definitions, relationships with other ontology concepts (associative relationships), and links to concepts from external data sets. The OntoMathEdu ontology is built on a set of OntoMathPro basic ontology relationships such as taxonomic relation (ISA); the relationship between the mathematical object and the field of mathematics; the relationship between mathematical objects is “determined by”; the relationship between the task and the method of solving it; a new set of didactic relations was also introduced [49].
When creating the top level of ontology OntoMathEdu, the planimetry section of the school mathematics course was selected as a pilot: the current version of the ontology contains 585 concepts related to the planimetry course of 5–9 classes of secondary school. The ontology structure contains type hierarchies; hierarchies of materialized relationships; hierarchy of roles and network of points of view. The specificity of school geometric knowledge was taken into account, therefore, when designing ontology, a number of relations between the concepts were singled out: “whole–part”, “determined”, relation of ontological dependence, “theorem– property”, “theorem–characteristic”, “found by formula” (see also [50, 51]). 5
This paper describes approaches to managing large collections of digital mathematical documents that are based on semantic methods and are consistent with the principles of the World Digital Mathematical Library (WDML). These approaches and methods fully relate to the areas that make up the new concept of Big Math. They are being developed and practically implemented as part of the creation of the Lobachevskii-DML digital math library. The main results mentioned are as follows.
Methods for the formation of digital collections from a set of documents – scientific articles, monographs, reports presented in various storage formats are proposed. Based on the analysis of the structure of documents and the stylistic features of their design, an algorithm for extracting their metadata has been developed.
In connection with the increasing role of metadata in the improvement of modern scientific communications, both methods for extracting missing metadata from documents and methods for converting already created metadata into the formats of relevant scientometric databases have been developed and described.
A software tool has been developed for dividing collections of articles into separate documents and forming their semantic presentation. For example, the set of “Proceedings of N.I. Lobachevskii Mathematical Center”, which have a different format and structure, describes an algorithm for creating a digital collection and its inclusion in the Lobachevskii-DML digital mathematical library.
Algorithms for enriching the electronic collections of the Lobachevskii-DML digital library and generating metadata of documents of these collections in selected formats are presented.
Services for the normalization of the collection metadata of the LobachevskiiDML digital library have been developed in accordance with the DTD rules and NISO JATS and DBLP XML schemas. By normalization, we mean the use of methods for generating or transforming document metadata in accordance with the rules and XML schemas of digital libraries and scientometric databases.
Algorithms for creating a mandatory and fundamental collection of metadata collections are presented in accordance with the rules of the European digital mathematical library EuDML.
The digital ecosystem OntoMath, which is the most important part of the Lobachevskii-DML digital library, is described. The main components of this ecosystem are: mathematical ontologies Mocassin, OntoMathPro and OntoMathEdu, the semantic publishing platform, the semantic search service OntoMathSearch, recommender systems for the selection of scientific classifiers, search for related articles and terminological annotation.
This work was funded by the subsidy allocated to Kazan Federal University for the state assignment in the sphere of scientific activities, grant agreement No.1.2368.2017. The work partially contains the results of the project “Monitoring and standardization of the development and use of technologies for storing and analyzing big data in the digital economy of the Russian Federation”, carried out as part of the program of competence of the National Technological Initiative “Center for storing and analyzing big data”, supported by the Ministry of Science and Higher Education Of the Russian Federation under the Treaty of Moscow State University named after M.V. Lomonosov with the Project Support Fund of the National Technological Initiative dated 08/15/2019 No. 7/1251/2 019. The work was also carried out with the partial support of the Russian Fund for Basic Researches and the Government of the Republic of Tatarstan within the framework of scientific projects 18-29-03086, 18-47-160012. 35. Batyrshina, R.R.: Method for extracting terms in digital mathematical collections.
Proc. of the N.I. Lobachevskii Math. Center. Kazan: Kazan Math. Soc. Publ. 55, 24–26 (2017). 36. Standard ECMA-376 Office Open XML File Formats. http://www.ecmainternational.org/publications/standards/Ecma-376.htm. Last accessed 16 May 2019 37. Ingersoll, G.S., Morton, T.S., and Farris, A.L.: Taming Text. How to Find, Organize, and Manipulate It. Manning Publications Co. (2013). 38. Sabitova, E.M.: Algorithm for extracting connections in scientific digital collections. Proc. of the N.I. Lobachevskii Math. Center. Kazan: Kazan Math. Soc.
Publ. 55, 123–126 (2017). 39. Bouche, T. and Ra´kosn´ık, J.: Report on the EuDML External Cooperation Model.
In: Kaiser K., Krantz S.G., Wegner B. (eds.) Topics and Issues in Electronic Publishing, JMM, Special Session, San Diego, 99–108 (2013). 40. Jost, M., Bouche, T., Goutorbe, C., and Jorda, J.P.: D3.2: The EuDML metadata schema. http://www.mathdoc.fr/publis/d3.2-v1.6.pdf. Last accessed 16 May 2019 41. EuDML metadata schema specification (v2.0-final). https://initiative.eudml.org/eudml-metadata-schema-specification-v20-final.
Last accessed 16 May 2019 42. Journal Article Tag Suite. NISO JATS V1.0. https://jats.nlm.nih.gov/1.0/. Last accessed 16 May 2019 43. Khaydarov, S. and Yamalutdinova, G.: Recommender System of Physical and Mathematical Documents Classification. CEUR Workshop Proceedings 2260, 480– 486 (2018). 44. Elizarov, A., Kirillovich, A., Lipachev, E., and Nevzorova, O.: Digital Ecosystem OntoMath: Mathematical Knowledge Analytics and Management. CCIS 706, 33– 46. Springer (2017). https://doi.org/10.1007/978-3-319-57135-5 3 45. Lange, C.: Ontologies and languages for representing mathematical knowledge on the Semantic Web. Semantic Web 4 (2), 119–158 (2013). https://doi.org/10.3233/SW-2012-0059 46. Solovyev, V. and Zhiltsov, N.: Logical Structure Analysis of Scientific Publications in Mathematics. Proc. of the Int. Conf. on Web Intelligence, Mining and Semantics (WIMS’11). ACM 21, 1–9 (2011) 47. Elizarov, A.M., Zhizhchenko, A.B., Zhil’tsov, N.G., Kirillovich, A.V., and Lipachev, E.K.: Mathematical Knowledge Ontologies and Recommender Systems for Collections of Documents in Physics and Mathematics. Doklady Mathematics 93 (2), 231–233 (2016). https://doi.org/10.1134/S1064562416020174 48. Nevzorova, O., Zhiltsov, N., Kirillovich, A., and Lipachev, E.: OntoMathPRO Ontology: A Linked Data Hub for Mathematics. CCIS 468, 105–119. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11716-4 9 49. Elizarov, A., Kirillovich, A., Lipachev, E., Nevzorova, O., and Shakirova, L.: Open Linked Data and Ontologies in Mathematics Education. CEUR Workshop Proceedings 2260, 186–196 (2018). 50. Kirillovich, A., Shakirova, L., Falileeva, M., and Lipachev, E.: Towards an Educational Mathematical Ontology. L. G´omez Chova, et al. (eds). 13th International Technology, Education and Development Conference (INTED2019), Valencia, Spain, March 11-13, 2019. IATED, 6823–6829 (2019). 51. Elizarov, A.M., Lipachev, E.K., and Khaydarov, S.M.: Method of automated selection of reviewers of scientific articles, implemented in the scientific journal information system. Proceedings of the 21th Conference Scientific Services & Internet (SSI 2019), Novorossiysk-Abrau, Russia, September 23-28, 2019.