Towards Formula Concept Discovery and Recognition

Abstract

Citation-based Information Retrieval (IR) methods for scientific documents have proven to be effective in academic disciplines that use many references. In science, technology, engineering, and mathematics (STEM), researchers cite less often but employ mathematical concepts to refer to prior knowledge (Moed et al.). Our long-term goal is to generalize citation-based IR-methods and apply the generalized method to both classical references and mathematical concepts. In this paper, we suggest how mathematical formulae could be cited and define a Formula Concept Retrieval challenge with two subtasks: Formula Concept Discovery (FCD) and Formula Concept Recognition (FCR). While the former aims at the definition and exploration of a Formula Concept that names bundled equivalent representations of a formula, the latter is designed to match a given formula to a prior assigned concept ID. Moreover, we present first Machine Learning based approaches to tackle the FCD and FCR tasks, which we apply to a standardized test-collection (NTCIR arXiv dataset). Our FCD approach yields a recall of 68% for retrieving equivalent representations of frequent formulae, and 72% for extracting the formula name from the surrounding text. FCD and FCR will enable citing formulae within mathematical documents and facilitate semantic search as well as similarity computations for plagiarism detection or document recommender systems.

Keywords

• Natural Language Processing
• Mathematical Language Processing
• Mathematical Information Retrieval
• Feature Analysis
• Machine Learning

Introduction

Documents from Science, Technology, Engineering, and Mathematics (STEM) often contain a significant amount of mathematical formulae. Since they are vital to understanding the content of these documents, semantic search engines or recommender systems need to process and analyze them alongside the text. In information science and technology, the semantics of natural language is typically grasped via conceptualization [ 25 ]. In the case of mathematical language, we argue for the introduction of a definition for a mathematical Formula Concept as a collection of equivalent formulae with different representations (see [ 15 ] for a discussion of the definition difficulties). Once defined, the technical implementation of a Formula Concept can be Formula Concept Discovery (FCD) and Formula Concept Recognition (FCR). The first term (FCD) refers to the exploration of formula concepts by examining a multitude of formula examples from various sources and occurrences. Figure 1 illustrates how the same equation, in this case, the Klein-Gordon equation from Quantum Physics, can be represented in different formats that seem very diverse at first glance but actually represent the same mathematical concept. We will present first implementations of FCD and FCR in the following.

Related Work

Mathematical Information Retrieval (MathIR) addresses the information need in STEM fields by retrieving, processing and analyzing mathematical formulae. Up until now, various formula search engines have been developed, and translations between different markups (LaTeX, Presentation, and Content MathML) and standards elaborated [ 5 ]. Since Wikipedia is only semi-structured, Wikidata4 was launched to provide direct access to specific interlingual facts (RDF5 triples) and retrieve information systematically. Wikidata is a free and open semantic knowledge-base that can be read and edited by humans and machines [ 23 ]. Wikidata stores items with statements and their references. In the case of mathematical knowledge, this includes formulae, e.g., pressure (Q39552) with a defining formula property (P2534) p = FS . To scalably seed information into Wikidata, a Primary Sources tool6 was introduced, allowing active users to quickly browse through new claims and their references to approve or reject them. The arXiv.org e-Print server [ 10 ] makes available free preprints for a large collection of publications from Physics, Mathematics, Computer Science, Economics and more. Many authors provide their LaTeX source code. Both Wikipedia and arXiv articles were extracted as part of the NTCIR MathIR Task [ 1 ]. In 2017, the Special Interest group for Math Linguistics (SIGMathLing)7 was initiated as a forum and resource cooperative for the linguistics of mathematical/technical documents. For Mathematical Language Processing (MLP), the formula parts (operators, identifiers, numbers) have to be annotated using the Mathematical Markup Language (MathML). There are several tools available, most prominently the LaTeXML converter8. Furthermore, the occurring symbols (variables, constants) need to be disambiguated, i.e., their meaning inferred from the context and semantically annotated. There have been attempts to automatically retrieve the semantics of identifiers from the surrounding text [ 18 ]. While Wikipedia articles more commonly contain variable definitions in the text, in general, many paper articles often omit them. This leaves the task of manual annotation inevitable for building machine-interpretable datasets. The NIST Digital Repository of Mathematical Formulae (DRMF) [ 3 ] and NIST Digital Library of Mathematical Functions (DLMF) [ 9 ] are two examples of maintained high-quality semantic datasets. At this moment, Wikidata contains approximately 3600 items with a "defining formula" property. Moreover, the benchmark MathMLben [ 17 ] was created to evaluate tools for mathematical format conversion (from LaTeX to MathML to Computer Algebra Systems), containing approximately 300 formulae from Wikipedia, the arXiV and the DLMF, which were augmented by Wikidata macros [ 16 ]. 4 http://www.wikidata.org 5 https://www.w3.org/RDF 6 https://www.wikidata.org/wiki/Wikidata:Primary_sources_tool 7 https://sigmathling.kwarc.info 8 https://dlmf.nist.gov/LaTeXML

Formula Concept Retrieval Challenge

We define as the goal to be eventually able to map all of the various representations of a formula to a unique and open concept ID, e.g., linking all occurrences of the Klein-Gordon equation shown in Figure 1 to the Wikidata item Q868967 9.

We define two subtasks of the Formula Concept Retrieval challenge: – Formula Concept Discovery (FCD) as a method to find common equivalent representations and a name candidate for a given formula, and – Formula Concept Recognition (FCR) as the approach to recognize formulae in documents as being instances of prior defined formula concept. 4

Our Approach

In the following, we present our first efforts to implement and evaluate a Formula Concept Discovery (FCD). We approach FCD by retrieving equivalent formulations with different representations (see Figure 2) as well as name candidates from the surrounding text. The initial step is to identify formula candidates which occur most often within a given dataset, assuming that they are potential seeds of popular formula concepts. We first tried formula clustering but discovered that it was not a suitable method for FCD since the number of clusters is a priori unclear and the tested algorithms were not able to group equivalent formulae. Subsequently, we decided to start with a ranking of formula duplicates (with the same LaTeX string), which yielded reasonable results. We employed the NTCIR arXiv dataset [ 1 ] which is comprised of 104062 document sections containing over 60 million formulae. We confined our computations to the subject class of astrophysics (680 astro-ph documents), employing a domain expert to semantically evaluate the results. From the duplicate ranking, we selected a formula length range between 10 and 30 characters and restricted our selection to duplicates occurring in at least two different documents. This yielded 3495 formulae. We then manually selected all equations, and discarded all stubs without a right-hand-side, as well as simple variable dependence definitions, such as x = x (t) and x = y or x = const. For the first 50 samples from the duplicate ranking, we retrieved the operators and identifiers from the provided MathML <mo> and <mi> tags, as well as the surrounding text (words within a window of ±500 characters around the formula). We encoded both tags using the TfidfVectorizer from the Python package Scikit-learn [ 13 ] and Doc2Vec model [ 8 ] from the Python package Gensim [ 14 ]. We then compared the performance of a k-nearest neighbor classifier (Scikit-learn) on the four resulting vector encodings (math2vec [ 24 ] and math tf-idf for the formulae, semantics2vec and semantics tf-idf for the surrounding text) to retrieve equivalent representations. 9 https://www.wikidata.org/wiki/Q868967

Our Results

Having launched FCD as a method for tagging formulae with Wikidata QIDs, we can now employ FCR to identify formulae within STEM documents using their constituting parts (operators and identifiers) in a SPARQL query10. However, since at the moment only less than 4000 formulae are seeded into Wikidata [ 19 ] and storing multiple representations as "defining formula" of the same formula concept item is not endorsed, we argue for the creation of a specific Wikidataattached Formula Concept Database. It should include formalized augmentation to generate equivalent forms using, e.g., commutations, additional sub- and superscripts, unit and reference frame variations, etc. Most importantly, a method for inferring substitutions or implicit terms needs to be developed.

Hubble’s law (Q179916) equation of state (Q214967) p = ωρ a˙ = aH p = κρ ω = p/ρ pd = ωρd

Hi = R˙ /R

H = a˙/a H(t) = a˙/a 10 W3C Recommendation: https://www.w3.org/TR/rdf-sparql-query

References

1. Aizawa , A. , Kohlhase , M. , Ounis , I. , Schubotz , M. : NTCIR-11 math -2 task overview . In: NTCIR. National Institute of Informatics (NII) ( 2014 )

2. Arbab , A.I. : Derivation of dirac, klein-gordon, schrödinger, diffusion and quantum heat transport equations from a universal quantum wave equation . EPL (Europhysics Letters) 92 ( 4 ), 40001 ( 2010 )

3. Cohl , H.S. , McClain , M.A. , Saunders , B.V. , Schubotz , M. , Williams , J.C. : Digital repository of mathematical formulae . In: CICM. Lecture Notes in Computer Science , vol. 8543 , pp. 419 - 422 . Springer ( 2014 )

4. Detweiler , S. : Klein-gordon equation and rotating black holes . Physical Review D 22 ( 10 ), 2323 ( 1980 )

5. Guidi , F. , Coen , C.S.: A survey on retrieval of mathematical knowledge . Mathematics in Computer Science 10 ( 4 ), 409 - 427 ( 2016 )

6. Haroun , K.M. , Yagob , A.A.M. , Allah , M.D.A. : Derivation of klein-gordon equation for frictional medium . American Scientific Research Journal for Engineering , Technology, and Sciences (ASRJETS) 38(1) , 1 - 6 ( 2017 )

7. Kaloyerou , P. , Vigier , J.: Evolution time klein-gordon equation and derivation of its nonlinear counterpart . Journal of Physics A: Mathematical and General 22 ( 6 ), 663 ( 1989 )

8. Le , Q.V. , Mikolov , T. : Distributed representations of sentences and documents . In: ICML. JMLR Workshop and Conference Proceedings , vol. 32 , pp. 1188 - 1196 . JMLR.org ( 2014 )

9. Lozier , D.W.: NIST digital library of mathematical functions . Ann. Math. Artif. Intell. 38 ( 1-3 ), 105 - 119 ( 2003 )

10. McKiernan , G. : arxiv.org: the los alamos national laboratory e-print server . International Journal on Grey Literature 1 ( 3 ), 127 - 138 ( 2000 )

11. Morawetz , C.S.: Time decay for the nonlinear klein-gordon equation . Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences 306 ( 1486 ), 291 - 296 ( 1968 )

12. Pecher , H.: Nonlinear small data scattering for the wave and klein-gordon equation . Mathematische Zeitschrift 185 ( 2 ), 261 - 270 ( 1984 )

13. Pedregosa , F. , Varoquaux , G. , Gramfort , A. , Michel , V. , Thirion , B. , Grisel , O. , Blondel , M. , Prettenhofer , P. , Weiss , R. , Dubourg , V. , VanderPlas , J., Passos , A. , Cournapeau , D. , Brucher , M. , Perrot , M. , Duchesnay , E.: Scikit-learn: Machine learning in python . Journal of Machine Learning Research 12 , 2825 - 2830 ( 2011 )

14. Rehurek , R.: Scalability of Semantic Analysis in Natural Language Processing. Ph.D. thesis , Masarykova univerzita, Fakulta informatiky ( 2011 )

15. Scharpf , P. , Schubotz , M. , Gipp , B. : Representing mathematical formulae in content mathml using wikidata . In: BIRNDL@ SIGIR . pp. 46 - 59 ( 2018 )

16. Scharpf , P. , Schubotz , M. , Gipp , B. : Representing mathematical formulae in content mathml using wikidata . In: BIRNDL@SIGIR. CEUR Workshop Proceedings , vol. 2132 , pp. 46 - 59 . CEUR-WS.org ( 2018 )

17. Schubotz , M. , Greiner-Petter , A. , Scharpf , P. , Meuschke , N. , Cohl , H.S. , Gipp , B. : Improving the representation and conversion of mathematical formulae by considering their textual context . In: JCDL . pp. 233 - 242 . ACM ( 2018 )

18. Schubotz , M. , Grigorev , A. , Leich , M. , Cohl , H.S. , Meuschke , N. , Gipp , B. , Youssef , A.S. , Markl , V. : Semantification of identifiers in mathematics for better math information retrieval . In: SIGIR . pp. 135 - 144 . ACM ( 2016 )

19. Schubotz , M. , Scharpf , P. , Dudhat , K. , Nagar , Y. , Hamborg , F. , Gipp , B. : Introducing mathqa-a math-aware question answering system . In: Proceedings of the ACM/IEEECS Joint Conference on Digital Libraries (JCDL) , Workshop on Knowledge Discovery, Fort Worth, USA ( 2018 )

20. Strauss , W. , Vazquez , L. : Numerical solution of a nonlinear klein-gordon equation . Journal of Computational Physics 28 ( 2 ), 271 - 278 ( 1978 )

21. Tiwari , S. : Derivation of the hamiltonian form of the klein-gordon equation from schrödinger-furth quantum diffusion theory: Comments . Physics Letters A 133 ( 6 ), 279 - 282 ( 1988 )

22. Tretyakov , O.A. , Akgun , O. : Derivation of klein-gordon equation from maxwell's equations and study of relativistic time-domain waveguide modes . Progress In Electromagnetics Research 105 , 171 - 191 ( 2010 )

23. Vrandecic , D. , Krötzsch , M. : Wikidata: a free collaborative knowledgebase . Commun. ACM 57 ( 10 ), 78 - 85 ( 2014 )

24. Youssef , A. , Miller , B.R. : Deep learning for math knowledge processing . In: Rabe, F. , Farmer , W.M. , Passmore , G.O. , Youssef , A . (eds.) Intelligent Computer Mathematics - 11th International Conference, CICM 2018, Hagenberg, Austria, August 13-17 , 2018 , Proceedings. Lecture Notes in Computer Science , vol. 11006 , pp. 271 - 286 . Springer ( 2018 ). https://doi.org/10.1007/978-3- 319 -96812-4_ 23 , https: //doi.org/10.1007/978-3- 319 -96812-4_ 23

25. Yucong , D. , Cruz , C. : Formalizing semantic of natural language through conceptualization from existence . International Journal of Innovation, Management and Technology 2 ( 1 ), pages- 37 ( 2011 )