Abjad numerals, also called hisa¯b al-jumal, is a numeral system based on the twenty eight letters of Arabic but not in the dictionary order. In ancient Arabic mathematics, all problems and solutions sentences were completely expressed in natural language with no mathematical symbolism. The present paper is the first attempt to automatically analyze and recognize Abjad numerals in medieval Arabic mathematical texts. Since that hisa¯b al-jumal system has no ambiguity, we also translate Abjad numeral written in natural language to modern numeral system. We construct a new dataset named Hj-Tagged corpus to facilitate our study. According to the experimental results, the proposed method is eficient for automatically analyze and recognize Abjad numerals and mathematical components (such as numerical constants, Abjad numbers, mathematical operations,.. etc). We also translate Abjad terms detected in the previous step to modern numeral system, where it achieves an F1 score of 98.1%.
dãl(X), the first four letters in the order [
In several medieval Arabic manuscripts such as mathematical, geographical, and astronomical
texts [
When a number is written in hisa¯b al-jumal notation, it becomes dificult to recognize
it as number, especially if the hisa¯b al-jumal word make sense. For example, the equation:
" éJKAK hA¿ ú¯ é®J¯X ñË H. Qå à@ AKXP@" [We wanted to multiply law minutes by kah seconds ]
(Mifta¯h al Hissa¯b [
In this work, we explore the use of a Bi-directional Long Short Term Memory (BI-LSTM) network with a conditional random field (CRF) layer to automatically analyze and recognize Abjad numerals in mathematical expression in medieval Arabic mathematical texts. Additionally, we also translate hisa¯b al-jumal terms detected in the previous step which written in natural language to modern numeral system (such as decimal numbers).
In the past few years, recurrent neural networks (RNN) [
Following this introduction, the remainder of this paper is organized as follows. Section 2 explains the LSTM networks, Bi-LSTM networks, and Bi-LSTM with CRF networks. Section 3 describes how to translate hisa¯b al-jumal to modern numeral system. Section 4 shows the experimental setup such as dataset construction, model architecture, and the training process. Finally section 5 summarizes our methods, results, and discusses the future work.
Recurrent neural networks (RNNs) have proved to be eficient to learn sequential data including
language model [
However, standard RNNs sufer from both exploding and vanishing gradients problems [ 21]. On the other hand, the RNNs with the gating units such as LSTM-RNN [22] are the most efective sequence models in practical applications by adding extra memory cell inherent in RNNs.
The LSTM cell can be described mathematically with the following six fundamental operational stages: ∙ Input Gate: = ( () + ℎ− 1) ∙ Forget Gate: = ( () + ℎ− 1) ∙ Output/Exposure Gate: = ( () + ℎ− 1) ∙ New memory cell: ˜ = tanh( () + ℎ− 1) ∙ Final memory cell: = ⊙ ˜− 1 + ⊙ ˜ ∙ Final hidden state: ℎ = ⊙ tanh() where is the input vector at time , and ℎ denote the hidden state vector storing all the useful information at (and before) time . The and terms denote weight matrices for each gate. The symbol represents the Sigmoid activation function, ⊙ is the element wise multiplication.
In this paper, we propose to apply Bi-LSTM neural network [23] instead of a single forward network. In doing so, we can eficiently make use of past features (via forward states) and future features (via backward states) for a specific time frame. Finally, we construct our neural network model by feeding the output vectors of Bi-LSTM into a Conditional Random Field (CRF) layer [24] to jointly decode the best sequence of tags. Consider an input sentence = {0, 1, .., } and = {1, 2, ..., } is the corresponding sequence of tags for sentence . We consider to be the matrix of scores output by the Bi-LSTM network. is of size × , where k is the number of distinct tags, , is transition probability which represents the score of the ℎ tag of the word ℎ, its score defined with the following form [25]:
(, ) = ∑︁ A,+1 + ∑︁ P,
=0 =1 where A is a matrix of transition scores such that A, represents the score of a transition from the tag i to tag j. We use 0 and are the start and end tags of a sentence, then we add to the set of possible tags. A is therefore a square matrix of size + 2.
() denotes the set of possible sequence of tags for . The probabilistic model for sequence CRF defines a family of conditional probability (|) with all possible sequence of tags under the given with the following form: (|) = ∑︀˜∈ () (,˜) (,) During the training, log probability of correct tag sequence log (|) is maximized.
Figure 1 illustrates the main architecture of our neural network model for medieval mathematical entity recognition system in which each word is tagged with other (O) or one of six entity types: hisa¯b al-jumal (H-jumal), root (Root), square (Square), cube (Cube), equal (Equal), and operation (Op). The sentence of "AÒëPX ÈYªK èP@Yg. @ l k. ð ÈAÓ" [A square and ℎ¯ roots are equal to thirty sïn dirhems], is tagged as {Square Op H-jumal Root Equal H-jumal O}.
In the following, we translate some of the Arabic basic mathematical terms and notations used throughout medieval period into modern symbols: ∙ Shay’ ("Zúae ") or jidhr ("PYg. "), refer to unknown value(). ∙ Ma¯l ("ÈAÓ") and ka’b (" I.ª»") represent respectively 2 and 3. ∙ Powers greater than or equal to four can be formed by combining the two words ma¯l and ka’b. For example, ma¯l ma¯l (4), ma¯l ka’b (5), and so on. ∙ Dirhams ("AÒëPX") or mina al’adad ("XYªË@ áÓ") represent a simple number. ∙ The verb ’a¯dala ("ÈXA«") is used to indicate equality ("=") in an equation. ∙ The one-letter word wa ("ð") take the meaning of the modern addition (“+”) depend on the context. ∙ Hisab al-jummal system is often used to describe numbers during the medieval Arabic period.
The numerological calculation of hisa¯b al-jumal terms requires a dictionary approach to relate every letter in the Arabic alphabet to its equivalence in a number format (see Table 1). A dictionary approach is necessary to recognize each letter and its numerical value as shown in the table below.
Let be the sequence of words {1, 2, .., }, to capture a correspondence between the word and its numerical value , we define an alignment to be a set of pairs (, ), where is a token in and is sum total of letters value. For example, consider the following sentence : Ma¯lan wa ka¯b‘ ‘ashya¯‘ ta’dilu nüd‘ dirhaman (“AÒëPX YK ÈYªK ZAJ@ I.» ð àBAÓ”) [Two ma¯ls and kab things equals nad dirhams], given the above definitions, and knowing that the terms kab (" I.»") and nad ("YK") are hisa¯b al-jumal, once is calculated over all hisa¯b al-jumal words {1=(YK, 50+4) ,2=( I.», 20+2)}, can be written as “AÒëPX 54 ÈYªK ZAJ@ 22 ð àBAÓ” [ Two ma¯ls and twenty-two things equals fifty-four dirhams ]. We have shown that the passage from a sequence of words of any length to its numerical value notation is quite easy. However, no ambiguity is possible because there are exactly one unique translation of to .
Consider the previous example sentence (see section 2):
"AÒëPX ÈYªK èP@Yg. @ l k. ð ÈAÓ".
Which was tagged with: {Square Op H-jumal Root Equal H-jumal O}. By applying the numerological calculation of hisa¯b al-jumal to the words tagged as H-jumal, this sentence is transformed
In this section we first present our proposed architecture, shown in Figure 2, for automatically recognize and translate hisa¯b al-jumal entity in medieval algebraic equations and expressions. Next, we will discuss the construction of a new Hj-Tagged Corpus and the training detail followed by their results.
We evaluate our proposed system on Hj-Tagged Corpus, constructed from the AMAK Dataset
[
Table 3 shows some examples from the Hj-Tagged Corpus which consists of sequences of words and their tags. For evaluation, we report the precision, recall, and F1 scores for all tagged entities in the test set.
To ensure that the model does not see the context from the testing set during training, we ifrst split the training, validation, and testing set on our collected dataset. The size of the split of our collected data into training, validation, and testing is 2,062, 100, 100 respectively.
The Bi-LSTM-CRF model were implemented using the TensorFlow and Keras [29], a flexible neural network library written in Python. The general settings of our neural network model are listed below: ∙ Dimension of word embedding vector: 20. ∙ Dimension of hidden layer: 50 (for each LSTM: forward layer and backward layer). ∙ Learning method: SGD optimizer, learning rate: 0.01 ∙ Number of examples used in each iteration(BATCH SIZE): 30. Tgas sequences (from left to right)
∙ We fix dropout rate at 0.5 for all dropout layers through all the experiments. ∙ Supervised learning was applied with up to 100 epochs for training the network.
In what follows (see Table 4), we use classification metrics such as precision, recall, and F1-score to evaluate our methods. F1-score can be computed to evaluate the performance of the system based on the detection results of the machine and the results of a human evaluator. F1-score is the harmonic mean of Precision and Recall computed from the number of mispronunciations detected by both the computer and human evaluator. They are defined as
2 × × 1 = (1)
+
Precision = TP/(TP + FP) is the fraction of all positive predictions that are true positives, while Recall = TP/(TP + FN) is the fraction of all actual positives that are predicted positive. More precisely, the True Positive (TP), in this system, is the number of Abjad numbers the system got right, False Positive (FP) is the number of Abjad numbers wrongly selected, and False Negative (FN) are the Abjad numbers wrongly classified as no Abjad numbers.
First of all, we notice that Bi-LSTM-CRF network performs remarkably well on the Hj-Tagged Corpus with a mean F1 score of 98.1%. Additionally, using the same parameters, we compare Bi-LSTM-CRF model performance to a Bi-LSTM network. We show the precision, recall, and F1 scores of the models. One can see that adding CRF layer significantly improved prediction. Besides that, the training phase require less than 60 epochs to converge and it in general takes a few minutes. Finally, our experimental results suggests that Bi-LSTM-CRF network are less sensitive to training data size and the impact of noise from the tags.
This paper focuses on recognizing and tagging components of a mathematical expression in medieval Arabic text. First, we want to mention here that our model was trained only on the Hj-Tagged Corpus. The training set is small, this limits the amount of ensemble diversity, which may reduce the network ability to generalize on new testing examples. Second, we did not perform any dataset preprocessing, apart from replacing every decimal number in the collected equations with a random numerical entity in the hisa¯b al-jumal system.
An other important point is that manually tagging such dataset with limited-vocabulary makes the system extremely sensitive to noise.
On the other hand, our model was able to correctly predict sentences which contain ambiguities in the test phase. For example, the word wa ("ð") can mean the addition operator such as (" èP@Yg. @ ð ÈAÓ" [Square Op H-jumal Root]), or hisa¯b al-jumal entity such as (" èP@Yg. @ ð ð ÈAÓ"[Square Op H-jumal Root]).
Finally, we implement a simple tag-based method to translate all hisa¯b al-jumal terms detected in the previous step to modern numeral system using the methodology described in Section 3. For example: The sentence of " èP@Yg. @ l k. ð ÈAÓ AÒëPX ÈYªK" [Square Op H-jumal Root Equal
This paper experimented the first attempt to automatically analyze and recognize Abjad numerals in medieval Arabic mathematical texts using the Bi-LSTM CRF model. We also translate hisa¯b al-jumal terms detected in the previous step to modern numeral system. An additional key strength of this work is the time and efort spent on manually building a new dataset named Hj-Tagged Corpus, which consists of 2,262 tagged medieval mathematical sentences.
In the future, we can improve the intermediate representations learned in our network by training this model jointly with named entity recognition (NER) tags. We also plan to enrich the training examples by expanding Hj-Tagged Corpus. Another interesting direction is to apply our model to data from other Arabic sources in many diferent fields, such as geography, physics, chemistry, medicine, architecture, Astronomy, and so on.
Experimental results on the Hj-Tagged Corpus demonstrate that the proposed method ofers an important step in medieval Arabic mathematics analysis to enable scientists to understand and explore medieval mathematical texts. [18] Mikolov, Tomáš, et al. (2011) "Strategies for training large scale neural network language models." 2011 IEEE Workshop on Automatic Speech Recognition & Understanding. IEEE. [19] Jackson, Richard G., et al. (2017) "Natural language processing to extract symptoms of severe mental illness from clinical text: the Clinical Record Interactive Search Comprehensive Data Extraction (CRIS-CODE) project." BMJ open 7.1 : e012012. [20] Swartz, Jordan, et al. (2017) "Creation of a simple natural language processing tool to support an imaging utilization quality dashboard." International journal of medical informatics 101 : 93-99. [21] Bengio, Yoshua, Patrice Simard, and Paolo Frasconi. (1994) "Learning long-term dependencies with gradient descent is dificult." IEEE transactions on neural networks 5.2 : 157-166. [22] Hochreiter, Sepp, and Jürgen Schmidhuber. (1997) "Long short-term memory." Neural computation 9.8 : 1735-1780. [23] Graves, Alex, and Jürgen Schmidhuber. (2005) "Framewise phoneme classification with bidirectional LSTM networks." Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005.. Vol. 4. IEEE, . [24] John Laferty, Andrew McCallum, and Fernando CN Pereira. (2001). Conditional random ifelds: Probabilistic models for segmenting and labeling sequence data. In Proceedings of ICML2001, volume 951, pages 282–289. [25] Lample, Guillaume, et al. "Neural architectures for named entity recognition." arXiv preprint arXiv:1603.01360 (2016). [26] MUŠARRAFA, Alı Mustafı et AHMAD, Muhammad Mursı. Al-Khwa¯rizm¯ı. Kitab almukhtasar fı hisa¯b al-jabr wa’l-muqabalah, 1939. [27] Roshdi Rashed, Al-Khwa¯rizm¯ı, Le commencement de l’algèbre, éd. 2009. [28] "Muhammad Ba¯qir Zayn al-’ A¯bid¯ın al-Yazd¯ı", ’Uyun al-Hissab (Les Fontaines du calcul), Manuscript undated - Harvard University, http://pds.lib.harvard.edu/pds/view/11328976? n=1&imagesize=1200&jp2Res=.25&printThumbnails=no. [29] TensorFlow’s implementation of the Keras, https://www.tensorflow.org/guide/keras,