Syntactic Parsing present a fundamental step in the process of automatic analysis of the language since it is the crucial task of determining the syntactic structures sentences. In this paper, we propose to syntactically analyze sentences for the Arabic language using deep learning techniques. We present our methodology and expose evaluation results using several deep learning architectures.
implementing the deduced grammar (results of the first phase) to perform the syntactic analysis. They tested this analyzer with a set of sentences taken from the ATB Treebank (1650 sentences). The authors found an accuracy of 83.59%, a recall equal to 82.98% and an F-measure equal to 83.23%.
[2] proposed to use an Arabic property grammar to evaluate and enrich the Stanford Parser. In fact, this parser is based on the verification of the satisfaction of the syntactic constraints, also called properties, based on the analysis results of the corpus. Moreover, they enriched the simple representation of the result of the analysis with syntactic properties. This makes it possible to clarify several implicit information which present the relations between syntactic units.[2] obtained an F-score of 77.62%. with a recall value of 70.2% and a precision of 86.81%.
Recently,[
[
The statistical approach is based on statistical models. We have not found recent works using deep
learning for syntactic parsing of MSA. On the other hand, we found works dealing with French and
English. We can cite the work of [
[
[
[
= 92%, = 84% − = 88.27% F-score = 80.7 % for the data tags and 78.3 % for the predicted tags
F-score=95% F-score=83,24% F-score=85.32%
This section presents the general architecture of our proposed approach.
Our statistical method for parsing Arabic based on deep learning techniques is divided into three steps: the preprocessing step, the training step and the validation and testing step. The first step presents the data preprocessing by extracting the syntactic levels for each sentence of the data. The second, does the training of our models and finally, the third step is the step of validation and testing. The steps of our proposed method are illustrated in the following figure:
The preprocessing step consists of preparing data for the training step.
In this work, we used the annotated Penn Arabic Treebank (ATB) corpus [
The ATB corpus comprises data extracted from linguistic sources written with the standard and
modern Arabic language. It behaves 599 texts taken from the Lebanese newspaper ≪ An Nahar≫. The
texts are non-vowel or partially vowel and segment. Each word is annotated with several information
such as the morphological trait, the part of speech and the English translation. Also, it contains the
syntax trees for each sentence [
In addition, The ATB corpus is very rich in information, such as gender, number and rationality. It deals with large and important annotations. On the one hand, the corpus encompasses a set of 498 annotations to describe all the morphological functionalities, it also uses 22 annotations to describe grammatical classes. There are 20 other notations, which describe the semantic relationships between words. On the other hand, stop words are also marked (stop words exist with a large number in Standard Arabic) with specific annotations. In our work, we use version 3.2 which contains 402,291 words. This version has been created with several different formats to help the user for his research: SGM, POS, XML, penntree, Integreted.
For the preprocessing step of our ATB corpus, we mainly used the two formats ≪ XML ≫ and ≪ Tree ≫ because they have a hierarchical and readable representation and in addition, they contain the essential information for our work objective and for experimentation.
We have presented the eight syntactic levels with a set of morpho-syntactic labels for each sentence
extracted from the corpus ATB [
Features. These features indicate the information used from the annotated corpus during the training step, which is the morphological annotations in the syntactic annotations. In fact, we extracted the morpho-syntactic labeling from ATB. And, we have reduced the number of labels in order to avoid complexity and to be prepared to learn and predict.
For example: NP+ ADJP: NP. 3.2.
Many machine learning algorithms need a vector representation as an input. In this work, we have used embedding resources such as input from our neural systems. we have represented our data resource with word2vec.
Word2vec is a popular method of constructing word embedding. It was introduced by [
The skip-gram aims to predict the words of the context given an input word [
In this work, we applied the bag of words (BOW) representation because it is the most popular method for NLP applications.
After obtaining the word embedding with the word2vec method, we start the training by applying the deep learning techniques: LSTM, GRU, BI-LSTM. These models will present results after training and we will compare them to choose the model having a better result to apply it to perform parsing.
Today, neural networks present the systems most used in different machine learning tasks. They are widely used, for example, in the fields of computer vision (image classification, detection of an object, segmentation, etc.) and automatic language processing (automatic translation, voice recognition, language models, etc.).
In this work, we are interested in the creation of a model for each syntactic level. This model has an important objective, is to determinate the different constituents of a sentence and the different relations between them.
• Long Short-Term Memory Network (LSTM) presents a building unit for the layers of a recurrent neural network (RNN). An RNN made up of LSTM units is often referred to as an LSTM network. A common LSTM unit is presented with a cell, an input gate, an output gate and a forget gate. The cell is responsible for” storing” values. Each of the three gates can be considered as a” conventional” artificial neuron, as in a multilayer neural network (or feedforward): that is to say, they calculate an activation (using an activation function) of • • a weighted sum. Indeed, they can be considered as regulators of the flow of values. There are connections between these doors and the cell.
Gated Recurrent Unit (GRU) is a gated recurrent network similar to the LSTM network, but it receives fewer parameters than it.
BILSTM is a Bidirectional LSTM, present a sequence processing model that include two LSTMs: one is receives the input in a forward direction, and the other in a backwards direction. BILSTMs increase the amount of information available on the network and improve the context available for the algorithm.
The evaluation of our method is realized in a validation and test step. To achieve this, we have divided the ATB corpus in two parts, one for learning (70%) and one for evaluation (30%) between validation (0.15%) and testing (0.15%).
We used deep learning techniques for train and test our models for our eight levels. we applied the RNN models: LSTM model, GRU model and BILSTM model.
The results are illustrated in table 2. The table shows analysis performance by levels.
LSTM-model 99,00% 99.18% 98,84% 99,09% 99,36% 99,21% 99.25% 99.26%
GRU-model 99,08% 99.31% 98,91% 99.15% 99.39% 99.28% 99.32% 99.35%
BILSTM-model 99,60% 99.53% 99,17% 99.36% 99.60% 99.49% 99.57% 99.60%
We have obtained good results which are encouraging to realize the syntactic parsing for the standard Arabic language with a deep learning model. We can compare the results obtained for the models: LSTM, GRU, BILSTM. We deduced that that the models BILSTM have best results.
In this paper, we presented our numerical method to perform parsing for Standard Arabic using deep learning techniques. We have built a model and we obtained encouraging results. As a perspective, we think that we can develop the learning stage by adding other features besides the POS features. we believe that the enrichment of the features can give better results. We plan to evaluate our method with another external Arabic corpus.
[1] C. Aloulou.Une approche multi-agent pour l’analyse de l’arabe: Modélisation de la syntaxe.PhD thesis, Thèse de doctorat en informarique, Ecole Nationale des Sciences de l ..., 2005.
[2] R. B. Bahloul, N. Kadri, K. Haddar, and P. Blache. Evaluation and enrichment of stanfordparser using an arabic property grammar. In A. F. Gelbukh, editor,Computational Linguisticsand Intelligent Text Processing - 18th International Conference, CICLing 2017, Budapest,Hungary, April 17-23, 2017, Revised Selected Papers, Part I, volume 10761 ofLecture Notes inComputer Science, pages 170–182. Springer, 2017.