Native Language Identi cation is the process of determining the language native to the author from the written text. We have proposed a system that uses machine learning algorithms to identify the native language from the written text. We extracted Term frequencyinverse document frequency (Tf-idf) as the feature from the given document and used statistical based measures such as Analysis of Variance -F value measure, Chi - square measure for selecting the best features. The selected features are fed to Multi Layer Perceptron and Stochastic Gradient Descent classi er to classify the native language into one of 6 listed Indian languages. This work was submitted to Indian Native Language Identi cation task INLI@FIRE2018. We have investigated the performance of the proposed system using three classi ers namely Multi Layer Perceptron (MLP) classi er with Analysis of Variance -F value measure, MLP classi er with Chi - square measure and Stochastic Gradient Descent classi er with Chi - square measure. From the results we have observed that SGD classi er with Chi - square measure has performed better than the other two classi ers.
Native Language Identi cation (NLI) is the process of identifying the native language of the author based on written texts in an another language. In uence of one language can a ect the usage of other language by a same speaker referred to as cross-linguistic in uence (CLI). CLI has played an important role in Second Language Acquisition (SLA) which studies and examines the e ects of one language on other learned languages.
The in uence of the native language will be re ected in the text through the usage of patterns. Identifying such patterns forms the basics of NLI. NLI has been identi ed as a multiclass classi cation task, for which various traditional machine learning approaches can be used to identify the native language. NLI nd its applications in educational scenarios, where feedback can be provided to language learners, in authorship pro ling, security, personalized grammar correction etc.
The proposed work for NLI of 6 languages has used three models such as
Multi Layer Perceptron classi er with Analysis of Variance -F value measure,
MLP classi er with Chi - square measure and Stochastic Gradient Descent
classi er with Chi - square measure. This proposed work is submitted for the shared
task on INLI @FIRE 2018 [
This section gives the overview of similar work done in the led of NLI. Most
of the widely used features for NLI [
In [
In [
Three advanced ensemble models are evaluated in [
The other work that supports the ensemble of classi ers for NLI is [
In [
Another work that used ensemble of classi ers that also a participant of NLI
Shared Task 2017 is reported in [
The task of Indian Native Language Identi cation (INLI) has been carried out
using machine learning algorithms. Three runs have been submitted for this
INLI@FIRE2018 task. First submission has used Tf-idf features with Analysis
of Variance (ANOVA) F-values for selecting best features. These features are
trained using Multi Layer Perceptron (MLP) classi er. Second submission used
Tf-idf features with Chi - square value feature selection method. These features
are trained using same MLP classi er. Third submission used the Tf-idf features
with Chi - square value feature selection method. These features are trained using
Stochastic Gradient Decent (SGD) classi er. The details of the tasks have been
explained in the following sections. The proposed approach uses the following
steps to carry out the INLI task.
(i) Data preparation
(ii) Feature extraction
(iii)Feature selection and
(iv) Classi cation
The architecture of the proposed system is shown in gure 1.
The dataset used for this research are Facebook comments which are presented
in the form of XML les. From this XML le, the relevant text portions, that
is, text given between the comment tags are extracted. From the extracted text,
special symbols, punctuation symbols are removed before processing the input
text. Minidom library from XML.dom package [
Tf-idf (Term frequency-inverse document frequency) is used in the proposed approach to represent the feature of the given text. The Tf-idf weight is the one that is used widely in natural language processing applications. It is a statistical measure which is used to assess how signi cant is a word is to a document in a corpus. The signi cance increases as the number of times a word occurs increases in the document. The feature extraction is done using the tool TF-IDF vectorizer method from the scikit learn library.
From the extracted Tf-idf features, the best features are selected using two statistical feature selection algorithms namely Analysis of Variance -F value measure (ANOVA) and Chi-square methods in the proposed approach. ANOVA and Chisquare methods are lter based feature selection methods. These methods select the best discriminative features from the training data. These statistical feature selection methods are applied on the categorically independent features. The analysis of variance is a statistical inference test that compares multiple groups at the same time. The chi-square test is a statistical test of independence to determine the dependency of two variables. Chi-square statistics between every feature variable and the target variable observes the existence of a relationship between the variables and the target. If the target variable is independent of the feature variable, those features are discarded. If they are dependent, the feature variable is considered to be very important. The implementation of feature selection is carried out using f classif and chi2 packages from scikit-learn Python library. 3.4
The proposed approach uses Multi Layer Perceptron (MLP) classi er for submission 1 & 2. Stochastic Gradient Decent algorithm is used for submission 3. MLP is a kind of feedforward neural network with layers namely, input layer, hidden layers and output layer. MLP forms a directed graph where the input vector ows only in one direction through the intermediate layers. Each node has a neuron with a nonlinear activation function excepting the input nodes. With multiple layers, a MLP performs back propagation which is a supervised learning technique that classi es non linearly separable samples. For training with multi layer perceptron network, the following con gurations were used. Number of hidden layers for the network is 2, number of hidden nodes in each hidden layer is 25 as shown in gure 2, the activation function used in the hidden layer is RELU (Recti ed Linear Unit) and Adam optimizer is used for weight optimization.
Stochastic Gradient Decent classi er is a linear classi er used for supervised learning. SGD is the variant of Gradient Decent algorithm which is used to minimize the least square error. The problem with Gradient Descent is when the data set is huge, parameter calculation becomes expensive. The weight is optimized using the following equation.
w = w
OQi(w) (1) In equation 1, Qi(w) refers to the gradient of the prediction error for the model on the training data, w is the weight being optimized and is the learning rate. In SGD, a sample of training set or one training value is used to calculate the parameters instead of the entire sample space on each iteration. When compared to other classi ers, this works much faster. By combining several binary classi ers together in one versus all manner, SGD performs a multi-class classi cation. A binary classi er is trained to learn to distinguish between one and that of the remaining N-1 classes of the total N classes. During the testing phase, for each of the classi er, signed distance to the hyperplane which is referred to as con dence score is calculated and the class with the maximum con dence is chosen. 4
The proposed approach used the data set provided by the INLI@FIRE 2018
shared task [
Two test sets have been provided by INLI@FIRE 2018 shared task for testing. The number of documents in test set 1 is 783. The number of documents in test set 2 is 1185. The cross validation accuracy (with 5 folds) of the proposed work for the three submissions are given in Table 2.
The lack of accuracy or the errors are mainly due to the lack of reasonable number of training samples in the given data set. The performance of the proposed system for Indian language identi cation task have been measured using the metrics namely precision, recall and F1-measure for each language for the two test sets. The overall accuracy of proposed system has also been estimated. The performance of the proposed system using MLP classi er with Analysis of Variance -F value measure is given in Table 3. Class
Similar to the previous proposed model, the model using MLP classi er with Chi - square measure has also identi ed Bengali language with highest F1 score than the other languages in both the test sets.
The performance of the proposed system using SGD classi er with Chi square measure is given in Table 5. SGD classi er with Chi - square measure has classi ed Bengali language with highest F1 score in both test datasets with 65% and 44.4% respectively. When compared with the other two models of the proposed work, SGD classi er with Chi - square measure has achieved overall accuracy of 37% for test set 2 and 46.2% for test set 1 in NLI task. This might be because the linear classi er, SGD which is actually a Bayesian that relies on prior distribution that improves mean squared error and thereby the improving prediction accuracy. This performance of the proposed system using SGD with Chi-square measure has been evaluated to be the best performing model among the INLI@FIRE2018 submitted tasks. Our proposed approach has used a machine learning algorithm to identify the native language of the given document using Tf-idf features. From the extracted features we used statistical measures such as Analysis of Variance -F value measure, Chi - square measure for selecting the best features. These are given to MLP and SGD classi ers to classify the Indian Native Languages. We observed that SGD with chi-square measure has performed better than the MLP classi er with Analysis of Variance -F measure and MLP with chi-square measure. Performance of the proposed work can be improved further by using socio linguistic features from the text.