Native Language Identification (NLI) is the process of identifying the native language of non-native speakers based on their speech or writing. It has several applications namely authorship profiling and identification, forensic analysis, second language identification, and educational applications. English is one of the prominent language used by most of the non-English people in the world. The native language of the non-English speakers may be easily identified based on their English accents. However, identification of native language based on the users posts and comments written in English is a challenging task. In this paper, we present a neural network approach to identify the native language of an Indian speaker based on the English comments that are posted in microblogs. The lexical features are extracted from the text posted by the user and are used to build a neural network classifier to identify the native language of the user. We have evaluated our approach using the data set given by INLI@FIRE2017 shared task.
Native Language Identification (NLI) is the process of automatically
identifying the native language of a person based on her/his speech
or writing in another language. It has several applications namely
authorship profiling and identification [
We have implemented a supervised approach for this INLI task. The steps used in our approach are given below.
• Preprocess the given text • Extract linguistics features for training data • Build a neural network model from the features of training data • Predict class label for the instance as any of the six languages namely Tamil, Hindi, Kannada, Malayalam, Bengali or Telugu using the model
We have implemented our methodology in Python for the INLI task. The data set used to evaluate the task consists of a set of training data for six Indian languages and test data. The number of training instances are 207, 211, 203, 200, 202 and 210 for the languages Tamil, Hindi, Kannada, Malayalam, Bengali and Telugu respectively and number of test instances are 783. The steps used in our approach are explained in detail in the following subsections. 2.1
As a preprocessing step, all the ’xml’ tags are removed from the given text and only the body part of the given input is considered for further processing. The punctuations like “, ”, - , _, ‘, and, ’ are removed from the text and the terms such as n’t, &, ’m, ’ll are replaced as ‘not’, ‘and’, ‘am’, and ‘will’ respectively before extracting the features. Each term of the text is annotated with parts of speech (POS) information such as noun, verb, adjective, adverb, and determiner. In general, nouns present in the text can be used as features. However, adjectives may also be helpful to identify the native language. For example, from the post ‘I attended my kutty brother Rams birthday party’, the adjective ‘kutty’ may used to identify the language as Tamil. So, in our approach, we have considered nouns and adjectives as features. All forms of nouns (NN*) namely NN, NNS and NNP, and all forms of adjectives (JJ*) JJ, JJR and JJS are extracted from the text. The feature set is constructed by lemmatizing each extracted term and by eliminating all the duplicate terms. We have obtained the bag of words (BOW) by processing all the text of given training data.
We have used the NLTK tool kit1 to preprocess the given data and to annotate the text with POS information. The Wordnet Lemmatizer was used to lemmatize the terms that are extracted from POS information. We have obtained a total of 12067 features from training data. We have used the boolean model to construct the feature vectors for the instances of training data. 2.2
We have applied a neural network approach to identify the native language of the user. The set of BOW features along with the class labels namely Tamil, Hindi, Kannada, Malayalam, Bengali and Telugu from training data are used to build a model using a simple neural network with two hidden layers. The features are extracted for each instance of test data with unknown class label ’?’, similar to training data using the features identified from training data. One of the label from the given labels namely Tamil, Hindi, Kannada, Malayalam, Bengali and Telugu is identified for the test data instances using the built model.
We have used the Keras framework2 with Tensorflow backend to implement a neural network classifier for this problem. The number of BOW features (12067) constitutes the number of neurons for the input layer of the network. We have used a sequential model of Keras to construct our neural network. We have added two hidden layers with number of neurons as 64 and 32 respectively with ’RELU’ activation function. The output layer was added by specifying the number of neurons as 6 (to classify the instance to one of the 6 languages) with an activation function ’SOFTMAX’. We used ’sparse_categorical_crossentropy’ loss function with ’SGD’ optimizer to compile the model. We trained the model with a batch_size of 10 for 100 epochs and obtained a training accuracy of 98.1%. 3
Our approach for native language identification has been evaluated
based on the metrics namely precision, recall, and F1 measure for
each language and also overall accuracy. The results obtained by
our approach are presented in Table 1. A comparative study of
results of all the participants of INLI@FIRE2017 is available in [
We have obtained an overall accuracy of 38.80% using our neural network approach for Indian native language identification task. This is very poor compared to the training accuracy of 98.1% and is an indication of over-fitting. We need to explore using regularization techniques such as dropout during training to avoid this. 4
We have presented a system that uses a neural network model for identifying the native language, namely Tamil, Hindi, Kannada, Malayalam, Bengali or Telugu, of Indians from the English comments posted by them in social media. We have extracted the linguistics features from training data to build a neural network model with two hidden layers. The data set given by INLI@FIRE2017 shared task has been used to evaluate our methodology. We have obtained an overall accuracy of 38.80%. This is very poor compared to the training accuracy and indicates over-fitting. Regularization techniques such as dropout may be used to improve generalization. A lexical database may be used to correct terms such as pls, sry, fyi, etc., present in social media text for improving the performance of the system. The performance may improve if we select only the significant features using
χ 2 feature selection [14].
We would like to thank the management of SSN Institutions for funding the High Performance Computing (HPC) lab where this work is being carried out.