Natural Language processing (NLP) in its pure sense, is a platform that provides the ability for transforming natural language text to useful information. Named Entity Recognition (NER) is a key task in NLP for classification of named entities in natural languages. Though, there are several algorithms for named entity classification, identifying named entities in twitter data is a demanding task. Loads of information are being shared by people in twitter on a daily basis. This information is unstructured and often contains important information about organizations, politics, disasters, promotional advertisements etc. In this paper, we provide a NER that can effectively classify named entities in twitter data for Indian Languages such as English, Hindi and Tamil. POS, Chunk, Suffix, Prefix information has been used for training in Conditional Random Fields (CRF) based NER Model. CRF is a popular model for labeling and classification in text mining. Performance analysis was done using n-fold validation and F-measure. A maximum precision of 93.82 for English, 92.28 for Hindi and 86.94 for Tamil twitter data was achieved through N fold validation. Results provided by ESM-IL share task in terms of precision for English is 50.48, for Hindi is 81.49 and for Tamil 70.42. The proposed algorithm has a higher classification accuracy and it is achieved through n-fold validation.
NLP is gaining prominence due to the importance given to social
media data such as twitter and Facebook. Twitter data are
predominantly extracted and monitored by public and private
organizations for analysis of various trends in the industry and for
opinion mining. For efficient NLP, corpus should be from the
same domain of the NLP application [
NER is important for Indian language twitter data, as
there is no clue in identifying named entities in them. Recognizing
named entities in social media data is quite challenging due to the
unstructured clauses in a sentence and entities are more diverse in
nature. There are a variety of approaches for recognizing named
entities. Supervised approaches include Hidden Markov Model
(HMM), Decision Trees, Maximum Entropy Model (MEnt),
Support Vector Machines (SVM) and CRF etc.. Here, for the
ESM-IL task, CRF has been used to develop NER for English,
Hindi and Tamil twitter data [
Twitter is one of the popular social networking sites where people share their opinions through their tweets. Each tweet or post can contain a maximum of 140 characters, including smilies, hash tags, other symbols and website links. We attempted to develop a twitter based NER for English, Hindi and Tamil data. Twitter training data were provided by ESM-IL for this task. Training data included raw data files and annotated named entities. Before pre-processing, annotated named entities were mapped with the corresponding tweets in the raw data file, to label the named entities. Group of labels is called as Tag set. A total of 22 tags were identified from the training data, namely person, location, organization, date, quantity, money etc.. Pre-processing has been done before applying the methodology. It included tokenization and removing noisy data. 3.
CRF, a probabilistic approach for developing NER is used for classification. CRF is a popular approach for effectively classifying named entities. It takes into account the neighboring samples or the context information of the sentence. But the disadvantage in twitter data is, lack of context information. Tweets in the raw data file were tokenized and pre-processed by removing noisy data website links, hash tag, and smilies etc. was done. Initially, neighboring samples were only considered in developing the model for NER. Later to include lexical information, POS taggers, Chunker and 1st, 2nd, 3rd degree suffixes and prefixes were also added as features. The methodology of the proposed algorithm is shown in Figure 1.
CRF generated NER tagger
Input considered for NER were raw and annotated data provided by ESM-IL shared task. To avoid misclassification of named entities, labelling was done by assigning the first word of the entity with B-tag (Begining of the tag) and remaining words in the same entity with I-tag (Inside the tag). With the help of the annotated data, named entities in the raw data were mapped. For pre-processing and in order to identify the noisy data rules were framed. For eg., detection and removal of website links was performed by checking for the presence of tokens such as http//:, .com.
For English parts of speech tagging and chunking,
pattern.en module was used [8]. It is a python based POS tagger
cum chunker for twitter data. Training in a twitter based POS
tagger provides better accuracy than a normal tagger. So pattern
POS tagger was preferred. Due to the unavailability of twitter
based POS tagger, for Hindi parts of speech tagging the tagger
provided by Society for Natural Language Technology Research
was used [9]. It is a CRF based open source software. Apart from
the POS and Chunk taggers, other features like suffixes and
prefixes were also used. The list of features used are provided in
Table 1.
For n-fold cross validation, the annotated data was randomly
portioned into five sub sets. Validation was performed by n
rounds of testing the system by providing one set of the annotated
data for testing each time and the remaining n-1 subsets for
training [
The performance metrics used for analysis are Precision, Recall and F-measure. Precision and Recall are the most effective and frequently used measures in case of information retrieval.
F-measure is defined as the weighted harmonic mean of precision and recall. where, True positives are the total number of NE’s tagged correctly with boundaries.
False positives are the total number of words that are wrongly tagged by the system which are not tagged manually. False negatives are the total number of untagged words by the system which are manually tagged [12].
In this section, the results obtained with the CRF model is discussed. Twitter data for experiment was provided by FIRE 2015. Table 2 compares the results obtained using n-fold validation and the results provided by ESM-IL shared task. As a part of the initial pre-processing website links, hash tags and smilies were removed from the raw data. Further, POS tagging and chunking for English data was done. This was given to the CRF model and a test run was generated. A maximum F-measure of value 80.81 was obtained. To improve the accuracy features like prefixes and suffixes of the NE were included and a maximum F-measure of 86.66 was obtained.
For Hindi, after initial pre-processing, POS tagging was done. An efficient chunker is not available for Hindi data, and therefore chunking was not done. This was supplied as input to the CRF model and a maximum F-measure of 83.49 was obtained. Prefixes and suffixes of NEs were also combined with the existing POS tags and 78.47 F-measure value was obtained. When considering only prefixes and suffixes of NE as feature, the model resulted in an F-measure of value 73.12. The classification accuracy reduced for Hindi when compared to English due to the unavailability of twitter based POS tagger and chunker.
For Tamil data, since there is no proper POS tagger and chunker available of twitter and generic data, features like suffixes and prefixes of NE were considered and the model obtained a maximum F-measure 79.87. Excluding the suffixes and prefixes, considering only the tokens as a feature, the model gave an Fmeasure of 62.95. The classification accuracy reduced for Tamil data similar to Hindi due to the unavailability of Twitter based POS tagger and chunker. Table 3 shows the examples for the results obtained from CRF generated model for English, Hindi and Tamil twitter data. Sample tweets contain tweet-id, user-id and the tweet. 1. 2. 3. 1. 3. 624133739023446016 917553836 As outrage builds, Karnataka Vikas Grameena Bank looks at rescuing farmers: Much of the farming community's outrage… http://dlvr.it/BcS9Kh 623537594798739456 3016932104 विश्व फाइनल की मेजबानी करेगा ईनड गान्सड http://hindi.webdunia.com/latest-cricketnews/twenty20-world-cup115072100078_1.html … pic.twitter.com/pqzyc mT9b8 टी20 621569431932530688 மாவட்டம் கலல்ிடலயறிசக்ு பாைதத்ால் நெலல்ை
நபாமதுகக்ள் … http://goo.gl/fb/Efqp0N 317752766 நெலல்ை றிதக்கபப்ட்ட மழ்சகி :
NEs identified in tweets
2. विश्व
टी20: I-ENTERTAINMENT ईनड गान्सड
நெலல்ை மாவட்டம் கலல்ிடலயறிசக்ு B-LOCATION
B-LOCATION
Named Entity Recognizers for English, Hindi and Tamil were developed for the twitter data. A CRF based model was generated by POS tagging, chunking and applying other feature information to the given data. To test the accuracy of the CRF model n fold validation was done. N fold experiment for training data gave a maximum precision of 93.82 for English, 92.28 for Hindi and 86.94 for Tamil twitter data. ESM-IL evaluation for English, Hindi and Tamil data resulted in competitive precisions of 81.49, 70.42 and 50.21 respectively. Due to the increase in the percentage of recall, F-measure was reduced. This increase in recall can be reduced by providing pre-processed rules based on lexical resources.
The authors thank Hindustan Institute of Technology & Science for their continuous support.
11. https://taku910.github.io/crfpp/