This working note presents a statistical based classi er for text classi cation using entity relationship information present in the input text. We observed that parts-of-speech tags and named entities information will help us to predict the relationship between entities. We also presented the procedure for predicting POS tags and named entities, which we considered as the sources of information for entity relationship. These features (POS tags, NE) along with the words, in input text sentence, are used as input features to classify the given input into any one of the prede ned relationship class. It also presents the experimental details and performance results of this classi er on ve Indian language datasets such as hindi, kannada, malayalam, tamil and telugu.
Relation Extraction(RE) is an important subtask in Natural Language Processing(NLP) pipeline to convert the unstructured data format to structured data format by extracting the relationship information among the entities in the natural text. The main goal of a RE task is to identify and extract the relationships between two or more entities existing in the given unstructured data and classies the given text based on the extracted relationship.
With the increase in the usage of internet, the unstructured digital data has been increasing exponentially in the form of blogs, research documents, posts, tweets, news articles, and question-answering forms. This unstructured digital data consists important information in the hidden form. The objective of Information Retrieval(IR) is to develop tools for extracting this information automatically. To extract this information, it requires the conversion of unstructured digital data into structured form by predicting the named entities and relationships existed among those entities in the unstructured data.
For example consider the sentence shown in table-1 along with its POS tags and named entities. The sentence has three entities person, occupation, and organization. We can extract these entities information using NER tools. With these entities information, RE algorithm identi es the existence of entities in the given text, but not the relationship information among those entities. In this example there are two relationships among the three entities. The First relationship is woking as between john-person and assistant professor-occupation entities. The second relationship is working in between john-person and IITD-organization entities.
Named entities in the second example, shown in table-2, are person, location, event and datenum. Its corresponding POS tags and words are the sources to nd the relationship among these entities. In this example two relationships discovered and discovered in are exist. The discovered relationship between Columbus-person and America-location. second relationship discovered in between America-location and 1492-datenum.
As shown in the above examples, our approach is to consider POS tags along
with named entities, as they are good source for relationship extraction and text
classi cation. Since, most of the times verbs, prepositions and verbs followed
by preposition can provide the relationship information, in this work we built
a machine learning based classi er to extract generic relationship pattern using
POS tags and named entities along with words in a given training example in
training phase. There are many applications in which we can use RE task. A
bio-medical tool EDGAR, presented in [
RE is a pivotal sub task in several natural language understanding
applications, such as question answering [
This section describes the overall approach we followed in this work to classify the sentences according to the relationship information that exists among the entities in that sentence. For implementation we divided the approach into two stages. In rst stage, as shown in the gure-1, we extracted the list of features which carries the entity information and relationship information for each sentence in train and testset. In second stage we train a sentence level classi er and build the model by feeding training sentences along with features extracted in the previous stage. These features are parts of speech tags and named entities in an input sentence. Later we applied this trained classi er to classify test sentences with its POS features and entity features. 3
We experimented on ve Indian language data sets using di erent classi ers. This section describes the problem de nition, details of feature extraction, classi cation and datasets details with experimental results on those datasets. 3.1
Given a large collection of sentences C and each sentence is represented with a sequence of words, such as w0; w1; w2; :::wn, which contains the set of entities and relation among those entities. Classify these sentences into any one of the class labels given in training data using entity relationship information existing in an input sentence. 3.2
As shown in the example mentioned in introduction section, the adverb, verb, conjunction and preposition type of words in a sentence are the sources of information for extracting the relation among the existing entities. Hence, we predicted the POS tag for each word in a sentence using di erent POS taggers for di erent languages. These POS tagging tools are freely available in IIITHyd web site4. Though, they provided the POS taggers for these ve Indian languages we utilized only hindi, tamil and telugu POS taggers. The details of these POS taggers and its source code is available in IIIT-Hyd website5. We predicted the named entities using a deep learning based Recurrent Neural Network called Bi-directional Long Short-Term Memory. The detailed description of this model is explained in our work submitted along with this paper in ArnektIECSIL@FIRE2018 shared-task workshop. 3.3
As described in the previous section, we applied di erent types of machine
learning based text classi ers to predict the entity relationship class labels. Initially
we veri ed the performance of these classi ers on telugu language dataset. As
logistic regression based classi er gave the highest accuracy(76.35%) compared
to other classi ers NBEM(67.5%) and MLP(75.98%) we applied this classi er
for the remaining datasets.
4 http ltrc.iiit.ac.inshow le.php? lename=downloadsshallow parser.php
5 http:ltrc.iiit.ac.indownload.php
Naive Bayes EM (NBEM) classi er: NBEM is a semi-supervised learning
based classi er[
jDlj maximize X log( (xj ; yj )) + j=1 jDuj X log( (xk; ybk)) k=1 (1) It learns the model parameters by applying Expectation and Maximization(EM) algorithm. In EM algorithm it predicts the labels of each test sample in Expectation step(E-step) and updates the parameters in Maximization step(M-step) in each iteration.
Logistic Regression classi er: In logistic regression[
jDnj maximize X log( (yj =xj ))
j=1 (yj =xj ) = exp( yTj :xj ) PjLj l=1 yTl :xj Multilayer Perceptron: It is a special type of feedforward neural network in which it uses a non-linear activation function in each neuron of hidden layer and output layer. We used the back propagation algorithm while learning the MLP classi er on training data. In training phase, it back propagates the error occurred for each input observation. While back propagating the error, it adjust the weights by minimizing the following objective function shown in equation-4, where Ei(xn) gives the error at ith output node for the nth input observation. We applied the gradient descent algorithm to change the weights.
Ei(xn) = 1 X(yin 2 n yin)2 b (2) (3) (4)
Arnekt-IECSIL@FIRE2018 shared task [
Accuracy =
N o:Of sentences are assigned with the correct label
N o:Of sentences in the dataset P recision(Pi) =
Recall(Ri) =
N o:Of sentences are labeled with labeli
(5) (6) (7) f score(Fi) = 2
Pi
Ri
Pi + Ri Overall f score(F ) = 1 L j j iinL
X Fi (8) (9) The results in table-4 and 5 summarizes the accuracy and F-Scores of the model on testset-1 and 2 of ve languages. In Per-Evaluation and Final-Evaluation this model has high accuracy for kannada dataset compared with other models presented in this competition. For the remaining four languages this model is in the second position. This same order is repeated in F-Score performance of this model. We presented an approach which makes use of POS tag information for relation categorisation. Our approach was general and applicable for all the languages used in the shared-task. We could not get the POS tags of malayalam and kannada which can be further improved with such additional information which is crucial for the success in the shared task.