English. This paper describes our approach on Part-of-Speech tagging for Italian Social Media Texts (PoSTWITA), which is one of the task of EVALITA 2016 campaign. EVALITA is a evaluation campaign, where teams are participated and submit their systems towards the developing of tools related to Natural Language Processing (NLP) and Speech for Italian language. Our team NLP-NITMZ participated in the PoS tagging challenge for Italian Social Media Texts. In this task, total 9 team was participated and out of 4759 tags Team1 successfully identified 4435 tags and get the 1st rank. Our team get the 8th rank officially and we successfully identified 4091 tags as a accuracy of 85.96%.
the 5th evaluation campaign, where following six tasks are organized such as:
ArtiPhon – Articulatory Phone Recognition FactA – Event Factuality Annotation NEEL–IT – Named Entity Recognition and Linking in Italian Tweets PoSTWITA – POS tagging for Italian Social Media Texts QA4FAQ – Question Answering for Frequently Asked Questions SENTIPOLC – SENTIment POLarity Classification
In addition, a new challenge to this event is also organized by IBM Italy as IBM Watson Services Challenge. Among these challenges our team NLP–NITMZ is participated in 4th task i.e. POS tagging for Italian Social Media Texts (PoSTWITA).
The main concern about PosTWITA is,
Partof-Speech (PoS) tagging for automatic evaluation
of social media texts, in particular for micro–
blogging texts such as tweets, which have many
application such as identifying trends and
upcoming events in various fields. For these applications
NLP based methods need to be adapted for
obtaining a reliable processing of text. In literature
various attempts were already taken for developing
of such specialised tools
For this task, we used a supervised leaning for PoS tagging and the details of system implementation is given in section 2. We discuss the performance of the system in section 3. Finally, we conclude our task in section 4. over (x; y) pairs. In this case, we further break down the probability p(x; y) as follows: 2
For this task, we used supervised learning approach to build the model. First we implement the conditional model for PoS tagging and then to simplify the model we used Bayesian classification based generative model. Further this generative model is simplified based on two key assumptions to implement the HMM model using bigram. 2.1
In machine learning supervised problems are defined as a set of input called training examples (x(1); y(1)) (x(m); y(m)), where each input x(i) paired with a output label y(i). In this task, our goal is to learn a function f : X ! Y , where X and Y refers to the set of possible input and labels.
For PoS tagging problem, each input represents a sequence of words x(1i); ; x(nii) and labels be a sequence of tags y1(i); ; yn(ii), where ni refers to the length of ith training example. In this machine learning each input x be a sentence of Italian language and each label be the possible PoS tag. We use conditional model to define the function f (x) and we define the conditional probability as p (yjx) for any x, y pair. We use training examples to estimate the parameters of the model and output of the model for a given test example x is measured as f (x) = arg max p(yjx) y2Y (1) Thus we consider the most likely label y as the output of the trained model. If the model p(yjx) is close to the true conditional distribution of a labels given inputs, so the function f (x) will consider as an optimal. 2.2
In this model, we use the Bayes’ rule to transform the Eq.1 into a set of other probabilities called generative model. Without estimating the conditional probability p(yjx), in generative model we use the Bayesian classification p(x; y) p(x; y) = p(y)p(xjy) and then we estimate the model p(y) and p(xjy) separately. We consider p(y) as a prior probability distribution over label y and p(xjy) is the probability of generating the input x, given that the underlying label is y.
We use the Bayes rule to derive the conditional probability p(yjx) for any (x; y) pair: p(yjx) = p(y)p(xjy) p(x) where p(x) =
X p(x; y) = y2Y
X p(y)p(xjy) y2Y
We apply Bayes rule directly to a new test example x, so the output of the model f (x), can be estimated as follows: f (x) = arg max p(y)p(xjy)
y
To simplify Eq.5, we use Hidden Markov Model (HMM) taggers with two simplifying assumptions. The first assumption is that the probability of word appearing depends only on its own PoS tag as follows: p(w1n t1n) n Y p(wi ti) i=1 where p(w1n tn) means probability of tag ti with 1 word wi. The second assumption is that the probability of a tag appearing is dependent only on the previous tag, rather than entire tag sequence. This is known as bigram assumption and can be measured as follows: (2) (3) (4) (5) (6) (7) p(t1n) n Y p(ti; ti 1) i=1
Further, we incorporate these two assumptions in Eq.5 by which a bigram tagger estimates the most probable tag as follows: t^1n = arg max p(t1n w1n) t1n
n arg max Y p(wi ti)p(ti ti 1) (8) t1n i=1 3.1
For the proposed task organizers re-uses the tweets being part of the EVALITA2014 SENTIPLOC corpus. Both the development and test set first annotated manually for a global amount of 4; 041 and 1; 749 tweets and distributed as the new development set. Then a new manually annotated test set, which is composed of 600 and 700 tweets were produced using texts from the same period of time. All the annotations are carried out by three different annotators. Further a tokenised version of the texts is also distributed in order to avoid tokenisation problems among participants and the boring problem of disappeared tweets. 3.2
For this task, total 13 runs were submitted 9 teams and among these runs 4 Unofficial runs also submitted. In Table 1 we list out all results for this task.
Team 2, 3, 5 and 6 submitted one Un-Official run with compulsory one and these Un-Official submissions are ranked as 12th, 3rd, 7th and 9th respectively. We also listed these submissions in Table 1 with other runs. Our team NLP–NITMZ represent as Team8 and ranked as 11th in this task. 3.3
In this competition, a total of 4759 words were given for tagging purpose. These words were categories into 22 PoS tags and our team successfully tags 4091 words with 668 unsuccessful tags. The 1st ranked team successfully tags 4435 words and the last positioned team i.e. Team9 successfully identified 3617 tags. In Table 2, we provide our system tag wise statistics. This PoS tagging task of EVALITA 2016 campaign is for Italian language and our system ranked 11th position for the task of POS tagging for Italian Social Media Texts. We also want to mentioned that, authors are not native speaker of the Italian language. We build a supervised learning model based on the available knowledge on training dataset.
This work presented here under the research project Grant No. YSS/2015/000988 and supported by the Department of Science & Technology (DST) and Science and Engineering Research Board (SERB), Govt. of India. Authors are also acknowledges the Department of Computer Science & Engineering of National Institute of Technology Mizoram, India for proving infrastructural facilities.