This paper describes the system we developed for IberEval 2017 on Classi cation Of Spanish Election Tweets (COSET) task. Our approach is based on a weighted average ensemble of ve classi ers: 1) a classi er based on logistic regression; 2) a support vector machine classi er; 3) a Naive Bayes classi er for multinomial models; 4) a Guassian Naive Bayes classi er; and 5) a classi er implementing the k-nearest neighbors vote. Each such classi er was choice taking into account its contributes to the success of the system. The aim is to design a approach by using a voting method, where individual classi ers can have weaknesses. The performance of the ensemble is compared to the individual classi ers, and the experimental results show that the ensemble has better results.
Nowaday, Twitter1 have become an important part of the daily life of many of users. This microblogging services are used as communication media, recommendation services, real-time news sources and information sharing sites. The large amount of new data created as result makes automatic analysis essential for processing this data. Thus, Twitter has become an attractive area for many studies such as text classi cation.
Text classi cation aims at labeling natural language texts into a xed number of predetermined categories [1{3]. On Twitter, users post short text messages called tweets which makes quite di cult this task because of their features. Tweets are small (only 140 characters) and are charectized by their informal style language, many grammatical errors and spelling mistakes, slang and vulgar vocabulary, and abbreviations.
The Classi cation Of Spanish Election Tweets (COSET) task, in IberEval
2017 workshop [
The paper is organized as follows. Section 2 describes our system. Next, in Section 3, the experimental results are discussed. Finally, we present our conclusions in Section 4 with a summary of our ndings. 2
Our system carries out a sequence of steps, which goes from text preprocessing to tweet classi cation. The classi cation approach, based on the comination of ve classi er in order to assign to a tweet a class label, is described below. 2.1
In the preprocessing step, tweets are cleaned. First, taking advantage of regular expressions, the emoticons are detected and removed from the text. Also, we eliminate all links, urls and user names which can be identi ed because their rst character is the symbol @. As regards the words starting with hashtags (that is, the symbol #), we do not realize modi cations because they can be related directly with the topic of the text. Finally, we convert the words to lowercase and remove all non-letters characters and all stopwords present in tweets. 2.2
Classi er based on Logistic Regression (LR): Logistic regression belongs
to the family of classi ers known as the exponential or log-linear classi ers. It
works by extracting some set of weighted features from the input, combining
them linearly, and then applying an exponential function to this combination.
Thus, Logistic regression is a discriminative model that assigns a class to an
observation by computing a probability from the function of a weighted set of
features of the observation [
Logistic Regression is good at dealing with very high dimension data. Text classi cation is a classic problem.
Support Vector Machine classi er (SVM): Support Vector Machine uses linear models to implement nonlinear class boundaries. It transforms the input space using a nonlinear mapping into a new space. Then a linear model constructed in the new space can represent a nonlinear decision boundary in the original space. It plots each data item as a point in n-dimensional space (where 3 n is number of features) with the value of each feature being the value of a particular coordinate. Then, it performs classi cation by nding the hyper-plane that di erentiate the classes very well.
Support Vector Machine classi er works really well with clear margin of
separation and is e ective in cases where number of dimensions is greater than the
number of samples [
Guassian Naive Bayes classi er (GaussianNB): Naive Bayes methods are a set of supervised learning algorithms based on applying Bayes theorem with the assumption of independence between every pair of features. In spite of their apparently over-simpli ed assumptions, naive Bayes classi ers have worked quite well in many real-world situations. They require a small amount of training data to estimate the necessary parameters.
Naive Bayes classi ers can be extremely fast compared to more sophisticated
methods. The decoupling of the class conditional feature distributions means
that each distribution can be independently estimated as a one dimensional
distribution. This in turn helps to alleviate problems stemming from the curse
of dimensionality [
Naive Bayes classi er for Multinomial Models (MultinomialNB): Multinomial Naive Bayes is used for multinomially distributed data. It is one of the classic naive Bayes variants most used in text classi cation.
Classi er based on k-nearest neighbors vote (KNN): Neighbors-based
classi cation is a type of instance-based learning or non-generalizing learning.
This is it does not attempt to construct a general internal model, but simply
stores instances of the training data. Classi cation is computed from a simple
majority vote of the nearest neighbors of each point, where a query point is
assigned the data class which has the most representatives within the nearest
neighbors of the point. The number of nearest neighbors taken into account can
be a constant k. In our case, we take k = 1 for its contributes to the success of
the system. Despite its simplicity, it is often successful in classi cation situations
where the decision boundary is very irregular [
Ensemble classi er (Ensemble): Our design of classi er is an ensemble of the
above classi ers. In this way, the predicted class probabilities for each classi er
are collected, multiplied by the classi er weight, and averaged. The nal class
label is then derived from the class label with the highest average probability.
We used scikit-learn with these purposes [
We nd the best setting of weights via brute force grid search, limiting the
coe cient values in the interval [
In order to evaluate the advantages of the ensemble-based approach, we have used the F1 macro measure. This metric considers the precision and the recall of the systems predictions, combining them using the harmonic mean. Speci cally, we rely on the macro for preventing systems biased towards the most populated classes. Table 1 shows the results for the ensemble classi er we have proposed as well as for the base classi ers on the validation set. As expected, according to the measure, the ensemble classi er reveals a marked performance improvement with the highest score.
Further, we have studied the impact of other text processing beyond the
preprocessing described above. Thus, once the set of simple rules of preprocesing
has been applied, we have realized other preprocesses. We have assigned, to each
word in the text, its lemma and have tried with some feature selection methods.
Feature selection The main objective of feature selection methods is to
decrease of the dimensionality of the dataset by eliminating features that are not
related for the classi cation [
This paper has described an ensemble-based approach for IberEval 2017 on Spanish Election Tweets Classi cation task. We combined ve classiers by a weighted average, with the aim of designing an approach which improves the performance of the base classi ers. The results showed that, indeed, the ensemble classi er reveals a remarkable performance improvement with the highest score. Also, as part of experiments, we studied some preprocessing for texts: lemmatization and features selection methods. We achieved the best results with lemmatized texts and the feature selection method that removing features with low variance.