A huge amount of textual data is generated due to the boom of microblogging. Microblogging sites such as Facebook, Twitter and Google+ are used by millions of people to express their views and emotions on different subjects. In this paper, we discuss sentiment analysis on a Twitter dataset having various tweets from different users. Sentiment analysis is useful for gaining the opinion of people using large volumes of text data where texts are highly unstructured and heterogeneous. In this paper, different classification techniques like Support Vector Machine, Logistic Regression, Logistic Regression with Stochastic Gradient Descent optimizer, Decision Tree Classification, Naive Bayes, Bidirectional LSTM and Random Forest Classification have been applied to analyze the sentiment of people, i.e., whether their tweets are positive or negative. The corpus has been analyzed by plotting descriptive insights such as the word cloud and frequency of positive and negative tweets. The best classifier was selected by comparing the results of accuracy, recall, precision, F1 score, AUC score and ROC curve.
With the universality of microblogging and social networking sites, Twitter, with 319 million monthly users has now become a valuable resource for several individuals and organizations for posting blogs and expressing their views and opinions on different subjects like politics, sports, movies, etc. [1]. Stimulated by the growth of social media, many companies and media organizations are trying to mine Twitter to observe people’s views to understand what they feel and think about their products [2]. As a result, sentiment analysis on Twitter is an effective way of reckoning public opinion. Sentiment analysis provides the potential of observing numerous social networking sites in real-time.
Twitter has a limitation of 140 characters [3] in each tweet, which causes individuals to use phrases in their tweets. Sentiment Analysis automatically detects whether a text section contains emotions or opinioned content. It also determines the polarity of the text. Generally, the dataset consists of a group of tweets where each tweet is interpreted with a sentiment label. Commonly sentiments are labeled positive, negative or neutral. However, some datasets have mixed or irrelevant tags too, which ranges from -5 to 5 and depicting negative to positive polarity [4]. Twitter sentiment analysis is helpful to understand public temperament about different social or cultural events and forecasting the inconsistency within the stock exchange [5].
Sentiment analysis on Twitter is a sort of challenge due to its short length. The unstructured and heterogeneous data compelled us to apply the preprocessing step before feature extraction [6]. The various preprocessing steps include URLs removal, replacing negation, stopwords removal, removing numbers and expanding acronyms. The preprocessing has been done with the help of the Natural Language Tool Kit (NLTK). Then feature extraction is of two phases. First, the normal text was formed by eliminating the Twitter-specific features and then feature extraction was accomplished to extract more features [1].
This research paper is organized into different segments as follows. Section 2 briefs about related works of sentiment analysis. In Section 3 we talk about the methodology and materials which explains the data exploration, data preprocessing and feature extraction. We have also described the different classification algorithms used in the implementation namely Support Vector Machine, Logistic Regression, Logistic Regression - Stochastic Gradient Descent, Decision Tree, Naïve Bayes, Bidirectional Long Short-Term Memory (BiLSTM) and Random Forest. In Section 4 we show the results, analyses and comparison of models. Section 5 comprises the conclusion and future work.
With the advancement of Natural Language Processing (NLP), research on Sentiment Analysis ranges from document-level classification [7] to words and phrase-level classification [8]. The method to retrieve semantic information from a large corpus was presented by Hatzivassiloglou and McKeown. This method separates domain-dependent details and conforms to a novel domain when the corpus is substituted. Their model focuses on adjectives, intending to identify nearsynonyms and antonyms from their model.
For increasing the efficiency and accuracy
of the model [
The authors from the paper [10] discussed about social network analysis and Twitter being a rich source for sentiment analysis and proposed a model to implement Twitter sentiment analysis by fetching the data from Twitter APIs. Their analysis is based on different queries of job opportunities. The dataset has positive, negative, and neutral labels. They noted that the neutral sentiments are high in comparison to positive or negative which shows that there is a need to improve Twitter sentiment analysis [10]. Twitter has become increasingly popular in the field of politics. A real-time sentiment analyzer towards the incumbent of Ex. president Barack Obama and the nine other challengers have been designed by [11]. They used IBM’s InfoSphere Streams platform (IBM, 2012) for speed and accuracy and pipelining real-time data. Using the Twitter “firehouse” they constructed logical keyword combinations to recover relatable tweets about candidates and events. They achieved an accuracy of 59% [11].
Some researchers have tried to determine the public point of view on different subjects like politics, movies, news, etc. from the Twitter posts [12]. The authors of the paper [13] used IMDB, a popular Internet database containing movie information and Blippr, a social networking site where reviews are in the form of ‘Iblips’. Their analysis gave the F-score as high as 0.9 using SVM and demonstrated domain adaptation as a useful technique for sentiment analysis. They introduced a new feature reduction technique, Relative Information Index (RII), which combines with another popular technique ‘thresholding’ to form a good feature reduction technique that not only reduces the features but also improves the F-score [13]. The importance of sentiment analysis has increased so much that it has been in use in various industries, such as hotel management. In this regard, [14] classified the public reviews of a hotel into positive and negative. They collected 800 reviews from TripAdvisor and performed the preprocessing step by NLTK in Python. They used various classifiers like Logistic Regression, Random Forest, Stochastic Gradient Descent Classifier, Naïve Bayes and Support Vector Machine. Their analysis was that Naïve Bays classifier was best among them but Stochastic Gradient classifier also worked well. The analysis was based on the results of accuracy, recall, precision and F1-score [14].
F-score as high as 0.9 using SVM and demonstrated domain adaptation as a useful technique for sentiment analysis.
Naïve Bays classifier was best among them but Stochastic Gradient classifier also worked well. The analysis was based on the results of accuracy, recall, precision and F1score.
The study of computer algorithms that improves automatically by learning from itself is known as machine learning. The data and output are fed into the machine learning model and the machine creates its programming logic to predict the result. The dataset is split into two halves i.e., training part, which contains input feature vectors and their labels, and the testing part. A classification model with the help of a specific algorithm is developed using the training part to observe a pattern. The testing part is used to obtain the accuracy of the model, which tells whether a model is a good fit, underfit or overfit. 3.1.
The dataset used in this work was taken from UCI/Kaggle [15] in csv (comma separated values) which contains 1.6 million tweets. Preprocessing the data was done which includes tokenization, stemming, stopword removal to clean the text. A feature vector was created using relevant features. Data mining classification algorithms such as Decision Tree, Logistic Regression, Random Forest, SVM, Naive Bayes and LR-SGD classifiers were used to gather the accuracy by classifying the tweets into positive or negative tweets. Fig. 1 shows the algorithm adopted for sentiment analysis.
Exploring the data has a key role in machine learning as it helps us to visualize the types and statistics of data [16]. Here, the dataset consists of 0.8M positive and 0.8M negative tweets shown in Fig. 2 (a). As it is text data, the word cloud can also be visualized, as shown in Fig. 2 (b).
As the Twitter datasets are composed of unstructured, heterogeneous, ill-formed words, irregular grammar and non-dictionary terms, the tweets were cleaned by various NLTK methods before feature extraction [1]. Preprocessing steps are [12] • Eliminating all non-English characters and non-ASCII from the text. • Removal of all URL links as they do not provide any information about the sentiment. • Numbers are removed as they are not useful in finding sentiment. • Stop words are the most frequent words in a language, such as "as", "an", "about”, “any" etc. There are many stopwords in English literature. These stopwords do not play any role in finding the sentiments so they are removed from the dataset. • Stopwords also contain “not”, but are not removed from the tweets as they are crucial in analyzing negative reviews. • Stemming is the process to bring back the words into their original form such as “loved” becomes “love”, “worst” becomes “bad” and so on. 3.3.
In feature extraction, the vector space model is used for document representation. A vector is created whose dimension is equal to the size of English vocabulary and each element is initially initialized to 0. If a text data features that vocab word, one ‘1’ will be put in that dimension, as shown in Eqn. 1, Eqn. 2 and Eqn. 3. Every time, a text that features the vocab word is encountered, the count will be increased, leaving 0’s everywhere for the words which were not found even once. The 2nd Edition of the Oxford dictionary contains 171,476 words [17] in current use. So, if a vector is made with all these words the model will be of high variance and here feature selection comes into account. For proper weighting and feature extraction, the count vectorizer method was used which keeps track of the frequent terms as well as rare words. The vector space model improves the accuracy. The feature extraction method is used for dimensionality reduction by removing the non-informative words and rare words. Bag of Words model is created which contains the most frequent words from the feature vector to improve the accuracy [1].
= [0,0,0,1,0,0,0 … … … … . .0] = [0,0,0,0,2,0,0 … … … … … . .0] = [0,0,0,0,0,5,0 … … … … … … 0] (1) (2) (3)
Other than Bag-of words model Tokenization was also used for Bidirectional Long Short-Term Memory, in which raw texts are broken up into unique texts i.e., tokens.
Each of the tokens has its unique token id’s. In tokenisation, a vector is created with a size equivalent to the number of unique words in the corpora. A sequence of tokens is created and they are represented as a vector as shown in Eqn. 4 and Eqn. 5. As each of the tweets has a different length so its token represented sequence has also a different length which makes it difficult to feed into the Deep Learning algorithms as it requires sequences of the same length [18] . To counter this problem, padding and truncating steps come into account where the length of the padded sequence is defined. If the length of the tokenised sequence is larger than the padded sequence then the tokens of the sequence after the length of the tokenised sequence would be truncated, i.e., they are removed. If the length of the tokenised sequence is smaller than the padded sequence then the tokens of the sequence after the length of the tokenized sequence would be padded with “0”. If the length of the padded sequence is chosen to be 6 then Eqn. 4 will be truncated as shown in Eqn. 6 and Eqn. 5 will be padded as shown in Eqn. 7.
What consumes your mind controls your life = [
Classification algorithms are the most important part of supervised learning in machine learning. The classification algorithm is used to indicate the class of the data. In this paper, classification algorithms play a crucial role in labeling the tweets positive or negative. 3.4.1. Bidirectional Long Short-Term Memory (BiLSTM)
A traditional neural network can’t remember the previous inputs, for predicting the next word previous information is a must. Recurrent Neural Network (RNN) has the potential of remembering everything from the past as they have the loop and hidden layer in them. The loops in RNN allows the network to persist information. Recurrent neural network translates the independent activations to dependent activations by furnishing equal biases and weights to complete layers, thus the complexity of increasing the parameters is reduced and the result of one layer is the input to the following hidden layers [19]. Long ShortTerm Memory (LSTM) is a special form of Recurrent Neural Network (RNN) which has the potential to learn long-term dependencies. LSTMs are accomplished to abstain from the long dependencies problem. In LSTM the hidden layer of RNN is restored by the Long Short-Term Memory cell. The LSTM memory cell can be achieved by the Eqn. 8-12.
Where represents a logistic sigmoid function, c, o, i and f represent cell vectors, output, input and forget gate. These have the same dimension as the hidden vector k [19].
Bidirectional Long Short-Term Memory (BiLSTM) is an extension of LSTM, which can be designed by putting two independent LSTM. The structure permits the neural network to have both forward and backward information at every time step. This will run the data in two ways, one from future to past and one from past to future so by this method the model will be able to preserve information from both the (11) (12) future and past. Fig. 4 shows the Bidirectional LSTM [20].
Logistic regression is an example of a linear classifier that is used to classify the class of data. Logistic regression determines the link between the independent and dependent variables by estimating probabilities [16]. It returns the probability by transforming the output with the help of the logistic sigmoid function. Fig. 5 shows the linear regression graph and its equation is given by Eqn. 13 as, (13) (14) (15) The equation of sigmoid function [22] is, = 0 + 1 =
1 1 + − Now, applying Eqn. 14 to Eqn. 13 and solving for to get Eqn. 15 i.e., logistic regression equation ln (
1 −
) = 0 + 1 The graph is now converted into a logistic regression graph shown in Fig. 6. 3.4.3. Logistic Regression-Stochastic
Descent (LR-SGD) is a type of linear model, known as Incremental Gradient Descent [14].
Descent (LR-SGD) classifier is an effective way to selective learning of linear classifiers under different loss functions and penalties such as Logistic Regression and Support Vector Machines. The ‘log’ loss function is used to optimize Logistic Regression while the ‘hinge’ loss function is used for optimizing the Support
recently gained much significance in the field
of large-scale learning although it has been
around in the machine learning association for
a long time [
can be regarded as a linear model for regression and classification tasks [23]. The Support Vector Machine finds the optimal separable hyperplane to separate the tweets into two parts [24]. It is applied to noisy data. The hyperplane line separates the tweets in a very efficient way shown in Fig. 7. Support Vectors are the locations which are quite close to the line from both the classes. The distance between them is often called a margin [25]. The Support Vector Machine is easier to implement and scales well for high dimensional data. It is implemented with kernels that transform non-separable problems into separable problems by adding more dimensions to it. The most commonly used kernel is the Radial Basis Function (RBF) kernel. Mathematically, it can be defined by Eqn. 16, ( , ) = (− ∗ ( − 2) ) (16) Figure 7: SVM
classifier graph showing hyperplane 3.4.5. Naïve Bayes Classifier
Naïve Bayes [26] is the most common supervised
machine learning technique for classification. It is also known as the probabilistic classification technique as it is based on probability [27]. It is completely dependent on the famous probability theorem i.e.,
correlated to conditional probability. It finds the probability of an occurring event when the probability of another occurred event is already given [27]. Mathematically, it can be stated by Eqn. 17, ( ) = ( ) ( ⁄ ) ( ) (17) Where, ( ) refers to posterior i.e. probability of M when N is given, ( ⁄ ) represents likelihood i.e., probability of N when M true, P(M) is the prior i.e., probability of M and
P(N) represents marginalization i.e. probability of N [28]. After implementing the model in classifier the equation is [16], [29] given by Eqn. 18 as, = the
Tree classifier has been implemented [32]. It is robust, easy and simple to implement and not sensitive to irrelevant features [33]. Fig. 8 (a) shows how the dataset was split into different categories using the Decision Tree classifier and (b) demonstrates a general Decision Tree. feature space (b) Overview of a Decision Tree 3.4.7. Random Forest Classifier The Random Forest classifier is a supervised ML technique and a very popular classifier. Just like the Decision Tree, it can also be implemented on both classification and regression models. It is an ensemble learning method of classification that builds a set of multiple decision trees from the training data and outputs mode of class [34]. It is used in applications like search engines, image classification, etc. It constructs a decision tree from each sample and gives the output. The best solution is selected by voting. It is easier to implement, fast and scalable but it easily overfits the data [34]. Fig. 9 shows the complete sketch of the Random Forest classifier.
The dataset was collected from Kaggle. Implementation was done on Python and NLTK was used for cleaning and training the model. The various classifiers used are Logistic Regression, Naïve Bayes, Support Vector Machine, Random Forest, LR-SGD classifier, Bidirectional Long Short-Term Memory and Decision Tree. The dataset consists of 1.6 million out of which 1,280,000 were used for training and 320,000 for testing [15].
Evaluating the models is very important for observing the performance and correctness of the different models on the test data and finding the best among them. The performance of a classifier can be described by the confusion matrix on a set of data for which true values are known. With the help of the confusion matrix, different evaluating metrics such as accuracy, recall, precision, F1-score and AUC score have been evaluated to validate and verify the quality of the results [35], [36]. The confusion matrices for various classifiers have been shown in Fig. 10, Fig. 11, Fig. 12 and Fig. 13. Table 2 compares the different classification models based on these evaluating metrics. Fig. 14 graphically depicts the performance of the different classifiers concerning the accuracy, recall, precision, F1-score and AUC score.
Accuracy: It is the percentage of tweets that have been classified correctly by the model. The accuracy of the model can be calculated using Eqn. 19. Precision: It is the ratio of actual positive tweets to predicted positive tweets. The precision of the model can be calculated using Eqn. 20. + +
Recall: It is the ratio of predicted positive tweets to total positive tweets. The recall of the model can be calculated using Eqn. 21.
F1-score: F1-score can be defined as the harmonic mean of recall and precision. The Fmeasure of the model can be calculated using Eqn. 22.
1 = 2 ∗ ∗ +
Where, TP is the True Positive, TN refers to True Negative, FP is the False Positive, FN means False Negative, P refers to Precision and
R is the Recall.
AUC score: AUC score can be calculated by finding the area under the ROC curve [11]. The AUC score of the model can be calculated using Eqn. 23. ∗ + 1)⁄2 (23)
Where, SP is the Sum of positive observations, PE refers to Positive Examples and NO is the Negative Observations.
The Receiver Operating Characteristic curve (ROC) curve is a tool which predicts the probabilistic value of binary outcome [37]. The relationship between the sensitivity which is the true positive rate and the specificity which is the false positive rate is represented graphically by the ROC curve. It is a significant metric as it covers the whole spectrum between zero and one. The true positive rate is exactly equal to the false positive rate at 0.5, and this represents a random or no skilled classifier [38]. The AUC score can be calculated by finding the area under the ROC curve. The ROC curves for different classifiers have been plotted in Fig. 15.
With the help of the confusion matrix of the various classifiers showing the values of a true negative, true positive, false negative and false positive, we have calculated precision, accuracy, F1-score, recall and roc-auc score as shown in Table 2. In this paper, we have compared various classifiers like Random Forest, Logistic regression, Support Vector Machine, Decision Tree, LR-SGDC and Naïve Bayes with the state-of-the-art approach BiLSTM. On observing the results of Table 2 it had been found that the Bidirectional LSTM was the best classifier with an accuracy of 78.90%, and decision tree came out as runner up with an accuracy of 72.49%, followed by Support Vector Machine and LR-SGDC Classifier with an accuracy of 72.45% and 72.09% respectively. Random Forest and Naïve Bayes also predicted well with an accuracy of 71.29% and 71.24%. it was also observed that decision tree classifier didn’t came up to the expectation with just an accuracy of 68.49%. On examining carefully, it can be observed that prediction of true positive class with respect to predicted positive class i.e., precision score of Bi-LSTM was also highest among all with a precision score of 78.91%. LR-SGDC and SVM classifier were the runner ups with a precision score of 72.74% for each, followed by Logistic Regression with 72.72% precision score. Naïve Bayes and Random Forest classifier also predicted the positive class well with a precision score of 71.33% and 71.31% respectively. The precision score of decision tree classifier was least with a score of 68.5%. Prediction of true positive class with respect to actual positive class i.e., recall score of BiLSTM was best with score of 78.89%, with Logistic Regression as the runner up with sore of 72.49% followed by SVM, LR-SGDC, Random Forest and Naïve Bayes with score of 72.44%, 72.08%, 71.29% and 71.24% respectively. Even here, Decision Tree was not as good with precision score of 68.49%. The F1-score and AUC score of Bi-LSTM was best of among all the classifiers. All these results of various classifiers can be visualized graphically as shown in the Fig. 14. Fig. 15 depicts the ROC curve of all the classifiers implemented in our experiments, which also shows that Bi-LSTM is the best classifier. The model can also be very useful for analyzing the tweets related to medical data [39], [40],[41], [42], [43], [44].
There are various methods of machine learning, symbolic and deep learning for the analysis of the tweets or reviews. But machine learning techniques are most common, efficient and simpler than others. In this paper, machine learning techniques were used for the analysis of tweets on a Twitter dataset. The tweets were cleaned in the preprocessing step by removing the stopwords, URL, numbers and various Twitter-specific features with the help of NLTK. To deal with the miss-spelling and noninformative words, feature extraction was done and a Bag of Words model was created with the most frequent words. The tweets were, then, classified into positive and negative by various classifiers like LR-SGD Classifier, Naïve Bayes, Random Forest, Logistic Regression, SVM, Bidirectional LSTM and Decision Tree. By observing the ROC curve and accuracy score, it was clear that Bidirectional LSTM is the best classifier with an accuracy of 78.90%. Hence, it was found that Bidirectional LSTM is very useful in finding sentiment analysis.
The model can be implemented in a website or Android applications for classifying the sentiments of people on different subjects. As the microblogging sites are blooming, sentiment analysis is very important for many organizations in implicating social intelligence and social media analytics.
The future of this research paper is to explore the data on a wider genre of different social networking sites and e-commencing sites where people do online shopping for many things like books, games, etc. Accuracy rates of these products can be found by sentiment analysis. It can also be implemented to build the human confidence model.
I would like to express my heartiest gratitude to all the co-authors and special thanks to Prof. Mahendra Kumar Gourisaria and Mr. Harshvardhan GM who have been a constant source of knowledge, inspiration and support. I would equally thank my parents and friends who inspired me to remain focused and helped me to complete this research paper. 8. References [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [30] [31] [32] [33] [34] [35] [36] [37] [38] [39]