From the beginning of 2020, we saw a rise of a new virus called the Coronavirus and ultimately a pandemic that anyone reading this paper must have been through. With the rise of COVID,many vaccines were found, the global vaccination drive as a result of this naturally fueled a possibility of Pro-Vaxxers and Anti-Vaxxers strongly expressing their support and concerns regarding the vaccines on social media platforms and along with this came up the need of quick identification of people who are experiencing COVID-19 symptoms. So in this paper, an efort has been made to facilitate the understanding of all these complications and help the concerned authorities. With the help of data in the form of Covid-19 tweets, a (machine-learning) classifier has been built which can classify users as per their vaccine related stance and also classify users who have reported their symptoms through tweets.
Globally, as of 6:28pm CEST, 7 October 2022, there have been 617,597,680 confirmed cases of
COVID-19, including 6,532,705 deaths, reported to WHO. Fortunately, since December 2020 /
January 2021, multiple pharmaceutical companies have put forward vaccines (e.g., AstraZenca,
Pfizer, Moderna ,Covishield to name a few) that are claimed to reduce the chance of COVID
infection and fatality. Naturally, governments across the world are procuring and administering
these vaccines to their citizens. But with the rise in these vaccination drives there has been
increase in vaccine hesitancy and anti-vaccination speeches all over the world, undermining
the eforts to control the spread of the novel coronavirus. So, it is quite evident that there is a
need for us and especially the authorities to analyse the societal angle i.e. the public sentiments
towards the vaccines. A major motivation for this approach would be [
Another matter of importance is the rapid identification of people who are experiencing COVID19 symptoms, because it is extremely necessary for authorities to arrest the spread of the disease.So for such purposes,we specifically explore if tweets that report about someone experiencing COVID-19 symptoms (e.g., ‘fever’, ‘cough’) can be automatically identified.We call such tweets symptom-reporting tweets.
For both the purposes mentioned above,we built and train a machine learning classifier to classify the tweets into their respective classes to a certain precision.
• For the 1st task of identifying vaccine stance :
We build a classifier for 3-class classification on tweets with respect to the stance reflected towards COVID-19 vaccines. The 3 classes are described below: – 1. AntiVax - the tweet indicates hesitancy (of the user who posted the tweet) towards the use of vaccines. – 2. ProVax - the tweet supports / promotes the use of vaccines. – 3. Neutral - the tweet does not have any discernible sentiment expressed towards vaccines or is not related to vaccines • For the 2nd task of detection of reporting symptoms :
Build an efective classifier for 4-class classification on tweets that can detect tweets that report someone experiencing COVID-19 symptoms. The 4 classes are described below: – 1. Primary Reporting - The user (who posted the tweet) is reporting symptoms of himself/herself. – 2. Secondary Reporting - The user is reporting symptoms of some friend / relative / neighbour / someone they met. – 3. Third-party Reporting - The user is reporting symptoms of some celebrity / third-party person. – 4. Non-Reporting - The user is not reporting anyone experiencing COVID-19 symptoms, but talking about symptom-words in some other context. This class includes tweets that only give general information about COVID-19 symptoms, without specifically reporting about a person experiencing such symptoms.
The data can be credited to [
As the data collected is primarily tweets, in both the cases the data needs to be cleaned, transformed and in some cases even further sampling was needed to produce fair results. In regards to both the tasks, we followed same steps to clean the data.
Following steps were taken to clean the data: 1. The column representing tweet ids were dropped as it was of no consequence after the data had been crawled. 2. The user handles were removed from the tweet using the ntx or the neattext library. 3. The urls or hyperlinks were removed as they did not provide any form of insight, from the tweet using the ntx or the neattext library. 4. The special characters for example: currency symbols,hashtags (#),percentages e.t.c were removed for further refinement of the data using the neattext library.
After the tweets were cleaned, some natural language processing was done on the tweets: 1. The emojis were also removed from all the individual tweets and replaced with their encoded meaning using demojize method of emoji library of python. 2. And finally contractions used in natural language like "I’ll" were fixed to become more meaningful like "I will" using the contractions library of python. 3. The tweets were all changed to lower case to maintain more uniformity during the classification task. 4. Lemmatization was performed on the tweets to convert each word to their base forms, further enhancing the classification job by preserving the context of the tweets, using the nltk or the natural language toolkit in python. 5. Using the nltk library, stopwords like "the","is" e.t.c were removed as they do not provide or preserve any context in the tweet.
Task Vaccine Stance
@NikkiHaley @pfizer @realDonaldTrump Nothing to do with Trump. Delete tweet, you look foolish.
I used to go "no no, the cough is not because of my smoking, it’s just a mild cold". Now I go "don’t worry about my cough! It’s because
I’m a smoker!" #Covid_19 #smoking 3.3. Checking if the data is balanced or not 1. For vaccine stance task: The data provided was balanced for each of the 3 classes and needed no under-sampling or synthetic up sampling, to be used without a particular bias forming towards any of the classes. 2. For reporting symptoms classification:The data provided was quite imbalanced for each of the 4 classes. The 4 classes had such value distributions.
a) non-reporting - 814 tweets b) primary - 437 tweets c) third-party - 196 tweets d) secondary - 127 tweets So to avoid any unnecessary bias in the classification task, we under-sampled the data to 127 tweets each class, which ensured better results and a 25 percent distribution for each class.
After the data has been processed, it’s time we build a classifier catered to each task. To build a classifier we must first vectorize, i.e. convert the text data to a more machine learning applicable numerical format. The choice of model is the crucial step and depending on that hyper parameter tuning for the complete machine learning model can be done.
4.1.1. Classifier models:
To classify tweets into the 3 aforementioned classes, we implemented various classification
models like Naive Bayes, XGBoost along with TF-IDF vectorizers, Count Vectorizers, cosine
similarity and word2vec models. Among all these models we specifically focused on XGBoost
for it produced good results on training data and some other techniques to further supplement
its performance. The models considered are listed below:
1. XGboost with count vectorizer and word2vec model
2. XGboost with count vectorizer and cosine similarity
3. XGboost with count vectorizer
Along with the machine learning models listed some other techniques including a word2vec
model was trained on the given corpus of data. Cosine similarity was also used to further
supplement the XGBoost classifier, which was referenced from this article [
XGBoost with Cosine similarity XGBoost with Count Vectorizer
So, from the above evaluation it is very clear that the best model was the one where XGBoost was used with Count Vectorizer (as per the macro f1 score). The reason for which the other two models couldn’t perform well are yet to be found, although some outliers or some mislabelled tweet may have resulted to the decrease in their performance. In all cases Count Vectorizer, also proved to be much more eficient when it came to classifying ProVax or AntiVax tweets. To classify the tweets into 4 diferent classes we have yet again focused on XGBoost as our primary classifier. Along with Xgboost we have used Count Vectorizer but this time we have focused on a range of words rather than individual words for better detection of symptoms. 4.2.2. Evaluation of models: No hyper parameter tuning was performed on this model so we jump to the results part of the model. The model after being trained and tested on the training data, was tested on the actual test data which shows:
The work done here is purely based on machine learning classifiers, through this we have attempted to make an eficient classifier using the XGBoost machine learning algorithm to classify tweets with a good macro - F1 score and also explore other similar text based classifiers which could be applicable to our context. Future work: In future I plan to explore problems like this with more sophisticated and state of the art deep-learning based classifiers like BERT, neural nets , etc.
Thanks to https://www.kaggle.com/code/prashant111/a-guide-on-xgboost-hyperparameters-tuning/ notebook with the hyper parameter tunning of the XGBoost model and https: //machinelearningmastery.com/ for being a general guide throughout the process.