Corona Virus and Conspiracies Multimedia Analysis Task is the task in MediaEval 2021 Challenge that concentrates on conspiracy theories that assume some kind of nefarious actions related to COVID-19. Our HCMUS team performs diferent approaches based on multiple pretrained models and many techniques to deal with 2 subtasks. Based on our experiments, we submit 5 runs for subtask 1 and 1 run for subtask 2. Run 1 and 2 both introduces BERT[5] pretrained model but the diference between them is that we add a sentimental analysis to extract semantic feature before training in the first run. In run 3 and 4, we propose a naive bayes classifier[ 4] and a LSTM[8] model to diversify our methods. Run 5 ultilize an ensemble of machine learning and deep learning models - multimodal approach for text-based analysis[3]. Finally, in the only run in subtask 2, we conduct a simple naive bayes algorithm to classify those theories. In the final result, our method achieves 0.5987 in task 1, 0.3136 in task 2.
The COVID-19 pandemic has severely afected people worldwide, and consequently, it has dominated world news for months. Thus, it has been the topic of a massive amount of misinformation, which was most likely amplified by the fact that many details about the virus were unknown at the start of the pandemic. In the Multimedia Evaluation Challenge 2021 (MediaEval2021), the purpose of Corona Virus and Conspiracies Multimedia Analysis Task is to develop methods capable of detecting such misinformation. By this way, this task aid in preventing misinformation outspread causing social anxiety and vaccination doubts. We propose diferent methods which are mainly based on deep learning model to solve the problem in various aspects which would be described in the later sections. 2.1
In subtask 1, we have recieved two datasets in total: * dev1-task1.csv: unbalance dataset consisting of 500 tweets in which (Non-Discuss-Promote) respectively is (340,76,84) * dev-task1.csv: unbalance dataset consisting of 1011 tweets in which (Non-Discuss-Promote) respectively is (414,186,411) 2.2
Text-Based Misinformation Detection exists similar objectives to the text classification task. Hence, we take advantage of pretrained NLP models and fine-tune them for this task. However, the validation result is biased towards non-conspiracy class since given dataset is small and unbalance. Therefore, we adapt those models to generate new data by crawling data from Twitter and assigning a label for a tweet if it gets the most voting which increase the efectiveness and balance on the dataset as well. 3 3.1
From the pure train data, we need to preprocess the tweets to make
it easier for our model to learn. The first step is replace those words
in short form into its original form - I’m to I am. The second step is
to remove the stopwords - about, above, ...; lemmatize the family
words - roofing, roofers,... into roof. Finally, we also try to remove
any other meaningless feature in the tweets such as the "https",
"(amp)" and the emoji to get a perfect tweet for training.
Firstly, we use sentiment analysis method to categorize all tweets
into two classes - optimism and anger. Based on the observation,
non conspiracy tweets contain a higher rate in optimism while
discuss/promote tweets dominate the anger rate. Therefore, we
decide to pick out the tweets with the opt rate greater than 0.8 and
anger rate less than 0.2 in the test set and directly label them as
non conspiracy. The remained tweets are predicted by BERT[
We use tf-idf vectorizer to extract feature from the text. After trying
both Logistic Regression and Naive Bayes[
We use pretrained glove to transform each word in the sentence
into an array of 300 numbers represent the "meaning" of the word.
Finally we built a 2D LSTM[
We combine all the results run by those deep-learning and
machinelearning algorithm and label the tweet by its highest-voted class[
Similar to the method in run 3, we used tf-idf vectorizer to extract
feature from the text. Base on our observation, test set is extremely
unbalanced regrading to multilabel problem, so we try to resolve it
by downsampling the data - keeping only the dominant sentences
in the biased class. In order to handle multilabel problem, we utilize
three diferent methods: Binary Relevance[
In summary, we identify challenges of the dataset and propose diferent approaches to address the issues. We conclude that classifying a tweet promotes/supports or discusses sentiment task is heavily biased towards the writers attitude, therefore making it dificult for NLP model to learn the true label. In recent study, we can only extract basic state of sentiment of a tweet such as sad or optimism, so we aim to tackle the challenge in a higher level in the future.
This research was funded by SeLab-HCMUS and VNUHCM-University Of Science.