In the wake of ongoing COVID-19 pandemic, a parallel stream of misinformation and conspiracies rises on the internet. People around the world are being flooded with texts and visuals claiming false statements linked with coronavirus disease. This paper presents a multi-modal fake news detection system that uses text and image features to detect conspiracy tweets. This research has been performed in context with the FakeNews: Coronavirus and 5G Conspiracy task of MediaEval 2020. The NLP subtask we have performed utilizes an ensemble of machine learning and deep learning algorithms for the analysis of textualvisual data. We demonstrate the performances of experiments performed for each modality and results obtained after their fusion.
Scientists, Economics, Mathematicians, Analysts and many other professionals have made their claim by formulating theories on origination and spread of the Coronavirus Disease 2019 (COVID-19). Research and Investment are made both on cure and tracing the cause of the origination of pandemic. And along with the rising number of these theories, the spread of misinformation related to COVID-19, termed as 'Infodemic' has been on the rise too, a lot of times from internet users, public figures and potentially trusted sources. Messages and media carrying such misinformation are spread both intentionally and unintentionally. Several times, they have been linked with existing theories that make them sound true despite not involving either substantial proof or logic. People also get amused by the superficial texts and images carried by the misinformation and tend not to verify the credibility that it carries. Moreover, they pass it further to their friends and families whom they are trusted by and ultimately the misinformation manages to convince a large group of audience that is connected via this network and thereby impacting the habit and lifestyle of the people that accept it. These changes can have an adverse effect or tend to be of no use and consume time and other material resources of people. Hence it becomes necessary to identify, evaluate and share the authenticity of every information, especially those involving conspiracy claims.
One such misinformation that has impacted the thoughts and lifestyle of people and the emergence of technology and revenue of several brands is 5G Corona Conspiracy. This conspiracy has played its significant path to impact the minds of consumers by creating ambiguity about the safety of using 5G communication technology.
To fight the ongoing misinformation wave amidst the pandemic, our NLP subtask at MediaEval 2020 uses ensemble technique with multiple ML and DL models to identify 5G related coronavirus conspiracies prevalent on Twitter. Detailed overview of the task and dataset has been described by Pogorelov et al. [1], [2].
We adopt an ensembling approach incorporating several machine learning and deep learning-based text and image classifiers. We divide our approach into three routines: text-based classification, image-based classification and fusion of text and image models.
The proposed architecture uses a combination of features obtained from multiple classifiers. We experimented with several text classifiers on the development dataset and decided to use a fixed subset of them based on the results obtained on each one of them separately. We have used Support Vector Machine (SVM), Naïve Bayes (NB), K-Nearest Neighbour (KNN), LSTM (Long-Short Term Memory) and Bi-LSTM (Bidirectional-LSTM) for the NLP classification task. Each tweet undergoes preprocessing steps before being passed to these classifiers. These include URL removal, punctation removal, lowercasing, tokenization, stopword removal, stemming/lemmatization and padding. We incorporate LSTM and Bi-LSTM with series of Dense layers and setting Dropout value to 0.5. RMSprop optimizer has been used while training LSTM and Bi-LSTM models for 15 epochs each with a batch size equal to 64. For text-based approach, classification results obtained from SVM, NB, KNN, LSTM and Bi-LSTM are used for majority voting to obtain final predictions.
For visual classification, we filtered tweets containing images and obtained 171 images with the label 5G Coronavirus Conspiracy, 118 belonging to Other Conspiracy class and the rest 791 were Non-Conspiracy tweet images. The test set consisted of 617 images. We fine-tune and use three deep learning models namely, VGG16 [3], Xception [4] and InceptionV3 [5] for classifying images and use their results for majority voting to make final predictions.
MediaEval'20, December 14-15 2020, Online C. Raj, M. Mehta These models have been pre-trained previously and we fine-tune them setting the dropout value to 0.5 and an added batch normalization layer after the dropout layer. We have used sigmoid activation in the last layer for binary predictions and softmax for multiclass predictions. We have used Adam optimizer for all visual classification models and trained them for 15 epochs each setting batch size to 64.
For multi-modal classification, we ensemble all text and image-based classifiers utilized and employ max-voting for final classification. Figure 1 demonstrates the ensembling architecture. The class with the highest number of votes is selected as the predicted class for each tweet. Development on all runs has been performed by splitting the dataset into 7:3 ratio for training and validation. We provide the details of models used and results obtained on validation in Table 1 and Table 2.
The MediaEval 2020 FakeNews: Coronavirus and 5G Conspiracy NLP subtask requires classification of tweets related to coronavirus and 5G conspiracy from other conspiracy and nonconspiracy tweets. Table 2 and Table 3 show the classification results on the development set and test set respectively. We perform five runs on the given task which include three- Observing the trend of results obtained in development and training phases, we observe that binary classifier performed better than three-class classifier. Our binary text classifier achieved third highest score (0.4179) in the challenge. This demonstrates that our model finds it easier to distinguish 5G coronavirus conspiracies from all other conspiracies and real tweets. Ternary text-based classification achieved a score of 0.3408. Image-based detection quality can be further improved significantly. Low scores of models using image modality owe to the small size of visual data. Proposed method with larger dataset would perform eminently. We suggest the use of data augmentation techniques for better performance.
In this paper, we employ machine learning and deep learningbased ensembling technique that uses majority voting to deduce predictions if a tweet is related to 5G Coronavirus conspiracy or not. We perform a multimodal analysis utilizing text-based NLP features from the tweet and visual features from the images posted along with those tweets. We build a fusion model that incorporates both textual and visual features and generates prediction based on each modality separately and their combination. Our classification approach plays with both binary and ternary classifiers to experiment with the efficiency of the ensemble models. The limitation we encounter is the lack of sufficient training data and propose to fix it in future works using data augmentation techniques on both text and image data to receive better performance and healthier conspiracy detection.