The issue of hateful or toxic content on social media platforms such as Twitter and Parler is on the rise and demands attention. The aim of this research is to compare and analyze toxicity between Twitter and Parler for COVID-19 discourse. Highly toxic individuals and their networks are analyzed for the two platforms. Data from January 1, 2020, to December 31, 2020, is analyzed to ascertain and compare overall network health and the evolution of toxicity over time. We found evidence that Twitter contained a higher level of toxicity regarding COVID-19 discourse than Parler. When analyzing COVID-19 vaccine discussions within the Twitter network, prominent conspiracy theory themes emerged among highly toxic users. Within the Parler COVID-19 vaccine discussion, we identified clusters of highly toxic users and important bridges aiding the spread of misinformation. These toxic conversations could impact the public health response to various non-pharmaceutical interventions (NPI's). The research demonstrates a computational method to evaluate toxicity and means for policymakers to improve the overall health of our online discourse by stemming the flow of toxicity in communities through online social networks.
While major social media platforms like Facebook, Twitter, and YouTube have implemented guidelines and enforcement
measures to manage toxic content and misinformation, "free speech" platforms such as Parler have been more lenient in
permitting hate speech and conspiracy theories. And potentially harmful misinformation. One study showed % of active
users in Parler post hateful content [
The spread of abusive language and toxic content on social media can have negative impacts on communities. Analyzing
toxic content provides additional insights and helps address the challenge of managing safety on social platforms. Our
analysis contributes to the current research on the health of social media. For this paper, we consider misinformation to
be a claim that contradicts or distorts the common understanding of verifiable facts [
This work answers two research questions: 1) Does Twitter and Parler differ in terms of Toxicity on each platform? (Section 5.1) 2) Was toxic content on the COVID-19 vaccine narrative used to spread toxicity to other narratives? (Section 5.2) The remainder of this paper is organized as follows. In section 2, the related work that has been published regarding toxicity on social media is presented. Section 3 describes the data collection process and the methodology used in this paper. Section 4 presents the highlights from our results and analysis. Finally, Section 5 concludes with the contributions of this work and presents our plans and ideas for future work.
The key findings and contributions of this research are: ● ● ● ●
Twitter contained statistically significant (p-value < 0.05) higher level of toxicity compared to Parler regarding COVID-19 discourse. (Section 5.1) Prominent conspiracy theory themes were identified within the Twitter network originating from the COVID19 vaccine narrative, such as those regarding Bill Gates and the QAnon group. (Section 5.2) Well defined user communities with highly toxic content were identified, including a misinformation echo chamber within the Parler network. (Section 5.2) Significant bridge nodes were identified that spread toxic COVID-19 vaccine misinformation throughout the Parler network. (Section 5.2) In the next section we present research from previous studies, background about existing approaches to detecting toxicity, and previous research discussing the impact of toxicity on public health discourse.
Several researchers have attempted to characterize the behavior of toxic users, while also attempting to predict their
future behavior. Cheng et al. investigated the long-term patterns of users displaying anti-social behavior in online
forums and whether anti-social users can be identified early in their posting history using text quality metrics [
Pascual-Ferrá et al. claim that social media has an important effect on strengthening public health issues. They
focused on online conversations regarding COVID-19 and wearing masks to understand toxicity's role in this
discourse [
Researchers have developed multiple methods and models to detect toxicity in online text. Watanabe et al. proposed
a machine-learning method to detect hate speech on Twitter using sentiment and semantic-based features [
In a study by Obadimu et al. a Non-negative Matrix Factorization (NMF) technique as an optimization problem was
used to forecast commenter toxicity on YouTube. Their findings showed that the NMF model performed more
accurately than other models in forecasting toxicity scores and had better computation time [
The data from both Twitter and Parler analyzed in this work consists of a corpus of user posts collected based on a list of seed hashtags related to COVID-19 from January 1, 2020, through December 31, 2020 (Table 1) Eight datasets were created, four for each platform with corresponding hashtags and keywords. For Parler data, we used an open dataset created by Aliapoulios et al. which was a complete dataset of all Parler data from August 2018 to when Parler was shut down in January 2021 [22]. The Twitter Developer API was used to collect data from Twitter for the hashtags in (Table 1) post-hoc. Because of this, tweets and accounts removed from Twitter for being labeled misinformation were not collected. Data collected in the study will be made available upon request according to data sharing guidelines of Twitter and Parler. In the next section, we discuss our methodology and approach to calculating toxicity.
Prior to executing the toxicity analysis, we removed our seed keywords and hashtags from each record in the datasets so their presence would not influence the calculated toxicity score. Non-English posts for Parler and Twitter were removed as more than 90% of our data are in English and Detoxify model can generate toxicity score effectively for English text.
We computed toxicity scores for each Parler post and Twitter tweet in the dataset using Detoxify. Detoxify is a model created by Unitary AI (https://github.com/unitaryai/detoxify) that uses a Convolutional Neural Network that is trained with word vector inputs to determine whether the text could be perceived as “toxic” to a discussion. Given a text input, the Detoxify API returns a probability score between 0 and 1, with higher values indicating a greater likelihood of the toxicity label being applied to the text. Since toxicity scores are based on a probability score of 0 to 1, toxicity scores of 0.5 or greater indicate a piece of text labeled as "toxic". Detoxify returns seven categories of toxicity scores in terms of level and type: 1) toxicity, which is the overall level of toxicity for a piece of text, 2) severe toxicity, 3) obscene, 4) threat, 5) insult, 6) identity attack and 7) sexually explicit. The reason for using Detoxify is it is an open-source comment detection python library that identifies harmful and inappropriate texts online. This is a multilingual model that has been trained in English, French, Italian, Spanish, Russian, Turkish, and Portuguese. Although it can predict toxicity by providing a score, it is not effective while there are some words related to swearing, insults, or profanity present in the text. They may predict a non-toxic text as toxic if there are certain words However, this inefficiency level is very low, and we can ignore this as it is same for both platforms. For comparison, we also explored using Google’s Perspective API, which is a related type of model with similar outputs used for determining toxicity and the scores were similar.
For our co-hashtag social network analysis, we used NetworkX (https://networkx.org/), a Python library for creating and analyzing network graphs, to generate co-hashtag graph networks for the Twitter and Parler vaccine category datasets. We used the Girvan-Newman algorithm to identify distinct communities within each network [23]. We removed low toxic posts less than 0.5 to focus on the analysis of highly toxic content [18]. The next section discusses our analysis and results of these Twitter and Parler datasets.
In this section, we discuss our analysis and results. First, we discuss the overall posting frequency of our seed hashtags (and keywords) and the results of our toxicity analysis. This is followed by a discussion of our social network analysis using co-hashtag graph networks and present visualizations of some of the most interesting highlights from our findings within each discussion.
5.1
The analysis was conducted using Twitter data, and the hashtags used as seeds were initially seen in March 2020. From March to December 2020, the COVID category had the highest number of posts compared to other Twitter datasets. The number of tweets showed a peak in mid-April, near the end of June, and a significant increase in midNovember (Figure 1). It wasn't until the end of May that posting activity using the seed keywords was observed on Parler, as shown in Figure 1 (right). In November, there was a significant increase in posting frequency across all Parler datasets, which initially appeared to be indicative of artificial behavior. However, upon further inspection of the dataset, it was discovered that Parler users tended to use all four seed hashtags within a single post, unlike Twitter users. Our analysis revealed that there were differences in the presence of toxicity (toxicity score > 0.5) in user-generated text content between Twitter and Parler from January 1, 2020, to December 31, 2020.
When comparing the amounts of harmful content on each site, Twitter had a greater proportion of toxic tweets overall (Figure 2). This means that the majority of the Twitter content had a higher probability of being labeled as toxic than compared to the Parler content. Surprisingly, in the overall toxicity category, the Twitter content for all datasets had a higher percentage of content with toxicity scores greater than 0.7 and greater than 0.9 than did the Parler content (see Table 2). Again, Parler only exceeded Twitter with regard to the percentage of toxic content for the COVID dataset. This is an interesting result because we expected to see more toxic content on Parler due to the free-speech nature of the platform and how they tout claim their lack of censorship as a selling point for users. We also looked at the “obscene” and “insult” toxicity categories for each tweet and post for all datasets. Of the seven categories of toxicity scores obtained from Detoxify, only three contained enough data to warrant inclusion in the
Parler discussion: toxicity (overall), obscene, and insult. More Twitter content fell into the obscene category than Parler content for all datasets, with the highest percentage being within the Lockdown dataset (28.6% versus 13.06%). However, more Parler content fell into the insult category than Twitter content for the COVID dataset (18.01% versus 10.93%). The percentage of toxic content (overall toxicity category) within the Vaccine datasets varied considerably between platforms (30.98% for Twitter versus 11.93% for Parler). Overall, the toxicity analysis revealed that Twitter was more toxic than Parler in all but one case, the COVID dataset. The toxic content was more obscene and insulting for both platforms. But the toxic content on Twitter was obscener than that of Parler, especially within the Lockdown dataset. The toxic content on Parler was more of an insulting type within the COVID dataset.
Next, we wanted to test the statistical validity of our findings. We conducted statistical significance testing between Twitter and Parler, using a t-test for the 4 datasets. The null hypothesis of the t-test is that the means of two groups are the same. P values for these t-tests are shown in Table 3. The p values for all these tests are significantly lower than 0.05 which implies the null hypothesis can be rejected for all four pairs. So, the alternative hypothesis is accepted which is there are significant differences between the mean toxicity scores for Twitter and Parler for all four datasets. The Twitter data, for example, shows that there are a few conversations that are very toxic, and those few highly toxic conversations are driving up the overall toxicity level of the platform. This has important implications for platform administrators, who may be able to significantly reduce the strongest drivers of toxicity by moderating the relatively few, highly toxic users, rather than attempting larger platform-wide changes to all users. Table 3 also gives a good summary comparison and contrast between the two platforms in terms of the context of discussions, especially in those of vaccine and lockdown, for which Twitter is clearly more toxic. The toxicity standard deviation metrics revealed some additional unique contrasts between the two platforms (Table 3). The standard deviation of toxicity values for content within the lockdown, mask, and vaccine categories are higher on Twitter than on Parler, indicating that there is more variation in toxicity for these datasets, although values were higher for Parler for content within the COVID category. Since the percentage of toxic content in the vaccine datasets contrasted considerably between Twitter and Parler, we next drill down into those datasets and look at their network structures to identify some possible explanations for this drastic difference.
The 5-point statistical analysis in Figures 3 to 6, shows that in general, the term “f*ckcovid” on Parler is more toxic than on Twitter. However, for the other negative terms related to COVID, Twitter is more toxic. We use the top 3 most severe classes to compare in our statistical analysis. For ‘f*cklockdown’ and ‘f*ckvaccine’ hashtags, Parler is less toxic than Twitter if we consider the mean toxicity of from the boxplot for both platforms. On the other hand, for ‘f*ckcovid’ and ‘f*ckmask’ hashtags there is a significant increase in toxicity in Parler. On Twitter, the most toxic hashtag category is ‘f*ckmask’ whereas for Parler it is ‘f*ckcovid’. Conducting social network analysis allows us to identify some important characteristics of the users within these datasets for each platform. Using NetworkX, we created co-hashtag network graphs to compare the Twitter and Parler Vaccine datasets filtered down to focus on the tweets and posts that scored greater than 0.5 on toxicity. This allowed us to identify the user communities, look at the context of the hashtags, and see what other topics and information toxic users shared and actively associated with the Vaccine hashtags/keywords. Our results show that the overall structure of the Twitter and the Parler co-hashtag networks vary considerably, as do the structure of their internal components. At the highest level, the Twitter co-hashtag network appears to be unstructured and somewhat scattered out with a few small clusters of users (Figure 7, left); whereas the Parler co-hashtag network is clustered and shows clear connective bridges and communities of users (Figure 7, right). Upon drilling down into these co-hashtag networks, we can identify some of the contexts within these clusters of users. When analyzing the Twitter network, we identified a mass of users sharing hashtags indicative of various Bill Gates conspiracies, anti-vaccination ideas, and the far-right conspiracy group QAnon (Figure 8). QAnon is a movement that has followers that spread false information on a variety of topics, including COVID-19 [24]. On Parler, QAnon followers often use the #wwg1wga hashtag in posts. The hashtag #wwg1wga is an abbreviation for the phrase, “where we go one, we go all” [25]. This hashtag was identified as being a clear bridge node in the Parler network, meaning that it is an important connector node and conduit by which other information flows within and throughout the network (Figure 9).
The #wwg1wga bridge node can be seen connecting election conspiracy theorists (Figure 10, right), who was actively associating with the #f*ckvaccine hashtag with other hashtags indicative of the 2020 U.S. presidential election such as #maga, #trump2020, and #stopthesteal, which is related to the misinformation narrative regarding wide-spread election fraud. The other bridge nodes identified in the Parler network were #sheep and #fightback. Our drill-down also allowed us to identify a misinformation echo chamber operating within this Parler co-hashtag network (Figure 10).
This echo chamber is completely separated from the rest of the network components. Instead of the users within it exchanging new information with one another, they are repeatedly sharing the same ideas, such as #plandemic, #masksmakeyousick, #vaccineskill, and #coronavirushoax. The #plandemic hashtag was used after the “Plandemic” movie was posted online, which spread numerous conspiracy theories about COVID-19 [26]. Overall, the Twitter network showed prominent conspiracy themes such as those regarding Bill Gates and QAnon. In contrast, the Parler network showed defined user community clusters, the most concerning consisting of a misinformation echo chamber. Also, within the COVID-19 vaccine discussion within the Parler network, important bridge nodes were identified that serve to spread information throughout the rest of the network. One interesting observation was the bridge node that connected the QAnon group connected with an identified proTrump group. In the next section, we discuss our conclusions and ideas for future work.
In the specified timeframe, both Twitter and Parler exhibited considerable degrees of toxicity concerning COVID19 topics. This study examines and contrasts the degree of toxicity and network patterns of these toxic communities on both platforms. The methods were applied to different datasets for Twitter and Parler. The results obtained from this research are preliminary, and they are restricted to a particular data arrangement. The findings indicate toxicity levels were higher overall on Twitter for all datasets except for the COVID category. It was unexpected to observe higher toxicity levels on Twitter since it is stringent content guidelines and moderation policies. Conversely, Parler’s guidelines highlight a lack of moderation. One possible explanation for the unexpectedly high toxicity on the Parler COVID dataset is that Twitter began removing users and posts sharing COVID-19 misinformation in April 2020, sparking anger and prompting many users to migrate to Parler instead [27]. In addition to being detrimental to the overall health of social networks, the moderate proportion of toxic content on these platforms surrounding COVID19 topics may affect users’ perceptions of the effectiveness and importance of periodic lockdowns, wearing face masks, and becoming vaccinated. The contributions of this work include 1) Evidence that Twitter contained a higher level of toxicity regarding COVID-19 discourse than Parler; 2) When analyzing COVID-19 vaccine discussion within the Twitter network, prominent conspiracy theory themes were identified, such as those regarding Bill Gates and the QAnon group; 3) For COVID-19 vaccine discussions within the Parler network, defined clusters of users were identified, including a misinformation echo chamber; 4) In Parler, important bridge nodes were identified that spread COVID-19 vaccine misinformation throughout the rest of the network. Of specific interest was a bridge node that connected the QAnon group with an identified pro-Trump group.
The approach employed to gather and scrutinize data via the chosen seed hashtags is a possible constraint of this paper. The classification model used in this paper encounters challenges in distinguishing the intended meaning of profanity in the semantic context. Consequently, it frequently categorizes obscene language as toxic, regardless of the user's intent. In our future research, we will be mindful of this limitation and account for it accordingly. In future work, we plan to create and compare Twitter and Parler mentions, shared URL, and retweet/echo networks. These additional analyses will improve our ability to identify misinformation/conspiracy theories and to identify the users and communities that spread them. The other concerns that we can pursue for future work are users who are suspicious of being a bot. By using Botometer, we are able to detect the users who have the highest probability of being a bot. So by eliminating them we can build our network again, calculate the toxicity and expand our analysis. We will also further explore the vaccine and lockdown topics due to their exhibiting notably higher toxicity. We will also expand the analysis with the addition of topic modeling and models for the diffusion of information on OSNs. These efforts provide an important perspective on the effects of the differences between platform moderation and are the first step in a cross-platform analysis of toxicity with implications for public health and public trust.
This research is funded in part by the U.S. National Science Foundation (OIA-1946391, OIA-1920920, IIS1636933, ACI-1429160, and IIS-1110868), U.S. Office of the Under Secretary of Defense for Research and Engineering (FA9550-22-1-0332), U.S. Office of Naval Research (N00014-10-1-0091, N00014-14-1-0489, N00014-15-P-1187, N00014-16-1-2016, N00014-16-1-2412, N00014-17-1-2675, N00014-17-1-2605, N68335-19C-0359, N00014-19-1-2336, N68335-20-C-0540, N00014-21-1-2121, N00014-21-1-2765, N00014-22-1-2318), U.S. Air Force Research Laboratory, U.S. Army Research Office (W911NF-20-1-0262, W911NF-16-1-0189, W911NF-23-1-0011), U.S. Defense Advanced Research Projects Agency (W31P4Q-17-C-0059), Arkansas Research Alliance, the Jerry L. Maulden/Entergy Endowment at the University of Arkansas at Little Rock, and the Australian Department of Defense Strategic Policy Grants Program (SPGP) (award number: 2020-106-094). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding organizations. The researchers gratefully acknowledge the support.
[18] Obadimu, A., Khaund, T., Mead, E., Marcoux, T., and Agarwal, N., 2021. “Developing a socio-computational approach to examine toxicity propagation and regulation in COVID-19 discourse on YouTube,” Information Processing and Management, pp. 58, https://doi.org/10.1016/j.ipm. 2021.102660. [19] Pascual-Ferrá, P., Alperstein, N., Barnett, D., and Rimal, R., 2021. “Toxicity and verbal aggression on social media: Polarized discourse on wearing face masks during the COVID-19 pandemic,” Big Data and Society, pp.8, https://doi.org/10.1177/20539517211023533. [20] Chandrasekara, D., Sedera, D., & Gao, C., 2021. “Determining Boundary Conditions of Social Influence for Social Networks Research,” Australasian Journal of Information Systems (25). [21] Trinkle, B. S., Warkentin, M., Malimage, K., and Raddatz, N., 2021. “High-risk deviant decisions: does neutralization still play a role?.” Journal of the Association for Information Systems ( 22:3). [22] Aliapoulios, M., Bevensee, E., Blackburn, J., Bradlyn, B., De Cristofaro, E., Stringhini, G., and Zannettou. S. 2021. “A Large Open Dataset from the Parler Social Network,” Zendo, https://doi.org/10.5281/zenodo.4442460. [23] https://networkx.guide/algorithms/community-detection/girvan-newman/ [24] Roose, K., 2021. “What Is QAnon, the Viral Pro-Trump Conspiracy Theory?,” NY Times, https: //www.nytimes.com/article/what-is-qanon.html. [25] Rahn, W., and Patterson, D., 2021. “What is QAnon? What does WWG1WGA mean? The conspiracy theory that explains everything and nothing,” CBS News, https://www.cbsnews.com/news/what-is-the-qanon-conspiracytheory/. [26] McDonald, S., 2021. “New ‘Plandemic’ Video Peddles Misinformation, Conspiracies,” FactCheck, https://www.factcheck.org/2020/08/new-plandemic-video-peddlesmisinformation-conspiracies/. [27] Peters, J., 2020. “Twitter will remove misleading COVID-19-related tweets that could incite people to engage in ‘harmful activity,” The Verge, https://www.theverge.com/2020/4/22/21231956/twitter-remove-covid-19-tweetscall-to-action-harm-5g.