In the short amount of time, many COVID-19 datasets have been released. Most of these datasets are generic, and lack annotations or labels. Examples include multilingual corpus on a wide variety of topics related to COVID-19 [CLF20, AMEP+20, HJB+20], longitudinal Twitter chatter dataset [BTW+20], multilingual dataset with location information of the users [QIO20], Twitter dataset for Arabic tweets [AAA20], Twitter dataset for popular Arabic tweets [HHSE20], and dataset for identi cation of stance, replies, and quotes [VCKBC20]. Most of these datasets either have no annotations at all, employ automated annotations using transfer learning or semi-supervised methods, or are not speci cally designed for misinformation.

In terms of datasets collected for COVID-19 misinformation analysis and detection, examples include CoAID [CL20] which contains automatic annotations for tweets, replies, and claims for fake news; ReCOVery [ZMFZ20] is a multimodal dataset annotated for tweets sharing reliable versus unreliable news, annotated via distant supervision; FakeCovid [SN20] is a multilingual cross-domain fake news detection dataset with manual annotations; and [DSW20] is a large-scale Twitter dataset also focused on fake news. A survey of the di erent COVID-19 datasets can be found in [LUM+20] and [SAAA20].

In terms of the diversity of the classes, and the size of the dataset, the most relevant dataset is by Alam et al. [ASN+20] who, like our study, present a comprehensive codebook to annotate tweets on a ner granularity. Their dataset, however, is limited to a few hundred tweets, and our dataset is much more diverse in the range of topics covered. Dharawat et al. [DLMZ20] present a similar dataset with focus on the severity of the misinformation. However, their dataset does not consider the di erent \types" of misinformation. Finally, Song et al. present a dataset in [SPJ+20] which contains a diverse set of 10 categories, but still is not as large, and contains fewer categories in relation to the dataset collected within our study. 2.2

A plethora of research has already been conducted for analysing COVID-19 misinformation online. Some examples include categorization and identi cation of misinformed users based on their home countries, social identities, and political a liation [HC20, SSM+20], characterization of di erent types conspiracy theories propagated by Twitter bots [Fer20], characterization of the prevalence of low-credibility information related to COVID-19 [YTLM20], exploratory analysis of the content of COVID-19 tweets [OPR20, SDM20], understanding the types, sources, and claims of COVID-19 misinformation [BSHN20], and comparison of the credibility of COVID-19 tweets to datasets pertaining to other health issues [BKF+20]. To the best of our knowledge none of the studies have characterized COVID-19 misinformation communities in terms of their sociolinguistic patterns. In this study, we do not characterize the misinformation content directly. Instead, we conduct a set of analysis to understand and characterize the competing COVID-19 communities through their content, and content-sharing behaviors and interactions.

To collect Twitter dataset, we use Twitter search API using a diverse set of keywords as shown in table 1 to collect data. We collected our data on three days: 29th March 2020, 15th June 2020, and 24th June 2020. Each of these collections extracted a set of tweets from their corresponding week. For the annotation process, tweets were randomly sampled from that set.

Terms bleach, vaccine, acetic acid, steroids, essential oil, saltwater, ethanol, children, kids, garlic, alcohol, chlorine, sesame oil, conspiracy, 5G, Keywords cure, colloidal silver, dryer, bioweapon, cocaine, hydroxychloroquine, chloroquine, gates, immune, poison, fake, treat, doctor, senna makki, senna tea #nCoV20199, #CoronaOutbreak, #CoronaVirus, #CoronavirusCoverup, #CoroHashtags navirusOutbreak, #COVID19, #Coronavirus, #WuhanCoronavirus, #coronaviris, #Wuhan Our annotation task aims to determine the category to which a given tweet belongs to. After many discussions and revisions, we identify 17 categories that a particular tweet could classify to. These 17 categories are de ned in table 2. These categories are de ned in further detail along with their de nitions and examples in our codebook which we make available for the public to use.

Based on these categories, tweets were randomly and uniformly sampled from the data collection to maintain diversity in terms of topics covered. In the rst phase around 4573 tweets were annotated by a single annotator. Table 2 shows the distribution of the data in terms of the di erent categories as annotated by the rst annotator. In the second phase, 651 of these annotated tweets were assigned randomly to 6 other annotators.

Conducting analyses for a competing set of communities requires identifying those communities rst. Because we have already annotated data across a set of true and false information categories, we identify the membership of the users by assigning a valence of +1 to the categories True Treatment, True Prevention, Correction/Calling Out, Sarcasm/Satire, and True Public Health Response, and a valence of -1 to the categories Conspiracy, Fake Cure, Fake Treatment, False Fact or Prevention, and False Public Health Response. Note that we assign the valence to the categories (or annotations) and not the tweets themselves. This is so that we can leverage the annotations from multiple annotators. At the end, we compute the valence of each user as a weighted sum of the valence of the annotations assigned to their tweets. Then we use the valence assigned to each user to identify their membership i.e. if valence is greater than 0, the user is assigned to the informed group, and if the valence is less than 0, the user is assigned to the misinformed group. Out of 3629 users, the community detection process assigns 47% (1697) of the users to the informed group, 29% (1043) of the users to the misinformed group, and 24% (889) of the users to ambiguous or irrelevant category1. 5.2

Because our goal is to characterize communities and their behaviors, once we identify the two communities, we collect the timelines of users in each community to augment our data. Our hypothesis is that these additional posts can be used to mitigate survivorship bias [BGIR92] within our analyses. To conduct network analysis, bot analysis, and sociolinguistic analysis, we rst extract only the COVID-19 related tweets from 1Irrelevant users are users who have only posted tweets within other categories such as \Politics" or \Emergency". Because these categories do not have an assigned valence related to misinformation, they are not relevant for the purposes of this study. the timelines of each user. We do this by ltering all the tweets by the case-insensitive keywords \corona" and \covid". This yields a total of 330609 tweets with an average of 91 tweets per user. 5.3

To conduct network analysis, we extract the retweet, mention, and reply networks of the two target communities, and combine those networks together. We then compute the network density for each of the two groups. As described in [MTMC20], network density is de ned as the ratio of actual connections and potential connections. In dense networks, conformity of the ideas is highly encouraged, and di erence of opinions is discouraged. We also use ORA-PRO [CRC, ACR17, ACR18] to plot the network graph as shown in gure 2

We note that both the informed and misinformed users display echo-chamberness with misinformed subcommunities being much denser than the informed sub-communities as shown in table 3. We do, however, notice some two-way communication from both sides.

We also plot the retweet, mention and reply network separately as shown in gure 3. While retweet, and mention network show little to no two-way communication, we can observe that the reply network, while small in size, does in fact have much more intergroup engagement. We hypothesize that this is likely a consequence of the \corrective" or \calling-out" behavior. To understand the role of bots within the two competing groups, we used Bot-Hunter [BC18b, BC18a, BCB+18, BC20], which has a precision of .957 and a recall of .704, to identify potential bot-like accounts. We use the probability of greater than or equal to .75 as our con dence threshold to identify bots. We use a two-sample z-test for the di erence of proportions ( = 0:05) to test the di erence in proportion of bots between the two competing groups of users. The results of our analyses can be found in table 4.

We observe that from a total of 3629 users, 14% (505) of the users are identi ed as bots. The percentage of bots within identi ed misinformed users, however, is much higher (19%) than within identi ed informed users (11%). We nd our results to be statistically signi cant (p < 0:001; z = 6:23). This indicates that more than 1/5th of the misinformation related posts in our dataset are potentially a result of disinformation campaigns related to COVID-19. 5.5

To understand the linguistic di erences between the two competing communities, we conduct a linguistic analysis based on the tweets of the two groups by using the Linguistic Inquiry and Word Count (LIWC) program [PBJB15]. LIWC is a text analysis tool which looks at the di erent lexical categories each of which is psychologically meaningful. For a given text, LIWC calculates the percentage of each LIWC categories. All of these categories are based on word counts.

We run the LIWC program on the timelines of all the members for each of the two competing groups. We only use tweets relevant to COVID-19. We also remove users identi ed as bots. Because some users may be more active than others, using the results of the program as is may introduce biases in our analyses. To account for those biases, we rst normalize the percentages by the size of the data for each user. We use the mean of the normalized LIWC indices of tweets of individual users for a given lexical category as our test statistic. We use an independent z-test for the di erence in means to establish statistical signi cance. For all our tests, = 0:05. Our analyses are summarized in table 5.

For this part, we focus on investigating three linguistic dimensions, each of which, along with its linguistic correlates, is described below. 5.5.1

Narratives play a central role in how individuals process information, communicate, and reason [Ves17]. We set to test the di erences in the usage of narratives or anecdotes between the two COVID-19 misinformation communities. The LIWC correlates for narrative discourse structure include high usage of function words, pronouns, analytic summary dimension, and authenticity. High usage of function words and pronouns happens more often when expressing feelings and behaviors which tends to happen frequently in narratives [Pen11]. Moreover, low analytical thinking also suggests narrative language [PBJB15]. Furthermore, authentic individuals tend to be more personal, humble, and vulnerable [PBJB15]. Therefore, we use all of these as proxies to identify variation in the use of narratives across communities.

In the past [MTMC20], it has also been suggested that misinformed communities (eg. anti-vaxxers) tend to use many more pronouns suggesting highly narrative discourse structure. In this analysis, however, we nd that informed users in the COVID-19 discourse use signi cantly more pronouns, more functional words, mention more family-related keywords, are less analytical, and more authentic and honest in comparison to misinformed users. All of these suggest that informed users may use many more narratives than misinformed users. This is an interesting nding as it presents a dichotomy between the di erent misinformation communities (eg. anti-vaxxers and COVID-19 misinformed community). In hindsight, this is also an intuitive result, as our informed group is obtained from corrective discourse where users present their stories of family members or friends su ering from COVID-19 to call out conspiracies and false information. Because the two communities still seem to have less two-way communication, this also suggests that just the content and framing of the message (i.e. narratives) may not be enough, and perhaps there is a need to connect the two groups by identifying an e ective medium of communication. 5.5.2

Tone describes how positive a given text is. According to the de nition by LIWC, the higher the tone index, the more positive the tone. Indices less than 50 typically suggest a more negative tone. While we do not see signi cant di erences in the emotional tone of the competing groups, we nd both the communities to be highly negative. 5.5.3

Formality of the language has often been considered as one of the most important dimensions for stylistic variation. In [GMC+14], authors de ne linguistic formality as a style of writing that is meant to be precise, coherent, articulate and convincing to an educated audience, as opposed to informal discourse which is lled with deictic references (eg. here, there), pronouns, and narration. The LIWC correlates to this dimension are swear words (swear), and informal language (informal). Informal language in LIWC is computed on the bases of swear words, netspeak (eg. btw, lol), non uencies (eg. err, hmm), assents (eg. agree, OK), and llers (eg. youknow).

From table 5, it can be observed that misinformed users tend to be more informal than informed users, though informed users tend to use more swear words than misinformed users. This is intuitive as many of our informed users post corrective or sarcastic tweets to call out misinformation. However, our results are not signi cant, and, hence inconclusive. 5.6

To understand the interplay between the di erent kinds of misinformation themes and communities, we identify the vaccination-related stance of the members of the misinformed sub-community. To do that, we rst identify the subset of misinformed community who have posted at least one tweet related to \vaccines" in the past. We then collect the user-to-hashtag co-occurrence network. We use the valence of the vaccination hashtags obtained via the label propagationbased method mentioned in the study in [MTMC20] to identify the stance of each member (pro vs. anti) based on the weighted sum of the valences of the hashtags. If the weighted sum is greater than 0, we identify the member as pro-vaxxer, and if the weighted sum is less than 0, we identify the member as anti-vaxxer. The distribution of the pro- and anti-vaxxers within the COVID-19 misinformed group is as shown in table 6.

We observe that from 1027 COVID-19 misinformed users in our dataset, 41% of the members are identied as anti-vaxxers, whereas only 22% of the members are identi ed as pro-vaxxers. The di erence between the proportions of the two communities is signi cantly high. We also identify the proportion of bots within each of the two groups: misinformed pro-vaxxers, and misinformed anti-vaxxers. As shown in table 6, 17% of the misinformed pro-vaxxers are bots, which is signi cantly lower than the proportion of bots within the misinformed anti-vaxxers. The rst thing this suggests is that a big chunk of COVID-19 misinformation online may in fact be disinformation, and hence, intentional. The existence of bots within both the informed and misinformed communities also suggests that much of the disinformation online may be an organized e ort to amplify the COVID-19 debate to create discord in the communities as seen in the past with Twitter bots and Russian trolls [BJQ+18]. 6

This work was partially supported by a fellowship from Carnegie Mellon University's Center for Machine Learning and Health to Shahan A. Memon. We thank David Beskow for access to his Bot-Hunter model for bot analysis. We also thank members of CMU's Center for Computational Analysis of Social and Organizational Systems (CASOS) for insightful comments and discussions related to the data codebook and its revisions. [AAA20]