Understanding Community Rivalry on Social Media: A Case Study of Two Footballing Giants Sopan Khosla Siddhant Arora Abhilash Nandy skhosla@adobe.com cs5150480@iitd.ac.in nandyabhilash@gmail.com Adobe Research IIT Delhi IIT Kharagpur Bangalore, India New Delhi, India Kharagpur, India Ankita Saxena Anandhavelu N ankitasonu24@gmail.com anandvn@adobe.com IIT Roorkee Adobe Research Roorkee, India Bangalore, India ABSTRACT ACM Reference Format: Detection of hate speech in online user generated content Sopan Khosla, Siddhant Arora, Abhilash Nandy, Ankita Saxena, and Anandhavelu N. 2019. Understanding Community Rivalry on has become of increasing importance in recent times. Hate Social Media: A Case Study of Two Footballing Giants. In Joint speech can not only be against a particular user but also Proceedings of the ACM IUI 2019 Workshops, Los Angeles, USA, March against a group. Rivalry between two communities with 20, 2019 , 8 pages. opposing ideologies has been observed to instigate a lot of hate content on social media during controversial events. 1 INTRODUCTION Moreover, this online hate content has been observed to have Social media is a powerful communication tool that has facil- power to shape exogenous elements like communal riots [4, itated easy exchange of points of view. While it has enabled 6]. In this paper, we aim to analyze community rivalry in the people to interact with like-minded people, share informa- football domain (Real Madrid FC vs FC Barcelona) based on tion and support during a crisis [25], it has also resulted in the hate content exchanged between their supporters and a rise of anti-social behavior including online harassment, understand how events affect the relationship between these cyber-bullying, and hate speech [14]. With more and more clubs. We further analyze the behavior of key instigators of people sharing web content everyday, in particular on online hate speech in this domain and show how they differ from social networks (OSNs), the amount of hate speech is also general users. We also perform a linguistic analysis of the steadily increasing. In recent years, we have witnessed a hate content exchanged between rival communities. Overall, growing interest in the area of online hate speech detection our work provides a data-driven analysis of the nuances and particularly the automatization of this task. Social net- of online hate speech in the football domain that not only working communities like Facebook and Twitter are putting allows a deepened understanding of its social implications, forth hateful conduct policies [15, 36] to tackle this issue. but also its detection. User-defined communities are an essential component of many web platforms, where users express their ideas, CCS CONCEPTS opinions, and share information. These communities also • Information systems → Social networks; • Computing facilitate intercommunity interactions where members of methodologies → Natural language processing; Neural net- one community engage with members of another. Studies of works; • Social and professional topics → User character- intercommunity dynamics in the offline setting have shown istics. that intercommunity interactions can lead to the exchange of information and ideas [3, 17, 29] - or they can take a negative KEYWORDS turn, leading to conflicts [31]. If the participating commu- social media; OSN; hate speech; online communities; NLP; nities have opposing views, their interactions can lead to time-series; causal inference exchange of hate content that maybe directed towards the community’s ideology (or interest, team) or its members. These online exchanges can also lead to on-the-ground com- IUI Workshops’19, March 20, 2019, Los Angeles, USA munal violence [4, 6]. Copyright © 2019 for the individual papers by the papers’ authors. Copying permitted for private and academic purposes. This volume is published and In this work, we present the first comparative study on the copyrighted by its editors. exchange of hate content during inter-community interac- tions in football fan communities. This hate maybe directed IUI Workshops’19, March 20, 2019, Los Angeles, USA Khosla et al. Users (Tweets) Silva et al. [30] use regex patterns like "I hate General Community Mentions HS(S:all) HS(S:rival) ." to identify hate target groups in terms of 564,770 10,105 their class and ethnicity. Their system has very low recall as Real Madrid FC 3,502 (4,234) (2,065,846) (12,827) they only rely on very specific sentence structures. Another 307,931 6,855 line of work [13] identifies individual targets using mentions FC Barcelona 1,774 (2,112) (894,479) (12,568) in the hate tweets and uses Perspective API’s toxicity and Table 1: Data Statistics attack_on_commenter scores to detect if the hate speech is against the mentioned individual. In this work, we leverage the prior art in the area of stance detection for target-specific towards the club in general or the members (e.g. players, sup- hate speech detection [32]. porters, manager etc.) of that club. Specifically, we analyze the rivalry between two Spanish football communities, Real 3 BACKGROUND Madrid FC and FC Barcelona, on Twitter. Real madrid FC We define hate speech (HS) and hateful users (HU) accord- and FC Barcelona are considered two of the biggest football ing to the guidelines put forth by Twitter [36]. Any content clubs in the world and enjoy large support all over the world. that promotes violence against or directly attacks or threatens Prior work [19] studies how members of one community other people on the basis of race, ethnicity, national origin, attack the members of other community on Reddit, which sexual orientation, gender, gender identity, religious affiliation, provides an explicit platform for users to create and partici- age, disability, or serious disease is considered as "hate speech". pate in interest-based communities (called subreddits in case On the other hand, "hateful user" is a user that perpetrates of Reddit). However, for our study, we choose Twitter as it such type of content. Target Community (T) is defined as the provides a larger cross-section of general public. ideology, team, interest, ethnicity etc. which the intended In order to characterize the dynamics of hate between recipients of hate speech belong to. Whereas, source com- these communities, We first try to understand how hate munity (S) is the ideology, team, interest, ethnicity etc. that speech changes over time. Specifically, does hate speech characterizes the group of users who post one or more hate increase over time and does it spike during external events. tweets. For example, if a FC Barcelona supporter posts hate Next, we try to understand the characteristics of users who against Real Madrid or its members then FC Barcelona is spread hate and then we try to understand the hateful tweets considered the "source community" and Real Madrid is the themselves. We also try to analyze how hate from the rival "target community". community is similar or dissimilar to the general hate against a target community and its members. 4 DATA COLLECTION In this section, we provide details about the data collection 2 RELATED WORK and pre-processing pipeline. There have been notable contributions in the area of hate Data Sources We collect data from three main sources. We detection in Online Social Networks (OSN) and websites. [22, collect tweets from Twitter using tweepy API, match fixtures 24] use lexical features like word and character n-grams, av- and outcomes from Foxsports.com and other match statistics erage word embeddings, and paragraph embeddings. Other from espn.com. works [6, 11, 21, 30, 37] have leveraged profane words, part- Target specific tweets We collect relevant tweets in English of-speech tags, sentiment words and insulting syntactic con- language (via Twitter Search API) that mention the target structs in pre-processing and as features for hate classifica- community of interest, from June 2017 to May 2018. We tion. Models used in the existing literature include supervised create a list of entities (players, managers, owner, etc.) that classification methods such as Naive-Bayes [20], Logistic belong to the target community of interest and use them as Regression [38] [10], Rule-Based Classifiers [18], Random our search keywords for this. Forests [7] and Deep Neural Networks [1]. Preliminary hate filtering We adopt a high recall data col- There have been fewer efforts towards characterizing hate- lection mechanism to represent a fair sense of hate speech ful users online (people who post hate speech in OSNs). in our domain. Similar to [13], we use a lexicon of abusive Chatzakou et al. [8] study the Twitter users in the context words adopted from [39] to retrieve English hate terms. Af- of #GamerGate controversy. In another work, Chatzakou ter removing phrases that are context dependent, we use et al. [9] use a supervised model to classify Twitter users the resultant list of hate words to filter the extracted target into four classes: bully, aggressive, spam, and normal. Rudra specific tweets. el al. [27] characterize Twitter users, who post communal Source Community Identification We identify the com- tweets during disaster events, based on their popularity, in- munity to which the Twitter user belongs by extracting their terests, and social interactions. friends. We check if the user follows the official pages of Understanding Community Rivalry on Social Media IUI Workshops’19, March 20, 2019, Los Angeles, USA Figure 1: Schematic diagram of our target-specific hate speech detector the community of interest (or its members) to categorize Is this hate directed towards the target community? him as member of that community. Using this differentiation Stance is used to define target-specific opinion (as against process, we categorize users into Barcelona supporters/ Real a general opinion) which can be favor, against or neutral. Madrid supporters/ neither. Stance helps to disambiguate between the generic sentiment or opinion of an individual with what the individual is refer- ring to. In this work, we leverage stance detection algorithms 5 TARGET-SPECIFIC HATE SPEECH DETECTION as the second step in our target-specific hate detector. Tweets collected in the previous section merely mention To perform stance detection in tweets, we leverage a state- the target community. This does not guarantee that they of-the-art TC-LSTM model introduced by [33] for target- are actually talking about it. Also, despite the qualitative dependent sentiment classification. inspection of keyphrases, the filtered dataset still contained non-hate speech tweets. To mitigate the effects of obscure contexts in the filtering process, we propose a two-step clas- sifier that would provide us with tweets that contain hate 6 ANALYSIS speech against the target entity. In this section, we present the analysis on the hate dynam- Figure 1 shows the workflow for the proposed framework. ics between two footballing giants - Real Madrid FC (also A tweet is first passed through the hate-speech detection referred to as madrid) and FC Barcelona (barca). Tweets model. If the model classifies the text as positive for hate categorized by our model as hate speech against the target speech, then it is input to the stance detector along with community t by users of source community s are referred the target entity of interest. If the stance detection model to as HS(S:s, T:t) and the corresponding users who posted classifies it as negative towards the target, then we assert them as HU(S:s, T:t). Furthermore, S:all is used to represent that tweet contains hate-speech against the target entity. all hate against a target community. Does this tweet contain hate speech? There has not been much prior work in modeling hate speech Is hate exchange a year round event? specific to sports domain. Therefore, we use an existing hate We plot the time-series (Figure 2) of total number of hateful speech detection model trained on dataset from a different tweets exchanged between Real Madrid FC and FC Barcelona domain and see how it fares in our domain. in the period ranging from June 2017 to May 2018 (football We use an LSTM model [1] trained on two popular datasets season 2017-18). We observe that the number of tweets with of general tweets, manually annotated for hate speech, to de- hate speech spike in isolation. A close inspection maps these tect if hate speech is present in the tweet. Dataset introduced spikes to football matches (also called events in rest of the in [16] consists of 70K tweets manually annotated as abusive, paper) in which the target community was playing. In the hateful, normal and spam whereas the dataset proposed in absence of these events, we do not find a substantial amount [38] categorizes 20K tweets into sexist, racist and neither. of hate speech and hate speech in general does not seem to We consider the tweets labeled as abusive, hateful, sexist or increase with time. racist in the datasets as positive for hate speech. Spam sam- Studies [6, 28] have shown that online hate speech has ples are not used for training. We use an LSTM for modeling increased over the years. However, it is difficult to address as it has been shown to capture the long-range dependencies this question in retrospection as several offensive tweets are in tweets, which may play a role in hate-speech detection. taken down by Twitter soon after they are posted. IUI Workshops’19, March 20, 2019, Los Angeles, USA Khosla et al. 1,000 1,000 HS(S:barca, T:madrid) HS(S:madrid, T:barca) 800 800 600 600 400 400 200 200 0 0 0 100 200 300 400 0 100 200 300 400 Figure 2: Number of hate tweets exchanged between Real Madrid FC and FC Barcelona in 2017-18 season. Vertical lines denote a sample of events during the season. Green corresponds to el-classico matches; black and orange lines represent matches which saw smallest and largest amount of hate respectively from the rival community. ·10−2 ·10−2 5 HS(S:all, T:madrid) HS(S:barca, T:madrid) 1 4 0.8 3 0.6 ratio ratio 2 0.4 1 0.2 0 0 0 100 200 300 400 0 100 200 300 400 Figure 3: Ratio of ‘hate speech towards Real Madrid FC’ to ‘General Mentions of Real Madrid FC’ during the 2017-18 season. Impact of offline events on hate speech online A. Observed Time-Series: We define our treatment series with Following the inference from the previous section, we now a timespan of two days before the match as pre-treatment try to quantify the impact an event had on online hate speech. period and two days after the match as post-treatment period. We posit that an event (a football match in our case) has high B. Synthetic Control Group creation: We then identify possible impact on hate speech if it results in an increased relative control groups as time-series that occur in history, during amount of hate speech against non-event days. To analyze same days of the week as the treatment series, with no event the impact of events on online hate exchange between the taking place during this time interval. We rank all control two communities of interest, we plot a time-series of the groups based on their similarity to the treatment group using ratio of HS(S:s,T:t) to general tweets mentioning target com- Wilcoxon signed rank test and select the top 2 ranked time munity t. (Figure 3 shows the time-series for T:madrid). We series for creating our synthetic control time series. quantify their effect by treating them as interventions on ob- C. Impact Estimation: We finally use the difference between served time series. Following [25], we use Brodersen et al.’s observed post treatment time series and the synthetic con- technique [5] for causal inference on time-series to quantify trol time series to calculate the impact of event. The relative the impact of football events on hate speech. The behavior increase in online hate speech during an event is given by: of observed time-series (treatment) after a football match is tk − c k Í relef f ect = 100 ∗ Í (1) compared with a counterfactual time series (control). Since, ck we do not observe the control time-series, we model it from where tk is the value of the treatment time series at time k, observed time-series (in different time ranges) that correlate and c k that of the control time series. with the treatment series but were not affected by the event. Finally we use this setup to model the counterfactual of the Do all events contribute to the hate speech equally? Our re- treated time series using difference-in-differences approach. sults show that outcome of matches seems to affect the hate Understanding Community Rivalry on Social Media IUI Workshops’19, March 20, 2019, Los Angeles, USA Hateful General Hate Event Overlap Users Followers (% of total tweets) (% of total events) (HU) % Users who % Users who % Users % Users with Mean(%) Mean(%) Mean post > 10% post in > 20% with < 100 > 10, 000 T: madrid S:all 9.87 43.73 1.176 0.08 2570.58 21.66 2.46 S:barca 10.09 45.92 1.145 0.01 2062.83 18.29 2.64 T: barca S:all 9.10 36.17 1.480 0.05 3000.74 37.00 3.70 S:madrid 9.36 39.40 1.384 0.00 1069.66 30.45 1.60 Table 2: Characterizing hateful users Characterizing hateful users In this section we analyze HU(S:s, T:t). Users who post hate- ful content against the target community of interest. Do they post hate speech in general? We investigate if the hateful users in our domain use hate speech in their general tweets as well. We extract their 3200 most recent tweets and classify them using our hate speech detection model (explained in the earlier sections) after excluding tweets that mentioned entities related to Real Madrid FC or FC Barcelona. We observe that users who post hate in our domain also propagate significant hate in general with around 10% of their general tweets being hateful (Table 2). Our results show that HU(S: rival) post more hate in general as against HU(S: all) with a higher percentage of HU(S: barca, T: madrid) crossing the 10% mark as compared to HU(S: all, T: madrid) Figure 4: Example of impact estimation with (45.92% vs 43.73%). A similar trend is observed for T:barca. counterfactual predictions, for the event "Champions League 2018 final" between Real Madrid FC and Liverpool Do they post hate in multiple events? We then analyze the FC. Top: Black is the observed series before/after the event, user overlap across different events to see if there is a com- blue (dashed) is the counterfactual. mon set of users who post hate during multiple events. Any user who posts in a 24hr interval after the event is assumed to have posted because of that event. We find that around exchange. We find that losses and draws trigger greater hate 90% HUs write hate tweets for only 1−2% of football matches speech than wins (p < 0.001). We also observe that Home throughout the year. Whereas, less than 0.1% HUs post hate Losses and Draws trigger greater hate in comparison to Away in more than 20% of the total football matches in 2017-18 sea- Losses and Draws (p < 0.001). In contrast, Away Wins trigger son (Table 2). Members of the rival community HU(S:rival), greater hate than Home Wins (HU(S:all, T:madrid): p < 0.01; on average, show lower event overlap compared to HU(S:all). HU(S:barca, T:madrid): p < 0.001;) particularly from the This is consistent with the findings in the Analysis section rival community. Moreover, group games trigger less hate as the rival community is only interested in events which in comparison to championship games (HU(S:all, T:madrid): impact them directly. p = 0.05; HU(S:barca, T:madrid): p < 0.05;). Matches which lead to elimination (HU(S:all, T:madrid): p < 0.001; HU(S:all, Are they popular? Next, we check if the users who write T:madrid): p < 0.05;) trigger greater hate. Matches played hate tweets enjoy popularity on Twitter. We use number against the rival community tend to bring more hate from of followers as a metric to do the popularity analysis. As the rival community (p < 0.001). More generally, we find shown in Table 2, target-specific hate in our domain is posted that rival community posts disproportionately higher hate by common masses (less than 100 followers) whereas, the for matches whose results directly impact them. A similar popular users (more than 10,000 followers) seldom (< 4%) trend is observed for t = barca (omitted for brevity). participate in this phenomenon. Popular members of Real IUI Workshops’19, March 20, 2019, Los Angeles, USA Khosla et al. Madrid FC community seem to avoid hate speech against FC Exp. Top 10 Salient Words Barcelona (1.6%). f**k, b**ch, f**ked, f**king, adore, c*nt, yer, kid, (i) cheating, idiot What are their key personality traits? To study the key char- f**k, f**ked, f**king, b**ch, c*nt, bulls**t, adore, (ii) acteristics of the personalities of HUs in our domain, we use sh**ty, sh*t, cheating the Twitter REST API to fetch the most recent 3200 tweets granada, rampant, messi, bartomeudimiteya, , for each account. We exclude retweets as they might not re- (iii) valverde, forcabarca, viscabarca, yer, madridiots, penaldo flect author’s point of view and use IBM Watson Personality Insights API1 for this analysis. It outputs a normalized per- Table 3: Top 10 salient words learned by SAGE for different centile score for the characteristic. We study the results of the experiments with T:madrid. Note the presence of barca Big Five personality model, the most widely used model for specific keywords (bold) in (iii). generally describing how a person engages with the world. The model includes five primary dimensions: Agreeableness, some base content (base). It creates clean topic models by Conscientiousness, Extraversion, Neuroticism, and Open- taking into account the additive effects and combines multi- ness [2]. ple generative facets like topic and perspective distribution Figure 5 shows the distribution of scores of the Big Five of words. In this analysis, we conduct three experiments (i) personality traits for T:madrid. We find that HS(S:all) and child = HS(S:all), base = tweets which mention the target HS(S:barca) have more similar personalities to each other community; (ii) child = HS(S:rival), base = tweets which men- than general mentions. Both HS(S:all) and HS(S:barca) ex- tion the target community; and (iii) child = HS(S:rival), base hibit lower Agreeableness than general mentions. Prior work = HS(S:all). We look at the top 10 salient words learned for [34] associates lower Agreeableness scores with suspicious the above-mentioned experiments (Table 3). and antagonistic behaviors. Our results indicate that HS(S:all) As a whole, both (i) and (ii) contain similar salient words. and HS(S:barca) are more self-focused, contrary, cautious of These words are mostly cuss words which is to be expected. others, and lack empathy. For Conscientiousness, HS(S:all) The top salient words in (iii) contain mentions of the entities and HS(S:barca) generally have lower scores than general of the source community. On closer look, we find that these mentions. Our results suggest that these users are laid back, tweets try to demean the target community (e.g. players, less goal-oriented, and tend to disregard rules. Low Extraver- managers or ideology etc.) in an attempt to glorify the source sion scores for both HS(S:all) and HS(S:barca) show that they community. For example, this hate tweet by a Real Madrid FC are less sociable, less assertive, and more within themselves. supporter against FC Barcelona, king dem ronaldo king dem HS(S:all) and HS(S:barca) have slightly higher, but statisti- left and right salute d king..i want to take this opportunity and cally significant, scores for Neuroticism which indicates that say f**k all barcelona fans @fcbarcelona _es, tries to glorify they are more susceptible to stress and are more likely to ex- Ronaldo (an ex Real Madrid FC player) by calling him a King. perience anxiety, jealousy and anger. However, for Openness, Such a pattern of comparison is not visible in HS(S:all) which the distributions for HS(S:all) and HS(S:barca) are close to mostly focuses on the negatives of the target community. general mentions (with median of approximately 0.19). We For example, this hate tweet against FC Barcelona from a observe a similar trend for T:barca but omit here for brevity. user who is not a Real Madrid FC community member, Dear @FCBarcelona , please take your sh*t (Bellerin) back. Please!. Characterizing hateful tweets Are these hate tweets popular? We next investigate the popu- Psycholinguistic Analysis. We use LIWC2015 [26] for a full larity of target-specific hate tweets in our domain. We use psycholinguistic analysis. We look at the following dimen- the retweet-count of tweets to judge their popularity. We ob- sions: summary scores, personal pronouns, and negative serve that for target community FC Barcelona (T:barca), hate- emotions.2 In Figure 6 we can see that tweets with general ful tweets from Real Madrid i.e. HS(S:madrid, T:barca), are (non-hate) mentions (NHM) of Real Madrid FC differ signifi- retweeted less than general hateful tweets HS(S:all, T:barca) cantly from hateful tweets. Summary scores suggest that gen- (with mean = 0.19 and 0.53 respectively) which in turn are eral tweets display higher values of tone than HS(T:madrid) substantially less popular than non-hateful tweets (mean = suggesting that targeted hate speech is more hostile. HS(T: 5.47). madrid) contains higher number of pronouns and is angrier than general tweets. Also, HS(T:madrid) is more informal and Content Characteristics. We use SAGE [12] to analyze salient expectedly contain more swear words. It contains shorter words that characterize different types of tweets. SAGE at- sentences and uses more dictionary words on average as tempts to find salient terms in a text (child) with respect to 2 LIWC2015 language manual [26] provides a detailed description for these 1 https://www.ibm.com/watson/services/personality-insights/ dimensions. Understanding Community Rivalry on Social Media IUI Workshops’19, March 20, 2019, Los Angeles, USA Figure 5: Distribution of scores for the Big Five personality traits for users who posted tweets that mention Real Madrid FC. General Mentions are tweets that mention the target community or its associated entities. 100 HS(S:all) HS(S:barca) NHM 2 HS(S:all) HS(S:barca) NHM media. We show how rival communities post disproportion- 80 1.5 ately high amount of hate during events which have a direct 60 impact on their team’s interests. We find that hateful users Score Score 40 1 in our domain also post hate speech in general. They do 20 0.5 not post hate in multiple events and do not enjoy generous 0 0 popularity on social media. We show that their personality Tone Analytic Clout WPS Dic i we they you shehe characteristics are significantly different from general users (a) Summary (b) Personal Pronouns who post about the target community. Our analysis shows that hate tweets from rival community members differ in HS(S:all) HS(S:barca) NHM their theme from general hate tweets towards the target 6 community. They try to glorify their team’s players, playing style or ideology while demeaning the target community. 4 However, the psycholinguistic analysis of the hateful tweets Score 2 suggests that content from rival community does not differ from general tweets in terms of the emotional content, tone, 0 anger sad anx pronoun usage or swear words. Nonetheless, our analysis has limitations. Recent studies (c) Negative Emotions by Tufekci [35] and Morstatter et al. [23] have discussed the sample quality of the Twitter API. Since our analysis relies on Figure 6: Psycholinguistic Analysis of tweets which keyword-based methods for retrieval of explicit hate speech, mention Real Madrid FC (T:madrid). we cannot claim to have captured a complete representation of the hate exchange on Twitter. However, our main objective against the non-hate tweets mentioning Real Madrid FC. A was to characterize hateful users and tweets in the sports similar trend is observed for T:barca (omitted for brevity). domain with high precision and we believe that our careful filtering and classification models were able to do so. 7 DISCUSSION AND CONCLUSIONS In this work, we provide a novel view of hate exchange REFERENCES between different communities in the football domain. We [1] Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, and Vasudeva design a two-step model to detect target-specific hate speech. Varma. 2017. Deep learning for hate speech detection in tweets. Pro- Using causal inference methodologies, we are able to mea- ceedings of the 26th International Conference on World Wide Web Com- sure the effect of external events on hate speech on social panion, 759–760. IUI Workshops’19, March 20, 2019, Los Angeles, USA Khosla et al. [2] Murray R Barrick and Michael K Mount. 1991. The big five personality [21] Altaf Mahmud, Kazi Zubair Ahmed, and Mumit Khan. 2008. Detecting dimensions and job performance: a meta-analysis. Personnel psychology flames and insults in text. (2008). 44, 1 (1991), 1–26. [22] Yashar Mehdad and Joel Tetreault. 2016. Do Characters Abuse More [3] Václav Belák, Samantha Lam, and Conor Hayes. 2012. Cross- Than Words? Proceedings of the 17th Annual Meeting of the Special Community Influence in Discussion Fora. ICWSM 12 (2012), 34–41. Interest Group on Discourse and Dialogue, 299–303. [4] Susan Benesch. 2014. Countering dangerous speech to prevent mass [23] Fred Morstatter, Jürgen Pfeffer, Huan Liu, and Kathleen M Carley. violence during KenyaâĂŹs 2013 elections. Final Report (2014), 1–26. 2013. Is the Sample Good Enough? Comparing Data from Twitter’s [5] Kay H. Brodersen, Fabian Gallusser, Jim Koehler, Nicolas Remy, and Streaming API with Twitter’s Firehose. ICWSM. Steven L. Scott. 2015. Inferring causal impact using Bayesian structural [24] Chikashi Nobata, Joel Tetreault, Achint Thomas, Yashar Mehdad, and time-series models. Annals of Applied Statistics 9 (2015), 247–274. Yi Chang. 2016. Abusive language detection in online user content. [6] Pete Burnap, Omer F Rana, Nick Avis, Matthew Williams, William Proceedings of the 25th international conference on world wide web, Housley, Adam Edwards, Jeffrey Morgan, and Luke Sloan. 2015. Detect- 145–153. ing tension in online communities with computational Twitter analysis. [25] Alexandra Olteanu, Carlos Castillo, Jeremy Boy, and Kush R Varshney. Technological Forecasting and Social Change 95 (2015), 96–108. 2018. The Effect of Extremist Violence on Hateful Speech Online. arXiv [7] Pete Burnap and Matthew L. Williams. 2016. Us and them: identifying preprint arXiv:1804.05704 (2018). cyber hate on Twitter across multiple protected characteristics. EPJ [26] James W Pennebaker, Ryan L Boyd, Kayla Jordan, and Kate Black- Data Science 5, 1 (23 Mar 2016), 11. https://doi.org/10.1140/epjds/ burn. 2015. The development and psychometric properties of LIWC2015. s13688-016-0072-6 Technical Report. [8] Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Emiliano [27] Koustav Rudra, Siddhartha Banerjee, Niloy Ganguly, Pawan Goyal, De Cristofaro, Gianluca Stringhini, and Athena Vakali. 2017. Hate Muhammad Imran, and Prasenjit Mitra. 2016. Summarizing situational is not binary: Studying abusive behavior of# gamergate on twitter. tweets in crisis scenario. Proceedings of the 27th ACM Conference on Proceedings of the 28th ACM conference on hypertext and social media, Hypertext and Social Media, 137–147. 65–74. [28] SafeHome.org. 2017. Hate on Social Media. https://www.safehome. [9] Despoina Chatzakou, Nicolas Kourtellis, Jeremy Blackburn, Emiliano org/resources/hate-on-social-media/. De Cristofaro, Gianluca Stringhini, and Athena Vakali. 2017. Mean [29] Muzafer Sherif. 2015. Group conflict and co-operation: Their social birds: Detecting aggression and bullying on twitter. Proceedings of the psychology. Psychology Press. 2017 ACM on web science conference, 13–22. [30] Leandro Araújo Silva, Mainack Mondal, Denzil Correa, Fabrício Ben- [10] Thomas Davidson, Dana Warmsley, Michael Macy, and Ingmar Weber. evenuto, and Ingmar Weber. 2016. Analyzing the Targets of Hate in 2017. Automated hate speech detection and the problem of offensive Online Social Media. ICWSM, 687–690. language. Proceedings of ICWSM (2017). [31] Henri Tajfel and John C Turner. 1979. An integrative theory of in- [11] Karthik Dinakar, Birago Jones, Catherine Havasi, Henry Lieberman, tergroup conflict. The social psychology of intergroup relations 33, 47 and Rosalind Picard. 2012. Common sense reasoning for detection, (1979), 74. prevention, and mitigation of cyberbullying. ACM Transactions on [32] Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2016. Effective Interactive Intelligent Systems (TiiS) 2, 3 (2012), 18. LSTMs for target-dependent sentiment classification. International [12] Jacob Eisenstein, Amr Ahmed, and Eric P Xing. 2011. Sparse additive Conference on Computational Linguistics (2016), 3298–3307. generative models of text. Proceedings of the 28th International Confer- [33] Duyu Tang, Bing Qin, Xiaocheng Feng, and Ting Liu. 2016. Target- ence on International Conference on Machine Learning, 1041–1048. Dependent Sentiment Classification with Long Short Term Mem- [13] Mai ElSherief, Vivek Kulkarni, Dana Nguyen, William Yang Wang, ory. Proceedings of COLING 2016, the 26th International Conference and Elizabeth Belding. 2018. Hate Lingo: A Target-based Linguistic on Computational Linguistics: Technical Papers abs/1512.01100 (2016), Analysis of Hate Speech in Social Media. ICWSM (2018). 3298âĂŞ3307. [14] Mai ElSherief, Shirin Nilizadeh, Dana Nguyen, Giovanni Vigna, and [34] Ginka Toegel and Jean-Louis Barsoux. 2012. How to become a better Elizabeth Belding. 2018. Peer to Peer Hate: Hate Speech Instigators leader. MIT Sloan Management Review 53, 3 (2012), 51–60. and Their Targets. ICWSM (2018). [35] Zeynep Tufekci. 2014. Big Questions for Social Media Big Data: Repre- [15] Facebook. 2016. Controversial, Harmful and Hateful Speech on Face- sentativeness, Validity and Other Methodological Pitfalls. ICWSM 14 book. https://goo.gl/TWAHdr. (2014), 505–514. [16] Antigoni-Maria Founta, Constantinos Djouvas, Despoina Chatzakou, [36] Twitter. 2016. Hateful Conduct Policy. https://support.twitter.com/ Ilias Leontiadis, Jeremy Blackburn, Gianluca Stringhini, Athena Vakali, articles/20175050. Michael Sirivianos, and Nicolas Kourtellis. 2018. Large Scale Crowd- [37] Cynthia Van Hee, Els Lefever, Ben Verhoeven, Julie Mennes, Bart sourcing and Characterization of Twitter Abusive Behavior. Interna- Desmet, Guy De Pauw, Walter Daelemans, and Véronique Hoste. 2015. tional Conference on Social Web and Media Log (2018). Detection and fine-grained classification of cyberbullying events. In- [17] Howard Giles. 2005. Intergroup communication: Multiple perspectives. ternational Conference Recent Advances in Natural Language Processing Vol. 2. Peter Lang. (RANLP), 672–680. [18] Njagi Dennis Gitari, Zhang Zuping, Hanyurwimfura Damien, and [38] Zeerak Waseem and Dirk Hovy. 2016. Hateful Symbols or Hateful Jun Long. 2015. A lexicon-based approach for hate speech detection. People? Predictive Features for Hate Speech Detection on Twitter. International Journal of Multimedia and Ubiquitous Engineering 10, 4 Proceedings of the NAACL Student Research Workshop, 88–93. http: (2015), 215–230. //www.aclweb.org/anthology/N16-2013 [19] Srijan Kumar, William L Hamilton, Jure Leskovec, and Dan Jurafsky. [39] Michael Wiegand, Josef Ruppenhofer, Anna Schmidt, and Clayton 2018. Community interaction and conflict on the web. Proceedings of Greenberg. 2018. Inducing a Lexicon of Abusive Words–a Feature- the 2018 World Wide Web Conference on World Wide Web, 933–943. Based Approach. Proceedings of the 2018 Conference of the North Amer- [20] Irene Kwok and Yuzhou Wang. 2013. Locate the Hate: Detecting Tweets ican Chapter of the Association for Computational Linguistics: Human against Blacks. AAAI. Language Technologies, Volume 1 (Long Papers) 1, 1046–1056.