H ATE C HECKER: a Tool to Automatically Detect Hater Users in Online Social Networks Cataldo Musto Angelo Pio Sansonetti Marco Polignano University of Bari University of Bari University of Bari Dip. di Informatica Dip. di Informatica (Bachelor Student) Dip. di Informatica cataldo.musto@uniba.it a.sansonetti6@studenti.uniba.it marco.polignano@uniba.it Giovanni Semeraro Marco Stranisci University of Bari Associazione ACMOS Dip. di Informatica Torino giovanni.semeraro@uniba.it marco.stranisci@acmos.net Abstract 1 Background and Motivations According to a recent study1 , 58% of the Italian In this paper we present H ATE C HECKER, population regularly uses online social networks a tool for the automatic detection of hater as Twitter, Facebook, Instagram and LinkedIn. users in online social networks which has Such a huge diffusion of these platforms is pro- been developed within the activities of viding the users with many new opportunities and ”Contro L’Odio” research project. services, just think that almost everyone now uses In a nutshell, our tool implements a social media to get information, discuss, express methodology based on three steps: (i) all opinions and stay in touch with friends. Unfortu- the Tweets posted by a target user are nately, due to the lack of control and the absence gathered and processed. (ii) sentiment of a clear management of the concept of identity of analysis techniques are exploited to auto- the users, social networks have become the perfect matically label intolerant Tweets as hate place to spread hate against minorities and people speeches. (iii) a lexicon is used to clas- having different cultures, values and opinions. sify hate speeches against a set of spe- As pointed out by several works (Mathew et cific categories that can describe the tar- al., 2018), the diffusion of hate speeches in on- get user (e.g., racist, homophobic, anti- line social media is continuously growing and semitic, etc.). the countermeasures adopted by the single plat- forms are neither effective nor timely, even if a Finally, the output of the tool, that is to say, big effort is done to make the process of remov- a set of labels describing (if any) the in- ing hate speeches faster and more precise2 . Ac- tolerant traits of the target user, are shown cordingly, the research line related to the devel- through an interactive user interface and opment of tools and methods for the automatic exposed through a REST web service for detection of hate speeches gained more and more the integration in third-party applications. attention. Techniques for detecting hate speeches In the experimental evaluation we crawled are obviously based on NLP techniques, and range and annotated a set of 200 Twitter profiles from simple lexicon-based approaches (Gitari et and we investigated to what extent our tool al., 2015) to more sophisticated techniques that ex- is able to correctly identify hater users. ploit word embeddings (Djuric et al., 2015) and The results confirmed the validity of our deep learning methods (Badjatiya et al., 2017). methodology and paved the way for sev- Similar research attempts were also proposed eral future research directions. for the Italian language. One of the most popu- lar initiative is the Italian HateMap project (Musto 1 https://wearesocial.com/it/blog/2018/01/global-digital- report-2018 2 Copyright 2019 for this paper by its authors. Use per- https://www.cnbc.com/2019/02/04/facebook-google- mitted under Creative Commons License Attribution 4.0 In- and-twitter-are-getting-faster-at-removing-hate-speech- ternational (CC BY 4.0). online-eu-finds–.html et al., 2016), a research project that exploits se- • We evaluate several configurations (on vary- mantic analysis and opinion mining to identify ing of lexicons and sentiment analysis algo- the most-at-risk areas of the Italian country, that rithms) of the pipeline and we identified the is to say, the areas where the users more fre- most effective one to tackle our specific task; quently publish hate speeches. The interest of the research community for the topic was confirmed • We share the first publicly available dataset by the recent work by Bosco et al. (Bosco et for automatic detection of hater users on al., 2017), who studied hate speech against immi- Twitter. grants, and by Anzovino et al. (Anzovino et al., 2018) who detected misogyny on Twitter. More- In the following, we will first describe the over, as shown by the organization of a specific methodology we designed to implement our sys- task in the EVALITA evaluation campaign, an im- tem, then we will discuss the effectiveness of the portant effort is now devoted to the automatic de- approach by analyzing the results we obtained on tection of misogyny (Fersini et al., 2018) and hate a (publicly available) dataset of 200 Twitter users. speeches in general (Bosco et al., 2018; Basile et al., 2019). 2 Methodology In order to continue the investigation in this The workflow carried out by the H ATE C HECKER research line ACMOS3 , a no-profit associa- tool is reported in Figure 1. tion based in Torino, recently launched ”Contro Generally speaking, the pipeline consists of l’Odio4 ”, a joint research project with the Uni- four different modules, that is to say, a S OCIAL versity of Bari, University of Torino and several DATA E XTRACTOR, a S ENTIMENT A NALYZER, local associations. The project aims to develop a P ROFILE C LASSIFIER anda S OCIAL N ETWORK tools and methodologies to monitor (and hopefully P ROCESSOR. All these components use a NoSQL tackle) online hate speeches and intolerant behav- database to store the information they hold and iors. expose the output returned by the tool through a One of the outcomes of the research is H ATE - REST interface as well as through a Web Applica- C HECKER, a tool that aims to automatically iden- tion. In the following, a description of the single tify hater users on Twitter by exploiting sentiment modules that compose the workflow is provided. analysis and natural language processing tech- niques. The distinguishing aspect of the tool with 2.1 Social Data Extractor respect to the work we have previously introduced The whole pipeline implemented in the H ATE - is the focus of the tool itself. Indeed, differently C HECKER tool needs some textual content posted from most of the literature, that focused on the by the target user to label the user as a hater or analysis of single Tweets, H ATE C HECKER aims to not. In absence of textual content, it is not possi- analyze the users as a whole, and to identify hater ble provide such a classification. To this end, the users rather than hate speeches. Clearly, both the first and mandatory step carried out by the tool is tasks are in close correlation, since techniques to the extraction of the Tweets posted by the user we detect hate speeches can be used to detect hater want to analyze. In this case, we used the official users as well. Twitter APIs to gather the available Tweets and to However, through this work we want to move forward it to the next modules of the workflow. the focus on the latter since, up to our knowledge, Given that the real-time execution of the work- this a poorly investigated research direction. Just flow is one of the constraints of the project, we think that no datasets of hater users is currently limited the extraction to the 200 most recent publicly available. Tweets posted by the user. This is a reasonable To sum up, the contributions of the work can be choice, since we aim to detect users who recently summarized as follows: showed an intolerant behavior, rather than users who posted hate speeches one or two years ago. • We present a workflow that allows to detect hater users in online social networks; 2.2 Sentiment Analyzer 3 http://www.acmos.net Once the Tweets have been collected, it is nec- 4 http://www.controlodio.it essary to provide the tool with the ability to go Figure 1: The workflow carried out by the H ATE C HECKER tool through the content posted by the target and to au- 2.3 Profile Classifier tomatically identify the hate speeches. In such a specific setting, the simple exploitation To this end, the S ENTIMENT A NALYZER mod- of sentiment analysis techniques that provide a ules exploits Sentiment Analysis techniques (Pang rough binary classification of the single Tweets et al., 2008) to basically classify each Tweet as (conveying/not conveying hate) is not enough. In- positive or negative (that it to say, conveying hate deed, the answers to two fundamental questions speeches or not). To get this output we integrated are still lacking: and compared two different implementations of sentiment analysis algorithms: • How can we label the user as hater or non- hater on the ground of the Tweets she posted? • SentiPolC: (Basile and Novielli, 2014) a sen- timent analysis algorithm that resulted as the • How can we return a more fine-grained clas- best-performing one in EVALITA 2014 in sification of the user (e.g., racist, homofobe, carrying out the task of associating the cor- etc.) on the ground of the Tweets she posted? rect sentiment to Tweets; Both these issues are tackled by the P ROFILE C LASSIFIER module. As for the first question, a • HanSEL: an algorithm based on a deep neu- very simple strategy based on thresholding is im- ral network C-BiLSTN (Zhou et al., 2015) plemented. In particular, we defined a parameter with an input layer of word embeddings. This , and whether the user posted a number of Tweets strategy is based on the work proposed by labeled as hate speeches higher than , the user Polignano et al. (Polignano and Basile, 2018) herself is labeled as an hater. Of course, several and it has been improved within the activities values for the parameter  can be taken into ac- of the ’Contro l’Odio’ research project. In count to run the tool. particular, the whole net has been trained for As for the second question, we used a lexicon- 20 epochs with early stopping criteria, Adam based approach to provide a fine-grained classi- loss function, and binary cross-entropy as op- fication of users’ profiles. The intuition behind timization function. our methodology is that for each category a spe- A complete overview of the algorithms is out cific lexicon can be defined, and whether a Tweet of the scope of this paper and we suggest to go posted by the user contains one of the terms in the through the references for a thorough discussion. lexicon, the user is labeled with the name of the For the sake of simplicity, we can state that the category. output of both the algorithms is a binary classifi- Formally, let C = {c1 , c2 . . . cn } be the set of cation of each Tweet posted by the target user as the categories (e.g., racism, homophobia, sexism, negative (that is to say, conveying hate speeches) etc.) and let VCi = {t1 , t2 . . . tm } be the vocabu- or positive. Such an output is then passed to the lary of the category Ci . Given a Tweet T written P ROFILE C LASSIFIER module whose goal is to as- by a user u, if one of the terms in VCi is contained sign a more precise label to the user, on the ground in T , the user u is labeled with the category Ci . of the nature of the hate speeches she posted (if To define the lexicon for each category, we re- any). lied on the research results of the Italian Hate Map (Lingiardi et al., 2019). In particular, we exploited A screenshot of the working prototype of the the categories as well as the lexicon used in the platform is reported in Figure 2. As shown in Italian Hate Map Project, which consists of 6 dif- the Figure, a user interacting with the platform ferents categories (racism, homophobia, islamo- can query the system by interactively providing a phobia, xenophobia, anti-semitism, sexism, abuse Twitter user name. In a few seconds, the inter- against people with disabilities) and 76 different face shows a report of the target user containing a terms in total. set of emojis reporting the behavior of the user for In order to (hopefully) enrich and improve the each of the categories we analyzed, a snapshot of lexicon used in the Italian Hate Map project, we her own Tweets labeled as hate speeches and some exploited Hurtlex, a multilingual lexicon of hate information about the percentage of hater profiles words (Bassignana et al., 2018). Specifically, that are in the social network of the target user. we manually selected a subset of relevant terms It is worth to note that such a web applica- among those contained in Hurtlex and we merged tion is very useful for both monitoring tasks (e.g., the new terms with those contained in the original to verify whether a third-party account is an on- lexicon. In total, the complete lexicon contained line hater) as well as for Quantified Self scenarios 100 terms, 76 coming from the original Italian (Swan, 2013), that is to say, to improve the self- Hate Map lexicon and 24 gathered from Hurtlex. awareness and the self-consciousness of the user Obviously, in the experimental session the ef- towards the dynamics of her social network. Our fectiveness of the tool on varying of different lex- intuition is that a user who is aware of not being icons and on different configuration of the work- an hater, can use the system to identify (if any) the flow will be evaluated. haters that are still in her own social network, and maybe decide to unfollow them. 2.4 Social Network Processor At the end of the previous step, the target user is la- 3 Experimental Evaluation beled with a set of categories describing the facets The goal of the experimental session was to eval- of her intolerant behavior. uate the effectiveness of the tool on varying of dif- However, one of the goals of the project was ferent configurations of the pipeline. also to investigate the role and the impact of the To this end, due to the lack of a dataset of hater social network of the users in the dynamics of on- profiles, we manually crawled and annotated a set line haters. Accordingly, the S OCIAL N ETWORK of 200 Twitter users, which we made available5 P ROCESSOR gathers the entire social network of for the sake of reproducibility and to foster the re- the target user and runs again (in background, of search in the area. course) the whole pipeline on all the following In particular, we compared four different strate- and followers of the target user, in order to detect gies to run our tool, on varying on two different whether other people in the social network of the parameters, such as the lexicon and the sentiment target user can be labeled as haters as well. The analysis algorithm. In particular, we exploited the goal of this step is to further enhance the compre- following combinations of parameters: hension of network dynamics and to understand whether online haters tend to follow and be fol- • Sentiment Analysis: SentipolC and lowed by other haters. HanSEL, as previously explained Unfortunately, due to space reasons, the discus- sion of this part of the workflow is out of the scope • Lexicons: HateMap lexicon and complete of this paper and is left for future discussions. lexicon (HateMap+Hurtlex) 2.5 Data Exposure and Data Visualization As for the parameters, the threshold  was set Finally, the output of the platform is made avail- equal to 3 and both the sentiment analysis algo- able to third-party services and to the user itself. rithms were run with the standard parameters in- In the first case, a REST web service makes avail- troduced in the original papers. To evaluate the able the output of the tool (that it so say, the hate effectiveness of the approaches, we calculate the categories and the number of haters in her own so- number of correctly classified user profiles over cial network), while in the latter the same data are the total of hater users in the dataset. 5 shown through an interactive user interface. https://tinyurl.com/uniba-haters-dataset Figure 2: A screenshot of H ATE C HECKER at work Facets Lexicon Algorithm Racism Anti-semitism Disability Sexism Homophobia Xenophobia HateMap SentiPolC 71.5 92.0 82.0 77.5 84.0 75.5 HateMap HanSEL 73.0 95.5 88.5 79.0 84.0 79.0 Complete SentiPolC 78.0 95.0 86.5 78.0 84.0 78.0 Complete HanSEL 75.0 97.0 88.5 78.0 84.0 79.0 Table 1: Results of the Experiment. The best-performing configuration for each facet is reported in bold. The results of the experiments are reported in ods as those based on word embeddings and deep Table 1. In general, we can state that our approach learning techniques. Moreover, we can state that to automatically detect hater users in online so- the results can be further improved since no par- cial network provided us with encouraging results, ticular tuning of the parameters was carried out in since more a percentage between 78% and 97% of this work. the online haters were correctly detected by the al- As for the lexicons, the extension of the original gorithm, regardless of the specific category. Italian Hate Map lexicons with new terms led to an It is worth to note that the worse results (both of improvement of the results for all the facets (ex- them are beyond 70%, through) were obtained for cept for homophobia) for at least one of the com- racism and xenophobia, that is to say, two facets parisons. Such improvement are often tiny, but characterized by a lexicon that quickly evolves and this is an expected outcome since just a few terms often adopts terms that are not conventional and coming from Hurtlex were added. However, even not necessarily conveying hate (e.g.,, expressions these preliminary results provided us with encour- as ’Aiutiamoli a casa loro’ or terms as ’clandes- aging findings, since they showed that the integra- tini’). However, even for these categories the re- tion and the extension of sensible terms with the sults we obtained were encouraging. information coming from recently developed lexi- Conversely, results were particularly outstand- cal resources can lead to a further improvement of ing for facets such as anti-semitism and homopho- the accuracy of the system. bia, that have a quite fixed lexicon of terms that 4 Conclusions and Future Work can be used to hurt or offend such minorities. As for the different configurations, we noted In this work we have presented H ATE C HECKER, that H AN SEL tended to obtain better results than a tool that exploits sentiment analysis and natural S ENTI P OL C. This is a quite expected outcome, language processing techniques to automatically since it exploits more novel and effective meth- detect hater users in online social networks. Given a target user, the workflow we imple- task. In EVALITA 2018-Sixth Evaluation Campaign mented in our system uses sentiment analysis tech- of Natural Language Processing and Speech Tools for Italian, volume 2263, pages 1–9. CEUR. niques to identify hate speeches posted by the user and exploits a lexicon that extends that of the Ital- Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Gr- ian Hate Map project to assign to the person one bovic, Vladan Radosavljevic, and Narayan Bhamidi- or more labels that describe the nature of the hate pati. 2015. Hate speech detection with comment embeddings. In Proceedings of the 24th interna- speeches she posted. tional conference on world wide web, pages 29–30. As future work, we plan to arrange a user study, ACM. specifically designed for young people, to evaluate Elisabetta Fersini, Debora Nozza, and Paolo Rosso. the effectiveness of the system as a Quantified Self 2018. Overview of the evalita 2018 task on auto- tool (Musto et al., 2018), that is to say, to improve matic misogyny identification (ami). In EVALITA@ the awareness of the users towards the behavior of CLiC-it. other people in their social network. Njagi Dennis Gitari, Zhang Zuping, Hanyurwimfura Damien, and Jun Long. 2015. A lexicon-based approach for hate speech detection. International References Journal of Multimedia and Ubiquitous Engineering, 10(4):215–230. Maria Anzovino, Elisabetta Fersini, and Paolo Rosso. 2018. Automatic identification and classification of Vittorio Lingiardi, Nicola Carone, Giovanni Semer- misogynistic language on Twitter. In International aro, Cataldo Musto, Marilisa D’Amico, and Silvia Conference on Applications of Natural Language to Brena. 2019. Mapping twitter hate speech towards Information Systems, pages 57–64. Springer. social and sexual minorities: a lexicon-based ap- proach to semantic content analysis. Behaviour & Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, Information Technology, 0(0):1–11. and Vasudeva Varma. 2017. Deep learning for hate speech detection in tweets. In Proceedings of the Binny Mathew, Ritam Dutt, Pawan Goyal, and 26th International Conference on World Wide Web Animesh Mukherjee. 2018. Spread of hate Companion, pages 759–760. International World speech in online social media. arXiv preprint Wide Web Conferences Steering Committee. arXiv:1812.01693. Pierpaolo Basile and Nicole Novielli. 2014. Uniba Cataldo Musto, Giovanni Semeraro, Marco de Gem- at evalita 2014-sentipolc task: Predicting tweet sen- mis, and Pasquale Lops. 2016. Modeling com- timent polarity combining micro-blogging, lexicon munity behavior through semantic analysis of so- and semantic features. Proceedings of EVALITA, cial data: The italian hate map experience. In Pro- pages 58–63. ceedings of the 2016 Conference on User Model- ing Adaptation and Personalization, pages 307–308. Valerio Basile, Cristina Bosco, Elisabetta Fersini, Deb- ACM. ora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. Cataldo Musto, Giovanni Semeraro, Cosimo Lovas- 2019. Semeval-2019 task 5: Multilingual detec- cio, Marco de Gemmis, and Pasquale Lops. 2018. tion of hate speech against immigrants and women A framework for holistic user modeling merging in twitter. In Proceedings of the 13th International heterogeneous digital footprints. In Adjunct Pub- Workshop on Semantic Evaluation, pages 54–63. lication of the 26th Conference on User Model- ing, Adaptation and Personalization, pages 97–101. Elisa Bassignana, Valerio Basile, and Viviana Patti. ACM. 2018. Hurtlex: A multilingual lexicon of words to hurt. In 5th Italian Conference on Computational Bo Pang, Lillian Lee, et al. 2008. Opinion mining Linguistics, CLiC-it 2018, volume 2253, pages 1–6. and sentiment analysis. Foundations and Trends R CEUR-WS. in Information Retrieval, 2(1–2):1–135. Cristina Bosco, Viviana Patti, Marcello Bogetti, Marco Polignano and Pierpaolo Basile. 2018. Hansel: Michelangelo Conoscenti, Giancarlo Francesco Italian hate speech detection through ensemble Ruffo, Rossano Schifanella, and Marco Stranisci. learning and deep neural networks. In EVALITA@ 2017. Tools and resources for detecting hate and CLiC-it. prejudice against immigrants in social media. In Melanie Swan. 2013. The quantified self: Fundamen- Symposium III. Social Interactions in Complex Intel- tal disruption in big data science and biological dis- ligent Systems (SICIS) at AISB 2017, pages 79–84. covery. Big data, 1(2):85–99. AISB. Chunting Zhou, Chonglin Sun, Zhiyuan Liu, and Fran- Cristina Bosco, Dell’Orletta Felice, Fabio Poletto, cis Lau. 2015. A c-lstm neural network for text Manuela Sanguinetti, and Tesconi Maurizio. 2018. classification. arXiv preprint arXiv:1511.08630. Overview of the evalita 2018 hate speech detection