=Paper=
{{Paper
|id=Vol-2481/paper52
|storemode=property
|title=HateChecker: a Tool to Automatically Detect Hater Users in Online Social Networks
|pdfUrl=https://ceur-ws.org/Vol-2481/paper52.pdf
|volume=Vol-2481
|authors=Cataldo Musto,Angelo Sansonetti,Marco Polignano,Giovanni Semeraro,Marco Stranisci
|dblpUrl=https://dblp.org/rec/conf/clic-it/MustoSPSS19
}}
==HateChecker: a Tool to Automatically Detect Hater Users in Online Social Networks==
https://ceur-ws.org/Vol-2481/paper52.pdf
H ATE C HECKER: a Tool to Automatically Detect
Hater Users in Online Social Networks
Cataldo Musto Angelo Pio Sansonetti Marco Polignano
University of Bari University of Bari University of Bari
Dip. di Informatica Dip. di Informatica (Bachelor Student) Dip. di Informatica
cataldo.musto@uniba.it a.sansonetti6@studenti.uniba.it marco.polignano@uniba.it
Giovanni Semeraro Marco Stranisci
University of Bari Associazione ACMOS
Dip. di Informatica Torino
giovanni.semeraro@uniba.it marco.stranisci@acmos.net
Abstract 1 Background and Motivations
According to a recent study1 , 58% of the Italian
In this paper we present H ATE C HECKER, population regularly uses online social networks
a tool for the automatic detection of hater as Twitter, Facebook, Instagram and LinkedIn.
users in online social networks which has Such a huge diffusion of these platforms is pro-
been developed within the activities of viding the users with many new opportunities and
”Contro L’Odio” research project. services, just think that almost everyone now uses
In a nutshell, our tool implements a social media to get information, discuss, express
methodology based on three steps: (i) all opinions and stay in touch with friends. Unfortu-
the Tweets posted by a target user are nately, due to the lack of control and the absence
gathered and processed. (ii) sentiment of a clear management of the concept of identity of
analysis techniques are exploited to auto- the users, social networks have become the perfect
matically label intolerant Tweets as hate place to spread hate against minorities and people
speeches. (iii) a lexicon is used to clas- having different cultures, values and opinions.
sify hate speeches against a set of spe- As pointed out by several works (Mathew et
cific categories that can describe the tar- al., 2018), the diffusion of hate speeches in on-
get user (e.g., racist, homophobic, anti- line social media is continuously growing and
semitic, etc.). the countermeasures adopted by the single plat-
forms are neither effective nor timely, even if a
Finally, the output of the tool, that is to say, big effort is done to make the process of remov-
a set of labels describing (if any) the in- ing hate speeches faster and more precise2 . Ac-
tolerant traits of the target user, are shown cordingly, the research line related to the devel-
through an interactive user interface and opment of tools and methods for the automatic
exposed through a REST web service for detection of hate speeches gained more and more
the integration in third-party applications. attention. Techniques for detecting hate speeches
In the experimental evaluation we crawled are obviously based on NLP techniques, and range
and annotated a set of 200 Twitter profiles from simple lexicon-based approaches (Gitari et
and we investigated to what extent our tool al., 2015) to more sophisticated techniques that ex-
is able to correctly identify hater users. ploit word embeddings (Djuric et al., 2015) and
The results confirmed the validity of our deep learning methods (Badjatiya et al., 2017).
methodology and paved the way for sev- Similar research attempts were also proposed
eral future research directions. for the Italian language. One of the most popu-
lar initiative is the Italian HateMap project (Musto
1
https://wearesocial.com/it/blog/2018/01/global-digital-
report-2018
2
Copyright 2019 for this paper by its authors. Use per- https://www.cnbc.com/2019/02/04/facebook-google-
mitted under Creative Commons License Attribution 4.0 In- and-twitter-are-getting-faster-at-removing-hate-speech-
ternational (CC BY 4.0). online-eu-finds–.html
et al., 2016), a research project that exploits se- • We evaluate several configurations (on vary-
mantic analysis and opinion mining to identify ing of lexicons and sentiment analysis algo-
the most-at-risk areas of the Italian country, that rithms) of the pipeline and we identified the
is to say, the areas where the users more fre- most effective one to tackle our specific task;
quently publish hate speeches. The interest of the
research community for the topic was confirmed • We share the first publicly available dataset
by the recent work by Bosco et al. (Bosco et for automatic detection of hater users on
al., 2017), who studied hate speech against immi- Twitter.
grants, and by Anzovino et al. (Anzovino et al.,
2018) who detected misogyny on Twitter. More- In the following, we will first describe the
over, as shown by the organization of a specific methodology we designed to implement our sys-
task in the EVALITA evaluation campaign, an im- tem, then we will discuss the effectiveness of the
portant effort is now devoted to the automatic de- approach by analyzing the results we obtained on
tection of misogyny (Fersini et al., 2018) and hate a (publicly available) dataset of 200 Twitter users.
speeches in general (Bosco et al., 2018; Basile et
al., 2019). 2 Methodology
In order to continue the investigation in this The workflow carried out by the H ATE C HECKER
research line ACMOS3 , a no-profit associa- tool is reported in Figure 1.
tion based in Torino, recently launched ”Contro Generally speaking, the pipeline consists of
l’Odio4 ”, a joint research project with the Uni- four different modules, that is to say, a S OCIAL
versity of Bari, University of Torino and several DATA E XTRACTOR, a S ENTIMENT A NALYZER,
local associations. The project aims to develop a P ROFILE C LASSIFIER anda S OCIAL N ETWORK
tools and methodologies to monitor (and hopefully P ROCESSOR. All these components use a NoSQL
tackle) online hate speeches and intolerant behav- database to store the information they hold and
iors. expose the output returned by the tool through a
One of the outcomes of the research is H ATE - REST interface as well as through a Web Applica-
C HECKER, a tool that aims to automatically iden- tion. In the following, a description of the single
tify hater users on Twitter by exploiting sentiment modules that compose the workflow is provided.
analysis and natural language processing tech-
niques. The distinguishing aspect of the tool with 2.1 Social Data Extractor
respect to the work we have previously introduced
The whole pipeline implemented in the H ATE -
is the focus of the tool itself. Indeed, differently
C HECKER tool needs some textual content posted
from most of the literature, that focused on the
by the target user to label the user as a hater or
analysis of single Tweets, H ATE C HECKER aims to
not. In absence of textual content, it is not possi-
analyze the users as a whole, and to identify hater
ble provide such a classification. To this end, the
users rather than hate speeches. Clearly, both the
first and mandatory step carried out by the tool is
tasks are in close correlation, since techniques to
the extraction of the Tweets posted by the user we
detect hate speeches can be used to detect hater
want to analyze. In this case, we used the official
users as well.
Twitter APIs to gather the available Tweets and to
However, through this work we want to move forward it to the next modules of the workflow.
the focus on the latter since, up to our knowledge, Given that the real-time execution of the work-
this a poorly investigated research direction. Just flow is one of the constraints of the project, we
think that no datasets of hater users is currently limited the extraction to the 200 most recent
publicly available. Tweets posted by the user. This is a reasonable
To sum up, the contributions of the work can be choice, since we aim to detect users who recently
summarized as follows: showed an intolerant behavior, rather than users
who posted hate speeches one or two years ago.
• We present a workflow that allows to detect
hater users in online social networks; 2.2 Sentiment Analyzer
3
http://www.acmos.net Once the Tweets have been collected, it is nec-
4
http://www.controlodio.it essary to provide the tool with the ability to go
Figure 1: The workflow carried out by the H ATE C HECKER tool
through the content posted by the target and to au- 2.3 Profile Classifier
tomatically identify the hate speeches. In such a specific setting, the simple exploitation
To this end, the S ENTIMENT A NALYZER mod- of sentiment analysis techniques that provide a
ules exploits Sentiment Analysis techniques (Pang rough binary classification of the single Tweets
et al., 2008) to basically classify each Tweet as (conveying/not conveying hate) is not enough. In-
positive or negative (that it to say, conveying hate deed, the answers to two fundamental questions
speeches or not). To get this output we integrated are still lacking:
and compared two different implementations of
sentiment analysis algorithms: • How can we label the user as hater or non-
hater on the ground of the Tweets she posted?
• SentiPolC: (Basile and Novielli, 2014) a sen-
timent analysis algorithm that resulted as the • How can we return a more fine-grained clas-
best-performing one in EVALITA 2014 in sification of the user (e.g., racist, homofobe,
carrying out the task of associating the cor- etc.) on the ground of the Tweets she posted?
rect sentiment to Tweets; Both these issues are tackled by the P ROFILE
C LASSIFIER module. As for the first question, a
• HanSEL: an algorithm based on a deep neu-
very simple strategy based on thresholding is im-
ral network C-BiLSTN (Zhou et al., 2015)
plemented. In particular, we defined a parameter
with an input layer of word embeddings. This
, and whether the user posted a number of Tweets
strategy is based on the work proposed by
labeled as hate speeches higher than , the user
Polignano et al. (Polignano and Basile, 2018)
herself is labeled as an hater. Of course, several
and it has been improved within the activities
values for the parameter can be taken into ac-
of the ’Contro l’Odio’ research project. In
count to run the tool.
particular, the whole net has been trained for
As for the second question, we used a lexicon-
20 epochs with early stopping criteria, Adam
based approach to provide a fine-grained classi-
loss function, and binary cross-entropy as op-
fication of users’ profiles. The intuition behind
timization function.
our methodology is that for each category a spe-
A complete overview of the algorithms is out cific lexicon can be defined, and whether a Tweet
of the scope of this paper and we suggest to go posted by the user contains one of the terms in the
through the references for a thorough discussion. lexicon, the user is labeled with the name of the
For the sake of simplicity, we can state that the category.
output of both the algorithms is a binary classifi- Formally, let C = {c1 , c2 . . . cn } be the set of
cation of each Tweet posted by the target user as the categories (e.g., racism, homophobia, sexism,
negative (that is to say, conveying hate speeches) etc.) and let VCi = {t1 , t2 . . . tm } be the vocabu-
or positive. Such an output is then passed to the lary of the category Ci . Given a Tweet T written
P ROFILE C LASSIFIER module whose goal is to as- by a user u, if one of the terms in VCi is contained
sign a more precise label to the user, on the ground in T , the user u is labeled with the category Ci .
of the nature of the hate speeches she posted (if To define the lexicon for each category, we re-
any). lied on the research results of the Italian Hate Map
(Lingiardi et al., 2019). In particular, we exploited A screenshot of the working prototype of the
the categories as well as the lexicon used in the platform is reported in Figure 2. As shown in
Italian Hate Map Project, which consists of 6 dif- the Figure, a user interacting with the platform
ferents categories (racism, homophobia, islamo- can query the system by interactively providing a
phobia, xenophobia, anti-semitism, sexism, abuse Twitter user name. In a few seconds, the inter-
against people with disabilities) and 76 different face shows a report of the target user containing a
terms in total. set of emojis reporting the behavior of the user for
In order to (hopefully) enrich and improve the each of the categories we analyzed, a snapshot of
lexicon used in the Italian Hate Map project, we her own Tweets labeled as hate speeches and some
exploited Hurtlex, a multilingual lexicon of hate information about the percentage of hater profiles
words (Bassignana et al., 2018). Specifically, that are in the social network of the target user.
we manually selected a subset of relevant terms It is worth to note that such a web applica-
among those contained in Hurtlex and we merged tion is very useful for both monitoring tasks (e.g.,
the new terms with those contained in the original to verify whether a third-party account is an on-
lexicon. In total, the complete lexicon contained line hater) as well as for Quantified Self scenarios
100 terms, 76 coming from the original Italian (Swan, 2013), that is to say, to improve the self-
Hate Map lexicon and 24 gathered from Hurtlex. awareness and the self-consciousness of the user
Obviously, in the experimental session the ef- towards the dynamics of her social network. Our
fectiveness of the tool on varying of different lex- intuition is that a user who is aware of not being
icons and on different configuration of the work- an hater, can use the system to identify (if any) the
flow will be evaluated. haters that are still in her own social network, and
maybe decide to unfollow them.
2.4 Social Network Processor
At the end of the previous step, the target user is la- 3 Experimental Evaluation
beled with a set of categories describing the facets
The goal of the experimental session was to eval-
of her intolerant behavior.
uate the effectiveness of the tool on varying of dif-
However, one of the goals of the project was
ferent configurations of the pipeline.
also to investigate the role and the impact of the
To this end, due to the lack of a dataset of hater
social network of the users in the dynamics of on-
profiles, we manually crawled and annotated a set
line haters. Accordingly, the S OCIAL N ETWORK
of 200 Twitter users, which we made available5
P ROCESSOR gathers the entire social network of
for the sake of reproducibility and to foster the re-
the target user and runs again (in background, of
search in the area.
course) the whole pipeline on all the following
In particular, we compared four different strate-
and followers of the target user, in order to detect
gies to run our tool, on varying on two different
whether other people in the social network of the
parameters, such as the lexicon and the sentiment
target user can be labeled as haters as well. The
analysis algorithm. In particular, we exploited the
goal of this step is to further enhance the compre-
following combinations of parameters:
hension of network dynamics and to understand
whether online haters tend to follow and be fol- • Sentiment Analysis: SentipolC and
lowed by other haters. HanSEL, as previously explained
Unfortunately, due to space reasons, the discus-
sion of this part of the workflow is out of the scope • Lexicons: HateMap lexicon and complete
of this paper and is left for future discussions. lexicon (HateMap+Hurtlex)
2.5 Data Exposure and Data Visualization As for the parameters, the threshold was set
Finally, the output of the platform is made avail- equal to 3 and both the sentiment analysis algo-
able to third-party services and to the user itself. rithms were run with the standard parameters in-
In the first case, a REST web service makes avail- troduced in the original papers. To evaluate the
able the output of the tool (that it so say, the hate effectiveness of the approaches, we calculate the
categories and the number of haters in her own so- number of correctly classified user profiles over
cial network), while in the latter the same data are the total of hater users in the dataset.
5
shown through an interactive user interface. https://tinyurl.com/uniba-haters-dataset
Figure 2: A screenshot of H ATE C HECKER at work
Facets
Lexicon Algorithm Racism Anti-semitism Disability Sexism Homophobia Xenophobia
HateMap SentiPolC 71.5 92.0 82.0 77.5 84.0 75.5
HateMap HanSEL 73.0 95.5 88.5 79.0 84.0 79.0
Complete SentiPolC 78.0 95.0 86.5 78.0 84.0 78.0
Complete HanSEL 75.0 97.0 88.5 78.0 84.0 79.0
Table 1: Results of the Experiment. The best-performing configuration for each facet is reported in bold.
The results of the experiments are reported in ods as those based on word embeddings and deep
Table 1. In general, we can state that our approach learning techniques. Moreover, we can state that
to automatically detect hater users in online so- the results can be further improved since no par-
cial network provided us with encouraging results, ticular tuning of the parameters was carried out in
since more a percentage between 78% and 97% of this work.
the online haters were correctly detected by the al- As for the lexicons, the extension of the original
gorithm, regardless of the specific category. Italian Hate Map lexicons with new terms led to an
It is worth to note that the worse results (both of improvement of the results for all the facets (ex-
them are beyond 70%, through) were obtained for cept for homophobia) for at least one of the com-
racism and xenophobia, that is to say, two facets parisons. Such improvement are often tiny, but
characterized by a lexicon that quickly evolves and this is an expected outcome since just a few terms
often adopts terms that are not conventional and coming from Hurtlex were added. However, even
not necessarily conveying hate (e.g.,, expressions these preliminary results provided us with encour-
as ’Aiutiamoli a casa loro’ or terms as ’clandes- aging findings, since they showed that the integra-
tini’). However, even for these categories the re- tion and the extension of sensible terms with the
sults we obtained were encouraging. information coming from recently developed lexi-
Conversely, results were particularly outstand- cal resources can lead to a further improvement of
ing for facets such as anti-semitism and homopho- the accuracy of the system.
bia, that have a quite fixed lexicon of terms that
4 Conclusions and Future Work
can be used to hurt or offend such minorities.
As for the different configurations, we noted In this work we have presented H ATE C HECKER,
that H AN SEL tended to obtain better results than a tool that exploits sentiment analysis and natural
S ENTI P OL C. This is a quite expected outcome, language processing techniques to automatically
since it exploits more novel and effective meth- detect hater users in online social networks.
Given a target user, the workflow we imple- task. In EVALITA 2018-Sixth Evaluation Campaign
mented in our system uses sentiment analysis tech- of Natural Language Processing and Speech Tools
for Italian, volume 2263, pages 1–9. CEUR.
niques to identify hate speeches posted by the user
and exploits a lexicon that extends that of the Ital- Nemanja Djuric, Jing Zhou, Robin Morris, Mihajlo Gr-
ian Hate Map project to assign to the person one bovic, Vladan Radosavljevic, and Narayan Bhamidi-
or more labels that describe the nature of the hate pati. 2015. Hate speech detection with comment
embeddings. In Proceedings of the 24th interna-
speeches she posted. tional conference on world wide web, pages 29–30.
As future work, we plan to arrange a user study, ACM.
specifically designed for young people, to evaluate
Elisabetta Fersini, Debora Nozza, and Paolo Rosso.
the effectiveness of the system as a Quantified Self 2018. Overview of the evalita 2018 task on auto-
tool (Musto et al., 2018), that is to say, to improve matic misogyny identification (ami). In EVALITA@
the awareness of the users towards the behavior of CLiC-it.
other people in their social network. Njagi Dennis Gitari, Zhang Zuping, Hanyurwimfura
Damien, and Jun Long. 2015. A lexicon-based
approach for hate speech detection. International
References Journal of Multimedia and Ubiquitous Engineering,
10(4):215–230.
Maria Anzovino, Elisabetta Fersini, and Paolo Rosso.
2018. Automatic identification and classification of Vittorio Lingiardi, Nicola Carone, Giovanni Semer-
misogynistic language on Twitter. In International aro, Cataldo Musto, Marilisa D’Amico, and Silvia
Conference on Applications of Natural Language to Brena. 2019. Mapping twitter hate speech towards
Information Systems, pages 57–64. Springer. social and sexual minorities: a lexicon-based ap-
proach to semantic content analysis. Behaviour &
Pinkesh Badjatiya, Shashank Gupta, Manish Gupta, Information Technology, 0(0):1–11.
and Vasudeva Varma. 2017. Deep learning for hate
speech detection in tweets. In Proceedings of the Binny Mathew, Ritam Dutt, Pawan Goyal, and
26th International Conference on World Wide Web Animesh Mukherjee. 2018. Spread of hate
Companion, pages 759–760. International World speech in online social media. arXiv preprint
Wide Web Conferences Steering Committee. arXiv:1812.01693.
Pierpaolo Basile and Nicole Novielli. 2014. Uniba Cataldo Musto, Giovanni Semeraro, Marco de Gem-
at evalita 2014-sentipolc task: Predicting tweet sen- mis, and Pasquale Lops. 2016. Modeling com-
timent polarity combining micro-blogging, lexicon munity behavior through semantic analysis of so-
and semantic features. Proceedings of EVALITA, cial data: The italian hate map experience. In Pro-
pages 58–63. ceedings of the 2016 Conference on User Model-
ing Adaptation and Personalization, pages 307–308.
Valerio Basile, Cristina Bosco, Elisabetta Fersini, Deb- ACM.
ora Nozza, Viviana Patti, Francisco Manuel Rangel
Pardo, Paolo Rosso, and Manuela Sanguinetti. Cataldo Musto, Giovanni Semeraro, Cosimo Lovas-
2019. Semeval-2019 task 5: Multilingual detec- cio, Marco de Gemmis, and Pasquale Lops. 2018.
tion of hate speech against immigrants and women A framework for holistic user modeling merging
in twitter. In Proceedings of the 13th International heterogeneous digital footprints. In Adjunct Pub-
Workshop on Semantic Evaluation, pages 54–63. lication of the 26th Conference on User Model-
ing, Adaptation and Personalization, pages 97–101.
Elisa Bassignana, Valerio Basile, and Viviana Patti. ACM.
2018. Hurtlex: A multilingual lexicon of words to
hurt. In 5th Italian Conference on Computational Bo Pang, Lillian Lee, et al. 2008. Opinion mining
Linguistics, CLiC-it 2018, volume 2253, pages 1–6. and sentiment analysis. Foundations and Trends R
CEUR-WS. in Information Retrieval, 2(1–2):1–135.
Cristina Bosco, Viviana Patti, Marcello Bogetti, Marco Polignano and Pierpaolo Basile. 2018. Hansel:
Michelangelo Conoscenti, Giancarlo Francesco Italian hate speech detection through ensemble
Ruffo, Rossano Schifanella, and Marco Stranisci. learning and deep neural networks. In EVALITA@
2017. Tools and resources for detecting hate and CLiC-it.
prejudice against immigrants in social media. In Melanie Swan. 2013. The quantified self: Fundamen-
Symposium III. Social Interactions in Complex Intel- tal disruption in big data science and biological dis-
ligent Systems (SICIS) at AISB 2017, pages 79–84. covery. Big data, 1(2):85–99.
AISB.
Chunting Zhou, Chonglin Sun, Zhiyuan Liu, and Fran-
Cristina Bosco, Dell’Orletta Felice, Fabio Poletto, cis Lau. 2015. A c-lstm neural network for text
Manuela Sanguinetti, and Tesconi Maurizio. 2018. classification. arXiv preprint arXiv:1511.08630.
Overview of the evalita 2018 hate speech detection