Text classi cation is a classical problem in NLP and has impactful applications. An essential business application is hate speech detection from online data. With the enormous amount of social media data getting generated continuously across the world, detection of hate speech is considered a very challenging task in NLP. In this paper, we describe our approaches for three shared tasks on hate speech and o ensive content identi cation in Indo-European languages (Mandl et al. [9]). We describe statistical machine learning-based approaches as well as deep learning-based approaches and present their comparisons. We observe that convolutional neural networks perform quite well in the classi cation task.
1
Social media has become an important communication medium today. Social
media technology enables a piece of information to be spread quickly. With the
exponential growth of social media users, public platforms are often used to
express satisfaction or grievance regarding any product, service or experience. A
huge amount of such data is generated continuously across the world.
Unfortunately, these data often contain o ensive words, which can be considered as
hate speech. The anonymity and mobility provided by the social media platforms
have made the breeding and spread of hate speech (Zhang et al. [
The term `hate speech' was formally de ned as `any communication that
disparages a person or a group on the basis of some characteristics (to be referred
to as types of hate or hate classes) such as race, colour, ethnicity, gender, sexual
orientation, nationality, religion, or other characteristics (Nockleby et al. [
In this paper, we use tweets in English, Hindi and German language and perform three classi cation tasks on them: { Sub-task A: Hate and O ensive, and Non Hate-O ensive { Sub-task B: Hate speech, O ensive, and Profane { Sub-task C: Targeted Insult, and Untargeted
We perform the above classi cation task using both statistical machine learning methods as well as deep learning methods - we use SVM and CNN, respectively. We perform feature engineering on our datasets before creating feature vectors for SVM. For CNN, we create embedding vectors using di erent word embeddings are use them in our model.
The rest of the paper is organized as follows. We discuss the related work in detail in the next section. Next, we describe our approaches in detail in Section 3. Then, we outline the experimental setup in Section 4 and present the results of our experiments in Section 5. Finally, we conclude the paper and discuss future work in Section 6. 2
Traditional machine learning methods have performed quite well in classi
cation tasks. There has been extensive work on classi cation tasks with feature
engineering. Nobata et al.[
With the constant generation of huge amounts of data, neural networks are
starting to take over statistical machine learning models. Gehrmann et al. [
Word embeddings in neural networks are quite in uential for classi cation
tasks. Akhtar et al. [
Hate speech detection in the English language has been an extensive area of
research. Xiang et al. [
We implement three di erent approaches for the given task: { Support Vector Machine without feature engineering { Support Vector Machine with feature engineering { Convolutional Neural Network 3.1
In this approach, we rst clean the given tweets by the following steps:
After the data is cleaned, the tweets are encoded as feature vectors and are directly used as input to our SVM. Results of this method are shown in Section 5. 3.2
In this approach, we perform feature engineering before creating feature vectors for SVM. We select a handful of features which carry some relevant information about a tweet. For example, if the user uses one or more angry emoticons in a tweet, the tweet is more likely to be carrying the hatred emotion towards something.
We select the following features and include them in our feature vector: { emoticons: we create a dictionary of happy, sad, anger, fear, surprise, disgust, others emoticons and count the number of each of them used in a tweet { hashtags: we extract out the words used in hashtags in a tweet. Users often summarize their opinion through a hashtag. This can help nd out the emotion expressed in a tweet. { intensi ers: words like exceptionally, incredibly, awful, insanely, etc are often used to emphasize on some descriptive word. We call them intensi ers. { negations: words like never, no. nothing, nowhere, etc are used to thwart the meaning of a piece of text. { hate words: we use a list of hate words and nd their occurrences in a tweet
Other features include char n-grams, word n-grams, etc 3.3
In the deep neural network approach, we tokenize each input sentence and nd the embedding of each word. Our hate speech dataset consists of tweets in three di erent languages. We use domain-speci c word embeddings which are trained on Twitter data.
We pass the embeddings through our convolutional layers with multiple lter widths and feature maps. After each convolutional layer, we perform max-overtime pooling before passing them through a nal fully connected layer with softmax output. We train this model by minimizing the categorical cross-entropy loss. 4
For statistical machine learning-based approaches, we use SVM with RBF kernel and c = 1:0 using grid-search and Random-forest with number of estimators = 50. We use nltk3 libraries for all data processing steps in our SVM model.
For our deep learning-based approaches, we use CNN. We have 2
convolutional layers with total 128 lters with size 5 and max-pooling of 5 and 30. We
make our CNN deeper by using multiple lters - 3, 4, and 5. We use di erent
word embeddings for each language which we feed into our model as input.
{ English: Used domain speci c word embeddings trained on Twitter domain
(Kamble et al. [
FastText considers sub-word embedding. It is helpful in our case, as tweets are often informal and sub-word information should be given importance to extract the semantics of tweets.
We experiment with word embeddings of di erent dimensions and nd that 100 dimensional word embeddings perform the best. 5
{ Task 1: 0 - HOF, 1 - NOT { Task 2: 0 - PRFN, 1 - OFFN, 2 - HATE, 3: NONE { Task 3: 0 - TIN, 1 - UNT, 2 - NONE In this paper, we present our approaches for the three tasks of HASOC2019. We describe feature engineering for our statistical machine learning-based approaches. We also use a list of hate words for each of the three languages and use them in feature engineering for creating feature vectors of our model. We also describe a deep learning-based approach. In this approach, we use domain-speci c word embeddings and nd that word embeddings trained on a particular domain performs better for a text from the same domain. We also see improvement in 3 https://www.nltk.org/ 4 http://www.c lt.iitb.ac.in/ the performance of text classi cation by convolutional neural networks on using domain-speci c word embeddings.
In this paper, we show that statistical machine learning-based methods perform well for classi cation tasks when the dataset is not very huge. However, with the constant generation of data over social media platforms, such tasks often need to cater to a wide variety with fast processing. Deep learning-based methods perform better on such massive data.
Our work is an e ort to improve the existing methodology of detecting hate
speech in social media comments. With the increase in social media usage by
people across the world, hate speech is getting generated and propagated at high
speed. Due to the massive scale of the web, methods that automatically detect
hate speech are required (Schmidt et al. [
For future work, we will add more feature engineering and compare results with the current model. We will run our deep neural network on a larger dataset, using domain-speci c word embeddings and observe its performance.