In this paper, team University of Botswana Computer Science (UB-CS) investigate the use of XLMRoBERTa, a multilingual model trained on 100 diferent languages for transfer learning in the identification of conversational hate-speech in code-mixed languages. We also investigate whether enriching the tweets with textual sentiments from emojis can help improve the classification performance. Our proposed solution outperformed other teams that participated at the HASOC (2022) Task 2 with a macro F1 score of 0.4939. The result suggest that enriching the tweets with textual sentiments and using a pre-trained multilingual model for transfer learning can help in the identification of conversational hate-speech in code-mixed languages.
it is supporting an ofensive preceding or parent message. Furthermore messages are often
expressed using a mix of languages, a property that needs to be factored in the development of
hate and ofensive content detection systems [
The Hate Speech and Ofensive Content Identification in Indo-European Languages (HASOC
(2019)) 1 shared task started in 2019 [
A new task was introduced in HASOC (2021) [
2https://competitions.codalab.org/competitions/20011 3https://projects.fzai.h-da.de/iggsa/ 4https://hasocfire.github.io/hasoc/2022/index.html visible from a single comment or reply but can be determined if parent content is considered. The aim of the task is thus to detect posts that are hateful or ofensive on their own, and those that support hate or ofensive content of their parent posts. Hence the task defines three classes for the identification of hate and ofensive language in posts as follows: • (SHOF) Standalone Hate - This tweet, comment, or reply contains Hate, ofensive, and profane content in itself. • (CHOF) Contextual Hate - Comment or reply is supporting the hate, ofence and profanity expressed in its parent. This includes afirming the hate with positive sentiment and having apparent hate. • (NONE) Non-Hate - This tweet, comment, or reply does not contain Hate, ofensive, and profane content in itself.
The dataset for Task 2: Identification of Conversational Hate-Speech in Code-Mixed Languages (ICHCL) comprises twitter postings, comments and replies to each comment based on controversial stories from diferent topics including Temple-Mosque Controversy, Taliban and Covid Controversy. The tweets use mix of both the English and Hindi languages referred to as Hinglish. The statistics of the dataset is shown Table 1. This data was randomly split into 80% training data and 20% validation data.
The tweets were first concatenated to create conversational threads comprising parent tweets and comments as well as parent tweets, comments and replies where available. A manual exploration of the training data indicated that the tweets contained a lot of special characters, urls and emojis. We perform data cleaning by removing urls, stopwords, extra spaces and newlines. We however retain emojis which we expand to text using the emoji library5 to augment the tweets.
In our emperical investigation we deploy a SimpleTransformers6 Library by HuggingFace7, which has implementation of task-specific SimpleTransformer models. In particular, we use a classification model called ClassificationModel , which uses a pre-trained model for the task of binary and multi-class classification. The model used is based on the HugginFace implementation of XLM-RoBERTa, a transformer based multilingual model pre-trained on CommonCrawl data containing 100 languages. XLM-RoBERTa is based on the BERT architecture and has a total of 12 layers for learning diferent semantic information with a classification layer built on top. Since we consider the influence of emojis in our experiments we first deployed the model with emojis omitted from the tweets using a learning rate of 1e-5 at 3 and 5 epochs respectively. We further experimented with augmented tweets similarly at a learning rate of 1e-5 at 3 and 5 epochs. All models used the AdamW optimizer. Base on the result in Table 2, we chose to use the parameter used in Run 4 enhanced tweet for our run submission the Task 2: Identification of Conversational Hate-Speech in Code-Mixed Languages (ICHCL).
Table 3 shows the leaderboard of the HASOC (2022) Task 2: Identification of Conversational Hate-Speech in Code-Mixed Languages (ICHCL). Our team UB-CS denoted by † managed to outperform other teams. The results suggest that using multilingual model trained on several languages can improve the identification of conversational hate speech in code mixed languages (HINGLISH - Hindi-English). In addition, the results suggest that we can further improve the performance by enriching the tweets with textual sentiments generated from emojis.
The results of our investigation suggests that enriching the tweets with textual sentiments and using a pre-trained multilingual model for transfer learning can help in the identification of conversational hate-speech in code-mixed languages. A natural progression of this work is to analyse whether a state-of-the-art performance can be attained by using an ensemble from several pre-trained multilingual models for transfer learning. FIRE ’19, Association for Computing Machinery, New York, NY, USA, 2019, p. 14–17. URL: https://doi.org/10.1145/3368567.3368584. [7] M. Zampieri, S. Malmasi, P. Nakov, S. Rosenthal, N. Farra, R. Kumar, SemEval-2019 task 6: Identifying and categorizing ofensive language in social media (OfensEval), in: Proceedings of the 13th International Workshop on Semantic Evaluation, Association for Computational Linguistics, Minneapolis, Minnesota, USA, 2019, pp. 75–86. URL: https://aclanthology.org/ S19-2010. doi:10.18653/v1/S19-2010. [8] M. Wiegand, M. Siegel, Overview of the germeval 2018 shared task on the identification of ofensive language, 2018. [9] T. Mandl, S. Modha, A. Kumar M, B. R. Chakravarthi, Overview of the hasoc track at fire 2020: Hate speech and ofensive language identification in tamil, malayalam, hindi, english and german, in: Forum for Information Retrieval Evaluation, FIRE 2020, Association for Computing Machinery, New York, NY, USA, 2020, p. 29–32. URL: https://doi.org/10.1145/ 3441501.3441517. doi:10.1145/3441501.3441517.