In this paper, we present a Web Application Platform for the Detection of Aggressiveness in Social Media using Natural Language Processing and Machine Learning techniques, describing its architecture, the development technologies used and the diferent language models that have been integrated into the system. Finally, we conclude that the platform is a powerful tool to tackle real time aggressiveness on social media such as sexism or hate speech.
the detection of aggression in real-time data.
The rest of the paper is structured as follows: In Section 2 we provide a description of the tool and its architecture. Language models implemented are explained in Section 3. Finally, Section 4 presents conclusions and future work.
The ALIADA Web application consists of five
internal modules that interact with each other to attend
incoming requests and provide resources to relevant
stakeholders (hereinafter, namely, users):
Users can retrieve social data through requests, in
which the social network used as source must be
specified along with other search parameters: (
Gathered data is anonymized before being stored in
the Elasticsearch data warehouse in string format,
structured as: (
Request’s retrieved data can be downloaded in • Data Storage Module, based on ELK’s Elas- comma separated format (.csv) however, importing ticsearch search engine it allows to index data new data into a request is not allowed. At the same under a non SQL approach. time, a request social data cannot be shared in other • Routing Module, relies on the FastAPI frame- requests or by any other users distinct from their work to attend requests asynchronously using original requester who is allowed to run diferent Python. ML classifying models against a same request in • User Interface Module, built using state-of- order to collect diverse statistics (e.g: in terms of the-art web technologies such as HTML5, sexism, ofensiveness, hate speech, etc.). CSS3 (specifically, Bootstrap 5 as CSS framework) and Javascript. 2.1.3. User creation and management • Internal Logic Module, implemented using
Python manages data retrievals from social networking sites and the classification of incoming users requests. • Artificial Intelligence: Machine Learning
Module, built upon the Torch library for Python, allows to perform inferences in ML and Deep Learning models.
Responses from the server require of authorized credentials that must be granted by an administrator, after requesting access through the contact form on the platform’s homepage.
Users must be logged in to request and classify
social data, this authorization is sent in each HTTP
Request through Javascript Web Tokens (JWT) and
serves two purposes: (
These modules are organized into Backend and Frontend, the former being responsible for routing and associated logic, and the latter of providing a graphical interface to interact with.
On the one hand, users’ requests for data retrieval and classification are segmented into separated 2.1. Backend queues and serviced according to the date on which Encompasses the routing management and handling they were sent to the server along with a priority of incoming endpoint calls: value that is reduced progressively as long as no new data is retrieved from the source, helping to 2.1.1. Stored Data and Storage Process determine when a certain topic is no longer relevant. On the other hand, the server trafic is handled Information regarding users, their related personal- asynchronously through FastAPI’s uvicorn library ization and data retrieval and classification requests, which allows to run an ASGI Web server. is stored in an Elasticsearch repository considering:
• The type of the submitted request: either data retrieval or classification. • Social network used as source: Twitter or
Youtube. • The language model applied.
Classification orders are associated to retrieval requests, they specify which ML model will be applied to the data and internally, they are ordered by the date in which they were sent to the server. Further on, the Pickle and Torch libraries are used to load the trained model architecture and state, as well as its associated vocabulary. Integrating new ML models into the server requires for the uploading of the trained model along with its corresponding word embeddings or bag-of-words structure and a categorical label dictionary to improve the comprehensibility of the model. A new function must be declared inside the Classifying module to load the model and use it against input data. 2.2. Frontend: User Interface ALIADA provides a minimalistic web interface to make use of all of its features in a fast and intuitive way. Right after logging in from the main webpage, access to all the application’s functions is provided: New data retrieval requests, statistics about the classification results, graphs of the total amount of downloaded posts, etc. In the following, these features are further described.
Dashboard. A dashboard (Figure 1) containing the current status of data retrieval and classification requests is displayed. Here, the client can see an ApexCharts’ graph3 that plots the total amount of data downloaded in a given time period, a list containing all active requests and a button to create a new one. Clicking on this button will pop up a form with all the information required to send a new data retrieval request as shown in Figure 2.
My requests and classification panel. In order to have a more in-depth view of active and completed requests, two diferent sections are provided: my requests and classification panel. The former shows the current state (queued, in progress or completed) of each data retrieval request, while the latter shows the classification results in the form of graphs as seen in Figure 3. This section also shows all anonymized texts with their predicted labels, some information about the data retrieval and buttons to both download the full retrieved corpus as a .csv file and reuse the data to infer new labels with a diferent ML model.
Administrator. Finally, only users with the administrator role have access to the administration panel. Here, an administrator can see the application’s log history or the list of active requests in real-time. Users can also be created and deleted from this panel.
The main objective of ALIADA is to monitor social media posts for the detection of aggressive content. Therefore, it is necessary to integrate diferent ML solutions to detect this behavior. Specifically, we have trained diferent models based on SVM for the detection of three phenomena: hate speech, sexism, and ofensiveness.
In order to train these solutions, we have taken into account most of the available corpora generated for aggressiveness detection in Spanish including
HatEval [12], HaterNet [13], EXIST [14], NewsComTOX [15] and OfendES [ 7]. A total of four models are available in the platform: hate_speech_svm has been trained on HatEval, HaterNet and NewsComTOX datasets, ofendes_svm has been trained on the large OfendES dataset, sexism_svm is trained on the EXIST dataset and finally all_concepts_svm combine all of the datasets.
ALIADA is a powerful and useful tool to tackle aggressiveness in social networking sites in realtime, allowing for the detection of such attitudes in social publications through ML algorithms. In the near future, we would like to go further and, in addition to post classification, we will develop an explainability tool in order to understand what sections within each post makes it more aggressive than others through what is known as Named Entity Recognition (NER) techniques, and an emotion or performance tool to determine which attitude causes a greater efect in terms of its associated social reactions (namely, likes and retweets).
This work has been partially supported by Big Hug project (P20_00956, PAIDI 2020) and WeLee project (1380939, FEDER Andalucía 2014-2020) funded by the Andalusian Regional Government, LIVING-LANG project (RTI2018-094653-B-C21) funded by MCIN/AEI/10.13039/501100011033 and by ERDF A way of making Europe, and the scholarship (FPI-PRE2019-089310) from the Ministry of Science, Innovation, and Universities of the Spanish Government.