The rise of digital platforms has led to an increase in social interactions, but it has also given rise to inappropriate behavior on the Web, including the spread of hate speech and ofensive language. This poses a threat to freedom of expression, exposing users to denigrating content based on personal characteristics. Such communication can have harmful psychological efects, especially on vulnerable communities. Detecting and preventing hate speech has become a crucial area of research in Natural Language Processing (NLP) and Machine Learning (ML). This project aims to develop efective solutions using advanced ML and NLP techniques to assess the ofensiveness and toxicity of Spanish text. A cloud service will automatically label and classify texts, providing an interface for detecting and substituting inappropriate language. The prototype aims to assist users in writing non-ofensive content and serve as real-time monitoring and filtering tools for social media, contributing to a more inclusive and fair discourse.
The significant increase in social interactions through digital platforms has leveraged the presence of inappropriate behavior on the Web, such as the propagation of hate speech or the use of ofensive language among the users of these platforms. Freedom of expression in these media has exposed their users to publications that are sometimes used to denigrate, insult or hurt with foul or rude language based on gender, race, religion, ideology or other personal characteristics. Unfortunately, this type of communication can be very harmful and can cause negative psychological efects among users, especially among vulnerable communities such as children and teenagers, women, LGTBI, immigrants, religious and cultural communities.
Governments and online platforms have been addressing this problem for some years now, and measures are continuously being adopted in the form of laws and policies that contribute to fostering healthy coexistence in these media. For example, since 2013, the European Council has promoted the "No Hate Speech" movement 1 with the aim of mobilizing young people to combat hate speech and promote human rights on the Internet. In May 2016, the European Commission reached an agreement with Facebook, Microsoft, Twitter and YouTube to create a "Code of Conduct on combating illegal hate speech online"2. Based on this agreement, the "Protocol on combating illegal hate speech online" has been drafted3.
Between 2018 and 2020, other platforms such as In1https://www.coe.int/en/web/committee-on-combatting-hate-speech/ home
2https://ec.europa.eu/info/policies/ justice-and-fundamental-rights/combatting-discrimination/ racism-and-xenophobia/eu-code-conduct-countering-illegal-hate-speech-online_ en
discursoodio/PROTOCOLO_DISCURSO_ODIO.pdf
stagram, Snapchat, Dailymotion and TikTok joined this considerable eforts in the generation of resources, most
Code of Conduct. The 2019 report4 highlighted that of them are only available for English. However, it is
necthreats, insults and discrimination are counted as the essary to spend eforts on the development of resources
most repeated criminal acts, with the Internet (54.9%) and systems adapted to other languages since, for
examand social networks (17.2%) being the most used means ple, in the specific case of hate speech detection, there are
to commit these actions. This situation has led the Span- important cultural diferences (jargons, slang, idiomatic
ish Parliament to approve in 2020 a law to prevent the expressions...) depending on the language or the social
spread of hate online. The 2020 report5 maintains the group under examination[
Thus, it seems clear that the problem of detecting in- to build or optimize such datasets, like the creation of
appropriate behavior in general, and hate speech in par- "core" sets to deploy augmentation approaches, or more
ticular, on the Web has worsened in recent years, making concern in the debugging of text labeling errors.
it necessary to study, analyze and implement solutions Finally, the availability of tools to facilitate the task
in all areas, including the field of language technolo- of moderation for human professionals dedicated to the
gies [
In addition, hope speech is a type of language that is xenophobic, homophobic, racist or simply bad-sounding
able to relax a hostile environment and that helps, gives expressions will help us to build a much more inclusive,
suggestions and inspires for good to a number of people friendly, social and fair discourse.
when they are in times of illness, stress, loneliness or We propose this project "MODERATE: MODERAtion
depression. Detecting it automatically, so that positive of ConTEnts in Social networks using Language
Techcomments can be more widely disseminated, can have a nologies" ((TED2021-130145B-I00)to address these issues.
very significant efect when it comes to combating sexual This work has been partially supported by MCIN/AEI/
or racial discrimination or when we seek to foster less 10.13039/501100011033 and by the “European Union
bellicose environments[
To train these systems, it is essential to generate and The main objective of this project is to study and decompile manually labeled linguistic resources as well as velop efective solutions based on advanced machine to adapt existing ones to fit the particular use case. Al- learning and NLP techniques to determine the degree though in recent years the NLP community has invested of ofensiveness/toxicity/hope speech/hate speech of a text in Spanish and identify the spans related to this type 4https://www.interior.gob.es/opencms/pdf/ of content. It is proposed to develop a cloud service capaaprucbhliivcaocsi-oyn-edso-cduemsceanrtgaacbiolens//dpoucbulimcaecniotanceiso-np-eyr-ipoudibclaicsa/ciones/ ble of automatically labeling/classifying texts according informe-sobre-la-violencia-contra-la-mujer/Informe_evolucion_ to their level of ofensiveness or toxicity. In addition, an delitos_de_odio_en-Espana_2019_126200207.pdf interface will be designed to detect terms or passages 5https://www.interior.gob.es/opencms/pdf/ that contain inappropriate language (abusive, ofensive, archivos-y-documentacion/documentacion-y-publicaciones/ foul language, hate speech...), marking them in such a ipnufbolrimcaec-isoonberse--dleas-ceavroglaubclieosn/-pdueb-lliocsa-cdioenlietos-s-pdeer-iooddiicoa-se/n-Espana/ way as to facilitate their substitution so that the final Informe_evolucion_delitos_odio_Espana_2022_126200207.pdf text eliminates or reduces the degree of ofensiveness. The service will work both as an assistant to guide the user who writes a text and avoid possible ofensive expressions, as well as for the construction of listening, monitoring and filtering tools in real time on social media (comments, responses, posts...). A prototype will be built with diferent modules that will be integrated into a single platform that will be freely accessible through an Application Programming Interface (API) and through a demonstrator that will allow the configuration of filters and search parameters, in order to obtain diferent types of reports.
The main objective of this project is to study and develop efective solutions based on advanced machine learning and NLP techniques to determine the degree of ofensiveness/toxicity of a text in Spanish. It is proposed to develop a cloud service capable of automatically labeling/classifying texts according to their level of ofensiveness or toxicity. In addition, an interface will be designed to detect terms or passages that contain inappropriate language (abusive, ofensive, foul language, hate speech...), marking them in such a way as to facilitate their substitution so that the final text eliminates or reduces the degree of ofensiveness. The service will work both as an assistant to guide the user who writes a text and avoid possible ofensive expressions, as well as for the construction of listening, monitoring and filtering tools in real time on social media (comments, responses, posts...). A prototype will be built with diferent modules that will be integrated into a single platform that will be freely accessible through an Application Programming Interface (API) and through a demonstrator that will allow the configuration of filters and search parameters, in order to obtain diferent types of reports.
To achieve this goal, innovative techniques in NLP and advanced machine learning algorithms will be studied, designed and evaluated, including the generation of language models, application of deep learning and transfer learning techniques, as well as the latest experiences in multitask learning and zero-shot learning that allow the detection of inappropriate behaviors, including ofensive language and hate speech, through the integration of sentiment analysis techniques, emotion recognition or toxicity detection, among others. These afective computing elements are proving to be very eficient in the detection of inappropriate language, as they go beyond simple pattern detection, surface learning or lexicon searches, integrating advanced knowledge extracted from large text collections into the trained models.
This global goal can be divided into the following subobjectives: • Construction and compilation of new tools and resources based on human language technologies to infer, create and utilize knowledge applied to digital content, focusing on the creation of semiassisted annotators and their application in the annotation process to generate labeled data sets. • Identification of valid technologies for “listening” the interactions of individuals with their digital and social environment, so these interactions can be further analysed. • Study, development and implementation of language technologies together with advanced machine learning algorithms focused on the detection of inappropriate behavior on social networks and hope speech. • Study and development of deep learning algorithms to model diferent targeted forms of aggressive communication or risky situations, building artificial intelligence solutions to protect citizens. • Development and implementation to determine the degree of ofensiveness/toxicity/hope speech/hate speech of a text in Spanish.
To achieve these objectives, a methodology and recommendations on search tools in social networks will be defined and established. Filters and search parameters will be defined, and artificial intelligence algorithms based mainly on language technologies (language models, entity recognition, external knowledge integration, author profiling...) and advanced machine learning techniques oriented to neural networks (deep learning, transfer learning, multitask learning, zero-shot learning) will be proposed, although classic techniques such as SVM or logistic regression will also be evaluated. Existing and new or adapted linguistic resources that can contribute to the discovery of aggressive language patterns will also be taken into account. We will analyze which social networks and social media are the most suitable for monitoring and how to approach the problem in each of them.
The first step will be an in-depth study of the state of the art as well as an analysis of the problem. We will also decide the scenarios and social media to work on, establishing a working methodology for each of them.
After the analysis and establishment of the methodology, we will start with the design and implementation of the platform to help to detect hate speech and ofensive language. A prototype with the integration of the diferent modules will be freely accessible, allowing the configuration of filters and search parameters, in order to obtain diferent types of reports. During the development, various learning algorithms, language models and linguistic resources will be tested and evaluated in order the project. Firstly, there exists an important lack of to refine the final result and adjust the application for the resources, even more when dealing with the Spanish lantask of analysis and detection of ofensive language dis- guage. In the project we will focus on generating data course. For each of the selected domains and scenarios, and developing methods, techniques and tools to secure the following activities will be carried out: our interactions in the digital society. New architectures for artificial intelligence represent the main approach to 1. Data collection: Good practices will be applied
address social media monitoring, but further questions for the retrieval, processing and storage of big
remain, as new linguistic resources, specific and adapted data. Since we will be working with textual in
language models and integration solutions are needed to formation obtained from social networks, it is
reach the objectives proposed by this project. important to identify those technologies that are
On the other hand, the impact of the COVID-19 panvalid for "listening" and storing the interactions
demic has leveraged our dependency to the digital chanof individuals with their digital and social envi
nels of communication such as social media and social ronment.
networks, increasing the potential negative efect on the 2. Design of techniques and tools based on NLP: population, especially among vulnerable communities Development of algorithms based on neural net- such as children and teenagers, women, LGTBI, immiworks to model diferent specific forms of commu- grants, religious and cultural communities. nication, building artificial intelligence solutions For all these reasons it is expected that the project will that will act as early detection systems. These have a scientific impact (both nationally and internationalgorithms will be trained from available labeled ally) in diferent fields, such as the creation and use of efdatasets. Furthermore, once these algorithms are fective linguistic, semantic and computational resources implemented and start collecting interactions, the for diferent languages, especially in Spanish; the develalgorithms will be fine-tuned by re-training with opment and integration of specific methods and tools for the captured and classified information to adapt the retrieval, extraction, automatic analysis, summarizato the variability and flexibility of language over tion and representation in digital entities of information time: slang, ill-formed expressions, typos, gram- coming from diferent textual genres as well as the immar that occur very often in social networks and provement or creation of resources and components such other interpersonal communications. as crawlers, sentiment and opinion analysis systems and 3. Integration of technologies: These trained mod- integration tools, which will have a strong impact on the els will be incorporated into a web application scientific community and society. It is also expected to that will act as a monitor of ofensive language have transferable results in the medium term, working messages in social networks, so that, for each with real practical cases. message retrieved, its content will be automatically analyzed to detect whether it contains hate speech, ofensive language, toxicity... and the re- 5. Social and Economic Impact sult will be visualized through diferent graphs.
The user will have the possibility to generate a This proposal focuses on a significant strategic societal report and analyze the content of the data based challenge given that the detection and prevention of inon filters. appropriate behavior on the Internet is of the greatest im4. Evaluation of automatic systems: An essential portance for the contribution to the goal of smart, sustainstep is the evaluation of the developed algorithms able and inclusive growth within Europe in accordance to estimate the accuracy and coverage that these with European, national and regional policy-making. algorithms could have in a real scenario, i.e. when Digital media have become a space where hoaxes, hate monitoring the Web to detect this type of inap- speech or abusive behavior proliferate, among other conpropriate behavior. tents that directly and negatively harm the users of this space in particular and society in general. Thus, this
The diferent activities described in the previous steps project will ofer the modeling of the behavior of digital can be visualized in Figure 1. content, being able to contribute, in the detection, mitigation and prevention of harmful digital content, in pursuit 4. Scientific and Technical Impact of a sanitation of social media on the Internet helping to ensure a respectful, safe and reliable communication environment.
The project will have an important social impact from the point of view of digital content modeling. This modeling will provide a framework for specialists to develop
and implement information systems to address negative of the project justification. Companies like Facebook or social phenomena, protect society from dangers that may Twitter spend huge amounts of money developing autobe posed to citizens, etc. Also, positive digital content matic models for detecting inappropriate content. Major can improve digital literacy and people’s e-safety skills social media companies spend huge amounts of money and awareness to prevent the risks of harmful content on developing automatic models for detecting inappropriate the Internet, and thus improve users’ experience on the content. The estimated investment in content moderaInternet. Given that service providers have legal respon- tion will rise from 5 billion today to almost 12 billion in sibility for content, having tools for monitoring illegal the next five years. Due to problems with human moderahate speech would help to comply with the protocols tion, companies already make use of artificial intelligence imposed by legislation . for automatic moderation in 60% of the content. Hav
In summary, a direct social impact is to reduce the ing platforms and services such as the one proposed by harmful efects that inappropriate content causes on so- the project would have a series of clear impacts on concial networks, especially among the most vulnerable sec- tent moderation: 1) Increase the performance of manual tors of society: young people and children, people with moderation, 2) Reduce the costs of developing and intedisabilities and people susceptible to harassment (women, grating automatic detection solutions, and 3) Promote LGTBI, immigrants, religious and cultural communities). the moderation of content in Spanish, a language that
In terms of economic impact, several sectors could ben- already accounts for 8% of all communications on social efit such as communication media, tourism, fake-news networks, with a constant increase each year (reaching detection, digital security, hate speech detection, con- 15.6% in the most widespread network, Facebook). structive discourse promotion or self-learning. On the Despite the clear interest of such large corporations, other hand, big tech companies could be interested in this numerous startups and technological mid-size companies technology, also providing a great economic impact on are ofering automatic tools for textual analysis in a wide the transfer of this knowledge as mentioned in Section 2 range of sectors such as press, social media, e-commerce... Content moderation is needed by many online services (anyone where clients of users are able to publish comments). The technology and solutions released by the project could help local and international companies in providing more robust and performance products, empowering Spanish oriented products and improving competitiveness for such as mid-size companies.