Detecting stereotypes is a challenging task, particularly when they are not expressed explicitly. In this study, we applied an annotation schema from the literature designed to formalize implicit stereotypes. We analyzed implicit stereotypes about immigrants in two datasets: StereoHoax-IT and SterheoSchool, which are created from diferent sources. StereoHoaxIT consists of reactions on Twitter to specific hoaxes aimed at discriminating against immigrants, while SterheoSchool includes comments from teenagers on fake news generated in psychological experiments. We describe the annotation process, annotator disagreements, and provide both quantitative and qualitative analyses to shed light on how implicitness characterizes stereotypes in diferent texts. Our findings suggest that implicit stereotypes are often conveyed through logical linguistic relations, such as entailment and behavioral evaluations of immigrants.
1Transl. "They throw away the food they are given only to go eat the
poor dogs. Where will we end up!"
2Transl. "Just as we respect them and the color of their skin, they, who
live in our countries, should show respect toward us."
phenomena, in particular those characterized by sub- the presence or absence of anti-migrant stereotypes, and,
jectivity, such as irony [23]. In this paper, we analyze if present, for other related categories such as whether
the implicit manifestation of stereotypes targeting immi- the stereotype was expressed implicitly or explicitly and
grants, using a well-defined annotation schema proposed which forms of discredit the stereotype could be
clasby Schmeisser-Nieto et al. [14] and tested on a subset sified at. This category is inspired by the Stereotype
of comments from Spanish newspapers (DETESTS [
Our main contributions consist of expanding the an- primarily for the presence of anti-migrant stereotypes.
notation with topics of stereotypes about immigrants [
The rest of the paper is organized as follows: Sections 2 are also associated with metadata, such as the age and and 3 describe the datasets and the annotation applied; declared gender of the author. By collecting data generSections 4 and 5 present quantitative and qualitative anal- ated by teenagers, this corpus aims to fill a gap in the yses of the annotated data; and Section 6 summarizes the literature in which teenagers are an underrepresented results and provides guidance regarding future work. category in data annotated for text classification tasks. We applied the annotation scheme mentioned above to the news and comments. This corpus consists of 1,147 2. Datasets comments, of which 337 (33.8%) are annotated as containing stereotypes, of which 152 (45%) are expressed in an implicit form.
taining implicit stereotypes developed within the
STERHEOTYPES project4 and the SterotypHate project5. Their
content is related to attitudes regarding immigrants and 3. Annotation
they share similar conversational structures and the same
annotation scheme. Each message in these datasets is The annotation scheme we applied on the two corpora
contextualized, i.e. collocated within a discourse thread is based on two diferent layers, topics of stereotypes and
or presented as a comment on a given news item. For implicitness strategies, as well as the need for context.
the annotation scheme, each message is annotated for The topics of stereotypes were firstly introduced
within an evaluation task, DETESTS [
4STERHEOTYPES (Studying European Racial Hoaxes and sterEOTYPES) is an international project funded by Compagnia di San Paolo and VolksWagen Stiftung. 5StereotypHate is a project funded by Compagnia di San Paolo. 6The datasets will be made available for research purposes after the acceptance of the paper in anonymized form. contained stereotypes, and when they did, classify the stereotype into ten diferent categories: linguistic devices used to convey implicit stereotypes, we have revised the criteria proposed in [14] as follows: • Xenophobia victims Immigrants are perceived as victims of xenophobia and discrimination. They enrich culture and diversity and should have the same rights as citizens. • Sufering victims Immigrants are portrayed as victims of poverty and violence in their places of origin and as having to face dificult situations in their host countries. • Economic resources Immigrants are seen as an economic resource. They do the jobs that locals do not want to do, pay taxes and solve the problems arising from low population growth. • Migration control Immigrants present a threat due to massive influxes and a lack of control at the borders. Immigrants are illegal and should be expelled. It is seen as an invasion. • Culture and religion diferences Immigrants suppose a loss of the in-group’s values and traditions and the replacement of the target group’s customs and religions. They are also seen as uneducated and should adapt to their host country. • Benefits Immigrants compete with the in-group for resources such as public subsidies, school places, jobs, health care and pensions. They are privileged over the in-group. • Public health Immigrants are thought to be carriers of infections and diseases such as COVID-19, Ebola and HIV. • Security Immigration brings security issues. Due to immigration, there is an increase in crime, domestic violence, robbery, drug use, sexual assault, murder, terrorist attacks and public disorders. • Dehumanization Immigrants are seen as inferior beings and are compared with animals, parasites or scum. Their lives have less value than those of the in-group. • Other topics Any other immigration stereotypes not covered in the previous categories.
posed as criteria that could help annotators to annotate implicitness, since their vagueness may decrease InterAnnotator Agreement (IAA) [14]. By context, we refer to information contained in previous messages, which is considered necessary to understand the meaning of the message to be annotated, as in the following example: "Sempre assolti...sempre misure e pesi diferenti". Context: "Uccide anziana ebrea al grido di Allah Akbar. Assolto perché drogato."7 (StereoHoax-IT). Regarding the strategies and
Context: "Kills elderly Jewish woman while shouting ‘Allah Akbar.’ Acquitted because he was on drugs." • World knowledge World knowledge refers to the shared cultural, social and historical knowledge needed to interpret messages, e.g., "La scuola si inchina all’islam: l’aceto è bandito dalle mense." 8 (StereoHoax-IT) • Figures of speech Every figure of speech except for irony and sarcasm, and humor and jokes.
For instance, metaphor, rhetorical questions, euphemisms or reported speech, e.g., "Chi è quel pazzo che si mette in casa uno di questi? Un suicidio" 9 (StereoHoax-IT) • Irony/Sarcasm The message expresses a meaning that is the opposite of what is said, e.g. in "Che bella gente fanno arrivare.....che bello avere un paese pieno di risorse pronte a tutto.....ma proprio a tutto." 10 (StereoHoax-IT) • Humor/Jokes Jokes about a target group often use stereotypes and may or may not include irony, e.g. in "Chissà se ha detto:"Cibo no buono"." 11 (StereoHoax-IT) • Extrapolation The target refers to an individual or specific members of a social group, not the group as a whole, e.g. in "Classico del sud-italia
Maleducata" 12 (SterheoSchool) • Imperative/Exhortative Calls to take certain actions related to the target group, e.g. "Come in
Cina FUCILATELO" 13 (StereoHoax-IT) • Entailment/Evaluation Logical relation between two sentences in which the condition of truth of sentence A implies the truth of sentence B. The implicit stereotype is implied in sentence A. An evaluation of the author’s or in-group’s thoughts, emotions and behaviors, rather than content about the out-group or target group, can be considered as a type of entailment, e.g. "Saranno fuori o liberi presto" 14(StereoHoax-IT) is the answer to a racial hoax in which a group of immigrants rape and murder a teenage girl. With the author’s evaluation of the situation, it is entailed that immigrants are immune from punishment. • Other implicitness Other types of implicitness not considered in the previous categories. e.g. "al giorno d’oggi non ci si può fidare di nessuno una persona ripugnante" 15(SterheoSchool) 8Transl. "The school bows to Islam: vinegar is banned from canteens." 9Transl. "Who’s that fool who takes one of these into his house? a suicide" 10Transl. "Such nice people they bring in... how nice it is to have a
country full of resources ready for anything... anything at all" 11Transl. "I wonder if he said: «Food no good»" 12Transl. "Typical of Southern Italy" 13Transl. "SHOOT HIM like in China" 14Transl. "They will be out or free soon" 15Transl. "nowadays you can’t trust anyone a repulsive person" Table 1 stereotypical topics that portray immigrants as threats, Inter-annotator agreement test using Fleiss’ kappa ( ) coefi- the security issue is highly prevalent in both datasets. cient on the categories of implicitness and stereotype topics A common trend shows that the most frequent implicof the StereoHoax-IT and the SterheoSchool corpora. itness strategy in both datasets is ‘entailment/evaluation’, Label StereoHoax-IT SterheoSchool accounting for 64% in StereoHoax-IT and 80% in SterheoSchool. To a lesser degree, ‘extrapolation’ appears in SXuenferoipnhgovbiciativmicstims 0.04.957 0.05.050 both datasets, with 13% in the former and 19% in the latEconomic resource 0.48 0.50 ter, respectively. Other represented strategies that exceed Migration control 0.77 0.55 10% of instances are only found in StereoHoax-IT. Culture & religion 0.75 0.71 The label ‘context’ has a high prevalence in both Benefits 0.75 0.62 datasets, accounting for 38% in StereoHoax-IT and 80% SPeucbulircithyealth 00..8816 00..5604 in SterheoSchool. This is expected, as it depends on the Dehumanization 0.71 0.71 methodology to produce the comments—spontaneous Other topics 0.52 0.43 versus controlled—and the variety of contexts: two fake news for StereoSchool and 50 racial hoaxes for Context 0.72 0.50 StereoHoax-IT. The limited amount of data unfortunately World knowledge 0.52 0.51 does not allow us to reliably evaluate a correlation beFigures of speech 0.68 0.70 tween ‘context’ and certain implicitness strategies, as Irony/Sarcasm 0.70 0.50 shown in Table 3, except for the association between ‘enHumor/Jokes 0.52 No cases tailment/evaluation’ and ‘context’ across both datasets. Extrapolation 0.51 0.53 The correlation between ‘implicitness’ and ‘context’ is IEmntpaeirlmateivnet//EExvhaoluratatitoivne 00..7435 00..4593 also shown in Bourgeade et al. [27], with significant assoOther implicitness 0.51 0.52 ciations of the aforementioned labels in three languages: French, Italian and Spanish. In StereoHoax-IT, the correlations between the ‘context’ and ‘irony/sarcasm’, ‘extrap
The annotation was carried out on the Label Studio olation’ and ‘imperative/exhortative’ are also significant, platform by three native Italian speakers with a back- whereas the category of other implicitness strategies is ground in linguistics, some of whom specialized in NLP. also significantly correlated in SterheoSchool, which can They achieved an acceptable to good IAA in the majority be analyzed qualitatively to determine if there is a pattern of cases, as reported in Table 1, which varies across cate- among them. The other strategies do not have represengories and corpora. By observing Table 2, we can see that tative instances that allow for analyzing them comparaonly a few topics have been marked by the majority of tively, except for ‘extrapolation’, which is significantly annotators , while not all the implicit criteria have been correlated in StereoHoax-IT but not in SterheoSchool. identified in the texts (i.e., ‘humor/jokes’). In terms of co-occurrences between topics and implicit 4. Quantitative Analysis strategies, we can observe from Table 4 that there is also a great disparity in both datasets. Focusing on the two Table 2 shows the distribution of the disaggregated anno- topics with the highest representation in SterheoSchool tations across both datasets. Columns 0%, 33%, 67% and (Culture & religion, 51%, and security, 35%), which ac100%, respectively, indicate the number of instances per count for the majority of the corpus, we can analyze label that were annotated by no annotator (0%), by one some diferences with StereoHoax-IT. Firstly, ‘culture & annotator (33%), by two annotators (67%) and by all three religion’ is expressed primarily through entailments or annotators (100%). Column % positive class shows the per- evaluations (65 co-occurrences) and secondarily through centage of the label voted by the majority of annotators, extrapolations in SterheoSchool. In contrast, the distriand its total number of cases in parentheses. bution of strategies used to represent ‘culture & religion’
Firstly, an inconsistency in the distribution of labels stereotypes is more evenly spread in StereoHoax-IT. A can be observed since SterheoSchool has a representation similar pattern is observed with the topic of ’security’, of labels of more than 10% on only four labels. This dispar- which, while concentrating strategies in ’entailment/eity is due to the extraction methods of each dataset: the valuation,’ also utilizes a range of other strategies, partictopics of the racial hoaxes used to extract the dataset were ularly ‘extrapolation’ and ‘imperative/exhortative’. With more balanced in StereoHoax-IT than in SterheoSchool, these co-occurrences, we can reafirm that the diferent with the latter focusing generally on security and cultural methods to extract the data have an impact on the characdiferences that are discussed in the two only contexts teristics of it, and therefore, its distribution of labels. For provided to the students for their comments. However, instance, the messages were written in a non-controlled while in the former there is a representation of all the environment, which gives the authors the freedom to express themselves without constrains. Moreover, the Labels Xenophobia victims Sufering victims Economic resource Migration control Culture & religion Benefits Public health Security Dehumanization Other topics Context World knowledge Figures of speech Irony/Sarcasm Humor/Jokes Extrapolation Entailment/Evaluation Imperative/Exhortative Other implicitness topics in StereoHoax-IT are more balanced, as seen in the distribution of ‘entailment/evaluation’, which is also used in ‘migration control’, ‘benefits’, ‘public health’ and ‘dehumanization’. On the other hand, in SterheoSchool, both initial fake news have the same narrative features, such as describing an aggression and highlighting the origin of the aggressor, thus eliciting a reaction in the readers related to these topics. The example "Siamo alla follia: ad Agrigento autobus gratis agli immigrati per evitare violenze e aggressioni." 16 (StereoHoax-IT) is related to security expressed through extrapolation. The example "Un cristiano che entrasse in una moschea in un paese arabo e sputasse per terra sopravviverebbe pochi secondi." 17 (StereoHoax-IT) highlights cultural and religious diferences by the evaluation of a hypothetical situation. 16Transl. "It’s crazy: in Agrigento, free buses for immigrants to prevent
violence and aggressions." 17Transl. "A Christian entering a Mosque in an Arab country and spitting on the ground would survive a few seconds."
Example 1 has been annotated with the topic ‘public health’ and ‘figures of speech’ and ‘Irony/Sarcasm’ for the strategy of implicitness; all labels achieved a 67% IAA.
1) Governo di involtini primavera!!! 18 (StereoHoax-IT) In the context given for this message, the author complains that the government did not use more restrictive measures against Chinese children during the early stages of COVID-19. First, an ironic reading, i.e., as stating A to mean not-A, is triggered by the metonymy “spring rolls” [29], identifying Chinese citizens through a traditional Chinese dish. Second, disapproval is conveyed showing a kind of favorable attitude of the Italian 18Trasl."Spring rolls government." government toward Chinese children. also interesting, and has been studied especially in social
Example 2 was annotated as ‘culture & religion’ by all media [32, 33], as a means to lower the negative social three annotators. In terms of the implicitness strategies, cost of what has been said. The two categories that most it was labeled as both ‘extrapolation’ and ‘entailment/e- frequently co-occur with ‘irony/sarcasm’ in StereoHoaxvaluation’ by two out of the three annotators. IT are ‘figures of speech’ (out of 35 instances, six are also 2) Venezia, donne velate sputano al crocifisso. 19 ironic) and ‘humor/jokes’ (out of three cases, two are (StereoHoax-IT) ironic), as in the next example: In this case, the noun phrase “veiled women” is a case of 5) @Belle facce intelligenti! Viva Lombroso!22 (67% Hulexical narrowing, i.e., a lexical item conveys a meaning mor/Jokes, 67% Irony/Sarcasm, StereoHoax-IT) that is more specific than the item’s encoded meaning. We found messages in which ‘entailment/evaluation’ coThe reader selects a more specific meaning on the basis occurs with ‘irony/sarcasm’, but this correlation should of stereotypes and world knowledge [30] of the mean- be analyzed in depth to be considered relevant, as 64% of ing of “veiled women”, which denotes a set of women instances were annotated as ‘entailment/evaluation.’ who wear a veil, narrowed to mean Muslim women. This equalization arises from the stereotype that posits that 6. Conclusions if a woman wears a veil, she is a Muslim. Furthermore, the absence of the determiner in the noun phrase, that In this paper, we applied an annotation scheme for analyzusually indicates a generic reference, combined with the ing the implicitness of stereotypes against immigrants acimperfective aspect and present tense of the verb, may cording to two main dimensions (i.e., topics and strategies suggest a habitual interpretation of the predicate "spit on for making the content implicit) to the Italian StereoHoaxthe crucifix" [ 31]. ‘Extrapolation’ strategy here refers to IT and SterheoSchool corpora. Adding these two layers the attribution of this action to the entire category. of annotation allowed us to observe that annotators need
Among the more frequently agreed implicitness strate- to use contextual information to determine the presence gies, there are ‘imperative/exhortative’ and ‘figures of of stereotypes especially, when specific strategies have speech’, which have linguistic and punctuation features been used by the author of the message (irony/sarcasm, closer to explicitness: the former is associated with a spe- extrapolation, entailment/evaluation, and imperative/excific grammatical mood and the exclamation mark, while hortative). Moreover, implicit stereotypes appear to be the latter is associated with a question mark (considering conveyed mainly through logical linguistic relations such that rhetorical questions are frequently annotated as a as the entailment and behavioral evaluation of immiifgure of speech), see e.g.: grants and, in fewer cases, via ‘imperative/exhortative’, 3) Se non fate niente Fra 10 anni l’italia sarà tutta musul- ‘irony/sarcasm’ and ‘extrapolation.’ mana!20 (StereoHoax-IT) As future work, we plan to perform a comparative 4) Come ci si può sentir sicuri in una società che permette analysis with the datasets in Spanish, which have already questo? meschina21 (SterheoSchool) been annotated with this schema, in order to understand The high IAA for the category of ‘irony/sarcasm’ is cultural analogies and diferences in portraying immigrants as threats, enemies or victims. 19Trasl."Venice, veiled women spit on the crucifix." 20Trasl."If you do nothing In 10 years Italy will be completely Muslim" 21Trasl."How can one feel secure in a society that allows this? mean" 22Trasl."Nice smart faces! Long life Lombroso!"
the project StereotypHate (Compagnia di San Paolo for the call ‘Progetti di Ateneo - Compagnia di San Paolo 2019/2021 - Mission 1.1 - Finanziamento ex-post’).
The work of Cristina Bosco is partially funded by the same project. for Italian. Final Workshop (EVALITA 2020), vol- https://aclanthology.org/E17-1025. ume 2765, CEUR Workshop Proceedings (CEUR- [24] G. Corbelli, P. G. Cicirelli, F. D’Errico, M. Paciello, WS.org), 2020. Conference date: 17-12-2020. Preventing prejudice emerging from misleading [17] F. Rodríguez-Sánchez, J. C. de Albornoz, L. Plaza, news among adolescents: The role of implicit actiJ. Gonzalo, P. Rosso, M. Comet, T. Donoso, vation and regulatory self-eficacy in dealing with Overview of EXIST 2021: sexism identification in online misinformation, Social Sciences 12 (2023). social networks, Procesamiento del Lenguaje Nat- [25] F. D’Errico, P. G. Cicirelli, G. Corbelli, M. Paciello, ural 67 (2021) 195–207. URL: http://journal.sepln. Addressing racial misinformation at school: A org/sepln/ojs/ojs/index.php/pln/article/view/6389. psycho-social intervention aimed at reducing eth[18] F. Rodríguez-Sánchez, J. C. de Albornoz, nic moral disengagement in adolescents, Social L. Plaza, A. Mendieta-Aragón, G. Marco-Remón, Psychology of Education (2023).
M. Makeienko, M. Plaza, J. Gonzalo, D. Spina, [26] C. Bosco, V. Patti, S. Frenda, A. T. Cignarella, P. Rosso, Overview of EXIST 2022: sexism M. Paciello, F. D’Errico, Detecting racial identification in social networks, Procesamiento stereotypes: An italian social media corpus del Lenguaje Natural 69 (2022) 229–240. URL: where psychology meets nlp, Information http://journal.sepln.org/sepln/ojs/ojs/index.php/ Processing & Management 60 (2023) 103118. pln/article/view/6443. URL: https://linkinghub.elsevier.com/retrieve/pii/ [19] L. Plaza, J. Carrillo-de Albornoz, R. Morante, S0306457322002199. doi:10.1016/j.ipm.2022.
E. Amigó, J. Gonzalo, D. Spina, P. Rosso, Overview 103118. of exist 2023–learning with disagreement for sex- [27] T. Bourgeade, A. T. Cignarella, S. Frenda, M. Lauism identification and characterization, in: Inter- rent, W. Schmeisser-Nieto, F. Benamara, C. Bosco, national Conference of the Cross-Language Eval- V. Moriceau, V. Patti, M. Taulé, A Multilingual uation Forum for European Languages, Springer, Dataset of Racial Stereotypes in Social Media Con2023, pp. 316–342. versational Threads, in: Findings of the Association [20] W. S. Schmeisser-Nieto, P. Pastells, S. Frenda, for Computational Linguistics: EACL 2023, AssoM. Taule, Human vs. machine perceptions on ciation for Computational Linguistics, Dubrovnik, immigration stereotypes, in: N. Calzolari, M.-Y. Croatia, 2023, pp. 686–696.
Kan, V. Hoste, A. Lenci, S. Sakti, N. Xue (Eds.), [28] E. Chierchiello, T. Bourgeade, G. Ricci, C. Bosco, Proceedings of the 2024 Joint International Con- F. D’Errico, Studying reactions to stereotypes in ference on Computational Linguistics, Language teenagers: an annotated italian dataset, in: ProceedResources and Evaluation (LREC-COLING 2024), ings of the Fourth Workshop on Threat, Aggression ELRA and ICCL, Torino, Italia, 2024, pp. 8453–8463. and Cyberbullying (TRAC-2024), 2014.
URL: https://aclanthology.org/2024.lrec-main.741. [29] G. Lakof, Women, fire, and dangerous things: What [21] A. Fokkens, N. Ruigrok, C. Beukeboom, G. Sarah, categories reveal about the mind, University of W. Van Atteveldt, Studying muslim stereotyping Chicago Press, Chicago, 1987. through microportrait extraction, in: Proceedings [30] Y. Huang, Implicitness in the lexis, in: P. Cap, of the Eleventh International Conference on Lan- M. Dynel (Eds.), Implicitness: From lexis to disguage Resources and Evaluation (LREC 2018), 2018, course, John Benjamins, Amsterdam/ Philadelphia, pp. 3734–3741. 2017, pp. 67–94. [22] J. J. Sánchez-Junquera, B. Chulvi, P. Rosso, [31] C. Lyons, Definiteness, Cambridge University Press, S. P. Ponzetto, How do you speak about im- Cambridge, 1999. migrants? taxonomy and stereoimmigrants [32] S. Frenda, V. Patti, P. Rosso, Killing me softly: dataset for identifying stereotypes about im- Creative and cognitive aspects of implicitness migrants, Applied Sciences 11 (2021). URL: in abusive language online, Natural Language https://www.mdpi.com/2076-3417/11/8/3610. Engineering 29 (2023) 1516–1537. doi:10.1017/ doi:10.3390/app11083610. S1351324922000316. [23] J. Karoui, F. Benamara, V. Moriceau, V. Patti, [33] S. Frenda, V. Patti, P. Rosso, When sarcasm hurts: C. Bosco, N. Aussenac-Gilles, Exploring the im- Irony-aware models for abusive language detecpact of pragmatic phenomena on irony detection tion, in: A. Arampatzis, E. Kanoulas, T. Tsikrika, in tweets: A multilingual corpus study, in: M. Lap- S. Vrochidis, A. Giachanou, D. Li, M. Aliannejadi, ata, P. Blunsom, A. Koller (Eds.), Proceedings of the M. Vlachos, G. Faggioli, N. Ferro (Eds.), Experimen15th Conference of the European Chapter of the tal IR Meets Multilinguality, Multimodality, and Association for Computational Linguistics: Volume Interaction, Springer Nature Switzerland, Cham, 1, Long Papers, Association for Computational Lin- 2023, pp. 34–47. guistics, Valencia, Spain, 2017, pp. 262–272. URL: