Multimodal Models: A Transnational Feminist Approach Considering Geographical Region and Culture

Abstract

Deep learning based visual-linguistic multimodal models such as Contrastive Language Image Pretraining (CLIP) have become increasingly popular recently and are used within text-to-image generative models such as DALL-E and Stable Difusion. However, gender and other social biases have been uncovered in these models, and this has the potential to be amplified and perpetuated through AI systems. In this paper, we present a methodology for auditing multimodal models that consider gender, informed by concepts from transnational feminism, including regional and cultural dimensions. Focusing on CLIP, we found evidence of significant gender bias with varying patterns across global regions. Harmful stereotypical associations were also uncovered related to visual cultural cues and labels such as terrorism. Levels of gender bias uncovered within CLIP for diferent regions aligned with global indices of societal gender equality, with those from the Global South reflecting the highest levels of gender bias.

Keywords

• Geographical
• Gender bias
• Multimodal models
• Computer vision

1. Introduction

Deep learning models used in computer vision have been shown to exhibit numerous social biases related to gender [ 1, 2, 3 ] and race [ 4, 1, 2 ]. Recently, such deep learning models have become more complex and moved towards multimodal operations with the capacity to work across modalities such as language and vision. Contrastive Language Image Pretraining (CLIP), for instance, is a large multimodal model by OpenAI trained on 300 million image-text pairs using contrastive learning [ 5 ] and used in popular generative models such as DALL-E and Stable Difusion [ 3 ]. Approaches to assessing bias in models are often informed by feminist theory and critical theories of race, and given that biases can occur at the intersection of multiple social identities, often adopt an intersectional perspective [ 2, 1, 4 ]. Geographical region, along with cultural diferences, are dimensions that afect gender inequality in society and are placed as a central focus of analysis within a transnational feminist perspective [ 6, 7, 8 ]. This research, therefore, builds on prior research on bias to incorporate consideration of geographical region and cultural features in an evaluation of how gender bias is manifested within large-scale multimodal models. In this paper, we have only considered binary gender for the purpose of our audit. This is done to reduce complexity and focus on specific parameters and does not reinforce or promote a binary view of gender.

2. Background and Related Work

Contrastive Learning Image Pretraining (CLIP) is a large multimodal visual-linguistic model developed by OpenAI, which connects text and images [ 5 ]. It is used in other multimodal models to create image or text embeddings which are further used down the pipeline [ 3 ]. The presence of social bias in CLIP can propagate downstream, be amplified and become evident in the final outputs of secondary models. Examples of such bias can be seen in generative models using CLIP, such as DALL-E 2 and Stable Difusion, where evidence of gender bias through the perpetuation of stereotypes was uncovered [ 3 ]. As CLIP forms the first stage of both these models generating image embeddings from text, it may well be the source of the bias or at least play a significant role in it.

2.1. Bias in Deep Neural Networks

2.1.1. Transnational Feminism

A transnational feminist perspective emphasises global diferences in the dynamics of gender inequalities in society [ 6, 9, 6, 8 ]. This standpoint necessitates consideration of the perspectives and contextual experiences of inequality from diferent regions and cultures. In relation to bias in large-scale multi-modal models, therefore, it is essential to study how such global geographical and cultural variations in gender inequality are reflected in multimodal models from diverse cultural and geographical contexts.

2.1.2. Bias in Computer Vision and Multimodal Models

Research on biases in computer vision evaluated the efect of skin tone and gender on facial recognition. For instance, Buolamwini and Gebru [ 1 ] found that classifiers from Microsoft, Face++, and IBM contained intersectional biases with the highest accuracy levels on the faces of men with lighter skin and the worst on the faces of women with darker skin tones. Further to consideration of skin tone, facial features are multifaceted and contain diverse visual cues such as those related to culture and ethnicity [ 10 ]. For instance, many people with fair skin but from diferent countries may difer in their appearances due to cultural norms in relation to clothing. [ 11 ] found that popular vision models often fail to detect and classify images from non-Western and developing countries. Drawing upon transnational feminism in our audit of CLIP addresses this issue, enabling the analysis of the efects of diversity with consideration to geographical region and culture.

2.1.3. Auditing Social Biases in CLIP

The authors of CLIP evaluated their own model and found evidence of social biases within it using datasets such as FairFace [ 4 ] and images of members of the US Congress. The racial classification within the FairFace dataset was compiled using the US Census with the addition of ‘Southeast Asian’ and ‘Middle Eastern’. Approaches to defining race itself and racial categories have been critiqued for being founded upon a predominantly Western perspective [ 12, 13, 14 ]. The use of certain race labels such as ‘Indian’, for instance, can be problematic given that it refers to nationality rather than one distinct race or ethnicity 1. This research, therefore, incorporates concepts from transnational feminism to audit CLIP in a way that considers race and gender from a trans-cultural perspective.

3. Methodology

To audit CLIP and understand how gender bias intersects with geographical region and culture, building on work by Mandal et al. [ 10 ], we created a dataset of images of men and women crawled from various geographical locations across the world. This method of basing the data gathering process in diferent regions of the world allows for the representation of gender that is presented to those diferent regions through internet searches to be captured and aligns with the importance of considering the issue of bias from multiple perspectives. We then created three sets of keywords denoting adjectives, occupations and negative and positive words. Using CLIP’s image and text encoders, the cosine similarity between the images and the keywords was then calculated to evaluate associations within the models.

3.1. The Image Dataset

We curated an image dataset using Google Image Search. The query terms were ‘man’ and ‘woman’ translated into diferent languages as per the location. We used Selenium to automate the image scrapping and used VPN to change the IP geo-location with each search happening in a new incognito browser profile. We used Western Europe, Eastern Europe, North Africa and West Asia, Sub-Saharan Africa, South Asia, Southeast Asia, East Asia, North America and Latin America as geographical regions as used by Mandal et al. [ 10 ]. The languages for the query terms and the country for the VPN location are provided in Table 1 along with language and location pairs and corresponding abbreviations. For each term and each region, 70 images were scraped, totalling a dataset of 1,260 images (630 each for men and women, 140 for each region).

3.2. The Keywords

We used three sets of keywords. The first set is based upon the bias analytics conducted by the developers of CLIP [ 5 ] and consists of five positive ( trustworthy, educated, smart, confident, and achiever ) and five negative ( criminal, terrorist, gangster, drug addict, and fraud) words. The next two sets of keywords: adjectives and occupations comprise five words associated with men and five with women each. For adjectives, the words honorable, dissolute, arrogant, heroic,

Indonesian Indonesia and boyish are associated with men, and romantic, submissive, elegant, caring, and delicate are associated with women. In the case of occupations, carpenter, mechanic, mason, architect, and mathematician are male-dominated and midwife, librarian, housekeeper, dancer, and teacher are female-dominated [ 15 ]. These sets of words are taken from Garg et al. [ 15 ], and five words were randomly chosen from the list for each of the subcategories.

3.3. Image-Text Similarity

CLIP is a multimodal model that creates embeddings for text and images using text and image encoders, trained using contrastive learning to find the most similar image-text pairs [ 5 ]. By calculating the cosine similarity of the image and text embeddings, we can find patterns that can point out bias in the CLIP embeddings. The similarity is calculated by adopting the approach developed by the authors of CLIP 2. Similarly, the image encoder used in our experiments is Vision Transformer ViT-L/32 and all keywords are prefixed with the sentence ‘An image of ’ following.

3.4. Visual Question Answering and Grad-CAM

We created a visual question-answering machine using CLIP that takes in an image and a text question (sentence) and answers the question based on the image. We then use Gradient Weighted Class Activation Mapping (Grad-CAM) [ 16 ] to create a heatmap superimposed on the original image to highlight the region of the image that the model uses the most to answer the question. 2https://colab.research.google.com/github/openai/clip/blob/master/notebooks/Interacting_with_CLIP.ipynb

4. Findings and Discussion

The image-text cosine similarity scores were calculated for the three sets of keywords: negative and positive traits, adjectives and occupations. The mean value of the scores for all the images from each region gender-wise is used for analysis. We also used Grad-CAM analysis for the negative and positive traits for further analysis. The findings are discussed in detail in the following subsections.

A summary of the trends in the scores is given in Table 2, where trend refers to the net positivity or negativity in the scores and is given as = ∑ − ∑ where is the mean cosine similarity of the positive words and is the mean cosine similarity of the negative words. Gender Diference is calculated as:

Diference = | ∑ − ∑ | where, M ∈ mean cosine similarity for images of men and W ∈ mean cosine similarity for images of women.

4.1. Negative and Positive Words

We see that the mean cosine similarity scores are higher for all the images, but images of women generally have less negativity than men but with geographical diferences. For images of women from Europe, North America, and East Asia, the trend results are zero (i.e. neutral). These regions generally comprise the ‘Global North’ and are generally wealthy, developed, and democratic [ 17 ]. Images of women from Sub-Saharan Africa, South-East Asia, and West Asia and North Africa have the highest levels of negative associations. These regions generally comprise the ‘Global South’ and lag behind the Global North in wealth and development [ 17 ]. The gender diference is highest for South Asia and West Asia and North Africa. These two regions also score the lowest in the Global Gender Gap Index 3.

The gender diference for Sub-Saharan Africa is low, but this region also ranks low in the Global Gender Gap Index. Fig 3 shows the relationship between the Global Gender Gap Index and gender diference, demonstrating a strong relationship between the two scores. The regions with the highest Global Gender Gap Index, such as Europe, North America, and East Asia, tend to have the lowest gender diference. The Global Gender Gap Index used in this paper is for the 3https://www3.weforum.org/docs/WEF_GGGR_2022.pdf country from where the images were scraped, as shown in Table 1. In the case of two countries, the average is used.

The similarity for the word ‘terrorist’ is highest for the images of women from South-East Asia and West Asia and North Africa (Appendix A). The predominant religion in these two regions also happens to be Islam4. Using Grad-CAM, we found that women from these regions have a higher chance of being assigned the label ‘terrorist’ (see Figure 1). On further analysis, we found that images of women wearing hijab (headscarf) are more likely to be associated with the label ‘terrorist’. In Figure 2, an image of two women from the same region (West Asia and North Africa), but with one wearing a hijab, was given to the visual question answering machine with the text ‘Who is the terrorist’. As seen in the Grad-CAM image, the region on the left with the woman wearing a hijab is highlighted more, indicating that the model focuses on that region to answer that question. This suggests that cultural artefacts such as clothing can lead to biases within multimodal models.

4.2. Adjectives

The cosine similarity scores for adjectives show stereotypical gender bias for men and women. The masculine adjectives have a higher similarity with images of men, and the feminine 4https://web.archive.org/web/20110209094904/http://www.pewforum.org/The-Future-of-the-Global-Muslim-Population. aspx, Last accessed: June 2023 adjectives have a higher similarity with the images of women. Figure 4 shows the mean cosine similarity of the keywords by region. Images of women from East and South-East Asia have higher similarity for the terms ‘caring’, ‘elegant’, and ‘delicate’. This may reflect a Western bias which considers Asian women as more ‘feminine’ [ 18 ]. The gender diference scores are the lowest for Europe, North America and East Asia. These regions tend to be developed and wealthier and score better in the Global Gender Gap Index [ 17 ]. West Asia and North Africa, and South Asia have the highest gender diference and perform worse in the Global Gender Gap Index 5. Fig 5 shows the relationship between the Global Gender Gap Index and gender diference, and a strong relationship is seen between the two scores.

4.3. Occupations

The cosine similarity scores for occupations show stereotypical gender bias for images of men and women for all regions. A heatmap of the similarity scores is given in Fig 6. Traditionally male-dominated occupations such as ‘mechanic’, ‘architect’, and ‘mathematician’ have higher 5https://www3.weforum.org/docs/WEF_GGGR_2022.pdf similarity scores for men, while traditionally female-dominated occupations such as ‘midwife’, ‘housekeeper’, and ‘librarian’ have higher similarity scores for women. Images of women from South, East, and South-East Asia had the highest similarity with occupations such as ‘midwife’, ‘housekeeper’, and ‘librarian’. Images of women from Europe and North America have lower similarity for traditionally female-dominated occupations such as ‘midwife’ but higher similarity for traditionally male-dominated occupations such as ‘architect’. The gender diference scores show a similar trend as seen earlier; Europe and North America show the least gender diference and are the regions with the best Global Gender Gap Index. Fig 7 shows the relationship between the Global Gender Gap Index and gender diference, and also reflects a strong relationship between the two scores.

5. Conclusion

Gender bias is a complex, multifaceted, and multidimensional issue comprising various dimensions such as race, ethnicity, culture, and geography. Thus it is dificult to analyse the issue using a singular theoretical lens or theories primarily developed in the Western world. Transnational feminism ofers places importance on the analysis of the issue of gender bias from a more inclusive lens, accommodating diverse global perspectives such as globalisation, income inequality, and the economic and digital divide between the global north and south among other contemporary issues. In incorporating this perspective in our research, we uncovered significant evidence of gender bias in CLIP with diferences in how such bias manifests regionally and culturally. Findings indicated that cultural components such as clothing can contribute to stereotypical associations. A strong correlation was also evident between the Global Gender Gap Index and gender diference scores, with Europe, North America, and East Asia scoring high on both the indices and South Asia, and West Asia and North Africa performing the worst. This may be related to levels of gender equality in society influencing the representation of gender within internet content from those regions, afecting levels of gender bias in training data. CLIP is also trained on data primarily curated from the English internet and biases exhibited are those inherited from it and this may explain the association of ‘hijab’ with ‘terrorism’ as has been explored in earlier research [ 10, 2, 11 ].

Acknowledgments

Abhishek Mandal was partially supported by the <A+> Alliance / Women at the Table as an Inaugural Tech Fellow 2020/2021. This publication has emanated from research supported by Science Foundation Ireland (SFI) under Grant Number SFI/12/RC/2289_2, co-funded by the European Regional Development Fund.

A. Consolidated mean scores - positive and negative traits R E i s t a i-sh .0520 .8210 .8610 .8910 .9810 .1990 .210 .8610 .9710 .6020 .02 .190 .8810 .9610 .1890 .1930 .2180 .1870 .2040 .2010 ittev r n a a g p A e S L n g E n E W - 8 7 7 8 8 8 9 7 18 .01

7 8 8 3 3 9 7 7 3 7 7 5 6 7 1 5 4 1 8 9 lish .910 .170 .01 .810 .190 .01 .910 .710 .810 .910 .10 .10 .10 .10 .10 .10 .10 .10 .0 u E d 3 li li-sh .910 .710 .810 .10 .10 .910 .910 .170 .01 .910 .109 .018 .01 .109 .109 .108 .200 .107 .019 .019

7 8 3 8 9 2 4 5 8 6 7 2 8 8 2 6 6 1 3 6 ra A A W i-cb AN .810 .107 .107 .106 .109 .108 .202 .108 .108 .109 .200 .108 .108 .108 .019 .019 .024 .01 .0

9 9 7 8 5 3 9 8 4 4 3 9 2 6 8 1 2 98 .20 022

References

[1] J. Buolamwini , T. Gebru, Gender shades: Intersectional accuracy disparities in commercial gender classification , in: Conference on fairness, accountability and transparency, PMLR , 2018 , pp. 77 - 91 .

[2] A. Birhane , V. U. Prabhu , E. Kahembwe, Multimodal datasets: misogyny, pornography, and malignant stereotypes , arXiv preprint arXiv:2110 . 01963 ( 2021 ).

[3] A. Mandal , S. Leavy , S. Little , Multimodal composite association score: Measuring gender bias in generative multimodal models , arXiv preprint arXiv:2304.13855 ( 2023 ).

[4] K. Karkkainen , J. Joo , Fairface: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation , in: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , 2021 , pp. 1548 - 1558 .

[5] A. Radford , J. W. Kim , C. Hallacy , A. Ramesh , G. Goh, S. Agarwal , G. Sastry , A. Askell , P. Mishkin , J. Clark , et al., Learning transferable visual models from natural language supervision , in: International Conference on Machine Learning, PMLR , 2021 , pp. 8748 - 8763 .

[6] L. Briggs , 991Transnational, in: The Oxford Handbook of Feminist Theory , Oxford University Press, 2016 .

[7] M. J. Alexander , C. T. Mohanty , Feminist genealogies, colonial legacies, democratic futures , Routledge , 2013 .

[8] I. Grewal , C. Kaplan , Scattered hegemonies: Postmodernity and transnational feminist practices , U of Minnesota Press, 1994 .

[9] J. Henrich , S. J. Heine , A. Norenzayan , The weirdest people in the world? , Behavioral and brain sciences 33 ( 2010 ) 61 - 83 .

[10] A. Mandal , S. Leavy , S. Little , Dataset diversity: Measuring and mitigating geographical bias in image search and retrieval , in: Proceedings of the 1st International Workshop on Trustworthy AI for Multimedia Computing , 2021 , pp. 19 - 25 .

[11] T. De Vries , I. Misra, C. Wang , L. Van der Maaten , Does object recognition work for everyone? , in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , 2019 , pp. 52 - 59 .

[12] S. O. Y. Keita , R. A. Kittles , C. D. Royal , G. E. Bonney , P. Furbert-Harris , G. M. Dunston , C. N. Rotimi , Conceptualizing human variation , Nature genetics 36 ( 2004 ) S17 - S20 .

[13] K. A. Kennedy , But professor, why teach race identification if races don't exist? , Journal of Forensic Sciences 40 ( 1995 ) 797 - 800 .

[14] R. F. Kennedy , C. S. Roy , M. L. Goldman , Race and ethnicity in the classical world: An anthology of primary sources in translation , Hackett Publishing, 2013 .

[15] N. Garg , L. Schiebinger , D. Jurafsky , J. Zou , Word embeddings quantify 100 years of gender and ethnic stereotypes , Proceedings of the National Academy of Sciences 115 ( 2018 ) E3635 - E3644 .

[16] R. R. Selvaraju , A. Das , R. Vedantam , M. Cogswell , D. Parikh , D. Batra , Grad-cam: Why did you say that? , arXiv preprint arXiv:1611.07450 ( 2016 ).

[17] N. Dados , R. Connell , The global south , Contexts 11 ( 2012 ) 12 - 13 .

[18] M. Ciurria , An intersectional feminist theory of moral responsibility , Routledge , 2019 .