This paper presents a data visualization tool developed during the investigation of the bias present in deep learning language models in Spanish. The tool allows us to explore in detail the outcome of the response of the models we present with a set of template sentences, allowing us to compare the behavior of the models when the templates are presented with a context that alludes to a man or a woman. The exploration of the data in the tool is performed at various levels of detail, from visualizing the model output itself with its weights to visualizing the aggregation of the results by categories. It will be this last visualization that will provide some interesting conclusions about how the models perceive mainly women by their bodies and men by their behavior.
HireVue malfunctioned on non-native candidates, internal probability of the model, the second one is since their accent would confuse the model. In itself a RSV (ranked status value) metric that represents it is not a problem that a model does not work the external state of the model, taking in this case initially for all cases, the problem comes when the the inverse position in the ranking. For example, if candidate is automatically discarded and does not we get 5 results for each template, the first result receive information about the reason. This makes will be the one with the highest probability and its us think that the application of non-explainable RSV will be 5, the second element will be the one models may be unfair in some situations. Amazon with the second highest probability and its RSV will also discarded[4] a similar tool for recruitment, as be 4, and so on. The interest of the first metric is it was found to be biased against women. to know precisely the state of the model, while the second metric approximates what happens when a model is applied to a real use case, in which we do 3. The problem of gender bias use the first N results with the highest probability ordered, independently of the weight of each result.
In this paper we will focus on the bias in language Subsequently, the adjectives proposed by the temmodels, specifically on the bias between men and plate will be categorized and the diferences between women (gender bias). There are previous studies male and female responses will be studied with the that show that language models do indeed capture tool. Categories are based on two diferent classifisignificant diferences between men and women, it is cation schemes: the work of Tsvetkov et al. [9] will the work of Bolukbasi et al. [5] the one that makes appear under the name Yulia on the tool, and the the first breakthroughs in this area. This work work by Wiggins [10] will be referred as Foa & Foa shows that the Word Embeddings model trained on the tool. from Google News conceives men and women difer- The results of the analysis are exported to a JSON ently. After experimenting with professions, he high- file and those JSON files are integrated into a web lights that the model creates associations such as application. The application is a reactive Vue client Man will be a computer programmer while Woman web application, the tool loads the results of the will be a homemaker. Later the work of Caliskan experimentation and allows to explore graphically et al. [6] will show that this bias is not only present its results with help from ChartJs, for generating on gender, but also other areas such as race. These diagrams and charts. types of diferences will later be found in more complex models such as BERT[7] or RoBERTa[8]. 4.1. Category viewer
The proposal that led to the creation of the proposed tool is the realization of a study on the bias in the main language models in Spanish. The main task is to know if gender bias is present in these models and try to characterize it. For the study we propose a series of template sentences that have a masked word, each template will have a masculine and a feminine version, the model will have to propose a set of words that would replace the masked word, as well as the probability of each word. We will have one set of words for the male version and another for the male version, which will allow us to compare 4.2. Tables how each version behaves. To focus the study we will use templates that should be completed with an In the tables tab you can explore the results of the adjective. For example,In the pair of sentences El model from another perspective. In this case we alumno es el más <mask> and La alumna es la más select the categorization, the category to explore <mask> for the first one the model suggests rápido, and what results we want to show in the table. The inteligente, joven while for the second template the most interesting visualization is "M-F Heat" which sugestion are joven, guapa, votada. will show the aggregate value for male minus female
We will obtain from each model, for each tem- and color the table as a heatmap, with the extreme plate a result with two metrics. The first is the From the charts tab you can choose a classification scheme, a model and a variable. Once chosen, the percentage of the words predicted by the model that fall into each category are displayed, in blue are shown the results for men, and in pink those for women. An interesting exploration is to choose the categorization Yulia and explore how systematically the value of the category BODY is higher for women, while the value of the category BEHA (Behaviour) is higher for men. This tells us that the models preferably associate women with attributes of their body while men with their behavior. value of each column being red for female and blue for male.
This will allow us to see at a glance whether the leaning in that category is towards male or female, or neither in particular. In addition we will be able to see which models have a higher level of bias given the color intensity. By default we have the RSV and Probability columns that show the external and internal state of the model, this will allow us to appreciate significant diferences in some cases.
Here we can open the recommended configuration of the table above and see how the Yulia - Body - M-F Heat table is mostly red, while the Yulia Beha - M-F Heat table is mostly blue. simply to be aware that a model yields a very low proportion of adjectives, so we suspect that given the data used in its training it may not allow us to study the bias in the model. On the other hand we can also see which models are the best performing for this type of task, as well as look for significant diferences in the number of adjectives proposed by each one.
research point of view, extending this type of tests 4.3. Adjective Stats to other domains such as race would imply that instead of having two dimensions (male/female) In the Adjective Stats tab you can study the ad- we would have multiple and would have to adapt jectives obtained over the total number of words them. It would also be interesting to incorporate proposed by the model. The interest of this tab is capabilities to load results from a remote URL or
This work is partially funded by grant P20_00956 (PAIDI 2020) from the Andalusian Regional Government and by grant RTI2018-094653-B-C21 for project LIVING-LANG by the Spanish Government. just drag and drop a local file, allowing that, once the experimental code is released, anyone can use the visualization tool as easily as possible.
Finally, it would be interesting to convert the tool into a complete client side application that puts a GUI not only to the results but also allows to graphically launch experiments through a connection with the experimentation software and to feeds back its results by incorporating them into the visualizations, so to speak, a no-code solution for bias analysis. [6] A. Caliskan, J. Bryson, A. Narayanan, Semantics derived automatically from language corpora contain human-like biases, Science 356 (2017) 183–186. [7] E. M. Bender, T. Gebru, A. McMillan-Major,
S. Shmitchell, On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?, in: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, Association for Computing Machinery, New York, NY, USA, 2021, p. 610–623.
URL: https://doi.org/10.1145/3442188.344592 2. doi:10.1145/3442188.3445922. [8] S. Sharma, M. Dey, K. Sinha, Evaluating gender bias in natural language inference, CoRR abs/2105.05541 (2021). URL: https://arxiv.or g/abs/2105.05541. arXiv:2105.05541. [9] Y. Tsvetkov, N. Schneider, D. Hovy, A. Bhatia, M. Faruqui, C. Dyer, Augmenting English Adjective Senses with Supersenses, in: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), European Language Resources As[1] J. L. Julia Angwin, Machine bias - there’s sociation (ELRA), Reykjavik, Iceland, 2014, software used across the country to predict pp. 4359–4365. URL: http://www.lrec-conf.or future criminals. and it’s biased against blacks., g/proceedings/lrec2014/pdf/1096_Paper.pd 2016. URL: https://www.propublica.org/artic f. le/machine-bias-risk-assessments-in-crimina [10] J. S. Wiggins, A psychological taxonomy of l-sentencing. trait-descriptive terms: The interpersonal do[2] Z. O. U. Berkeley, Z. Obermeyer, U. Berke- main. 37 (1979) 395–412. URL: https://doi.or ley, S. M. U. o. Chicago, S. Mullainathan, g/10.1037/0022-3514.37.3.395. doi:10.1037/ U. o. Chicago, O. M. A. Metrics, Dissect- 0022-3514.37.3.395. ing racial bias in an algorithm that guides health decisions for 70 million people: Proceedings of the conference on fairness, accountability, and transparency, 2019. URL: https: //dl.acm.org/doi/10.1145/3287560.3287593. [3] D. Harwell, A face-scanning algorithm increasingly decides whether you deserve the job, 2019.
URL: https://www.washingtonpost.com/tec hnology/2019/10/22/ai-hiring-face-scannin g-algorithm-increasingly-decides-whether-y ou-deserve-job/. [4] J. Dastin, Amazon scraps secret ai recruiting tool that showed bias against women, 2018.
URL: https://www.reuters.com/article/us-a mazon-com-jobs-automation-insight-idUSK
CN1MK08G. [5] T. Bolukbasi, K. Chang, J. Y. Zou,
V. Saligrama, A. Kalai, Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings, CoRR abs/1607.06520 (2016). URL: http://arxiv.or g/abs/1607.06520. arXiv:1607.06520.