Among the various types of biases that can be recognised in the behaviour of algorithms learning from data, gender-related biases assume particular importance in certain contexts, such as the Italian one, traditionally linked to a patriarchal vision of society. This becomes even more true considering the context of university education, where there is a strong under-representation of female students in STEM Faculties, and, particularly, in Computer Science Courses. After a brief review of gender biases reported in Machine Learning-based systems, the experience of the teaching “Gender Knowledge and Ethics in Artificial Intelligence” active since A.Y. 2021-22 at the School of Engineering of the University of Padova is presented.
Artificial Intelligence⋆
gender bias, gendered innovation, fairness, artificial intelligence, machine learning,
With the spread of applications that use Machine Learning techniques, increasing attention is
having the possible consequences that such applications have on an ethical and social level.
AI has certainly been a sector of big challenges but also a sector of questions related to the
repercussions on people’s rights and freedoms. Since our goal is to develop a trustworthy AI,
it is appropriate to face an analysis from the point of view of gender, ethnicity, personal and
social development of AI tools, algorithms and technologies. In particular, the Ethics guidelines
for a Trustworthy AI of the European Commission1 list seven key requirements that AI systems
should meet in order to be trustworthy: Human agency and oversight, Technical robustness and
CEUR
Workshop
Proceedings
2http://genderedinnovations.stanford.edu/index.html
take into account that studies in the sub-field of Machine learning have shown that these kinds
of algorithms can upload the gender biases difused in the society [
As bias we intend a system of shared knowledge in society, for or against something: biases are based on stereotypes and prejudices. Stereotypes can help us to deal with a diferent/unknown world. They become central when our world is threatened, they can take the status of judgments that categorize facts or people: in other words, they become prejudices. At this point a vicious circle can arise between stereotypes, prejudices that can lead to discrimination. Machine learning algorithms, like people, are vulnerable to these distortions.
The problem exists since Machine Learning algorithms, by their intrinsic nature, are trained on the basis of training examples, they learn from data, and therefore can subsume and capture the stereotypes related to people sharing a given characteristic, for example the gender identity, which run through the data. If used to make automatic decisions, these potentially biased systems could lead to unfair, incorrect decisions that could discriminate some groups over others. There is the risk of being discriminatory for certain categories of users.
All this information was the stimulus for us to design a Course “Gender Knowledge and Ethics in Artificial Intelligence” that was held for the first time in the academic year 2021/22 in the School of Engineering at the University of Padua to introduce an ethical dimension applied to AI discipline.
In the first part of the paper we will present some relevant case studies were gender biases were found in machine learning-based applications, in order to give a general overview of how the problem can lead to unfair and discriminatory results. Then, in the second part we will describe and discuss our experience of teaching the course as an important way to disseminate a gender culture and to address some of the problems connected to the use of ML-algorithms.
While the concept of bias is very broad, gender-related biases are considered an essential aspect
of fairness [
First of all, numerous studies have shown that gender biases are deeply rooted in our society. Therefore, the risk that the datasets used for many applications with great social impact (autonomous driving vehicles, recommendation systems, personnel selection systems, etc.) contain biases linked directly or indirectly to gender is very high.
Secondly, gender biases afect more or less half of the population, so their presence has an impact on a large number of people.
Thirdly, given the wide spread of gender bias in our societies, it is relatively easy to find datasets on which to experiment with analysis and debiasing techniques.
Fourthly, in comparison with other types of bias (racial, social, etc.), it is easier to define the categories subject to possible discrimination. Gender studies, while recognising the multiplicity of gender identities, validate the existence of two well-defined polarities, male and female. The existence of two prevailing categories facilitates the definition of experimental protocols for the validation of analysis and debiasing techniques.
Fifthly, following the usual practice of bringing our research experiences back into teaching, promoting studies on gender biases in AI can facilitate the introduction of gender issues into our computer science courses, with a twofold advantage: a) increasing the degree of involvement of our female students, and b) making our male students aware of stereotypes and biases that risk discriminating against their female counterparts, making their university and professional careers more dificult.
Gender bias in AI was reported in various services with an high social impact [
The outcomes of three commercial gender classification systems (by Microsoft, IBM, and
Face++), tested on the Pilot Parliaments Benchmark – a dataset with a balanced intersectional
representation on the basis of gender and skin type – showed that all classifiers perform better
on male faces than female faces (with a diference in error rate between 8.1% and 20.6%), and on
lighter faces than darker faces (diference between 11.8% and 19.2%) [
A field test on the Facebook platform found that an advertisement promoting careers in STEM
was showed more times to males than females [
Another case was reported by Amazon, that started using a hiring tool to help rank candidates
using data from previous hires [
Gender bias is also an open issue for applications based on Natural Language Processing [
A recent extensive review [
In the perspective of developing an Artificial Intelligence you can trust in an inclusive and ethical way, we have taught since A.Y. 2021-22 at the School of Engineering of the University of Padua the course “Gender Knowledge and Ethics in Artificial Intelligence” with the aim to provide the related basic knowledge and principles in a multidisciplinary and interdisciplinary approach. The course (6 CFU, 48 hour of teaching) is opened both to bachelor and master students. The first edition of the course, not compulsory for any course of study, was attended by about 100 students, with a gender distribution in line with that of the engineering school. This means that the course was chosen, with due proportion, by both males and females and the explicit reference in the title to gender knowledge did not alienate male students, as we initially feared. About 80% of the attendees were bachelor’s students, testifying to the great interest in the subject even among the youngest students. Despite the fact that we considered these topics more suitable for master’s students, the interest shown by the bachelor students made us realise the importance of introducing concepts relating to gender knowledge and ethics even in the first years of engineering, especially considering that many computer engineering students enter the world of work immediately after their bachelor’s degree. Most of the participants came from computer engineering and biomedical engineering courses, with some students from electronic engineering.
As concern the contents of the course, the encounter between machines and people in contemporary society raises very central ethical questions. To this aim, it is necessary to introduce an ethical dimension applied to this discipline with a special attention to Machine Learning, facing an analysis from the point of view of gender, ethnicity, personal and social development of the ML algorithms which can lead in some cases to unfair and discriminatory decisions. External experts have been invited to take lectures and seminars, for all the disciplinary fields outside the computer science area. The syllabus and other information on the teaching can be found on the webpage https://www.dei.unipd.it/node/35894.
In a first part of the course, particular attention has been given to some concepts concerning
gender equality and gender knowledge in order to contrast stereotypes and prejudices that
condition social interactions and to favor a change towards a more equitable and sustainable
society. After an analysis of the diferences between sex and gender [
The open ethical questions are many raised by the encounter between machines, intelligent systems and people in the contemporary society. They concern, for example, the definition of a new concept of privacy in face of a meticulous collection of data to which people are subject, the fairness of decisions made by systems based on ML algorithms, the ethics to be adopted for autonomous systems to take decisions in emergency situations, and so on. In our course we have addressed all these problems dealing with ethical and legal issues applied to Artificial Intelligence. These issues and the computational techniques useful to mitigate the possible negative efects, have been discussed in relations to some relevant case studies.
The participation of the students has been wide and lively. In general we can say that this teaching experience was very positive for both teachers and students.
As noted by [
It is very important to act on several perspectives. First of all, by reducing the strong under-representation of women in Artificial Intelligence. Advancing women’s careers in AI, therefore, is not only a right in itself; it is essential to prevent biases and improve the eficacy of AI-based systems. Then, it is necessary to disseminate a gender culture at diferent levels, especially toward the younger generation, as the experience of our course has clearly shown. And an updating of training and education programmes in computer science, following a multidisciplinary approach, is perhaps one of the most promising ways to achieve these goals. Moreover, in light of the principles of gendered innovation, we believe that the study of computational techniques to analyse the presence of gender bias and mitigate its efect on outcomes represents not only an interesting problem, but first and foremost a great opportunity.
In the perspective of developing a trustworthy AI able to learn fair AI models even in spite of biased data, it is then important to address the problem of framing the landscape of gender equality and AI, trying to understand how AI can overcome gender bias and showing how an interdisciplinary analysis can help in a re-calibration of the biased tools.
A special thanks to Francesca A. Lisi of the University of Bari ”Aldo Moro” for her collaboration to the present research and the lessons held during the Course. This work is partially supported by the project “Creative Recommendations to avoid Unfair Bottlenecks” of the Dept of Information Engineering of the University of Padova.