For several days in early July 2018, Google and Apple's search assistants wrongfully reported that the man behind the Marvel comic books, Stan Lee, had passed away . 4 It did not take long for news articles to start popping up noting the unjustified death declaration. Although Google and Apple never officially reported on this issue, its source is likely traced back to Wikidata. On June 27th, a Wikidata user ran their own script made to parse data from Wikipedia and insert them as claims into Wikidata. This script then mistakenly pronounced Stan Lee dead. Other users soon corrected the error, which resulted in an edit war that became so bad that the page had to be temporarily locked against vandalism. This is not the only occurrence of incorrect information in knowledge graphs causing issues in downstream search queries. In the second half of 2018, the former Guantanamo Bay detainee Omar Khadr was incorrectly reported by Google search for the query 'Canadian Soldiers'. Again the cause was the script written by the aforementioned user. Although the issue was quickly resolved after online outrage, it cropped up twice more over a period of several months. It eventually led Google to take manual action to fix the knowledge graph. 5In addition to the presence of incorrect information in knowledge graphs due to either an error in the KG construction or intentionally supplied by content curators, KGs can also be incomplete. As an example, in Freebase [2], over 70% of person entities have no known place of birth and over 75% have no known nationality [8]. In Wikidata [14], we observe a similar behavior, for instance, over 97% of humans have no known religion and over 83% of humans have no known spoken, written or signed languages. Subsets of both Wikidata and Freebase have been widely used for testing knowledge graph completion models. However, these subsets do not take into account the incompleteness of the KGs and are prepared in a way to test solely the accuracy of models. However, if the subsets are incomplete (or unbalanced), the models can be biased. For instance, the Wikidata12K [11,6] dataset contains 80% male and 20% female politicians. Clearly, this dataset is unbalanced and a model trained on it will likely overrepresent men in its predictions.
Indeed this is shown in our experiments using the TransE [3] model; when asked to predict people most likely to be politicians, the top 100 ranked answers contain just 12.4% while the remaining 83.6% are male. In order to mitigate such biased predictions, recently there has been a growing effort towards adapting/extending KG completion models [4,9,10]. These studies on bias analysis of KG embedding models focus primarily on altering the models such that sensitive features (such as gender, sexual orientation, etc) are dealt with differently from other features. The underlying implication is that the models cause bias, or that it is their task to solve it. However, we found out that the datasets on which the models are trained on are biased/unbalanced. Although algorithms for the automatic balancing of data do exist [5], these are not trivial to apply to graph datasets. Our experiments showed unsatisfactory results using these methods.
Furthermore, adapting models to remove bias means that the resulting embeddings will be bias-neutral with regards to the strength of the model used. That is, removing bias requires a bias detection model and the extent of the the bias removed depends on how much bias is detected. As a result, embeddings are not truly neutral: a more powerful model might still be able to detect biases. Therefore we argue that a domain expert must remain in the loop.
In this work, we address the problem by working directly on the data rather than altering KG embedding models. In other words, we investigate a new approach in order to balance (mitigate bias) a given dataset: we automatically extend a dataset by extracting additional facts to complete missing values of sensitive features. Moreover, so as to motivate the proposed approach, we carried out a comprehensive analysis of the distribution of sensitive features in Wikidata highlighting various skewed data distributions.
We group the related work into two classes of bias analysis: (i) knowledge graphs and (ii) embedding models.
Bias analysis of knowledge graphs. [7] proposes methods to trace the provenance of crowdsourced fact checking to enable bias transparency rather than aiming at eliminating bias from a KG. Furthermore, they investigate how paid crowdsourcing can be used to understand contributors' implicit bias. Specifically, they recruit click workers to verify controversial facts and study them as they do so. I.e., they track what search engines are used and which position the URL used to validate was ranked in the result page. An example verification task is the question of whether Catalonia is a part of Spain or an independent country. The paper proposes adding both facts to the knowledge graph, with a statement testifying how much support there is for each fact.
[15] introduces ProWD, a framework and tool for profiling the completeness of Wikidata. Completeness measure is based on Class-Facet-Attribute (CFA) profiles. For example one could compare how often the attribute "educated at" or"date of birth" compare between male, German computer scientists, and female, Indonesian computer scientists.
Bias analysis of embedding models. Bourli et al. [4] present an analysis method for investigating gender bias with regards to occupation in entity embeddings. Specifically, they subtract the male embedding from the female entity embedding to get the bias vector. Projecting an occupation on this vector then gives them the bias in this occupation. Furthermore, they introduce a de-biasing approach that generates new de-biased embedding vectors from the existing one by subtracting it from the bias vector.
[10] conduct experiments on Wikidata and Freebase, and show that harmful social biases related to professions are encoded in the embeddings with respect to gender, religion, ethnicity and nationality. They first explain how traditional word embeddings metrics do not apply to KG embeddings due to the transformations applied. They then provide a method for evaluating bias. Their method operates through increasing/decreasing an entities score of a sensitive attribute (e.g., make it more male and less female) and then recording how the likelihood of a certain target triple being true changes (e.g., whether they are a nurse of a lawyer). As a followup, the authors present a novel approach to KG embedding where embeddings are trained to be neutral with respect to sensitive features using an adversarial loss function [9] . To achieve this, they add a neural-network based classifier to the scoring function: scores are penalized when this classifier can predict the value of the sensitive attribute from the existing embedding. However, this means that the embeddings are only neutral with respect to the power of the model: a more powerful model might be able to infer the sensitive values.
These (and other initiatives) indicate that there is a growing attention to bias in knowledge graphs, and efforts to make bias visible. As knowledge graphs often are collaborative repositories, it is relevant to provide users with accessible means for identifying possible bias. The examples above are helpful but limited in two ways: they either are valid for a specific knowledge graph, and/or a limited number of attributes. A general framework might provide more possibilities to map bias in knowledge graphs and enable users to become aware of the dis-Fig. 1: How many humans have at least one occurrence of a given property? tribution of items and attributes in a given knowledge graph. With their own subject specific expertise, these users can then decide which bias is problematic, and how to address it.
Wikidata is the large, open knowledge graph which acts as central storage for the structured data of other Wikimedia projects such as Wikipedia. Data is stored as claims or triples, containing a subject item, a property and a value. Values are entities or literals such as a quantity, a string or even a coordinate. Items are identified through URIs starting with 'Q' (e.g. Q22686 for Donald Trump) and properties are identified through URIs starting with 'P' (e.g., P40 for Child ). Claims can be contextualized with additional data such as sources (for the data), ranks (in case of multiple values for a property) and qualifiers (e.g., to note that a fact was true at a specific point in time, or that a fact is disputed). A claim and its additional data are collectively referred to as a statement.
We investigated how several properties were distributed among the class of humans in Wikidata. An item x is a human when it is an instance of (P31) human (Q5), i.e., item x must have the claim (x, P31, Q5). Using the Wikidata dump from 2021/03/31, we extracted 9,028,271 such items. We will now give a brief overview of some of our preliminary findings.
To begin we, for each item in the subset we counted whether or not a property occurs among its claims. This gives us an overview of how often a property occurs at least once. The result is displayed in Figure 1. Ignoring the predicate instance of, which per our definition is present on all humans, the most occurring predicates are sex or gender (P21), occupation (P106), and given name (P735). These occur on 7,079,543 (78%), 6,359,256 (70%) and 5,635,238 (62%) humans respectively.
Additionally, we counted the number of languages each item had a label in. This gives us an overview of how complete Wikidata is over several languages. The result of this is displayed in Figure 2. Expectedly, the most common language is English, with 8.517.283 (94%) humans having an English label. More unexpectedly however is the fact that, in spite of being a small country with only 17 million inhabitants, the second most common label language is Dutch with 7.785.518 (86%) humans having a Dutch label.
Next, we can look at the distribution of object entities for a given predicate. I.e., given a predicate such as place of death (P20) we can count how many people have object values such has Moscow or Paris. From this data, we have created a bar graph for a selection of predicates in Figure 3.
Looking at this data, it is immediately clear that it is not representative of the common population. For instance, the most common occupation by far is researcher (20%). Yet in reality, even in the USA only around 2% of the population has a PhD. 6 We of course understand that an encyclopedia covers persons of interest and not the general population. Hence it is logical that there is a bias. However, the problematic bias is not the overrepresentation of scholars but the overrepresentation of of white male scholars at western universities. If we want to inquire to what extend the population of researchers in Wikipedia is skewed we need to inquire about the presence of other occupations for persons of interest for an encyclopedia, such as athletes, activists, politicians, engineers and inventors.
We hypothesize that there are two main sources of bias present in this data. The first is availability bias, i.e., much of the data present in Wikidata is there because it could be easily imported. For instance through the use of bots. The second is interest bias, where the interests of the people who work on Wikidata end up deciding what content will dominate the dataset. Examples of this bias are the most common occupation being researcher (imported through article papers) and the second most common place of death being a concentration camp.
Temporal information in Wikidata present itself in two ways. Firstly, predicates can directly have timestamps as their object value. For instance, the date of birth of a person. All predicates that can have a timestamp as object value must be instances of (P31) Wikidata property with datatype 'time' (Q18636219). There are 34 such predicates. Secondly, temporal information can be included in any other predicate through the use of qualifiers. I.e., predicates start time (P569) and end time (P570) can be applied to a triple through reification to add temporal information to that triple.
Since we are interested in how humans are represented in Wikidata, we restrict spatiotemporal analysis to the human class. Specifically, we ground data in space by looking at a persons place of birth (P19 ) and in time by looking at the place of birth (P569 ). Through this we can analyse the completeness of Wikidata over time. Some results are displayed in Figure 4. We observe that the further we go back in time, the less distinct countries are observed in Wikidata. I.e., facts seem to be more based on a few countries. Addtionally, we investigate the occurrences of the most common ethnic groups listed in Wikidata. Interestingly, the use of ethnic group seems to have fallen out of favour for people born more recently. In the 18th century the most common ethnic groups was Greeks with over 400 occurrences, whereas in the 21st century the most common ethnic group is African American with just over 100 occurrences.
In this section we perform bias analysis of the knowledge graph embedding models. Specifically, we analyze the effect of balancing the data on link prediction performance. For this task we utilize two popular models, TransE [3] and Dist-Mult [16]. We perform our experiments on two state-of-the-art knowledge graphs. The first is Wikidata12k, a subset of Wikidata extracted by [11,6]. The second is DBP15k, a subset of DBpedia [1] originally created by [13] to test Entity-Alignment models. As we are interested in link prediction rather than entity alignment, we select a single instance of the dataset (the English version) and perform our experiments on it. All of our code is available on Github7
For a triple (s, p, o), let (e s , e p , e o ) denote its embedding vectors. Taking a KG and a random initialization of the vectors as an input, a vector representation of the KG is gradually learned using a scoring function φ(s, p, o). The scoring function should reflect how well the embedding captures the semantics of the KG. The learned embeddings can be used in tasks such as classification, clustering, and link prediction. In this work, we are focussed on the last. Link prediction is the task of predicting the most likely element given a tuple where one element is missing, e.g., given a triple (s,p,? ), to predict the most likely object entity.
The most popular embedding model is TransE (Translating embeddings). Its scoring function is based on the intuition that the the subject and object vector should be close together after adding the predicate vector. Its scoring function is written as φ(s, p, o) = ||e s + e p − e o || 1,2 . While being very powerful, it has limited expressiveness due to its is simplicty. Therefore, we also perform experiments with DistMult, a multiplicative model. Its scoring function is written as φ(s, p, o) = ||e s * e p * e o || 1,2, . In our experiments we do not use pre-trained models and instead train the embeddings from scratch.
One way of balancing datasets is to use Synthetic Minority Over-sampling TEchnique (SMOTE) [5]. SMOTE is an over-sampling technique that allows one to construct new examples of a given class based on existing examples in order to address imbalances in the dataset. An example would be oversampling female football players. However, SMOTE is not intended for graph datasets and as such is not trivial to apply to knowledge graphs while maintaining the underlying structure.
One way to apply SMOTE to graph data is through embedding the graph first. We use this approach to evaluate how well SMOTE is suited for our scenario. Our method is as follows. After obtaining the embeddings, we create a categorical variable with a category for each possible combination of sensitive features. In the case of 5 occupations and 2 genders, this implies 10 categories. The embeddings vectors are then combined with this categorical value associated with the sample it represents. Finally, we instruct SMOTE to generate the maximum number of samples for each possible entry using the python imbalancedlearn library [12], i.e., given that there are 1850 male association football players, we create both male and female physicists until there are 1850 of each.
However, preliminary experiments found that this method did not suffice for generating a balanced set of embeddings. Applying our evaluation method to datasets produced by above procedure did not create balanced predictions. We hypothesize that it is because SMOTE generates new examples based on existing biases. By interpolating new 'female' examples from existing female embeddings, we are only creating new examples in the same cluster. That means that locations in the embedding space which are already female become much more so. Therefore, we instead extend the datasets by sampling additional triples from the original knowledge graphs. To ensure that the new triples have healthy connectivity with regards to the rest of the graph this is done in a three step process. Firstly, all women with the required occupations are selected from the complete knowledge graph. Secondly, from this selection the women which have the largest number of predicates which are also in the original dataset are picked. Finally, we select the women whose object values are already in the graph. The last step ensures that we do not add object entities which occur only a few times, and only with women.
Wikidata12k does not contain any information about gender or occupation. However, we can look up this data by querying the original Wikidata knowledge graph. As Wikidata12k is originally a temporal knowledge graph, we strip out the temporal information and remove and duplicate triples that may be created by this process.
The five most common occupations are association football player Q937857 (1867), politician Q82955 (918), actor Q33999 (211), writer Q36180 (184) and physicist Q169470 (143). These occupations are not uniformly distributed with regards to gender: there are only a handful of women football players, and there is not a single woman physicist in the entire dataset.
In total, we add around 10,000 triples with female entities as subject to the Wikidata12k knowledge graph, resulting a new graph over 50,0000 triples. This increases the average number of mentions as subject (i.e., the average number of outlinks) per female entity from 3.49 in the original graph to 4.74 in the balanced graph. However, the number of outlinks still falls short of that of men, which is 5.41.
DBP15k is a subset of DBpedia [1] created by [13] with them. To prevent the graph from being too sparse for an embedding model to learn we delete all predicates which occur less than 50 times. DBpedia does not store any information about peoples sex or gender in a structured way. I.e., although a person can be of rdf:type of Man or Woman, manual inspection of the data did not reveal that this information was consistently present. However, most entities do contain their Wikidata identifiers. Since Wikidata does list peoples gender, we determine a persons gender by querying Wikidata for the given identifiers.
The five most common occupations are OfficeHolder (2508), Athlete (1436), Royalty (1002), SportsManager (288), and Scientist (282). Like Wikidata12k, the male/female ratio in these occupations is unbalanced, skewing heavily towards men. In addition to balancing the data by adding additional samples, we remove some male entities and their triples to create the balanced dataset.
To evaluate whether an embedding model contains bias with regards to gender and occupation we perform the following procedure. Firstly, we count the fraction of men and women that have a certain occupation (P106 ) x. Then, we ask the model to predict the n most likely entities for the query (?, P106, x). If the fraction of men or women returned is consistently larger than that the fraction present in the data, the model is biased. Specifically, when more men are predicted the model is biased against women and vice versa. If this bias is only present in the unbalanced dataset and not the balanced datasets, then the model reflects the data it has been trained on. However, if the bias is present in both scenarios, the models are either inherently biased or manage to pick up some form of bias in the data which is not reflected in our analysis.
Our results are displayed in Tables 2 and 3 for DBpedia and Wikidata12k respectively using TransE [3], and in Tables 4 and 5 using DistMult [16]. We observe that in both original datasets, the percentage of women predicted is very low and close to the percentage of women in the dataset. The largest difference is observed on the occupation Royalty in the DBpedia15k dataset. Here the difference is just over 10 percentage points.
When we extend our view to the balanced datasets, we find that the percentage of women predicted has moved upwards with the percentage of women in the dataset. Balancing the datasets thus helps with improving the representation of minority classes in the model output. However, we do observe that the absolute differences between the number of men in the dataset, and the number of men predicted (and for women) has increased, suggesting that the model has become less accurate.
Even so, we believe a more likely explanation to be that the larger number of entities predicted induces more variance in the predictions. This explanation is strengthened by the fact that the difference is smaller when using DistMult, Table 5: Comparison between the male/female distribution and the resulting DistMult model predictions in the original Wikidata12k dataset (top) and our balanced dataset (bottom). Difference column contains the difference (in percentage points) between the % of women in the predicted and the % of women predicted.
which is a more expressive model and can thus model the information more accurately.
Another point of note is the observation that on both datasets and almost all occupations, the sign of the difference between the percentage of women predicted and the percentage of women in the dataset is mostly positive. This means that women are actually overrepresented in the models predictions, indicating that the model is actually less biased than the data.
In this paper we proposed a new approach to mitigate bias in knowledge graphs embedding models by leveraging the distribution of the datasets in which the models are trained on. Specifically, rather than adapting models to mitigate bias, we instead analyze and augment the data that is fed into the model. We carried out several experiments using state of the art embedding models (namely, TransE and DistMult) and two knowledge graphs (namely DBpedia and Wikidata) and showed that balancing the data with regards to specific sensitive features (e.g. gender and occupation) improves the overall prediction capabilities of the models. Additionally, to motivate our work, we have done a completeness analysis of Wikidata using a number of sensitive features.
As a future work, we will extend the proposed approach to build a system that takes as an input a dataset and a selection of sensitive features and automatically balances the data with respect to the given features.