Crowdsourced knowledge bases like Wikidata allow their contributors to describe information without a schema and therefore in a wide variety of diverse perspectives. This is primarily an advantage but could potentially result in lower data quality concerning uniformity and completeness of the data. Our research shows that despite fewer restrictions, there are structures and patterns becoming apparent concerning the predicates used to describe instances of specific classes like countries, which could be used to enhance the data quality of such knowledge bases in favour of the data consumer.
Using Linked Data to store semantically enriched information offers flexibility in that there is no need to follow pre-defined schemas. Every Linked Data publisher can decide on its own how to describe different subjects with different predicates and objects. This is especially true for Wikidata as an example of a crowdsourced knowledge base where thousands of individual contributors put information into this knowledge base.
Wikidata acts as a central storage for Linked Data, which then can be reused in other Wikimedia projects. Not only humans but also machines can read and edit the data, which is also accessible via an application programming interface (API) and a SPARQL endpoint.
Wikidata is a bottom-up approach in terms of not having to follow well-defined schemas for describing certain objects or concepts.
On the one hand, the possibility to describe subjects from very different perspectives can result in a broader, more diverse and more ubiquitous representation of the information. On the other hand, this flexibility poses a risk to the data quality (DQ) by potentially resulting in inconsistent and incomplete data.
The challenge is to maintain the flexibility and its associated benefits of Linked Data and at the same time mitigate the risk of inconsistent and incomplete information.
The goal of this work is to support Wikidata contributors in still having maximum flexibility with regard to entering new information without compromising DQ.
The problem is relevant both for Wikidata consumers, which is the general public, as well as for Wikidata contributors. Users need to be sure, that the data has a high DQ which means that the data is “fit for use” [1]. Contributors face the challenge to enter their information in a way that such fitness for use is ensured for a very broad variety of applications.
Keeping the flexibility of the non-schematic approach of Linked Data and at the same time, mitigating the risk of reduced DQ by inconsistent and incomplete data would be a beneficial contribution to the semantic web.
Also for publishers in a professional context who wish to publish Linked Data, our approach could be promising.
A last party interested in our approach could be semantic web ontologists who would benefit in that they will be able to examine emerging structures and ontologies from a crowdsourced Linked Data knowledge base representing multiple perspectives. 3
The various fundamental dimension of DQ have been well established before the emerging of linked Open Data [2], [3].
DQ issues have been identified as an impediment for the adoption of Open Data [4] in general.
Measuring DQ in the realm of the Web of Data is in comparison to the Web of Documents on the one hand simpler because many DQ metrics can be calculated automatically. On the other hand, measuring fitness for use for a specified use case is challenging [5].
Low DQ can concern the data itself or the associated meta data [6], [
Zaveri et al. [9] presented a systematic review of data quality assessment methodologies applied to linked Open Data (LOD) and classified different aspects of LOD DQ using accessibility dimensions, intrinsic dimensions, trust dimensions, dataset dynamicity dimensions, contextual dimensions and representational dimensions.
Improving DQ with help of a statistical approach which does not depend on domain experts or clean master data is shown in [10].
Our hereby-presented approach focuses on the completeness of data within the contextual DQ dimensions which refers to the degree to which all required information is present in a particular dataset [9]. It is comparable to the work shown in [11] but differs in that we use Wikidata as data source and that we are especially focused on assisting the individual contributors of Wikidata.
Are there any patterns becoming apparent in Wikidata concerning the predicates used to describe instances of a certain specific class and can these structures be used to help the Wikidata contributors to improve DQ? 5
The hypothesis is that in Wikidata, thanks to the flexibility of Linked Data, there will be a diverse and ubiquitous representation of information but still there will be some convergent patterns becoming apparent.
If predicates used to describe instances of the same class (e.g. “countries”) are compared, there will be predicates, which are more often used than others are. If a predicate used for almost every instance is missing for a certain instance, this is an indication for potentially missing data or an inconsistency in having used another predicate to describe the same information. 6 7
We have to show which predicates are used how many times in describing all the subjects of a certain class. We call this the predicate distribution of a certain class. Our goal, which is to assist the contributors of Wikidata, will work, when such a predicate distribution shows, that certain predicates are used for a large majority of subjects of the examined class.
For testing our hypothesis and thereby trying to answer our research question, we examine Linked Data from Wikidata. Wikidata is an open knowledge base with people and machines being able to read and edit data [12]. The approach of Wikidata is a collaborative effort by a community of contributors [13]. It profits from the flexibility of Linked Data in that every contributor can describe information in different ways and from different perspectives. There are no schemas given to enter certain information, which increases the danger of incomplete and inconsistent information. This puts Wikidata in an ideal position to test our approach.
Our approach consists of the following steps: Decision on the class (in the sense of an object-oriented approach, eg. “countries”) from which the instances belong. Collecting all the triples where the instances appear on the subject position. Listing all the predicates used in these triples and counting for which instances they occur how many times.
The technical realisation of these steps is done with help of the SPARQL endpoint of Wikidata (http://query.wikidata.org) which allows to query the knowledge base data in an easily manner. The aggregation of the data is done with some simple scripts within the statistical program ‘R’.
To explore the viability of our ideas, we did a manual examination of a few examples, which we present here. As a first step, we chose to examine instances of a class with only a few members. We decided to take a closer look at all the seven members of the Swiss Federal Council, which is the government of Switzerland.
The following tables 1-3 show our preliminary results using our approach for all the seven members of the Swiss Federal Council. Table 1 shows all the used predicates. It turns out that predicates exist which are used for every but one subject. This could be a hint that there is a missing value for the subject in question.
Indeed, it turns out, that the three predicates, which are missing only once, are really missing values: Alain Berset surely has a work location and a native language. Guy Parmelin on the other hand has no GND ID (yet). But this could (and probably should) also be given in a triple stating that there is no such value for mister Parmelin.
Table 2 shows summarized data concerning the predicates used to describe all the members of the Swiss Federal Council. r e t l a h k r u B r e i d i D 6 1 1 1 1 1 number of instances oSfwcliassssF“emdeemrableCrosuonfctihl”e 7
number of different predicates used number of predicates used for all members number of predicates used for all but one member number of predicates used for only one member 38 17 3 7 As a second class, we selected all the 202 countries, which are listed in Wikidata. Table 3 shows summarized data for the used predicates.
One interesting observation is, that if the number of potential missing values for a certain country is very high, it turns out, that this probably is not because of low DQ but is a special case of either a historical (non-existing anymore) or a disputed country. A high number of potential missing values for a certain instance could therefore mean, that this instance could belong to another class (“historical country” instead of “country”). number of instances of class “countries”
number of different predicates used number of predicates used for more than 95% but less than 100% of the countries number of predicates used for less than
1% of the countries number of countries with more than 3
potential missing values countries with more than 3 potential missing values 202 266 45 72 8 country
# values Eastern Kingdom of
Women Turkish Republic of
Northern Cyprus
Somaliland Sahrawi Arab Democratic Republic
Palestine
Transnistria Kingdom of the
Netherlands
Kosovo 45 22 22 22 22 15 11 9 Fig. 1 shows the distribution of predicates for six selected classes. For all the examined classes, there is only a small fraction of predicates, which do occur in conjunction with a high percentage of instances.
The yellow curve concerning the predicates used in countries, which is s-shaped with a long tail, is the one to be expected for a class with a high number of predicates and instances. It means that most predicates are only scarcely used. These predicates could be very interesting in depicting instances with special properties. Our preliminary results show that there are examples of classes, which show a potentially useful distribution of the predicates used to describe the various instances of the class. Of course, this finding has to be systematically expanded to other classes. This will show, for which kind of classes, the distribution of the predicates used is especially helpful in supporting the Wikidata contributors. This research has to take into account the fact, that information in Wikidata is not solely sourced by human contributors but also by automated importing from other sources. This will have an effect on the predicate distribution for sure.
Further research has to include practical aspects like how to pass information about other subjects of the same class to the Wikidata contributors during editing of information in the knowledge base in a helpful but unrestricting way.
Obviously, with our approach, low DQ concerning missing values can only be detected if the missing predicates are used for other instances of the same class. However, the community aspect of Wikidata increases the probability that the important predicates are present for other instances of the same class.
Our approach can contribute to high DQ in Wikidata and other crowdsourced Linked Data knowledge bases without limiting the freedom of different and diverse perspectives which is one of the strength of such crowdsourced approaches.
I would like to give thanks to my PhD advisor Prof. Dr. Thomas Myrach from the Institute of Information Systems of the University of Bern and to my mentor Dr. Yolanda Gil from the Information Sciences Institute of the University of Southern California. Furthermore, I would like to give credit to Mikhail Popov from the Wikimedia Foundation for his “WikidataQueryServiceR” library for accessing the Wikidata Query Service (WDQS) through ‘R’.