=Paper=
{{Paper
|id=Vol-2982/paper-3
|storemode=property
|title=Reference Statistics in Wikidata Topical Subsets
|pdfUrl=https://ceur-ws.org/Vol-2982/paper-3.pdf
|volume=Vol-2982
|authors=Seyed Amir Hosseini Beghaeiraveri,Alasdair Gray,Fiona McNeill
|dblpUrl=https://dblp.org/rec/conf/semweb/BeghaeiraveriGM21
}}
==Reference Statistics in Wikidata Topical Subsets==
https://ceur-ws.org/Vol-2982/paper-3.pdf
Reference Statistics in Wikidata Topical Subsets
Seyed Amir Hosseini Beghaeiraveri1[0000−0002−9123−5686] ,
Alasdair J.G. Gray1[0000−0002−5711−4872] , and
Fiona J. McNeill2[0000−0001−7873−5187]
1
School of Mathematical and Computer Sciences,
Heriot-Watt University, Edinburgh, UK
2
School of Informatics, The University of Edinburgh, Edinburgh, UK
Abstract. Wikidata is the only general-purpose open knowledge graph
with the capability of specifying references for every single statement.
Currently, about 68% of Wikidata statements have at least one refer-
ence but the quality of these references is rarely covered in data quality
studies. There is also a lack of a comprehensive framework for evaluating
references. In this paper, we investigate the statistics of Wikidata refer-
ences in 6 topical subsets of Wikidata. We compare these statistics over
two Wikidata dumps; one from 2016 and one from 2021.
Keywords: Reference quality · Wikidata · Data quality · Topical subset
· WikiProject · Gene Wiki
1 Introduction
Wikidata [31] is a knowledge graph that started in 2012 and is now the most
active Wikimedia project. It contains knowledge on a broad range of topics
with statements (data asserting a fact) being created and edited through crowd-
sourcing. A distinguishing characteristic of Wikidata is its ability to capture
additional information about statements, such as providing references for each
piece of data. According to the Wikidata project,“Wikidata is not a database
that stores facts about the world, but a secondary knowledge base that collects
and links to references to such knowledge” [7].
Our focus in this paper is on the references of statements. Having good
evidence of where the data came from improves the trust and reusability of the
data as errors can be traced, and data can be categorized according to where
they came from [25,26]. According to Wikidata policy [7], all statements need to
be referenced except statements about common human knowledge, statements
that refer to an external source, or statements of items that are a source for other
statements. Wikidata recommends that references be relevant and authoritative,
but these terms are not explicitly defined. Providing appropriate references is the
responsibility of the person who adds the statement. Assessment of references
Copyright © 2021 for this paper by its authors. Use permitted under Creative
Commons License Attribution 4.0 International (CC BY 4.0).
is the responsibility of the Wikidata user community. Currently, about 68% of
Wikidata statements already have at least one reference [4]. While there has
been some initial work to look at reference quality [27], there no systematic way
to assess the quality of a reference.
Wikidata aims to cover a wide range of topics via user collaborations. Users
interested in a particular topic form communities called WikiProjects [16]. Be-
sides human users, WikiProjects may use bots to collect and edit a mass of data,
including references. Wikidata enforces strict rules for accepting edits by bots [5].
WikiProjects reflect the activity of contributors in covered topics. Investigating
WikiProjects provides a topical comparison basis to analyze the functionality of
humans and bots in different quality metrics across different Wikidata topics.
In this paper, we perform a statistical analysis on the reference statements
of different WikiProjects to provide insight into their quality. Our contributions
in this paper are:
1. Creating a topical comparison platform for investigating the quality of ref-
erences. This is done by extracting 6 topical subsets from Wikidata corre-
sponding to 6 different WikiProjects. We also publish the subsets for further
community experiments.
2. Providing a statistical report of references in the 6 subsets.
In Section 2 we discuss related work on reference quality. Section 3 explains
reference nodes in the Wikidata RDF model. Section 4 details the process of
subsetting Wikidata to build topical subsets for the topical comparison platform.
In Section 5, we present the statistics of references in the extracted subsets.
Section 6 outlines our position on the importance of studying the quality of
references and the initial ideas of a reference quality checking framework. The
conclusion of the paper is presented in Section 7.
2 Related Work
Provenance of data in knowledge graphs and its quality is one of the criteria of
trustworthiness which is one of the main dimensions of data quality [22]. The
analysis by Farber et al. [22] gives Wikidata the full score for the trustworthiness
on statement level as Wikidata can provide references for each single statement.
They do not give an analysis of how Wikidata uses this feature. Farber et al.
reported that the coverage of references over the statements in October 2015 is
1.3% while Wikidata Stats says more than 50% of statements had a reference
at that date [4]. The reason for this difference is that Farber et al. counted
the number of distinct reference nodes, while a reference node might be shared
between more than one statement. We call this shared references.
Accuracy and trustworthiness have not been covered in Wikidata as much
as other data quality dimensions [28]. Piscopo et al. [27] proposed an approach
to evaluate the authoritativeness and the relevance of Wikidata external sources
based on the quality definitions set by the Wikidata community. The approach
consists of two main steps. First, a set of sample references is evaluated through
microtask crowdsourcing. Then, this data is fed to a machine-learning algorithm
to apply a large-scale evaluation over the whole Wikidata dump (from October
2016). They evaluated only English language sources, mainly because of the
limits of performing crowdsourcing for non-English sources. They show that
Wikidata external sources are of good quality as 70% are relevant and 80% are
authoritative.
Comparing between Wikidata and Wikipedia external references, Piscopo et
al. [29] showed that Wikidata has a more diverse pool of sources, in terms of
country of provenance, and employs a larger percentage of external databases
and reference sources, such as library catalogues, compared to the online ency-
clopedia. More recently, Shenoy et al. [30] developed a framework to detect and
analyze low-quality statements in Wikidata. Their work does not consider the
quality of references as a metric. Curotto and Hugan [21] proposed a method
of searching and indexing English Wikipedia references to create references for
Wikidata facts. This proposal like any other reference-suggesting tool needs to
be evaluated in terms of the quality of suggested references which indicates the
need for a comprehensive reference quality checking framework.
The few prior work on reference quality [27,29] were applied on the 2016
and 2017 dumps of Wikidata. Given the exponential growth of Wikidata in
recent years, there is a need for a comprehensive investigation on the diversity of
current Wikidata references, the extent to which bots and humans participate in
references, and comparisons between bots and humans regarding to the quality
of references. Also, no prior work studied the reference quality across different
topics in Wikidata. In this paper, by investigating reference statistics we start
a path to a comprehensive review of Wikidata references. We aim to develop a
broader framework by precisely defining other data quality criteria for references.
3 Wikidata Data Model
The Wikidata knowledge graph consists of items (entities from the real world)
and properties (relationships between items or between items and values). An
(item, property, value) triple in Wikidata is called a claim. A distinguishing
characteristic of Wikidata is its ability to capture the provenance of each claim.
This is achieved by enriching the claim with qualifiers (contextual information)
and/or references (the source of the claim) to create a statement.
The Wikidata RDF model uses reification [1] for adding references to state-
ments, as shown in Figure 1. Every statement in Wikidata has a statement
node (identified by a unique ID in the wds: namespace) from which all ref-
erences, qualifiers, ranks, and values are stored. References are linked through
prov:wasDerivedFrom edges to reference nodes (identified by a unique ID in the
wdref: namespace). Reference nodes provide the provenance of the fact by one or
more properties like retrieved date (P813), stated in (P248), and reference URL
(P248). If a statement has multiple references, there will be a separate reference
node for each reference. If two statement nodes share the same provenance, then
they link to the same reference node.
Fig. 1. The reification used in Wikidata for adding references into statements derived
from [1]. abc is an arbitrary QID. opq, rst, and xyz are arbitrary PIDs.
4 Topical Subsetting of Wikidata
Investigating reference quality requires access to the reference nodes. This can be
achieved by querying for the reference nodes by type, e.g. using the basic graph
pattern ?item rdf:type wikibase:Reference. The Wikidata Query Service
has blocked these queries for performance reasons [1]. Due to the enormous
size of Wikidata dumps, locally indexing a complete Wikidata dump is time
consuming, costly, and requires hardware beyond a standard desktop computer.
Therefore, we use topical subsets of Wikidata [19] which gives us substantially
smaller datasets over which we can research references locally. They also provide
a basis for comparing the richness and quality of referencing across Wikidata
topics; thus reflecting the work of different communities. These smaller, focused,
datasets are more likely to be reused [24].
4.1 Topic Selection Desiderata
A WikiProject [16] is a team of Wikidata contributors who aim to improve Wiki-
data by working on a specific topic or doing a specific task. A simple query3 shows
that there are 243 WikiProjects in Wikidata, many of which have been created
to enrich Wikidata (both A-Boxes and T-Boxes) on a particular topic, such as
music, scientific disciplines, or politics. WikiProject contributors typically list
classes and properties for their topic so data instances that match these defini-
tions can be added to Wikidata. We can use such definitions to determine the
boundaries and the scope of the topic and extract their subset. These subsets
are representative of their relevant WikiProject in different experiments (e.g. in
reference statistics as we present in this paper).
WikiProjects vary in purpose, scope, activity, and progress. Extracting sub-
sets for each of the projects is not feasible due to their number, nor are all of
them suitable. A candidate project must meet the following desiderata:
3
https://w.wiki/48Rn - accessed 27 September 2021
– It should be topical in nature. Task-based projects such as disambiguation
pages [9] are not suitable for topical subsetting.
– Contributors should provide information about items, classes, and proper-
ties that are added to Wikidata through the project. This information is
presented as lists, tables, entity schemas, or UML class diagrams. Using this
information, we can determine the boundaries of the covered topic.
– The topic of the project should not be too limited or too broad. For example,
in the Scholia project [13], just scholarly articles make up 30% of Wikidata
items [6] which is very broad. We would like our candidates to have the same
level of independence [12] from the whole Wikidata.
– We would like our experiment to be a good approximation of the whole
Wikidata so we need candidates from a wide range of subject areas.
4.2 Selected Projects
Based on our topic selection desiderata we identified the following projects for
topical subsetting to enable us to investigate reference quality. We have selected
some closely related projects to allow direct comparison, and then some less
related ones for contrast. We have selected a combination of scientific and non-
scientific topics. The projects are of similar size and scope.
Gene Wiki [20]: Gene Wiki aims to make and maintain Wikidata as a central
hub of linked knowledge on Genes, Proteins, Diseases, Drugs, and related
items. It is one of the most active WikiProjects. It has five active bots and
specified 24 classes of data to be added to Wikidata pictured in a UML
class diagram. We include all instances of these 24 main classes and their
subclasses into the subset.
Taxonomy [15]: The goal of this project is to populate Wikidata with tax-
onomic names and their classifications. This project consists of the class
of Taxon (Q16521) and its hierarchy plus 47 other related classes that are
specified in the wiki page of the project. The Taxon (Q16521) class and its
subclasses are also considered in the Gene Wiki project. Considering it as a
separate use case allows investigating the references in this focused part of
Gene Wiki as compared to the rest.
Astronomy [8]: The main goal of this project is to define classes and properties
for items related to Astronomy. Accurate referencing is one of the main goals
of the project. Besides that, an active community, well-structured ontology
definition, and usefulness of the project motivate us to consider this project.
This subset consists of all instances of astronomical object (Q6999) class and
its subclasses.
Law [10]: This project aims to cover anything that touches the law, e.g. eco-
nomic laws, evidences, and legal proceedings. The provided data would be
particularly useful for judicial systems. The project intends to be broad in
scope, but it has a detailed ontology definition. Law (Q7748), public order
(Q294199), and evidence (Q176763) are some of the included classes.
Music [11]: This project aims to map and import all music-related data from
diverse sources to feed Wikipedia music infoboxes. Referencing is also im-
portant in this project. Musician (Q639669), musical ensemble (Q2088357),
and musical work(Q2188189) are some of the main classes.
Ships [14]: This project aims to establish the most ideal structure for ship data,
and create and update claims for all ship items on Wikidata. The project
has a well-structured class hierarchy. Based on the mentioned items and
classes on the project’s web page, all instances of all subclasses of watercraft
(Q1229765) and ship class (Q559026) are in the subset.
Full programmatic definitions of the subsets can be found in the supplementary
material for this paper [17].
4.3 Subset Extraction Setup
We use the Wikidata WDumper [23] tool to extract subsets corresponding to
each project. For each project, the main classes are identified according to the
project’s wiki page. Identified classes are then used to write WDumper specifica-
tion files. The specification files are then enriched with subclasses via a Python
script [17]. Finally, the related A-Boxes are extracted via WDumper. Subsets
include all statements for A-Boxes along with references, qualifiers, and rank
data. T-Boxes have been ignored as referencing does not apply to them. The
WDumper specification files for each project are in [17].
For each project, two separate subsets are extracted: one from the 2016 dump
(3 October 2016) [3] and one from the 2021 dump (30 June 2021) [2]. The 2021
dump was downloaded from the Wikimedia dump store 4 . We chose the 2016
dump as it is used in prior work on Wikidata reference quality. Thus, it will
allow us to compare statistics between the two different snapshots and drawer
some conclusions with that earlier work. The extracted subsets in N-Triples are
in [18]. For this paper, the subsets were indexed and queried using Blazegraph5
2.1.6 triplestore.
5 Reference Statistics
We consider four experiments in which we perform a set of SPARQL queries over
each extracted subset to obtain a statistical overview of references in Wikidata.
The SPARQL queries for each experiment along with results can be found at
the GitHub repository of the paper [17].
5.1 Basic Statistics
Table 1 shows the initial statistics for references in each project. The first
two columns are the number of items and the number of statements. The
4
https://dumps.wikimedia.org/other/wikibase/wikidatawiki/ - accessed 2 July
2021
5
https://github.com/blazegraph/database/releases/tag/BLAZEGRAPH_2_1_6_RC
Table 1. The basic statistic of references in each subset.
Statement Reference Referenced Shared
Project Dump Items
Nodes Nodes Statements Reference Nodes
2016 2,647,174 17,656,669 169,493 8,789,246(50%) 42,902(25%)
Gene Wiki
2021 8,801,623 92,729,475 9,559,517 65,780,005(71%) 4,700,610(49%)
2016 2,214,088 16,056,914 95,714 8,146,218(51%) 5,971(06%)
Taxonomy
2021 3,225,102 32,536,083 498,535 19,423,938(60%) 204,602(41%)
2016 141,843 888,717 13,260 751,158(85%) 12,198(92%)
Astronomy
2021 8,416,958 144,637,511 157,558 128,394,763(89%) 112,365(71%)
2016 67,763 174,252 380 48,225(27%) 152(40%)
Law
2021 433,440 4,236,657 407,409 2,266,462(53%) 317,975(78%)
2016 598,074 3,742,474 80,857 2,298,330(61%) 35,574(44%)
Music
2021 948,266 11,702,021 1,329,746 6,342,019(54%) 374,440(28%)
2016 42,873 183,240 857 114,528(62%) 227(26%)
Ships
2021 126,896 1,101,802 59,282 315,381(29%) 16,396 (28%)
Fig. 2. An example of a reference node that is shared between three statement nodes
third column is the number of reference nodes (i.e. nodes with the type wik-
ibase:Reference). The fourth column shows the number and percentage of
statements with at least one reference. The difference between the number of
referenced statements and the number of reference nodes is significant. This is
because a number of reference nodes are common between statements. In other
words, a number of statements have exactly the same references. We call these
shared references, which is shown in Figure 2. The fifth column shows the num-
ber and percentage of those reference nodes that are shared between more than
two statements.
As we can see from the table, in all projects the number of items, statement
nodes, and reference nodes has substantially increased from 2016 to 2021. After
the extraction, we recognized that Taxonomy project makes up about 30% of
Gene Wiki. The percentage of referenced statements has increased in all cases
except Music and Ships. In the case of Ships, the percentage of referenced state-
ments has dramatically decreased. Considering the increase in statements in
both, the decrease in referenced statement can show that human users are more
active than bots in Ships and Music (if we intuitively accept that bots provide
references more and better than humans). The percentage of shared references
for Gene Wiki, Taxonomy, Law, and Ships has increased from 2016 to 2021, while
for Astronomy and Music this amount has decreased. Among the 2021 datasets,
the highest number of referenced statements belongs to the Astronomy project
and the lowest to the Ships project. The increase in shared references in the Gene
Wiki and Taxonomy subsets is likely due to the use of bots to populate Wiki-
data. Considering the 2021 datasets, the highest number of shared references is
allocated to the Law project and the lowest to the Taxonomy project.
5.2 Usage of Reference-specific Properties
Wikidata offers a set of properties such as stated in (P248) and reference URL
(P854) to be used in references. In addition, different projects may offer proper-
ties for their references, e.g. the Gene Wiki and Taxonomy projects use properties
such as IUCN taxon ID (P627) even though they are identifier properties. Fig-
ure 3 shows the frequency of reference-specific properties used in references in
each use case for 2021 subsets. Note that, Figure 3 illustrates only the most used
properties; the variety of properties is more but the abundance of the remaining
properties is less than 3% overall. For details, see the CSV file at the GitHub
repository of the paper6 .
In Gene Wiki, Taxonomy, and Law, the most frequently used properties
are stated in (P248), retrieved (P813), and reference URL (P854), while Music
makes most use of the first two. This indicates that most of the references in
these subsets rely on external sources that were likely populated by bots. For
Gene Wiki and Taxonomy, the next most frequently used properties correspond
to identifier properties for well known data sources in the life sciences. It is likely
that these are used to indicate these data sources as the provider of the claim.
The use of external sources accounts for about 60% (Music) to 100% (Taxonomy)
of the references. In Astronomy (58%) and Ships (56%), the most frequently used
properties are imported from Wikimedia project (P143) and Wikimedia import
URL (P4656). These properties indicate that the source of the statement is one
of the internal Wikimedia projects, e.g. Wikipedia. Mentioning the Wikipedia
article as a source for corresponding Wikidata item is not recommended [7], so
the extent of these should be carefully considered in future studies.
5.3 Distribution of Triples per Nodes
Via the reference-specific properties, each reference node uses one or more triples
to point to the provenance of the claim. Figure 4 shows that the most frequent
properties are probably used together. Having more triples in a reference node
provides more details about the source which is likely to increase the accuracy.
Figure 4 shows the distribution of the number of triples over the total refer-
ence nodes in each project in 2016 and 2021 dumps. In all projects except Law
6
https://github.com/seyedahbr/Wikidata_Reference_Statistics/blob/main/
QueryResults/UsageofReference-specificProperties/PropertyUsage.xlsx
Fig. 3. The frequency of reference-specific properties used in references in each project
(2021 subsets).
and Ships, the average number of triples in references has decreased from 2016
to 2021. The best average belongs to Gene Wiki. The similarity of Gene Wiki
statistics in both 2016 and 2021 dumps is interesting and is probably related to
the steady activities of the project bots. The uniform distribution of triples in
taxonomy might be due to the steady activity of the bots in a specific field (as
opposed to Gene Wiki, which consists of several fields such as biology, chem-
istry, and pharmacology). In 2021, Astronomy has the lowest average number
of triples in reference nodes, despite having the highest percentage of referenced
statements. In the Music project, there are reference nodes with 22 and 35 triples;
these outliers are omitted from the figure for presentation purposes. The average
number of triples ranges between 1.2 (Ships 2016) and 3.5 (Gene Wiki 2021).
5.4 Distribution of Reference Sharing
Shared reference nodes can affect the quality of references. Having shared refer-
ences is not necessarily negative as they can reduce redundancy. For example, in
Gene Wiki multiple statements about a protein might be taken simultaneously
from the UniProt dataset via a bot, so the reference node of all these statements
will be the same. Figure 5 shows the distribution of reference sharing of each
project in the 2016 and 2021 dumps. In all projects except Astronomy, the refer-
ence sharing rate has decreased from 2016 to 2021. Although Figure 5 shows the
8 year
2016
Number of Triples in Each Reference Node
7 2021
6
5
4
3
2
1
Gene Wiki Taxonomy Astronomy Music Law Ships
Subset
Fig. 4. The distribution of triples per reference nodes. The red lines are the medians.
Triangles are mean points. Outliers are omitted for presentation purposes.
Table 2. Rounded mean and maximum of reference sharing in 2021 dump for each
project.
Gene Wiki Taxonomy Astronomy Law Music Ships
Mean 13 93 1142 7 14 17
Max 1,281,307 408,522 42,876,186 155,508 1,385,109 96,659
normal distribution rate in shared reference nodes, there are exception reference
nodes in each project shared between a very large number of statements. Table 2
shows the mean and maximum of reference sharing in each project in the 2021
dump. In Astronomy, there are about 43 million statements connected to just
one reference node, however, there is only one reference node with such situation.
In all projects, there are reference nodes that are providing the source of more
than 50,000 statements. This amount of sharing might challenge the relevancy
condition [7] and should be carefully examined.
6 From Statistics to Quality
To the best of our knowledge, the Piscopo et al. study [29] is the only research on
the quality of references in Wikidata, but this work has considerable limitations.
They started with the Wikidata edit history in October 2016. They extracted
all statements containing external references. Then they excluded statements
that do not require reference according to Wikidata policy which leads them
to 1,629,102 references. At this step, 89% of references pointed to two specific
sources7 that are excluded from the evaluation as around 98% of these were
added by one bot. In the remaining 11%, they evaluated only English sources
that are about 46%. In the end, only 83,215 references were reviewed. The num-
ber of statements and references is completely changed now. We can see from
7
uniprot.org and ebi.ac.uk
Number of Statements Pointed to Each Refrence Node (>1)
year
40 2016
2021
30
20
10
0
Gene Wiki Taxonomy Astronomy Music Law Ships
Subset
Fig. 5. The distribution of reference sharing (for reference nodes that are connected
to ≥ 2 statement). The red lines are the medians. Outliers and means are omitted for
presentation purposes.
[4] that the number of statements has increased 10 fold and the percentage of
statements referring to external sources has also increased from 25% to 68%.
Figure 3 confirms that currently there is a diversity in the most used properties
in references. All of these mean that there is a possibility of greater diversity in
references and a need for a comprehensive evaluation.
The impact of bots on the quality of references should also be examined.
Although Wikidata has strict policies for using bots, the effect of bots on ref-
erences has not been studied. The challenge here is that tracking bot activities
requires processing Wikidata edit history, which is ten times larger than the cur-
rent Wikidata dump. Shared references can also be a potential factor in reference
quality because they can at least challenge the relevancy condition.
Currently, the most important shortcoming is the lack of a framework to
examine the quality of referencing in Wikidata and other knowledge graphs.
Our idea is a scoring system that can evaluate different criteria on references
and quantify the result. For this scoring system, different criteria should be
defined according to the references. Relevancy and authoritativeness have been
suggested by the Wikidata community for references. There are also data quality
criteria such as Accuracy, Accessibility, Consistency, and Completeness that need
to be accurately defined according to the context of the references and reference-
specific properties. For example, accessibility can be defined as the availability
of the links mentioned in the references.
The above criteria apply to single references, but criteria such as shared
references should be considered on the whole of Wikidata (or its subsets). Fur-
thermore, some criteria can be measured by the machine, while some others such
as relevancy and authoritativeness are subjective, and evaluating them requires
machine training with human intervention.
Statistical information can be effective in defining some criteria. For example,
using the information of Section 5.3, we can determine a minimum number of
triples that reference nodes should have. Section 5.2 also tells us what properties
are most commonly used in references so we will be able to define necessary
criteria appropriate to those properties. Our plan for this scoring system is to
identify the necessary criteria, provide a precise definition of them, and measure
them on the six subsets as well as a random sample from Wikidata. We also plan
to separate human references and bot references to compare the quality between
them.
7 Conclusion
In this paper, we performed a statistical review of the references in Wikidata.
We extracted six independent Wikidata subsets corresponding to 6 different
WikiProjects and reviewed reference statistics in them. These statistics can be
used by project contributors to improve Wikidata, e.g. correcting the properties
used in their project, reviewing shared references, and trying to provide a suffi-
cient number of triples. The subsetting method used can be replicated for other
Wikidata projects and other fields of study.
Our statistics show the importance of a more in-depth study of Wikidata
references. We stated our position of the need for a reference quality scoring
system based on data quality dimensions and provided basic ideas for the sys-
tem. Such an assessment system can provide precise and detailed suggestions to
Wikidatians/WikiProject holders. Our future work is to complete the definition
and development of the reference quality scoring system. We aim to perform
a comprehensive evaluation on Wikidata references, using WikiProjects along
with randomly selected subsets. The challenges for the future work are the large
volume of data, tracing bot/human edits, and the subjective nature of the con-
cepts.
Acknowledgements
We would like to acknowledge the useful guidance and fruitful discussions with
the ShEx Community Group8 ; Kat Thronton, Andra Waagmeester, Dan Brick-
ley, and Eric Prud’hommaux.
References
1. Wikibase/Indexing/RDF Dump Format - MediaWiki, https://www.mediawiki.
org/wiki/Wikibase/Indexing/RDF_Dump_Format, accessed 2021-06-30
2. Wikidata entity dump (JSON) generated on June 30, 2021 : Free Download, Bor-
row, and Streaming, https://archive.org/details/wikidata-20210630.json.
gz, accessed 2021-09-27
8
http://shex.io/ - accessed July 2021
3. Wikidata entity dumps (JSON and TTL) of all Wikibase entries for Wiki-
data generated on October 03, 2016 : Wikidata editors : Free Download, Bor-
row, and Streaming, https://archive.org/details/wikibase-wikidatawiki-
20161003, accessed 2021-06-30
4. Wikidata Stats, https://wikidata-todo.toolforge.org/stats.php, accessed
2021-06-30
5. Wikidata:Bots - Wikidata, https://www.wikidata.org/wiki/Wikidata:Bots, ac-
cessed 2021-06-30
6. Wikidata:Statistics/Wikipedia/Type of content - Wikidata, https:
//www.wikidata.org/wiki/Wikidata:Statistics/Wikipedia/Type_of_content,
accessed 2021-06-30
7. Wikidata:Verifiability - Wikidata, https://www.wikidata.org/wiki/Wikidata:
Verifiability, accessed 2021-06-30
8. Wikidata:WikiProject Astronomy - Wikidata, https://www.wikidata.org/wiki/
Wikidata:WikiProject_Astronomy, accessed 2021-06-30
9. Wikidata:WikiProject Disambiguation pages - Wikidata, https://www.wikidata.
org/wiki/Wikidata:WikiProject_Disambiguation_pages, accessed 2021-06-30
10. Wikidata:WikiProject Law - Wikidata, https://www.wikidata.org/wiki/
Wikidata:WikiProject_Law#Participants, accessed 2021-06-30
11. Wikidata:WikiProject Music - Wikidata, https://www.wikidata.org/wiki/
Wikidata:WikiProject_Music#Overview, accessed 2021-06-30
12. Wikidata:WikiProject Schemas/Subsetting - Wikidata, https://www.wikidata.
org/wiki/Wikidata:WikiProject_Schemas/Subsetting, accessed 2021-06-30
13. Wikidata:WikiProject Scholia - Wikidata, https://www.wikidata.org/wiki/
Wikidata:WikiProject_Scholia, accessed 2021-06-30
14. Wikidata:WikiProject Ships - Wikidata, https://www.wikidata.org/wiki/
Wikidata:WikiProject_Ships, accessed 2021-06-30
15. Wikidata:WikiProject Taxonomy - Wikidata, https://www.wikidata.org/wiki/
Wikidata:WikiProject_Taxonomy, accessed 2021-06-30
16. Wikidata:WikiProjects - Wikidata, https://www.wikidata.org/wiki/Wikidata:
WikiProjects, accessed 2021-06-30
17. Beghaeiraveri, S.A.H.: Wikidata reference statistics. https://github.com/
seyedahbr/Wikidata_Reference_Statistics (2021)
18. Beghaeiraveri, S.A.H.: Wikidata Subsets of 6 Wikiproject (Gene
Wiki, Taxonomy, Astronomy, Music, Law, Ships) (Jul 2021).
https://doi.org/10.5281/zenodo.5117928, https://doi.org/10.5281/zenodo.
5117928
19. Beghaeiraveri, S.A.H., Gray, A.J.G., McNeill, F.J.: Experiences of Us-
ing WDumper to Create Topical Subsets from Wikidata. In: CEUR
Workshop Proceedings. vol. 2873, p. 13. CEUR-WS (Jun 2021), https:
//researchportal.hw.ac.uk/en/publications/experiences-of-using-
wdumper-to-create-topical-subsets-from-wikid, iSSN: 1613-0073
20. Burgstaller-Muehlbacher, S., Waagmeester, A., Mitraka, E., Turner, J., Putman,
T., Leong, J., Naik, C., Pavlidis, P., Schriml, L., Good, B.M., Su, A.I.: Wiki-
data as a semantic framework for the Gene Wiki initiative. Database (Oxford)
2016 (2016). https://doi.org/10.1093/database/baw015, https://academic.oup.
com/database/article-lookup/doi/10.1093/database/baw015
21. Curotto, P., Hogan, A.: Suggesting citations for wikidata claims based on
wikipedia’s external references. In: Wikidata@ ISWC (2020)
22. Färber, M., Bartscherer, F., Menne, C., Rettinger, A.: Linked data quality of
DBpedia, Freebase, OpenCyc, Wikidata, and YAGO. Semantic Web 9(1), 77–
129 (Nov 2017). https://doi.org/10.3233/SW-170275, https://www.medra.org/
servlet/aliasResolver?alias=iospress&doi=10.3233/SW-170275
23. Fünfstück, B.: Wdumper. https://github.com/bennofs/wdumper (2019)
24. Koesten, L., Vougiouklis, P., Simperl, E., Groth, P.: Dataset Reuse: To-
ward Translating Principles to Practice. Patterns 1(8), 100–136 (Nov 2020).
https://doi.org/10.1016/j.patter.2020.100136, https://www.sciencedirect.com/
science/article/pii/S2666389920301847
25. Lehmann, J., Gerber, D., Morsey, M., Ngonga Ngomo, A.C.: DeFacto - Deep Fact
Validation. In: The Semantic Web – ISWC 2012. pp. 312–327. Lecture Notes in
Computer Science, Springer (2012). https://doi.org/10.1007/978-3-642-35176-1 20
26. Lucassen, T., Schraagen, J.M.: Trust in wikipedia: how users trust information from
an unknown source. In: Proceedings of the 4th workshop on Information credibility.
pp. 19–26. WICOW ’10, Association for Computing Machinery, Raleigh, North
Carolina, USA (Apr 2010). https://doi.org/10.1145/1772938.1772944, https://
doi.org/10.1145/1772938.1772944
27. Piscopo, A., Kaffee, L.A., Phethean, C., Simperl, E.: Provenance Information in
a Collaborative Knowledge Graph: An Evaluation of Wikidata External Refer-
ences. In: The Semantic Web – ISWC 2017, vol. 10587, pp. 542–558. Springer
International Publishing, Cham (2017). https://doi.org/10.1007/978-3-319-68288-
4 32, http://link.springer.com/10.1007/978-3-319-68288-4_32, series Title:
Lecture Notes in Computer Science
28. Piscopo, A., Simperl, E.: What we talk about when we talk about wiki-
data quality: a literature survey. In: Proceedings of the 15th International
Symposium on Open Collaboration. pp. 1–11. ACM, Skövde Sweden (Aug
2019). https://doi.org/10.1145/3306446.3340822, https://dl.acm.org/doi/10.
1145/3306446.3340822
29. Piscopo, A., Vougiouklis, P., Kaffee, L.A., Phethean, C., Hare, J., Simperl, E.:
What do Wikidata and Wikipedia Have in Common?: An Analysis of their Use
of External References. In: Proceedings of the 13th International Symposium
on Open Collaboration - OpenSym ’17. pp. 1–10. ACM Press, Galway, Ireland
(2017). https://doi.org/10.1145/3125433.3125445, http://dl.acm.org/citation.
cfm?doid=3125433.3125445
30. Shenoy, K., Ilievski, F., Garijo, D., Schwabe, D., Szekely, P.: A Study of the Qual-
ity of Wikidata. arXiv:2107.00156 [cs] (Jun 2021), http://arxiv.org/abs/2107.
00156, arXiv: 2107.00156
31. Vrandečić, D., Krötzsch, M.: Wikidata: a free collaborative knowl-
edgebase. Communications of the ACM 57(10), 78–85 (Sep 2014).
https://doi.org/10.1145/2629489, https://dl.acm.org/doi/10.1145/2629489