Wikidata is a general-purpose knowledge graph covering a wide variety of topics with content being crowd-sourced through an open wiki. There are now over 90M interrelated data items in Wikidata which are accessible through a public query endpoint and data dumps. However, execution timeout limits and the size of data dumps make it di cult to use the data. The creation of arbitrary topical subsets of Wikidata, where only the relevant data is kept, would enable reuse of that data with the bene ts of cost reduction, ease of access, and exibility. In this paper, we provide a working de nition for topical subsets over the Wikidata Knowledge Graph and evaluate a third-party tool (WDumper) to extract these topical subsets from Wikidata.
A Knowledge Graph (KG) is de ned as representing real-world entities as
nodes in a graph with the relationships between them captured as edges [
Wikidata [
Our motivation for research on Wikidata subsetting is a combination of research goals, exibility and ease of use. From the exibility and ease of use perspective, we are looking for Wikidata subsets that can allow users to run smaller versions of Wikidata on available platforms such as laptops and PCs. Wikidata, as a knowledge graph with an interesting data model, has signi cant features for inspiration and improvement, but the speed of research and the diversity of researchers will be reduced if any experiment on it requires powerful servers, processing clusters, and hard disk arrays. From the research point of view, our motivation is creating a type of subset we call Topical Subset which is a set of entities related to a particular topic and the relationships between them. Having topical subsets of Wikidata for example in the elds of art, life sciences, or sports, not only helps us achieve the rst goal ( exibility and ease of use) but also provides a platform for comparing and evaluating Wikidata features in di erent topics. The subset will also make the research experiments reproducible as the data used can be archived and shared more easily.
You can envision that such subsets can be generated through SPARQL
CONSTRUCT queries. While this is straightforward for small subsets focused
on a single entity type, e.g. politicians, it does not scale to interrelated topics
that make up a larger domain, e.g. the life sciences subset de ned by GeneWiki
[
KG subsetting, particularly in the context of Wikidata [
WDumper [
Shape Expressions (ShEx) [
General purpose KGs like Wikidata are valuable sources of facts on a wide variety of topics. However, their increasing size makes them costly and slow to use locally. Additionally, the large volume of data in Wikidata increases the time required to run complex queries. This often restricts the types of queries that can be posed over the public endpoint since it has a strict 60-second limit on the execution time of queries. Downloading and using a local version of Wikidata is a way of circumventing the timeout limit. However, this is not a cheap option due to the size of the data. A suggested system to have a personal copy of Wikidata includes 16 vCPUs, 128GB memory, and 800GB of raided SSD space7. 6 https://www.wikidata.org/wiki/Wikidata:Database_download 7 See this post: https://addshore.com/2019/10/your-own-wikidata-queryservice-with-no-limits/
There are a large number of use case scenarios where users will not need access to all topics in a large general purpose KG. A small and complete enough subset can be more suitable for many purposes. With a small subset inference strategies can be applied to the data and complete in reasonable time. Topical subsets could also be published along with papers, which provides better reproducibility of experiments. Therefore having a topical subset that is smaller but has the required data can enable complex query processing on cheap servers or personal computers | reducing the overall cost | whilst also providing an improvement in query execution times. 3.1
We now de ne four use cases for topical subsets in Wikidata that we will use to review WDumper, and can also be used in other reviews as a comparison platform. Note that the use cases are de ned in terms of English language statements. A subsetting approach, method, or tool would need to formalise these, as appropriate for their con guration, to extract the relevant data. Politicians: This subset should contain all entities that are an instance of the class politician, or any of its subclasses. In the case of Wikidata, this would be the class Q82955, while for DBpedia it would be the class Politician. The subset should contain all facts pertaining to these entities, i.e. in Wikidata all statements and properties.
General(military) Politicians: The subset should contain all entities that are an instance of the class politician(Q82955) or any of its subclasses, who also are a military o cer (Q189290) and have the rank of general (Q83460), i.e. politico-military individuals. The main goal of this use case is to see the e ect of having more conditions in the English de nition on the run-time and the volume of the output of subset extraction tools.
UK Universities: The subset should contain all instances of the class university (Q3918) or any of its subclasses, that are located in the UK. The subset should contain all statements and properties pertaining to these entities. This use case extends the complexity of the subset by having alternative properties and values to satisfy, e.g. the location can be captured in Wikidata with the properties country(P17), located in territory(P131), or location(P276). Likewise the country could be stated as one of the component parts of the UK, e.g. Scotland.
Gene Wiki: This case is based on the class-level diagram of the Wikidata
knowledge graph for biomedical entities given in [
The selection of these use cases is a combination of research and experimental goals. The Gene Wiki and Politicians use cases have been selected for future research purposes because of their hypothetical richness in references. The other Listing 1.1. An example of a function R which is a query to return all entities with type city SELECT ? entity WHERE {
? entity wdt : P31 wd : Q515 . # instance of ( P31 ) city ( Q515 ) } two use cases have been chosen to explore the expressiveness of a topical subset de nition, and then to explore the runtime execution of these. 3.2
We now provide a de nition for topical subsets based on the Wikidata data model. Wikidata consists of the following collections: { E: set of Wikidata entities { their ID starts with a Q. { P : set of Wikidata properties { their ID starts with a P.
{ S: set of Wikidata statements.
Now we de ne the lter function R : E ! E as a black-box that can be applied on E and selects a nite number of its members related to a speci c topic. Let ER E be the output of the function R. For entity e 2 E let Se S be all simple and complex Wikidata statements in which e is the subject. Note that in Wikidata, a simple statement is a regular RDF triple, while a complex statement is a triple that references and/or quali ers attached to it. Also, let Pe be all properties which are used in Se triples either for the statement itself or quali ers/references. With these assumptions, we de ne dump DR as a topical subset of Wikidata with respect to R:
DR := (ER; [ Pe; [ Se) e2ER e2ER From the de nitions of Pe and Se we can conclude that
S Pe P and e2ER
S and subsequently DR is a subset of Wikidata. We consider R S Se e2ER as black-box; the input of R is the set of all Wikidata entities and its output is a subset of Wikidata entities related to a speci c topic. The function R can be any set of de nitions, rules, or lters that describe a related group of entities. The de nition of R depends on the topic that is being described. One example of R is a simple SELECT query that describes all entities that have type city (Q515) (Listing 1.1). 4
WDumper is a tool provided by Wikidata for producing custom subsets from Wikidata8. The R function can be seen as a lter approach on entities. For each 8 https://www.wikidata.org/wiki/Wikidata:Database_download#Partial_RDF_ dumps - accessed 9 February 2021 topic, the appropriate lters on entities must be de ned. Once the lters are de ned and the subset ER is extracted, WDumper extracts all statements with origin e, where e 2 ER, along with their quali ers and references. A component overview of WDumper is given in Figure 1.
WDumper requires two inputs. The rst is the complete dump of the Wikidata in JSON. The second is a JSON speci cation le that contains rules and lters for determining which entities, properties and statements to extract from the full Wikidata dump. This is the de nition of the function R. The output of WDumper is an N-Triple (.nt) le that contains the entities and statements speci ed in the second input. There is also a GUI for creating the input speci cation le. We review WDumper through the following steps: 1. Writing WDumper speci cations for the use cases in Section 3.1. 2. Running WDumper with the above speci cations on two complete Wikidata dumps belonging to two di erent time points and compare the run-time and the volume of the extracted output. 3. Evaluating the extracted output via performing di erent queries both on the output and the input full dump. 4. Summarizing results and expressing strengths and weaknesses of WDumper. 4.1
Section 3.1 introduced use cases to evaluate subset extraction tools. In this
section, we describe the corresponding WDumper speci cation les [
Politicians: We de ne a lter on the occupation property(P106) of the entities to be a politician(Q82955).
General(military) Politicians: This extends the Politicians de nition with two more conditions. The rst is the occupation property(P106) to be military o cer(Q189290). The second is the military rank property(P410) to be general(Q83460).
UK Universities: We de ne a lter on entities with two conditions: the instance of property(P31) to be university(Q189290), and the country property(P17) to be United Kingdom(Q145).
Gene Wiki: For each item type of the class-diagram in [
We now give details of our experimental environment in which we will evaluate WDumper.
Input Dumps. We use two full dumps of the Wikidata database. The rst full dump is from 27 April 201510, and the second is from 13 November 202011. The selected 2020 dump was the latest JSON dump available when conducting our evaluation. The selected 2015 dump is the rst archive date for which both JSON and Turtle les are available (we need the JSON le for WDumper running, while the Turtle le is needed to import the full dump in a triplestore and evaluate output of WDumper based on the input). Table 1 provides summary information 10 Downloaded from https://archive.org/download/wikidata-json-20150427 - accessed 11 November 2020 11 Downloaded from https://dumps.wikimedia.org/wikidatawiki/entities/ - accessed 15 November 2020 about these two dumps. The 2015 dump is smaller, can be stored and processed locally even on PCs, and it takes a much shorter time to generate output. For this reason, it is very suitable for initial tests. The 2020 dump, on the other hand, is much richer and can provide insights on how WDumper deals with large datasets of the size that Wikidata now produces.
Experimental Environment. Experiments were performed on a multi-core server with 64 8-core AMD 6380 2.5GHz 64bit CPUs, 512GB of memory and 2.7TB disk space. Java openJDK version 11 (build 11+28) and Gradle 6.7.1 was used to compile and run WDumper.
Experimental Run. The calculated times have been extracted from the elapsed time mentioned in WDumper output log. For each of the execution cases, three independent runs were performed. The average and standard deviation from these times were calculated. 4.3
WDumper was run with the speci cation les described in Section 4.1 and the
two Wikidata dumps. The generated subsets are included in [
Initial observations from Table 2 show that the run-time on the 2020 dump is signi cantly longer than the 2015 dump. This can be justi ed by the larger volume data that must be processed to produce the subset. In all cases, generating the subset with additional statements took longer than generating the simple statements, and produced more volume in the output, indicating the addition of references, quali ers, and statements nodes in the output. For example, this change is very signi cant in the case of Gene Wiki in the 2020 dump. The added lters as well as the conditions added to the lters also have a direct e ect on the run-time and an inverse e ect on the output volume, which is to be expected. Of course within run-times, the amount of data that must be written in the output must also be considered. This is evident in comparisons between UK universities and the military politicians in which the volume of data has a greater impact than the number of conditions. Overall, considering the high volume of data, the time required to extract a topical subset by WDumper seems appropriate. 13 https://wikidata-todo.toolforge.org/stats.php Adding more lters does not have a huge e ect on runtime which is dominated by data volume. 4.4
The previous section considered the runtime performance of the WDumper tool and the size of the generated subsets. We now consider validating the content of the subsets. That is, we consider if the produced output has the information that it was supposed to have according to the de nition in Section 3.2. Our assessment will be based on the following conditions: Condition 1: The number of ltered entities in the output should be equal to the same number of entities in the input dump. For example, in the Politicians use case, the number of persons with the occupation of politician in output should be equal to the number of persons with the occupation of politician in the corresponding input dump. This condition can be tested with COUNT queries on the input and output datasets.
Listing 1.2. Commands used for xing syntax errors of the 2015 dump. sed -i -E 's /( <.*)}(.* >)/\1\2/ ' < dump_file > sed -i -E 's /( <.*)\\ n (.* >)/\1\2/ ' <dump_file > sed -i -E 's /( <.*)\|(.* >)/\1\2/ ' <dump_file > Condition 2: For each entity that is supposed to be in output, the number of its related statements must be equal in both input and output datasets. For example, in the Politicians use case, if the main dump has 50 statements about George Washington, we expect to see the same number of statements about this politician in the output too. This condition can be tested using DESCRIBE queries.
Condition 3: WDumper can extract intermediate statement nodes, references, and quali ers exactly as they are at the input dump. This condition can be tested by querying the quali ers and references of some given statements. Testing the conditions requires running di erent queries on the input and output of WDumper. For the output of WDumper, we use Apache Jena Fuseki version 3.17 to import data as TDB2 RDF datasets and perform queries. Data Corrections. We encountered problems loading the 2015 turtle dump14 into Fuseki. Errors arose from the inclusion of bad line endings in more than 100 cases, and the existence of characters such as `na'. Unacceptable characters such as `nn' and `nn' can also be seen in the WDumper outputs, which reinforces the possibility that this problem occurs due to the conversion of information from the JSON le to RDF format.
For the WDumper outputs and the 2015 dump, these errors were
manually xed using the sed commands given in Listing 1.2. The sanitized versions
are available in [
Validation of Condition 1. We use COUNT queries to validate this condition. The purpose of these queries is to count the entities that should be in the output according to the lter(s) of each use case. If WDumper is performing correctly, 14 Downloaded from https://archive.org/download/wikidata-json-20150427 - accessed 20 December 2020 15 https://www.wikidata.org/wiki/Wikidata:History_Query_Service Listing 1.3. COUNT queries for evaluating condition 1. Pre xes and most of Gene Wiki's query have been deleted for more readability. ############ Politicians ############################## SELECT ( COUNT ( DISTINCT ? item ) AS ? count ) WHERE { ? item wdt : P106 wd: Q82955 . # occupation of politician } ############ UK Universities ########################## ?}SEiLteEmCT ww(ddttC::OUPPN31T17 (wwddD::ISQQT134I95N18CT ;.?##itecimon)usnttaArSnyce? ocfoofunUtnu)intievWdeHrERsKEiit{nygdom ############ General ( military ) Politicians ############ ?SEiLteEmCT www(dddtttC:::OUPPPN114T001660 (wwwDdddI:::STQQQI881N238C949T5625090? i;;.tem###) mooAiccSlcciuut?ppaaacrttoyiiuoonnntr)anookffWHEompRfioElli{igttei.nceiroaafnlficer } {US#?NE#IL#iOEt#NCe#Tm###(w#dC#tO#U:#NPT31G(enwDedI:STWQIi4Nk2iC3T02#6?#i#t.#}e#m#)###A#S##i?#n#sct#oau#nn#tc#e#)##oWf#H#E#RaE#c#{t#i#v#e##s#i#te {? item wdt : P31 wd: Q4936952 .} # instance of anat . struct .
UNION # ...
UNION {? item wdt : P31 wd: Q50379781 .} # instance of therap . use }
Politicians General (military) Politicians UK Universities
Gene Wiki 2015 Dump 2020 Dump
246,009 246,044 641,387 646,401 165 165 597 602 73 73 183 186 19,432 19,432 3,282,560 3,283,471 the result of this count should be the same on both the input and output datasets. These queries will be di erent for each use case, depending on the de nition of that use case. For example, while in the Politicians use case we count the number of people with political jobs, in the case of Gene Wiki we count the union of entities of type disease, genes, proteins, etc. Listing 1.3 shows the queries executed for each use case. These queries run on the each use case's output \withRQFS" since these are included in the input dataset. The results of performing the COUNT queries are shown in Table 3.
Our results show that for the 2015 dump, the number of entities in the output and input is equal except for the Politicians use case. In both 2015 and 2020 dumps, the di erence between input and output is less than one percent in the cases of inequality. In the case of 2020 dump, the di erence can be attributed to the entry of new data in the interval between our tests and the dump date. This is reasonable especially in the case of Gene Wiki where bots are importing new information into Wikidata every day. In the case of the 2015 dump in the Politicians row, the 35 di erences between input and output is unjusti able. The reason for this di erence may be the inability of WDumper to parse the data of these entities in the input dump. WDumper uses the JSON le as input, and to be able to fetch an entity, it must see the speci c structure of the Wikidata arrays and sub-arrays in the JSON le. Some entities may not have this complete structure in the JSON le but they do exist in the Turtle le.
Validation of Condition 2. To validate this condition, in each use case we use DESCRIBE queries for an arbitrary entity that is in the WDumper output. DESCRIBE queries list all triples of the given entity. We expect that the result of the DESCRIBE queries should be the same on both the input and output datasets. For each use case, we selected an arbitrary entity (called Tested Entity) which is present in both the input and output dataset. We then run a DESCRIBE wd:Q... query and count the extracted triples. Table 4 shows our results.
From Table 4 it is clear that the number of triples in the DESCRIBE queries in both the 2020 and 2015 dumps are not equal. This di erence prompted us to explore the di erences using the compare module of the RDFlib library. It was found that in the case of the 2015 dump, the input dump contains predicates such as <http://www.w3.org/2004/02/skos/core#prefLabel> and <http://schema.org/name>, which are not extracted by WDumper. Table 5, shows the details and total numbers of predicates that are in the input dump (2015 dump) for the selected entities and WDumper could not fetch. As we can see, the total column is exactly the di erence between the describe queries. In the case of 2020 dump, some of predicates with <http://www.w3.org/2004/02/ skos/core#> pre x, such as dateModified, and all <http://www.wikidata. org/prop/direct-normalized/> predicates are not detectable by WDumper. However, in both dumps the statements whose predicate is a property of Wikidata (e.g. P31, P106, etc.), were completely extracted by WDumper.
Q23 Q355643 Q1094046 Q17487737 P26 P485 P355 P680
Property Output Input Output Input
Validation of Condition 3. To validate this condition, we selected an arbitrary entity from each use case, and for this entity, we considered one of its statements. We then counted the quali ers and the references of this statement in 2020 dump (over the WDQS) and in the output of WDumper. Table 6 shows the selected entity, selected property, and the number of quali ers and references for them. From Table 6, it is clear that WDumper can extract quali ers and references completely from the input.
The results of our evaluations show that WDumper, as a custom dump production tool, can be used to create some topical subsets. This tool can correctly and completely extract the entities speci ed by its lters. It also extracts almost all statements related to entities (except it is not designed to extract some pre xes). One of the features we have been looking for is the ability to extract references and quali ers of Wikidata statements, which WDumper can do. Setting up this tool is not very complicated; the user only needs to select the lters of the entities and statements, run the tool and it extracts all of the information at once. Its GUI is also somewhat helpful, while the JSON structure of its speci cation les is also simple and understandable.
Limitations and Weaknesses. The most important weakness of WDumper with regard to the topical subsets is the limitation in the de nition of entity lters. In WDumper, entities can only be ltered based on the presence of a Px property or having the value v for a Px property. Although it is possible to deploy any number of such lters, this is not enough to specify some kinds of use cases. For example, suppose we want to specify the Scottish universities subset. By reviewing some of these universities on the Wikidata website, we can nd that their corresponding entity does not have any property that directly indicates they belong to Scotland. Of course, we can de ne the R function of these subsets through indirect methods (for example, considering the Geo-location of entities in Scottish universities), but these type of lters are not available in WDumper.
The recognition of type hierarchies is another limitation of WDumper. In the case of UK universities, for example, the University of Edinburgh(Q160302) is not among the universities extracted by WDumper. The reason for this is that the instance of property(P31) in this university refers to public university(Q875538) instead of university(Q3918). In SPARQL queries, such cases are handled by the property paths like wdt:P31/wdt:P279 . These property paths are not available in WDumper. The strategy of considering more lters to cover all subtypes needs a comprehensive knowledge of Wikidata ontology. This strategy will fail if the class hierarchy changes.
Another limitation is the inability to communicate between di erent lters in multi- lter cases. For example, in the Gene Wiki use case, we may want diseases that are somehow related to a gene or protein, while in the WDumper output there are diseases that have nothing to do with genes, proteins, and other Gene Wiki types. The inability to choose another output format other than NTriples, especially the Wikibase JSON output, which is more suitable for using the subset produced in a Wikibase instance and also has a smaller volume, is another limitation.
The main implications of these limitations is the reduction of exibility of subset extracting with this tool. With these weaknesses, users have to spend much more time de ning the desired subset. 5
In this paper, we reviewed the issue of building topical subsets over Wikidata. Our motivation for topical subsets is to enable e cient evaluation of complex queries over the knowledge graph with lower costs, reproducibility of experiments through archiving datasets, ease of use, and exibility. We provided example use cases for topical subsets as well as a de nition for topical subsets. This de nition enables us to evaluate and compare subset creation tools.
In this study we used WDumper for topical subset extraction over Wikidata and tested it by measuring run-time and output volume on four di erent use cases. We evaluated the correctness of the subsets generated by WDumper by comparing the answers to queries over the subsets and the full knowledge graph. Our experience shows that WDumper can be used to generate topical subsets of Wikidata in some use cases but not for all use cases. WDumper can extract the entities speci ed by its lters and extract most statements related to those entities; it also fetches the statement nodes and references/quali ers. However, WDumper has some weaknesses regarding topical subsets. Its main problem is the way it de nes lters on entities that reduces the power of this tool to build topical subsets. The most tangible issue is the inability to de ne and fetch subclasses of a class of entities, which is important in many use cases. Our suggestion for the future works is to explore alternative subsetting approaches such as using SPARQL queries or Shape Expressions. With selectors like SPARQL queries or ShEx schemata, we can increase the expressivity of the subset creation. It will also allow for subsets to be created on Knowledge Graphs other than Wikidata.
Acknowledgement. We would like to acknowledge the fruitful discussions with the participants of project 35 of the BioHackathon-Europe 2020; Dan Brickley,