Modern large-scale knowledge graphs, such as DBpedia, are datasets which require large computational resources to serve and process. Moreover, they often have longer release cycles, which leads to outdated information in those graphs. In this paper, we present DBpedia on Demand { a system which serves DBpedia resources on demand without the need to materialize and store the entire graph, and which even provides limited querying functionality.
Knowledge graphs on the Web { such as DBpedia [
Another challenge related to such knowledge graphs is their sheer size. The creation of those graphs is often a longer running and computation-intensive process, and even serving the knowledge graph online requires quite a bit of computing power. The hosted services for hosting a static DBpedia copy or running DBpedia live cost around $1 per hour at the time of writing this paper.2
User Front end Cache Request: http://dbpedia.org/resource/Mannheim Get Ingoing Links Get Page Sources
KG on Demand
Engine Generate RDF Extract Abstracts
Linked Data Endpoint HTTP GET http://dbpedia.org/resource/Mannheim SPARQL Endpoint SELECT ?p WHERE {dbr:Mannheim dbo:population ?p} Extraction Framework
In this paper, we introduce DBpedia on Demand { an installation of DBpedia which can be run on a local machine as a service. It creates knowledge graph resources on request without materializing an entire graph, which makes its computing requirements rather minimal. It provides a Linked Data endpoint as well as limited SPARQL querying properties.3 2
DBpedia is built from Wikipedia by extracting information from infoboxes and
other bits of structured contents, such as categories. To that end, it uses
mappings from infoboxes to a central ontology, which are also stored in a Wiki. When
creating a release, a dump of Wikipedia is processed, and the page contents are
transformed to knowledge graph instances using the mappings de ned in the
Mappings Wiki [
The idea of KG on demand is di erent. Instead of creating a full DBpedia copy, it creates instances upon request: when a user requests a certain resource, the corresponding Wikipedia page is downloaded and processed in real time. Mappings can be downloaded from the mappings Wiki regularly, which allows for using at least recent, although not up to date mappings. Fig. 1 shows the overall architecture of the KG on Demand engine. The process runs in ve steps: 1. The URI for which to generate the RDF for is obtained, either trough the user interface, a HTTP request, or by extracting it from a SPARQL query. 2. The ingoing links (i.e., Wikipedia pages pointing to the page which corresponds to the requested resource) to that resource are obtained from the Wikipedia API. 3 Code available at https://github.com/mbrockmeier/KnowledgeGraphOnDemand 3. The page sources for both the requested resources and the resources of the pages linking to the resource are obtained via the Wikipedia API. 4. For all those pages, the corresponding RDF representation is generated via the DBpedia Extraction Framework. 5. Additionally, the abstract for the resource at hand is generated by extracting the rst few sentences from the Wikipedia page.
In the DBpedia knowledge graph, outgoing edges from a resource are generated from the corresponding Wikipedia page's infobox. Hence, for generating only the outgoing edges, processing one single Wikipedia page is su cient. In contrast, ingoing edges are outgoing edges of other Wikipedia pages. In order to generate those, we rst query the Wikipedia API for other pages that link to the one at hand, and then process those as well in order to generate the ingoing edges.
Since the abstract extraction in DBpedia requires the creation and processing of a local copy of the entire Wikipedia, we developed or own simple abstract extraction mechanism instead of using the original one.
The number of ingoing Wikipedia links is the key factor that in uences the performance of the proposed approach. Fig. 2 depicts the performance for extracting resources from Wikipedia pages.4 It can be observed that the systems scales linearly with the number of ingoing links.5
The approach of generating resources for URIs also determines the kind of SPARQL queries that our approach can process. Generally, all queries that only consider simple ingoing and outgoing edges of a xed resource can be handled, such as SELECT ?actor WHERE { ?actor dbo:starring dbr:Lost_Highway .} SELECT ?director WHERE { dbr:Lost_Highway dbo:director ?director .} On the other hand, our approach cannot handle queries without a xed resource, or queries with a path length greater than one, such as: 4 The tests were run on an Intel Core i7-8700K processor, 32GB of DDR4 RAM, a 1TB NVMe SSD, and an internet connection with a downstream rate of 100MBit/s.
The reported processing times are the averages of 10 runs. 5 Since the number of ingoing links are the main driver of performance, the maximum number of ingoing links to consider can also be reduced in the con guration of our implementation. By default, all ingoing links are processed. In other words, the queries are limited to patterns with at least one resource xed, and all query variables being at most one hop away from that resource. 3
DBpedia live builds a full copy of DBpedia. This copy is then updated by monitoring the update stream of Wikipedia. Whenever a change is made in Wikipedia, DBpedia live is noti ed and processes the change by re-computing the corresponding instance representation in the knowledge graph. Likewise, it is noti ed on changes in the Mappings wiki (e.g., if the mapping of an infobox to an ontology class is changed), and the a ected instances can then be changed as well.
By design, DBpedia live always stores a local materialized copy of the full knowledge graph, which leads to high computational costs of serving the knowledge graph endpoint (i.e., it can be run on commodity hardware). Since all requests are processed on that local copy, they can be handled faster than in DBpedia on demand, and also all types of SPARQL queries can be answered. At the same time, DBpedia Live requires a lot of online runtime to constantly process the stream of changes from Wikipedia. Table 1 summarizes the main di erences between DBpedia Live and DBpedia on Demand. 4
In this paper, we have shown an approach which generates a knowledge graph like DBpedia in an on-demand and lightweight fashion. The approach is very resource-e cient and can be run on commodity hardware. Hence, it is an interesting building block for applications which use a knowledge graph, but do neither want to rely on a public endpoint, nor materialize an entire knowledge graph. The approach can be seen as a complement to DBpedia Live, in comparison to which it has di erent advantages and disadvantages.
While the current implementation is a prototype only, there are some shortcomings which we have inherited from the DBpedia Extraction Framework. First, that framework is not capable of multithreading, which makes the current implementation suitable for a local service, rather than for setting up a public endpoint. Second, we have currently encapsulated the entire extraction framework, which, however, comes with a signi cant ramp-up time. Both issues could be addressed by branching and refactoring the extraction framework's codebase.
While this demonstration has been based on DBpedia, it can be transferred
to other approaches as well. With the same mechanism, it would be possible
to use the extraction code of other Wikipedia-based knowledge graphs, such as
YAGO [
While the SPARQL capabilities in our approach are still basic, the
approach could be extended towards supporting more complex queries. Possible
approaches would be incremental solving of queries, or rst using the Wikipedia
link graph for obtaining candidates, and then running the extraction on the
candidates to narrow down the solution set. For implementing a query endpoint,
it would also be interesting to build a (possibly limited) linked data fragments
endpoint [
Apart from adding querying capabilities, the endpoint could also be
enriched with an on-demand approach of generation of embedding vectors, such
as RDF2vec Light [