Knowledge graph embedding approaches represent nodes and edges of graphs as mathematical vectors. Current approaches focus on embedding complete knowledge graphs, i.e. all nodes and edges. This leads to very high computational requirements on large graphs such as DBpedia or Wikidata. However, for most downstream application scenarios, only a small subset of concepts is of actual interest. In this paper, we present RDF2Vec Light, a lightweight embedding approach based on RDF2Vec which generates vectors for only a subset of entities. To that end, RDF2Vec Light only traverses and processes a subgraph of the knowledge graph. Our method allows the application of embeddings of very large knowledge graphs in scenarios where such embeddings were not possible before due to a significantly lower runtime and significantly reduced hardware requirements.
Public knowledge graphs (KGs), such as DBpedia or Wikidata, provide deep
background knowledge that can be exploited for downstream tasks such as
questionanswering or recommender systems [
In this paper, we address the scalability aspect of knowledge graph embeddings: Our novel approach, RDF2Vec Light, allows to train partial, task-specific models with 3 For example, the 200 dimensional DBpedia RDF2Vec embedding model available at
KGvec2go [
Algorithm 1: Walk generation algorithm for RDF2Vec Light. only a fraction of the computation requirements compared to other embedding approaches, while retaining a high performance on multiple tasks. The resulting models contain only vectors for entities of interest. Internally, RDF2Vec Light only traverses a subset of the underlying knowledge graph which leads to processing times that are much shorter than the original RDF2Vec approach which always processes an entire knowledge graph. Moreover, the resulting models are much smaller.4
RDF2Vec is based on performing random walks on a graph [
Rather than starting random walks at all entities of interest, it is randomly
decided for each depth-iteration whether to go backwards, i.e. to one of the node’s
predecessors, or forwards, i.e. to the node’s successors (line 9 of Algorithm 1). As a
result, the entities of interest can be at the beginning, at the end, or in the middle
of a walk which better captures the context of the entity. This generation process is
4 RDF2Vec Light models are typically only a few kilobytes in size, compared to multiple
gigabytes of disk space required to persist classic embedding models.
described in Algorithm 1. The RDF2Vec method as well as the RDF2Vec Light
extension have been implemented in Java and Python.5 The implementation can handle
various RDF formats such as n-triples, RDF/XML, Turtle, or HDT [
In order to evaluate the approach presented in this paper, the classification and
regression experiments, as well as the entity and document relatedness experiments of
Ristoski et al. [
Six classic and six light embedding spaces have been trained each with the following parameters held constant: w i nd ow si ze Æ 5, ne g at i ve sampl e s Æ 25. The parameters that were changed are the generation mode (cbow and sg ) as well as the dimension of the embedding space (50, 100, 200). All walks have been generated with 500 walks per entity and a depth of 4. For the evaluation, the DBpedia knowledge graph as of 2016-106 has been used.
For the classification and regression tasks, we follow the same setup as in the
original RDF2vec paper [
In the results tables, strategy refers to the configuration with which the embeddings have been obtained. The structure can be read as follows: <mode>_<number_of_walks_per_entity>_<walk_depth>_<training_mode>_<dimension> where mode is either Light or Classic. For example, Light_500_4_CBOW_100 refers to RDF2Vec Light embeddings with 500 walks per entity, a walk depth of 4, CBOW configuration, and an embedding space dimensionality of 100.
For classification and regression, we can observe that except for the cities dataset, the difference between the two approaches is rather marginal. For entity and document relatedness, the results are less conclusive. Here, we see that the RDF2vec light approach is en par with the classic approach for the CBOW variant, but the results are reversed when looking at the SG variant, which also yields the best results globally.
In order to analyze those results more deeply, and to distinguish the cases where RDF2vec Light is en par with classic RDF2vec from those where it is clearly inferior, we looked at the linkage degree of the entities at hand, as well the homogeneity of the entities of interest.
For the linkage degree, we can observe that a higher degree of the entities of
interest leads to a worse performance of RDF2vec Light. This can be seen in the inferior
performance of RDF2vec Light for the cities datasets in classification and regression.
Cities are among the most strongly interlinked entities in DBpedia [
While for classification and regression problems, the set of entities is rather homogeneous (i.e., all are cities, albums, etc.), the homogeneity is lower for the document and entity relatedness, where the entities of interest are scattered across many classes. Both degree and homogeneity contribute to the density of the considered subgraphs, as depicted in Fig. 1. From the plots, we can observe a correlation of RDF2Vec Light performance and the density of the graph spanned by the random walks – the more dense the graph (i.e., the less head entities there are and the more homogeneous the entity set at hand), the better the performance of RDF2Vec Light.
The runtime of RDF2Vec Light is linear in the number of entities of interest. On commodity hardware, the runtime is roughly 1 minute per 10 nodes. In comparison, training RDF2Vec on the full DBpedia graph takes a few days.
In this paper, we presented RDF2Vec Light, an approach for learning latent representations of knowledge graph entities that requires only a fraction of the computing power compared to other embedding approaches. Rather than embedding the whole knowledge graph, RDF2Vec Light trains vectors for only few entities of interest and their context. For this approach, the walk generation algorithm has been adapted to better represent the context of the entities. Our experiments show that the results achieved with RDF2Vec Light are comparable to those obtained with the standard RDF2Vec, while requiring only a fraction of the runtime.