Public knowledge graphs (KGs), such as DBpedia or Wikidata, provide deep background knowledge that can be exploited for downstream tasks such as questionanswering or recommender systems [3]. KG embeddings (KGEs) represent vertices and, depending on the approach, also edges of a KG as numeric vectors. This representation is easily consumable by most algorithms and can be exploited in downstream tasks. Advantages of KGEs, once they have been trained, include simple applicability, fast run time, good performance on multiple tasks, and reusability in downstream applications. On the downside, KGEs produce very large models 3 , and are very expensive to train and re-train in the case of evolving knowledge bases. For very large knowledge graphs, such as Wikidata, computing a complete embedding typically takes up to a day or longer [2].
In this paper, we address the scalability aspect of knowledge graph embeddings: Our novel approach, RDF2Vec Light, allows to train partial, task-specific models with 3 For example, the 200 dimensional DBpedia RDF2Vec embedding model available at KGvec2go [5] requires more than 10GB of disk storage. only a fraction of the computation requirements compared to other embedding approaches, while retaining a high performance on multiple tasks. The resulting models contain only vectors for entities of interest. Internally, RDF2Vec Light only traverses a subset of the underlying knowledge graph which leads to processing times that are much shorter than the original RDF2Vec approach which always processes an entire knowledge graph. Moreover, the resulting models are much smaller. 4
RDF2Vec is based on performing random walks on a graph [6]. The underlying idea of RDF2Vec Light embeddings is to generate only local walks for entities of interest given a predefined task. After the walk generation has been completed, the training of vectors can be performed like in the original approach.
Rather than starting random walks at all entities of interest, it is randomly decided for each depth-iteration whether to go backwards, i.e. to one of the node's predecessors, or forwards, i.e. to the node's successors (line 9 of Algorithm 1). As a result, the entities of interest can be at the beginning, at the end, or in the middle of a walk which better captures the context of the entity. This generation process is described in Algorithm 1. The RDF2Vec method as well as the RDF2Vec Light extension have been implemented in Java and Python. 5 The implementation can handle various RDF formats such as n-triples, RDF/XML, Turtle, or HDT [1]. In addition, a REST API has been implemented and is provided on http://www.kgvec2go.org.
In order to evaluate the approach presented in this paper, the classification and regression experiments, as well as the entity and document relatedness experiments of Ristoski et al. [6] have been repeated. The evaluation follows the setup defined in [4]. Six classic and six light embedding spaces have been trained each with the following parameters held constant: wi nd ow si ze = 5, neg at i ve sampl es = 25. The parameters that were changed are the generation mode (cbow and sg ) as well as the dimension of the embedding space (50, 100, 200). All walks have been generated with 500 walks per entity and a depth of 4. For the evaluation, the DBpedia knowledge graph as of 2016-106 has been used.
For the classification and regression tasks, we follow the same setup as in the original RDF2vec paper [7]: For the classification tasks, four classifiers have been evaluated: Naïve Bayes, C4.5 (decision tree algorithm), k-NN with k = 3, and Support Vector Machines (SVM) with C ∈ {10 −3 , 10 −2 , 0.1, 1, 10, 10 2 , 10 3 } where the best C is chosen. A 10-fold cross validation has been used to calculate the performance statistics. For the regression tasks, three approaches have been evaluated: linear regression, k-NN, and M5rules. For the sake of brevity, we only report results for the best performing approaches (SVM and LR). 7Table 2. Results on the document relatedness task (LP50), reporting the harmonic mean of Pearson correlation and Spearman rank correlation, and on entity relatedness (KORE), using cosine similarity. The best value of each comparison group is highlighted in bold. The overall best value is additionally underlined. In the results tables, strategy refers to the configuration with which the embeddings have been obtained. The structure can be read as follows: <mode>_<number_of_walks_per_entity>_<walk_depth>_<training_mode>_<dimen-sion> where mode is either Light or Classic. For example, Light_500_4_CBOW_100 refers to RDF2Vec Light embeddings with 500 walks per entity, a walk depth of 4, CBOW configuration, and an embedding space dimensionality of 100.
For classification and regression, we can observe that except for the cities dataset, the difference between the two approaches is rather marginal. For entity and document relatedness, the results are less conclusive. Here, we see that the RDF2vec light approach is en par with the classic approach for the CBOW variant, but the results are reversed when looking at the SG variant, which also yields the best results globally.
In order to analyze those results more deeply, and to distinguish the cases where RDF2vec Light is en par with classic RDF2vec from those where it is clearly inferior, we looked at the linkage degree of the entities at hand, as well the homogeneity of the entities of interest.
For the linkage degree, we can observe that a higher degree of the entities of interest leads to a worse performance of RDF2vec Light. This can be seen in the inferior performance of RDF2vec Light for the cities datasets in classification and regression. Cities are among the most strongly interlinked entities in DBpedia [3]. At the same time, the document and entity similarity datasets contain a larger number of strongly interlinked head entities.
While for classification and regression problems, the set of entities is rather homogeneous (i.e., all are cities, albums, etc.), the homogeneity is lower for the document and entity relatedness, where the entities of interest are scattered across many classes. Both degree and homogeneity contribute to the density of the considered subgraphs, as depicted in Fig. 1. From the plots, we can observe a correlation of RDF2Vec Light performance and the density of the graph spanned by the random walks -the more dense the graph (i.e., the less head entities there are and the more homogeneous the entity set at hand), the better the performance of RDF2Vec Light.
The runtime of RDF2Vec Light is linear in the number of entities of interest. On commodity hardware, the runtime is roughly 1 minute per 10 nodes. In comparison, training RDF2Vec on the full DBpedia graph takes a few days.
In this paper, we presented RDF2Vec Light, an approach for learning latent representations of knowledge graph entities that requires only a fraction of the computing power compared to other embedding approaches. Rather than embedding the whole knowledge graph, RDF2Vec Light trains vectors for only few entities of interest and their context. For this approach, the walk generation algorithm has been adapted to better represent the context of the entities. Our experiments show that the results achieved with RDF2Vec Light are comparable to those obtained with the standard RDF2Vec, while requiring only a fraction of the runtime.