The original Semantic Web vision foresees to describe entities in a way that the meaning can be interpreted both by machines and humans. Following that idea, large-scale knowledge graphs capturing a signi cant portion of knowledge have been developed. In the recent past, vector space embeddings of semantic web knowledge graphs { i.e., projections of a knowledge graph into a lower-dimensional, numerical feature space (a.k.a. latent feature space) { have been shown to yield superior performance in many tasks, including relation prediction, recommender systems, or the enrichment of predictive data mining tasks. At the same time, those projections describe an entity as a numerical vector, without any semantics attached to the dimensions. Thus, embeddings are as far from the original Semantic Web vision as can be. As a consequence, the results achieved with embeddings { as impressive as they are in terms of quantitative performance { are most often not interpretable, and it is hard to obtain a justi cation for a prediction, e.g., an explanation why an item has been suggested by a recommender system. In this paper, we make a claim for semantic embeddings and discuss possible ideas towards their construction.
Knowledge Graphs are directed, labeled graphs that encode knowledge about
entities from multiple domains [
Clustering similar entities have a similar position in the vector space. Relation preservation relations between entities are represented by a constant vector representation. That way, (approximate) arithmetics such as German!y + capit!al Berl!in become possible.
Fig. 1 depicts selected entities of DBpedia and Wikidata in the two dimensional PCA projection of an embedding space and illustrates these properties. Countries and cities form clusters, and the displacement between a country and its capital is similar for all the countries.
Over the past years, quite a few approaches for generating such embeddings
have been proposed, including TransE [
While the quantitative performance on those tasks is undisputed, the semantics of the knowledge representation are given up in favor of an entirely numeric, non-semantic representation. Therefore, the use of embeddings does not allow the interpretation of results. However, interpretations are useful, e.g., for { justifying why a relation was added to a knowledge graph, { explaining why an item was suggested by a recommendation system, or { allowing for descriptive, not only predictive data mining.
With embedding methods such as the ones enumerated above, the only explanations a user can get look like The item is close in the latent vector space to other items you liked, or Resources which have a value larger than 0.412 on dimension 108 usually have the property. Compared to the original Semantic Web vision, which foresees maximal interpretability of data and justi cations, these methods exist at the opposite end of the design space, i.e., they are as non-semantic as can be. Hence, we argue for the need of semantic embeddings, which preserve the interpretability. 2
As a semantic embedding, we understand an embedding space where each dimension can be assigned a human interpretable meaning. For example, for cities, such dimensions could be size or economic wealth. In terms of the underlying knowledge graph, they can map to elementary properties (e.g., population) or complex constructs (e.g., number of headquarter relations from companies). 2.1
One possible approach to solve this issue could be to learn a semantic interpretation for each dimension a posteriori, i.e., training a symbolic regression model to predict the value of each dimension based on interpretable features extracted from the knowledge graph. In theory, this works on any possible embedding, but there is no guarantee that such an approach would actually yield a meaningful model, and, depending on the task and type of model, it would still not be a trivial task to exploit the model for coming up with a semantic explanation. 2.2
Another possible approach is deriving embedding dimensions from patterns observed in a knowledge graph. While the design space for embedding construction is arbitrarily large { i.e., each dimension can be any arbitrary function { patternbased embeddings would work di erently. They guide the process of computing an embedding by restricting that search space so that each dimension corresponds to a pattern in the data, or combinations thereof. This allows tracking back the origin of a dimension to the original pattern(s) from which it was derived, and thereby fosters the interpretation of embedding dimensions.
We propose to rst learn a set of universal patterns that hold for a knowledge
graph, e.g., by learning graph patterns [
However, at this stage, the embedding space might still be high-dimensional and sparse, because it might take a lot of patterns to describe a knowledge graph. Therefore, a second step is required for compacting the embedding space. This could be achieved, e.g., by identifying patterns that are completely or almost mutually exclusive, and combining them in compacted dimensions. For example, only movies may have Oscar-winning actors, whereas only cities may be located in Europe (but movies have no location, and cities have no actors). Therefore, we could de ne an embedding dimension combining those two patterns, exposing di erent semantics depending on the type of entity at hand. 3
While some works on pattern induction in RDF graphs exist, it is not trivial to assume that each pattern is also a good candidate for an embedding dimension (or a partial one). Therefore, we need to develop heuristics for identifying patterns that are suitable as latent features, and ideally adapt the pattern learners so that they focus on nding such patterns.
Since embedding spaces should be continuous, another challenge is to nd patterns that are not just binary (e.g., an instance has a type or not), but continuous in a sense that they can be ful lled to a certain extent. Here, involving numerical and date-valued literals would also be desired, since they are often neglected in current embedding approaches, but contain a lot of useful semantics.
Finally, the dimensionality reduction of the pattern-based embeddings also needs to be carefully designed. We can assume that compacting will ultimately lead to some information loss, but it may even lead to false information. If we, as in the example above, combine a lot of features for movies and cities, this will ultimately render movies and cities to be indistinguishable in the vector space. 4
While embeddings have been proven to yield superior quantitative performance on many tasks, a vector space embedding is a very non-semantic representation of a knowledge graph. Hence, we consider semantic embeddings as a research (a) DBpedia vectors (b) Wikidata vectors direction with a lot of potential. We assume that since semantic embeddings are more constrained than generic embeddings, they will not reach their full quantitative performance. We rather see a continuous design space for knowledge graph embeddings with a trade-o between interpretability and quantitative performance.