Knowledge graphs constitute a native neuro-symbolic experimental setting due to their logic foundations, which motivates the development of neuro-symbolic approaches for Link Prediction (LP). Since current LP reference datasets seldom involves ontological knowledge, benchmarking such approaches is dificult. That is why, starting from the widely accepted datasets DB100k, NELL-995 and YAGO3-10, we semantically enriched them with ontological knowledge, namely class hierarchy and relation signatures (domains and ranges), and inferred new entity type assertions to create DB100k+, NELL-995+ and YAGO3-10+. We also present a generic masking script to generate sub-graphs with variable proportions of triples with signed/partially signed (no domain or no range)/unsigned (no domain and no range) relations, to evaluate the impact of semantic information on learning performance.
A Knowledge Graph (KG) is a set of triples ⊆ × × where is a set of entities and a set of relations or predicates. Each triple (, , ) instantiates a relation between two entities (its subject and object), and represents a fact about a modeled world, e.g., (Paris, capitalOf, France). Besides triples, KGs can also be associated with ontologies, i.e., a formal representations of the vocabulary used to model a domain. In such ontologies, axioms include, among others, entity typing with classes, the class hierarchy, and the signature (domain and/or range) of predicates. These specific ontology axioms will be referred in the remainder of this document as semantic information. We leave the consideration of other axioms for future work.
KGs are inherently incomplete because of their
semiautomatic construction process, and this flaw is to be
considered all along their life cycle [
To fill this gap, we propose three new datasets, namely DB100k+, NELL-955+ and YAGO3-10+, which extend the original well-known LP datasets with their related semantic information. We also present a generic masking script to generate sub-graphs with variable proportions of triples with fully signed (domain and range)/partially signed (no domain or no range)/unsigned (no domain and no range) relations, in order to evaluate the impact of semantic information availability on learning performance. The datasets, the source code to create them and the masking script are available online under the LGPL2.1 License on GitHub1 and Zenodo2.
The rest of this paper is organized as follows: Sections 2-4
respectively present NELL-995+, YAGO3-10+ and DB100k+;
Section 5 presents the masking algorithm; Section 6
concludes.
2. NELL-995+
Never Ending Language Learning (NELL)3 is a system
running continuously since 2010 to extract facts from text of
millions of web pages and to improve its own reading
competence over time. The NELL-995 [
NELL-995+ is the semantically enriched version of
NELL995 that we constructed using the original 995th iteration of
NELL and the related 995th iteration of NELL ontology5. All
of the 75,492 entities in the original dataset NELL-995 have
declared types. We removed type assertions using the
general class concept:everypromotedthing to avoid too
general and thus potentially useless statements. We added
350,723 new entity type assertions inferred based on the
predicate domains and ranges and the class hierarchy
provided in the NELL ontology.
1https://github.com/Wimmics/semantically-enriched-link-prediction-datasets
2https://doi.org/10.5281/zenodo.15834518
3https://nell-ld.telecom-st-etienne.fr/
4http://rtw.ml.cmu.edu/resources/results/08m/NELL.08m.995.esv.csv.
gz
5http://rtw.ml.cmu.edu/resources/results/08m/NELL.08m.995.
ontology.csv.gz
3. YAGO3-10+
The Yet Another Great Ontology (YAGO) Knowledge Base6
is a KG automatically built from Wikipedia, WordNet and
GeoNames. YAGO3-10 [
YAGO3-10+ is the semantically enriched version of
YAGO3-10 that we constructed as follows. We considered
the entity type assertions in the yagoTypes.ttl file from
the YAGO3 archive8. Out of the 123,182 entities occurring
in this file, 121,009 have declared types (and 2,173 are
untyped). We inferred 1,806,190 new entity type assertions
based on the class hierarchy in the yagoTaxonomy.ttl file
and relation domains and ranges in the yagoSchema.ttl
ifle. Out of the 37 relations occurring in the assertions, 28
have both a domain and a range declared in YAGO schema
and 9 relations have only a domain and no range.
4. DB100k+
DBpedia9 is a cross-domain crowdsourced KG built from the
infobox of each Wikipedia article. DB100k [
DB100k+ is the semantically enriched version of DB100k that we constructed as follows. Entities in DB100k follow the Wikidata notation while entity types in the DBpedia dump follow the DBpedia notation11. Hence, we linked the DB100k entities to DBpedia entities using the archived interlanguage alignment12. This processing revealed that, out of the 99,604 entities in DB100k, 92,558 have a type and that 8 entities are obviously not Wikidata entities (e.g. index.html or ?autoplay=true). For the sake of compatibility with the original DB100k dataset and to avoid the risk of introducing wrong or noisy information, these entities were left untyped. In order to reduce noise, only entity type assertions using classes within the DBpedia ontology namespace were kept (e.g. assertions using class owl:Thing were removed). For semantic enrichment, predicate domains and ranges were obtained from the DBpedia ontology version of Oct. 201613). 6https://yago-knowledge.org/ 7https://yago-knowledge.org/downloads/yago-3 8https://yago-knowledge.org/data/yago3/yago-3.0.2-turtle-simple.7z 9https://www.dbpedia.org/ 10https://downloads.dbpedia.org/2016-10/core/ 11https://downloads.dbpedia.org/2016-10/core/instance_types_en.ttl.
bz2 12https://downloads.dbpedia.org/2016-10/core/interlanguage_links_ chapters_en.ttl.bz2 13https://downloads.dbpedia.org/2016-10/dbpedia_2016-10.owl Out of the 470 relations used in DB100k, 278 have a declared domain and range, 33 have neither domain nor range, 76 have a declared domain and no range, and 83 have a declared range and no domain. We added 216,738 new entity type assertions to DB100k by inferring types based solely on the class hierarchy. We decided not to infer types based on predicate domains and ranges since they produce noisy data on DB100k (e.g. England of type Country, but also Horse Race and Music Genre).
We designed and developed a generic masking algorithm to generate variations of the semantically enriched datasets, considering input target proportions of triples with signed, partially signed or unsigned relations. It is a greedy algorithm that: (1) represents each solution as a dictionary where each key is a predicate, and each value is a dictionary with two ’domain’ and ’range’ boolean keys (true meaning retained, false meaning masked); (2) evaluates the fitness of a solution as the sum of the diferences between the target proportions and the obtained proportions in the train, validation set, and test sets; (3) aims at improving the solution at each iteration by masking a domain or a range such that the fitness is increased for the best; (4) stops when a solution cannot be enhanced this way.
To illustrate, we created a variant of NELL-995+ targeting the following proportions: 10% of triples with fully signed relations (a domain and a range), 30% with relations having a domain and no range, 10% with relations having a range and no domain, and 50% with unsigned relations. Our algorithm removed 180 of the 380 domain or range declarations and achieved the following triple proportions: 9.97%, 29.99%, 10.00% and 50.04%.
We created three KG-based datasets DB100k+, NELL-995+ and YAGO3-10+, by semantically enriching link prediction reference datasets. Table 1 summarizes the statistics of the number of relations and triples in each dataset, with additional statistics per split available on GitHub and Zenodo. We also provide a masking script to create subgraphs with variable proportions of triples with signed, partially signed, or unsigned relations. By extending LP reference datasets with ontological knowledge, these datasets will support the development and evaluation of neuro-symbolic approaches that take into account such knowledge, and ease their comparison with other approaches. For instance, as future work, we will use these datasets to evaluate the impact of type/domain/range information on LP performance.
The author(s) have not employed any Generative AI tools.