Freebase.
LOD reference
alignments
OAEI 2011.5
conference track,
multifarm dataset
Concepts from
triples of Billion
Triple Challenge
dataset, expert
evaluation
Unlike the Ontology Alignment Evaluation Initiative (OAEI), no standard reference
alignment exists for LOD ontologies. Researchers [
4
] had experts develop a
benchmark, the LOD reference alignments between ontology schema pairs taken from
eight LOD ontologies: AKT Reference (A), BBC program(B), DBpedia (D),
FOAF(F), Geonames(G), Music(M), SIOC (S), and the Semantic Web Conference
(W) because of their substantial LOD coverage domain diversity, and publicly
available schemas. Experts produced both subclass and equivalence mappings
between the pairs listed in Table 2. BLOOMS and AM are compared in the last two
columns [
3
] since WM or HCM produce only equivalence mappings and few of these
% inferable
# mappings
exist in the LOD reference alignments. Both BLOOMS and AM use inferencing to
produce some subclass mappings, BLOOMS using post-processing with the Jena
reasoner and AM using its own inferencing techniques. To understand this influence,
we performed an analysis on the LOD reference alignment for each pair to see the
percentage of its mappings inferrable from its equivalence mappings, given in column
1. The * for M, B indicates an analysis was not possible since many BBC concepts
could not be found directly in its file or even when opening the file using Protégé.
BLOOMS has better recall except for F,D and G,D. F,D has 87% inferable mappings
from its three equivalence relations. AM’s use of other LOD ontologies and WordNet
contributes to finding more correct mappings. For the G,D pair BLOOMS does not
find the only equivalence relation SpatialThing = Place so that Jena cannot produce
any of the inferable mappings. AM finds this mapping, likely from the comment field
for SpatialThing including the word ‘places.’ AM finds 68% (recall) of the reference
alignment mappings, very close to the 71% inferable mappings. Of the five remaining
pairs, AM has better precision for M,D with the smallest percentage of inferable
mappings. BLOOMS with Wikipedia finds more correct mappings since very few are
from inferable relations. AM’s lower recall corresponds with fewer inferable
relations, but those that it does find are more likely correct with its 0.62 precision