In the context of Linked Open Data (LOD) [
Fig. 1 presents an example of relative incompleteness in the representation of movie entity \Holy Smoke!" in DBpedia and Freebase. In this example, the di erence between the actor sets indicates the \Movie.Actor" property might be incomplete. As we do not know the exact complete set of actors and the noise observed in linked datasets interferes completeness estimation, we call this phenomenon relative incompleteness. If we follow the \Movie.Cinematographer" link in the data graphs of the two datasets, we can observe further relative incompleteness in its \birthPlace" property.
In this paper we discuss incompleteness related measures that can be obtained by pairwise dataset comparisons exploiting entity co-resolution across these datasets and apply these measures in a case study. The basis for the proposed measures is the assumption that dataset speci c di erences in representation of equivalent entities, and in particular the values of multi-value properties, can provide valuable insights into the relative incompleteness of these datasets. To facilitate discovery of relative incompleteness, we assume correct schema mapping and make use of known equivalent entities. In the context of LOD, absolute incompleteness is di cult to judge, as it is di cult to obtain the ground truth of absolute completeness. Therefore, we choose to estimate relative incompleteness of the properties by following paths of limited lengths in the data graphs.
By ith-Order property, we mean a property that can be reached from the target entity through a path of length i in the data graph. For instance, \Movie.Actor" is a 1st Order property in Fig. 1, while \Movie.Cinematograher. Name" is a 2nd Order property. Then, we de ne ith-Order Value Incompleteness as follows:
ith-Order Value Incompleteness (Dx, Dy, P ) between the pair of datasets Dx, Dy with respect to a ith-Order multi-value property P is the proportion of entities in Dx and Dy having di erent values in P .
As P is a multi-value property, the di erence on P usually indicates that at least one of the datasets does not provide su cient information on P . In Fig. 1, we observe a 2nd Order Value Incompleteness in the \Movie.Cinematographer .birthPlace" property. To determine equivalent values, we rely on direct value comparisons and identity links.
Considering the LOD cloud as a large interlinked knowledge graph, relative
incompleteness of data across di erent datasets is a crucial and often
underinvestigated issue. Relative incompleteness can result e.g. from extraction errors,
lack of source maintenance [
Datasets and Schemas: We used the latest version of three datasets from LOD - LinkedMDB (LMDB), DBpedia and Freebase. The LMDB dataset contains eight concepts about movies, such as Movie, Actor and Country, and more than 200,000 records. The DBpedia and Freebase datasets contain around 150,000 and 1,000,000 movie records respectively. To perform the study, we randomly selected 200 Movie and 200 Actor entities shared between these datasets. To establish the relationship of entities across the three datasets, we obtained the existing interlinking information (i.e., the owl:sameAs predicate) of the Movie entities across all the three datasets as well as the Actor entities in the DBpedia and Freebase. We manually established schema mappings between the Movie and Actor concepts and their properties among the datasets.
Evaluation Results: We computed the 1st and 2nd Order Value Incompleteness for each property in each pair of datasets. Table 1 presents an aggregated 1st and 2nd Order Value Incompleteness results for the Movie and Actor entities. In this result, if there is a single property that is incomplete on an entity, we would count this entity as incomplete. As we can see in Table 1, the relative incompleteness in the DBpedia/Freebase pair reaches 100% in the rst order and 89% in the second order, meaning that all the Movie entities in the datasets are a ected by incompleteness issues. The overall 1st Order Incompleteness of the Movie entities in the other dataset pairs is also pretty high, e.g., 70% for LMDB/Freebase and 56% for LMDB/DBpedia.
Table 2 presents the details of the evaluation for each property. As we can see in the Table 2, the highest relative incompleteness among all datasets is observed in the DBpedia/Freebase pair on the property Actor, whose incompleteness is 73%. This is because DBpedia tends to include only key people in a movie, whereas Freebase tends include more complete actor lists. For example, for the movie \Stand by Me", DBpedia lists only ve actors: Wil Wheaton, Kiefer Sutherland, River Phoenix, Corey Feldman, and Jerry O'Connell. The \starring" property of Freebase includes many more actor names such as Gary Riley, Bradley Gregg, Frances Lee McCain, etc. We also observed that the \starring" property sometimes mixes actor and character names in a movie. For example, for \Stand by Me", it includes characters Teddy Duchamp and Waitress. Regarding the LMDB/Freebase, the incompleteness on the properties Producer, Release Date and Actor are 30%, 26% and 19% respectively. LMDB/DBpedia shows a similar distribution, i.e. 29%, 11% and 15%, on the same properties.
An exemplary evaluation performed on the Actor entities indicates a similar tendency as for the Movie type, with 76% incompleteness in the rst order in the DBpedia/Freebase pair. While Actor entities always agree on the names and very often on the birth dates (which makes us think that the existing interlinking of Actor entities was established using these properties), they frequently disagree on the property of birthPlace. This is because the values of property birthPlace from DBpedia are much more detailed than those from Freebase. DBpedia typically includes a country name in an address, whereas Freebase does not. For example, the place of birth of the person \Len Wiseman" from DBpedia is \Fremont, California, United States", while that from Freebase is \Fremont, California". As a result, we observe an increased incompleteness in the property birthPlace. Interestingly, the property deathPlace is much less incomplete, as most actors listed in these databases are still alive (we regard null values as incomparable). 3
In this paper we presented measures to automatically evaluate relative incompleteness in linked datasets and applied these measures in a case study. From the experiment performed using three linked datasets in the movie domain we can conclude that incompleteness is a very common phenomenon in these datasets, and its number increases signi cantly with increasing order, i.e. increase of investigated entity neighbourhood. The main causes of relative incompleteness observed during our experiment are due to di erent interpretations of properties in the datasets. Our method of classi cation and identi cation of quality issues provides not only insights into the level of agreement between datasets but also into the overall quality of datasets. In future work we intend to extend these approaches to infer knowledge about the correctness and agreement of schemas.
This work was partially funded by the NSFC Project No. 61272138, ERC under ALEXANDRIA (ERC 339233), the COST Action IC1302 (KEYSTONE) and 973 Program Project of China (Grant Nr.: 2012CB316205).