An increasing amount of data is published and consumed on the Web according to the Linked Open Data (LOD) paradigm. In such scenario, capturing the age of data can provide insight about their validity under the hypothesis that more up-to-date data is, more likely is to be true. In this paper we present a model and a framework for assessing the currency of the data represented in one of the most important LOD datasets, DBpedia. Existing currency metrics are based on the notion of date of last modi cation, but often such information is not explicitly provided by data producers. The proposed framework extrapolates such temporal metadata from time-labeled revisions of Wikipedia pages (from which data has been extracted). Experimental results demonstrate the usefulness of the framework and the e ectiveness of the currency evaluation model to provide a reliable indicator of the validity of facts represented in DBpedia.
Linked Open Data (LOD) can be seen as a Web-scale knowledge base consisting
of named resources described and interlinked by means of RDF statements [
The problems of dealing with change, providing up-to-date information,
and making users aware of how much up-to-date information is have been
3 http://dbpedia.org/
acknowledged in the Semantic Web. A resource versioning mechanism for linked
data has been proposed [
Data currency (currency, for short) is a quality dimension proposed in the data
quality domain to describe and measure the age of (relational) data, and the speed
at which a system is capable to record changes occurring in the real-world [
DBpedia starting from the corresponding Wikipedia pages; { an evaluation method for estimating the quality of proposed currency measures based on the validity of facts.
The paper is organized as follows: Section 2 discusses related work on the assessment of time-related data quality dimensions; Section 3 introduce the de nitions adopted in the paper, and the metrics for measuring and evaluating currency. Our framework for assessing currency of DBpedia documents is presented in Section 4. Section 5 shows results of our experimentation on DBpedia and, in Section 6, we draw conclusions and future work. 2
Data currency is a quality dimension well known in the literature of information
quality [
SIEVE is a framework for evaluating quality of LOD, proposed in the context
of, and applied to, a data fusion scenario [
A recent technique able to assess the currency of facts and documents is
presented in [
Other time-related metadata de ne the temporal validity of RDF statements
according to the Temporal RDF model [
Linked Open Data describes resources identi ed by HTTP URIs by representing
their properties and links to other resources using the RDF language. Given an
in nite set U of URIs, an in nite set B of blank nodes, and an in nite set L of
literals, a statement < s; p; o >2 (U [ B) U (U [ B [ L) is called an RDF
triple. As the use of blank nodes is discouraged for LOD [
Most resources described in the LOD Cloud represent real-world objects
(e.g., soccer players, places or teams); we use the term entities as a short form for
named individuals as de ned in the OWL 2 speci cation. According to the LOD
principles [
Let E = fe1; e2; : : : ; eng be a set of entities in a dataset G and D = fde1 ; de2 ; : : : ; den g be a set of entity documents. Let S be a non-structured or semi-structured web source in the web that consists of a set of web pages S = fp1; p2; : : : ; png. We assume that each entity ei represented by an entity document dei has a corresponding web page pi 2 S; in the following we use the notation pe to refer to the web page that describes an entity e. Entities descriptions and source web pages change over time; di erent versions of a web page corresponding to an entity exist. Formally, a version of a web page p can be represented as a triple v =< id; p; t >, where id is the version identi er, p is the target page, and t is a time point that represents the time when the version has been published. 3.2
Starting from the de nition of currency proposed for relational databases [
The above de nition can be easily extended to RDF documents. Let cT ime and lmT ime(de) be respectively the current time (the assessment time) and the last modi cation time of an entity document de. We rst de ne the age measure of an entity document, based on the above informal de nition of data currency. The age(de) : D ! [0 + 1] of an entity document de, with e 2 E, can be measured by considering the time of the last modi cation of the entity document according to the following formula:
age(de) = cT ime lmT ime(de) We introduce a measure called age-based currency, which depends on the document age and returns normalized values in the range [0; 1] that are higher when the document are more up-to-date and lower when documents are old. The age-based currency (de) : D ! [0; 1] of an entity document de is de ned as follows: where startT ime represents a time point from which currency is measured (as an example, startT ime can identify the time when rst data have been published in the LOD). Observe that this age-based currency measure does not depend on the particular nature of RDF document. In fact, the same measure can be adopted to evaluate currency of other web documents such as Wikipedia pages (and will be used to this aim in Section 5).
Age-based currency is based on age, as de ned in Equation 1, and strongly depends on the assessment time (current time). We introduce another measure, which is less sensitive to current time (current time is used only for normalizing the values returned by the measure). We take inspiration from, and adapt a de nition of currency proposed for relational databases.
De nition 2 (System-Currency). Currency refers to the speed with which the information system state is updated after the real-world system changes.
According to this de nition currency measures the temporal delay between changes in the real world and the consequent updates in the data. An ideal system of currency measure in our domain should evaluate the time elapsed between a change a ecting a real-world entity and the update of the RDF document that describes that entity. However, changes in the real-world are di cult to track because real-world is opaque to a formal analysis. Since the data under the scope of our investigation are extracted from a web source, we can de ne the system currency of an RDF document by looking at the time elapsed between the update of the web source describing an entity and the update of the correspondent RDF document.
We rst de ne the notion of system delay with respect to an entity, de ned by a function systemDelay(de) : D ! [0 + 1] as the di erence between the time of last modi cation of a web page pe and the time of last modi cation of its respective entity document de as follows: systemDelay(de) = lmT ime(pe) lmT ime(de) (3) Based on the measure of system delay, we introduce a new currency measure called system currency that returns normalized values in the interval [0; 1], which are higher when the data are more up-to-date and lower values when data are lessup-to-date with respect to the web source. The system currency (de) : E ! [0; 1] of an entity document de is de ned as: We provide an assessment framework that leverages the temporal entities extracted from Wikipedia to compute the data currency of DBpedia entity documents; the assessment framework follows three basic steps: (1) extract a document representing an entity from the DBpedia dataset, (2) estimate the last modication date of the document looking at the version history of the page that describes the entity in Wikipedia, (3) use the estimated date to compute data currency values for the entity document.
The main problems we have to face, in order to apply the currency model
described in Section 3, concerns the (un)availability of the temporal
metainformation required to compute the currency, namely the last modi cation
date of an RDF document. According to a recent empirical analysis, only 10%
of RDF documents and less than 1% of the RDF statements are estimated to
be associated with temporal meta-information [
A pseudocode of the algorithm that implements the strategy is described in Algorithm 1. The input of the algorithm is an entity e for which the currency has to be evaluated using the currency measures proposed in Section 3. To obtain the estimated last modi cation date of an entity document we need to extract its document description and its correspondent page from the web.
The description of the entity de is obtained by the function rdf F etcher(e), line 2, which reads the statements recorded in an N-Triples entity document and records them. Among all the statements in the document we keep only those that use DBpedia properties; considered documents represent relational facts and do not include typing information, links to other datasets (e.g., same as links and other links to categories); in other words we consider in entity documents statements that are more sensitive to changes.
From line 3 to line 4, the algorithm extracts the ID of the last revision of
the web page pe (i.e the infobox information) corresponding to the entity e and
builds a structured representation of pe. In order to build a structured RDF
content we need to identify properties and their values in the semi-structured
part of the document (pe). The algorithm creates RDF statements from pe by
using the same approach and mappings used to extract DBpedia statements from
Wikipedia infoboxes [
Given p and de, the algorithm nds whether the structured representation of p provided by dev matches the entity description de (see line 4-7); we use an exact match function. In case the last revision of the structured representation of p does not match to the entity document, the algorithm checks for older versions and stops only when a matching version is found. At this point, we associate the timestamp of the v version to the entity document de (line 8). In this way we compute the currency of de (see line 9-11). 5 5.1
Methodology and Metrics. When multiple metrics are de ned the problem of evaluating the quality of such dimensions arises. In this section we propose a method to evaluate the e ectiveness of data currency measures addressing entity documents extracted from semi-structured information available in web sources. Out-of-date documents may contain facts that are not valid anymore when data are consumed; intuitively a description can be considered valid when it accurately represents a real-world entity. A data currency measure that is highly correlated to the validity of fact is useful since it may provide insights about the reliability of data. We de ne two metrics, namely accuracy and completeness, to capture the intuitive notion of validity by comparison with the web source which data are extracted from. These metrics use the semi-structured content available in the web source - Wikipedia infoboxes in our case - as a Gold Standard against which entity documents are compared.
Let us assume to have a mapping function that map every Wikipedia fact
to an RDF fact. This mapping function can be the one used to extract DBpedia
facts from Wikipedia facts4; however other techniques to extract RDF facts from
semi-structured content have been proposed [
We de ne the accuracy A(de) of an RDF document de as the number of semantically accurate facts V F (de) in de divided by the total number of facts T F (de) in de: We de ne the completeness of an RDF document de as the number of semantically accurate facts V F (de) in de divided by the total number of existing facts W F (pe) in the original document pe:
A(de) =
V F (de)
T F (de) C(de) =
V F (de) W F (pe) Dataset. In order to produce an exhaustive and signi cant experimentation, we de ne a speci c subset of DBpedia entities that is a representative sample for the entire dataset. These entities belong to the Soccer Player category. The main facts describing a soccer player in the Wikipedia's infobox template are the facts of a soccer player such as his appearance, his goals or moving from one club to another, which denote evidences about the changes performed. The changes can usually vary from three to seven days, which imply a high frequency of modi cations. Furthermore, after observing for several months the modi cation set of the Wikipedia's infoboxes of the soccer players, we noticed that there is a high interest of the end-users to maintain the infoboxes up-to-date. Often due to the high frequency of changes in Wikipedia, the new modi cations are not replicated also to DBpedia. 5.2
To realize the experiments we de ne ten samples composed each one by thirty soccer players chosen randomly where each sample contains a set of entities that have a age-based currency evaluated on DBpedia de ned in a speci c interval (e.g., [0; 0:1], [0:1; 0:2], ... [0:9; 1]). 4 http://mappings.dbpedia.org (5) (6)
As provided in table 1, the results of the computations show that the Pearson's coe cients are lower than 0.75 which mean that there is no a linear relationship between currency and accuracy or currency and completeness. Even though the correlation between system currency and the other estimators is lower than 0.75, we notice that the correlation between system currency and completeness is higher than the other correlation.
In addition, we calculated the Spearman's rank correlation coe cient, which is a metric that can be used to verify if there exists a non-linear correlation between the quality metrics. Table 2 shows the Spearman's coe cient computation between the two classes of quality measures performed on the collected soccer player entities.
The evaluation results allow us to assert that there exists a non-linear correlation between system currency and completeness because Spearman's coe cient is higher than 0.75. The correlation between the harmonic mean and the system currency is low (even if it shows a value closed to the threshold), we can deduce that there exists a correlation between the two components, which is driven by completeness. Finally, the complete absence of the correlation among validity indicators and the age-based currency of the Wikipedia and DBpedia ones, con rms that the accuracy and completeness do not depend on the age of the document.
In order to gure out the behaviour of the system currency and completeness, we compute the LOWESS (locally weighted scatterplot smoothing) function. This function is able to compute the local regression for each subset of entities and to identify, for each interval of system currency, the trend of the two distributions.
Figure 1 shows the LOWESS line between system currency and completeness. As provided by the graph, the local regression cannot be represented by a well-known mathematical function (e.g., polynomial, exponential or logarithmic) because it has di erent behaviours over three system currency intervals. Notice that for system currency values greater than 0.6, the two distributions tend to increase and for system currency values up to 0.4, the metrics tend to decrease. A particular case is shown in the interval with system currency values going from 0.4 up to 0.6 where the LOWESS is constant. 4 . 0 0.2 0.4 0.6 0.8
1.0
DBpedia System Currency
In order to verify the linearity on the three entities subsets, we compute the correlation coe cients also on the identi ed system currency intervals. The results provided in table 3 show that even for the subsets there does not exist a linear correlation. In particular, there is no correlation in the central interval where the Pearson's coe cient is close to 0 and the distribution is disperse as shown in gure 1. Furthermore, the entities in the central interval show that there does not exist a non-linear correlation.
Instead, the distributions increases according to a non-linear function because Spearman's coe cient is greater than 0.75 for the entities having high system currency and the lower interval distribution decrease according to a non-linear function represented by the Spearman's correlation which is lower than -0.75. 6
Based on the experimentation we can provide several general observations about the relation between system currency and completeness. The rst one is that entities with high system currency tend to be more complete. Commonly, there is an update to the soccer player infobox each week. In particular, the two facts changing frequently in the infobox are appearances and goals associated with the current club. While these two facts change often, the fact that the soccer players moves to a new club can change at most two times a year. Thus, we can deduce that in a soccer player domain only two facts can change in a short time. High system currency implies low elapsed time between the last modi cation time of a document extracted from DBpedia and the correspondent document extracted from Wikipedia. In case the elapsed time is low, the probability that the player changes club is low too which means that only a few infobox modi cations occurred. According to the formula 6, the completeness increases if the number of facts changing in the Wikipedia infobox is small.
The second observation concerns to the behaviour of entities with low system
currency. A deep analysis of these particular entities shows that the associated
Wikipedia pages have two common characteristics: (i) low frequency of modi
cations (approximately between six and twelve months), and (ii) refer to ex-soccer
players or athletes with low popularity. The low frequency of changes in a
document, typically known as volatility [
In case of entities for which the players have low popularity, the scenario is quite di erent. The infobox modi cation for these entities implies high completeness. Information about the unpopular players can change weekly, as well as for famous players. Therefore, if a page is not frequently updated could provide incorrect information. Consequently, we can infer that DBpedia entities of unpopular soccer players with low system currency and high completeness refers to Wikipedia pages infrequently updated, therefore, could not represent current information of real world that is also re ected in DBpedia.
At the end of such deep analysis of our experiments, we can assert that: (i) our approach for measuring currency based on the estimated timestamp is e ective; (ii) there exist a non-linear correlation between system currency and completeness of entities; (iii) more the system currency of an entity is higher, more the associated DBpedia document has high completeness; and (iv) entities with low system currency, that are instances of DBpedia classes of which their information can change frequently (e.g., unpopular soccer player), are associated with Wikipedia pages that could not provide real world information.
We plan to investigate several research directions in the future; on one hand; we will extend the experiments to analyze the currency of every DBpedia entity and the correlation to the validity of their respective documents; on the other hand, we will study the correlation between volatility and currency to improve the estimation of the validity of facts. The work presented in the paper is supported in part by the EU FP7 project COMSODE - Components Supporting the Open Data Exploitation (under contract number FP7-ICT-611358).