One of the important problems of semantic web is checking whether two datasets describe the same quantity. The existing solution to this problem is to use these datasets' ontologies to deduce that these datasets indeed represent the same quantity. However, even when ontologies seem to confirm the identify of the two corresponding quantities, it is still possible that in reality, we deal with somewhat different quantities. A natural way to check the identity is to compare the numerical values of the measurement results: if they are close (within measurement errors), then most probably we deal with the same quantity, else we most probably deal with different ones. In this paper, we show how to perform this checking.
Checking whether two datasets represent the same data: formulation of the
problem. In the semantic web, data are often encoded in Resource Description
Framework (RDF) [
In general, an RDF-based scientific dataset can be viewed as a (large) graph
of RDF triples. One of the hard-to-solve problems is that triples in two different
datasets using the same predicate hasGravityReading may not mean the same
thing just because the predicates have the same name. One way to check this is
to use semantics, i.e., to specify the meanings of the terms used in both datasets
by an appropriate ontology, and then use reasoning to verify that the
meaning of the terms is indeed the same. In the gravity example, we conclude that
the predicate hasGravityReading has the same meaning in both datasets if in
both datasets, this meaning coincides with sweet:hasGravityReading, the
meaning of this term in one of the the Semantic Web for Earth and Environmental
Terminology (SWEET) ontologies [
The question is: Based on the semantically annotated measurement results and the known information about the measurement uncertainty, how can we use the uncertainty information to either reinforce or question whether two datasets namely representing the same data may not be the same data.
Probabilistic approach to measurement uncertainty. To answer the above question, we must start by analyzing how the measurement uncertainty is represented. In this paper, we consider the traditional probabilistic way of describing measurement uncertainty.
In the engineering and scientific practice, we usually assume that for each measuring instrument, we know the probability distribution of different values of measurement error Δx!i d=ef x!i − xi. This assumption is often reasonable, since we can calibrate each measuring instrument by comparing the results of this measuring instrument with the results of a “standard” (much more accurate) one. The differences between the corresponding measurement results form the sample from which we can extract the desired distribution.
Often, after the calibration, it turns out that the tested measuring instrument is somewhat biased in the sense that the mean value of the measurement error is different from 0. In such cases, the instrument is usually re-calibrated – by subtracting this bias (mean) from all the measurement results – to make sure that the mean is 0. Thus, without losing generality, we can also assume that the mean value of the measurement error is 0: E[Δx!i] = 0.
The degree to which the measured value x!i differs from the actual value xi
is usually measured by the standard deviation σi! d=ef !E[(Δx!i)2].
Gaussian distribution: justification. The measurement error is usually caused by
a large number of different independent factors. It is known that under certain
reasonable conditions, the joint effect of a large number of small independent
factors has a probability distribution which is close to Gaussian; the corresponding
results (Central Limit Theorems) are the main reason why Gaussian (normal)
distribution is indeed widely spread in practice [
Towards a solution. We do not know the actual values xi, we only know the measurement results x!i and x!i! from the two datasets. For each i, the difference between these measurement results can be described in terms of the measurement errors: Δxi d=ef x!i − x!i! = (x!i − xi) − (x!i! − xi) = Δx!i − Δx!i!. It is reasonable to Maximum Entropy in Support of Semantically Annotated Datasets assume that this difference is also normally distributed. Since the mean values of Δx!i and Δx!i! are zeros, the mean value of their difference Δxi is also 0, so it is sufficient to find the standard deviation σi = √Vi of Δxi. In general, for the sum of two Gaussian variables, we have σi2 = (σi!)2 + (σi!!)2 + 2ri · σi! · σi!!, where ri = E[Δx!i · Δx!i!] is the correlation between the i-th measurement errors. It is known σi! · σ!!
i
that the correlation ri can take all possible values from the interval [
First idea: assume independence. A usual practical approach to situations in which we have no information about possible correlations is to assume that the measurement errors are independent.
A possible (somewhat informal) justification of this assumption is as follows.
Each correlation ri can take any value from the interval [
We have no information why some values are more reasonable than others,
whether non-negative correlation is more probable or non-positive correlation
is more probable. Thus, our information is invariant with respect to the change
ri → −ri, and hence, the selected correlation value ri must be invariant w.r.t. the
same transformation. Thus, we must have ri = −ri, thence ri = 0. A somewhat
more formal justification of this selection can be obtained from the Maximum
Entropy approach; see, e.g., [
Once we know the values, we can use the χ2 criterion (see, e.g., [
On the other hand, there is another possibility: that the two datasets do describe the same quantity, but the measurement errors are indeed correlated. An alternative idea: worst-case estimations. If the above inequality holds for some values σi, then it holds for larger values σi as well. To take into account the possibility of correlations, we should only reject the similarity hypothesis when the above inequality does not hold even for the largest possible values σi. Since |ri| ≤ 1, we have (σi)2 ≤ Vi d=ef (σi!)2 + (σi!!)2 + 2σi! · σi!!. The value σ!!
i Vi is attained for Δx!i! = − σ! · Δx!i. So, the largest possible value of σi2 is i equal to Vi. One can easily check that Vi = (σi! + σ!!)2. Thus, in this case, if i "n # Δxi $2 ≤ χ2n,α, then we conclude that the two datasets indeed describe i=1 σi! + σi!! the same quantity. If this inequality is not satisfied, then most probably, the datasets describe somewhat different quantities. χ2n,α.
Conclusion. Based on the semantically annotated measurement results and the known information about the measurement uncertainty, how can we use the uncertainty information to either reinforce or question whether two datasets namely representing the same data may not be the same data?
We assume the some values from the two datasets contain the results of measuring the same quantity at the same locations and/or moments of time. Let n denote the total number of such measurements, let x!1, . . . , x!n denote the corresponding results from the first dataset, and let x!1!, . . . , x!n! denote the measurement results from the second dataset. We assume that we know the standard deviations σi! and σi!! of these measurements, and that we have no information about possible correlation between the corresponding measurement errors. In this case, we apply the Maximum Entropy approach, and conclude that if "n (Δxi)2
i=1 (σi!)2 + (σi!!)2 ≤ χ2n,α, where χ2n,α ≈ n is the value of the χ2-criterion for the desired certainty α, then this reinforces the original conclusion that the two datasets represent the same data. If the above inequality is not satisfied, then we conclude that either the two datasets represent different data (or, alternatively, that the measurement uncertainty values σi! and σi!! are underestimated).
If we have reasons to suspect that the measurement errors corresponding to two databases may be correlated, then can be more cautious and reinforce the original conclusion even when a weaker inequality is satisfied: i"=n1 # σi!Δ+xσii!! $2 ≤ Acknowledgments. This work was partly supported by NSF grant HRD0734825 and by NIH Grant 1 T36 GM078000-01. The authors are thankful to the anonymous referees for valuable suggestions.