Public authorities are increasingly sharing sets of open data. These data are often preprocessed (e.g. smoothened, aggregated) to avoid to expose sensible data, while trying to preserve their reliability. We present two procedures for tackling the lack of methods for measuring the open data reliability. The rst procedure is based on a comparison between open and closed data, and the second derives reliability estimates from the analysis of open data only. We evaluate these two procedures over data from the data.police.uk website and from the Hampshire Police Constabulary in the UK. With the rst procedure we show that the open data reliability is high despite preprocessing, while with the second one we show how it is possible to achieve interesting results concerning the open data reliability estimation when analyzing open data alone.
Open Government Data are valuable for boosting the economy, enhancing the transparency of public administration and empowering the citizens. These data are often sensitive and so need to be preprocessed for privacy reasons. In the paper, we refer to the public Open Government Data as \open data" and to the original data as \closed data".
Di erent sources expose open data in di erent manners. For example, Crime
Reports [
The analysis of open data is spreading, led by the Open Data Institute
(http://www.theodi.org) and others. For instance, Koch-Weser [
The rest of this paper is structured as follows: Section 2 describes a procedure for measuring the reliability of open data given closed data and a case study implementation; Section 3 presents a procedure for analyzing open data and a case study; lastly, Section 4 provides nal discussion. 2
Procedure for Comparing Closed and Open Data
The UK Police Home O ce aggregates (i.e., presents coarsely) and smoothens
(introduces some small error) the open data for privacy reasons. We represent
the open data provenance with the PROV Ontology [
Aggregation
Smoothing Closed Data
Open Data
Select the relevant data Closed data might be spurious, so we select the data items that are relevant for our analyses. The selection of the data might involve the temporal aspect (i.e. only data referring to the relevant period are considered), their geographical location (select only the data regarding the area of interest), or other constraints and their combination; Roll up categorical data There exists a hierarchy of categories because each level is available to a di erent audience: open data are presented coarsely to the citizens, while closed data are ne grained. We bring the categorization to the same level, hence bringing the closed data to the same level as the open data.
Compare the corresponding counts Di erent measures are possible, because
the di erence between datasets can be considered from di erent points of
view: relative, absolute, etc.. For instance, the ratio of the correct items over
the total amount or the Wilcoxon signed-rank test [
Case Study 1 We compare a set of crime counts per categories grouped per neighbourhood and month from data.police.uk with a limited set (30,436 relevant entries) of corresponding closed data from the Hampshire Constabulary by implementing the procedure above as follows: Data Selection Select the data for the relevant months and geographical area.
In this latter case, we load the KML le describing the Hampshire
Constabulary area using the maptools library [
Data Comparison We average the result of the Wilcoxon signed-rank test applied per neighbourhood, to compare open and aggregated closed data.
For each neighbourhood we compute a Wilcoxon signed-rank test to check the signi cance of the di erence between open and closed data and we average the outcomes (see Table 1a). We compute the test on the di erences of the two counts (open and closed data) to check whether the estimated average of the distribution of the di erences is zero (that is, the two distributions are statistically equivalent) or not.
The results at our disposal are limited, since we could analyze only two
complete months. Still, we can say that smoothing, in these datasets, introduces
a small but signi cant error. The highest error average (2.75) occurs with the
entry with the highest error variance: this suggests that the higher error is due
to a few, sparse elements, and not to the majority of the items. To prove this,
we checked the error distribution among the entities and we reported the results
in Table 1b. A 2 test [
Procedure for Analyzing Open Data We propose here a procedure for analyzing open data alone, to be used when closed data are not available, which provides weaker but still useful results, (b) Percentage of items in each open dataset presenting a relative error of at most 0%, 25%, 50%, 75% and 100% with respect to the corresponding closed data item.
Month % of Entries per Relative Error
0% 25% 50% 75% 100% month 1 35% 44% 65% 74% 96% month 2 34% 43% 57% 65% 91% compared to the previous one. It compares each dataset with the consecutive one, measures their similarity and pinpoints the occurrence of possible reliability changes based on variations of similarity over time. We use a new similarity measure for comparing datasets, that aggregates di erent similarity \tests" performed on couples of datasets. Given two datasets d1 and d2, their similarity is computed as follows:
sim(d1; d2) = avg(t1(d1; d2); : : : ; tn(d1; d2)) where avg aggregates the results of n similarity tests ti, with i 2 f1 : : : ng. We propose the following families of tests, although we are not restricted to them: Statistical test Check with a statistical test (e.g. Wilcoxon signed-rank test) if the data are drawn from signi cantly di erent distributions.
Model Comparison test Build a model (e.g. linear regression [
Acknowledgements This work is supported in part under SOCIAM: The Theory and Practice of Social Machines; the SOCIAM Project is funded by the UK Engineering and Physical Sciences Research Council (EPSRC) under grant number EP/J017728/1.