The paper describes the first project that Istat, the Italian National Institute of Statistics, has set up for publishing data in LOD on its own SPARQL endpoint. Both the publication process and the underlying technical architecture are described with a focus on design choices (e.g. the adoption of RDF Data Cube Vocabulary for multidimensional data representation and the usage of R2RML for mapping rules specification) and on the adopted technological platform.
National Statistical Institutes (NSIs) play an important role as data producers, by
publishing Official Statistics in the service of citizens and policy-makers. Statistical
production processes are indeed intended to produce “data” as their final output. An
important phase of such processes is the dissemination phase, dedicated to the design
and development of publication tools aimed to reach the widest possible range of
users. In this respect, the Linked Data paradigm [
The increasing Internet penetration and the subsequent proliferation of electronic data
exchanges, resulted in the study of new protocols, models and formats specific for the
statistical data domain. Ten years work on searching and defining “a common
language and a common perception of the structure of classifications and the links
between them” has originated the Neuchâtel model, the first relevant standard for data
and metadata representation [
In recent years, further standard statistical data models have emerged, namely GSIM
and SDMX. GSIM (Generic Statistical Information Model) [
With respect to data standardization activities, NSIs are moving towards two directions; on the one hand they are promoting the development of models that are “adhoc” to the statistical domain, like SDMX; on the other hand, they are instead working for the purpose of extending the base of data users, going also beyond the established statistical users. The approach to achieve this latter goal involves the use of worldwide standard models and formats for data sharing, mainly Semantic Web standards. Indeed, several NSIs have set up a SPARQL endpoint for publishing their data in Linked Open Data (LOD) format, including INSEE (France) [20], EL.STAT (Greece) [19] and CSO (Ireland) [21].
In this paper, we describe a project by Istat (the Italian National Institute of Statistics) aimed to publish indicators of the 2011 Population Census as Linked Open Data. The project, named Census-LOD, is a flagship project that also has the ambitious goal to pave the way for the introduction of LOD as a stable channel for Official Statistics dissemination.
The remaining of the paper is structured as follows. After providing an overview of the dissemination activities related to the latest Italian Population and Housing Census in Section 2, we present the main contribution of the paper in Section 3 by describing the process followed to publish Census data in LOD. In Section 4, we briefly discuss the certified publication of Istat data, while in Section 5 we conclude with final remarks. 2
The 15th Italian Population and Housing Census: an
The data collection phase of the 15th Italian General Population and Housing Census
begun in October 2011 and ended in early 2012. The first outputs were disseminated
during 2012 (namely, provisional data and legal population). After the dissemination
of the legal population, the correction and validation phase for data relating to
individuals, households and buildings started. A large amount of statistical analyses are
being available since May 2014 through the I.Stat system, the Istat Web warehouse
[
The reports disseminated via I.Stat have municipalities as the lowest detail of the territorial dimension. As for the previous Census editions, Istat plans to publish a defined subset of indicators also at the territorial level of Census section, which is a sub-municipality level.
Currently, the Italian territory is divided into 400000 Census sections, so this level of publication concerns a huge amount of data that, in the past Census editions, were distributed as CSV or MS Excel files. A provisional version of these files is already available on the Web with a list of 43 indicators related to the population measures. The number of published indicators is expected to grow to around 200 by December 2014 with the inclusion of population, households, dwellings and buildings. 3
The Census-LOD project aims to make available the Census data at the Census section level in LOD.
The Istat dissemination architecture is so far based mainly on SDMX; in particular,
the machine-to-machine data exchange is implemented by a set of Web services that
return SDMX data (the system is collectively called SEP - Single Exit Point).
Following the guidelines for the enhancement of the quality of public information [
Later, the possibility of publishing data directly according to LOD paradigm, was taken into consideration independently from the SDMX-based publishing process. In particular, with the Census-LOD project, we decided to develop a platform for the LOD dissemination of: (i) the Population Census indicators at the territorial level of Census sections, linked to (ii) the “Territory” dataset describing the Italian territorial structure including regions, provinces, municipalities and Census sections and to (iii) the “Geonames” ontology that is an international ontology for the description of physical and other territorial characteristics.
In more details, the Census-LOD project consisted of three main phases, specifically: 1. Domain analysis and ontology definition; 2. Triples generation; 3. LOD publishing.
In the following sections, we describe the above phases in details. 3.1
Domain Analysis and Ontology Definition Phase As a first step of the project, we made an in-depth domain analysis, resulting in a conceptual model of the domain of interest.
We started analyzing two datasets, named Censpop and Territory; such datasets were used to be published in previous Census editions as CSV and XLS files. The Censpop dataset describes the population Census indicators at the territorial level of Census section. Table 1 shows an excerpt of one of the Censpop files; only three of the total amount of about 200 indicators are shown, namely: (i) total population, (ii) male population and (iii) female population.
5 5 5 5 5 9 34 13 292 23 6 17 7 141 11
The Territory dataset describes the Italian territorial features from both administrative and geographical perspectives. Fig. 1 shows an excerpt of one of the Territory file composed by several sheets, describing the territory in its different aspects as localities, municipalities, administrative zones etc.
The amount of involved data is huge: there are about 402903 Census sections, 74482 localities, 2200 Census areas, 3631 geomorphological entities and 43 indicators for each entity (e.g., “Resident Population – Males”, “Resident Population – age > 74 years”, “Foreigners and stateless persons resident in Italy – Males” and so on). In order to design the ontologies necessary for the data publication, we met periodically domain experts and interviewed them. In addition, we followed an abstraction process starting from the concrete data representations that we had at hand (i.e. CSV and Excel files).
We designed two distinct ontologies: (i) the Territorial Ontology, and (ii) the Census Data Ontology.
The Territorial Ontology is an OWL [
In order to specify mapping rules we used R2RML [
The mapping implementation can be on-demand or made by data materialization; in the first case the triples are created as a response to a query while, in the second case, they are materialized into the triple store. The triple store loading phase is completed by adding the new triples obtained as result of a reasoning phase; these rules are specified by the defined ontologies and are not coded by explicit mapping rules. The publishing phase includes a first step related to the choice of the Web application layer needed to access the triple store: specifically we provide three access points to cover the requirements of the different possible users interested to LOD data. As shown in Fig. 5, the design of the Web site consists of three different components: (i) a SPARQL endpoint, (ii) a Linked Data Interface (Faceted/Graph browser) and (iii) an ad-hoc GUI for datasets downloading.
Advanced users are supposed to be SPARQL knowledgeable and can directly access the SPARQL endpoint, which in turn can also be used for machine-to-machine communication. Basic users can instead use the Linked Data Interface to browse data or the ad-hoc application that provides a set of predefined queries and functionalities to build customized queries.
The LOD publishing phase results in the publication of the SPARQL endpoint and the
related Web containers. This phase is currently in progress; the expected data for
Census-LOD Web site publication is December 2014.
This phase also includes the definition of the technological environment. Th e core of
the platform is the triple store. We analyzed several platforms for publish ing LOD,
both open and commercial. The result of such an analysis was the choice of Oracle
“Spatial & Graph” solution, as triple store and SPARQL query engine, bein g it fully
compliant with the IT infrastructure already existing in Istat. This solutio n has the
major advantage to scale up to billions of triples [
The other important architectural choice concerns the dissemination platform; as mentioned in the previous section, we plan to deploy a SPARQL endpoint and two different interfaces for basic and advanced users. The SPARQL endpoint we use is Apache Jena Adapter (Joseki API) an open platform integrated with the Oracle SPARQL query engine. As Graph Browser, we adopted the ELDA framework [17], the open source implementation of the Linked Data API specification released by Epimorphics. The Linked Data API provides a configurable way to access RDF data using simple RESTful URLs that are translated into queries to a SPARQL endpoint. ELDA customization consists of writing down, using the turtle syntax [18], an API that specifies how to translate URLs into queries.
Fig. 6 shows the complete technological architecture. People make trust judgments based on data provenance that may or may not be explicitly offered to them. Since the data represented in LOD format, are accessible via machine-to-machine communication, it is necessary to explicitly represent provenance information in an accessible way to machines, so that software agent can make trust judgments.
This issue has been recognized extremely important, and, indeed, the Italian
guidelines for the enhancement of public information [
Census-LOD is the first project that deploys Istat data on a SPARQL endpoint that is owned by Istat itself. The publication of Census indicators at the sub-municipality level will be available in December 2014. Published data will be enriched with information related to the administrative and geo-morphological division of the national territory. All published data will be accompanied by their own formal semantics specification through the Territory and Census ontologies.
The LOD-based data dissemination will provide several benefits, including: Fostering harmonization of information concepts also within Istat. Indeed, a schema integration step is necessary in order to publish data and this will also provide a positive feedback by forcing statistical processes to share a common data semantics. Improving machine-to-machine data provisioning by Istat, by providing an RDFbased channel in addition to the already available SDMX one. Providing final users with advanced functionalities like navigational querying and information discovery. 16. Michel F., Montagnat J., Faron-Zucker C: “A survey of RDB to RDF translation approaches and tools”, Research report - http://hal.archives-ouvertes.fr/hal-00903568 November 2013 17. ELDA: http://www.epimorphics.com/web/tools/elda.html 18. TTL: http://www.w3.org/TeamSubmission/turtle/, 28 March 2011 19. EL.STAT RDF site: http://linked-statistics.gr/ 20. INSEE RDF site: http://rdf.insee.fr/ 21. CSO RDF site: http://data.cso.ie/