Recently, many organizations have opened up their data for others to reuse. A major part of these data concern statistics such as demographic and social indicators. Linked Data is a promising paradigm for opening data because it facilitates data integration on the Web. Recently, a growing number of organizations adopted linked data paradigm and provided Linked Open Statistical Data (LOSD). These data can be exploited to create added value services and applications that require integrated data from multiple sources. In this paper, we suggest that in order to unleash the full potential of LOSD we need to facilitate the interaction with LOSD and hide most of the complexity. Moreover, we describe the requirements and design criteria of a JSON-QB API that (i) facilitates the development of LOSD tools through a style of interaction familiar to web developers and (ii) o ers a uniform way to access LOSD. A proof of concept implementation of the JSON-QB API demonstrates part of the proposed functionality.
Increasingly, many governments, organisations and companies are opening up
their data for others to reuse through Open Data portals [12]. These data can
be exploited to create added value services, which can increase transparency,
contribute to economic growth and provide social value to citizens [
A major part of open data concerns statistics (e.g. economical and social
indicators) [
Linked data has been introduced as a promising paradigm for opening up
data because it facilitates data integration on the Web [
Although LOSD potential is high, their exploitation is low for two reasons.
First, using LOSD requires skills and tooling (e.g. RDF, SPARQL) that are
less widespread than some other web technologies (e.g. JSON, Javascript). For
example, there are many Javascript visualization libraries that consume JSON
data (e.g D3.js, charts.js), while there are just a few that consume RDF and
their functionality is limited. Second, many portals that use the standard
vocabularies often adopt di erent publishing practices [
In order to unleash the full potential of LOSD there is a need to standardize the interaction (i.e. input, output and functionality) with LOSD in a way that facilitates the development of reusable software. This paper describes the requirements and design criteria of a JSON-QB API that aims to exploit the advantages of LOSD (e.g. easy data integration) while making data available in a structure and format that is familiar to a larger group of developers. Some of the exibility, and associated complexity, of linked data is removed, in favour of simplicity and ease of use. Moreover, the API o ers a uniform way to access the data, thus enabling the development of generic software tools that can be reused across datasets.
The rest of the paper is organized as follows, section 2 explains the motivation for the development of a JSON-QB API, section 3 presents related work, section 4 de nes the requirements and design criteria for JSON-QN API. Section 5 presents a proof of concept implementation of the API. Finally, section 6 draws conclusions. 2
Currently, many LOSD have been made available on the Web through o cial portals. For example Census data of 2011 from Ireland4 and Italy5 have been published as linked data. The Department for Communities and Local Government (DCLG)6 in the UK, the Scottish Government7, and the Statistics Bureau of Japan8 opened up their statistics as linked data etc.
Although the above portals use the same standard vocabularies, they
often adopt di erent publishing practices [
In addition to the exploitation tools, complete platforms (e.g. PublishMyData9) aim both to publish and exploit LOSD. In this case, published data can be consumed only by tools of the same platform, since di erent publishing practices are adopted. This leads to the creation of LOSD system silos (software & data) that cannot interoperate among each other.
All the above tools and platforms follow the same traditional architecture ( gure 1) where each tool has an integrated access layer. If several tools are created for the same portal (i.e. same publishing practices), then each tool has to develop separately a similar data access layer. In addition, if a tool has to be used at another portal, then a new data access layer has to be created leading to additional costs. More importantly, the development of data access layers requires signi cant programming expertise in LOSD, a skill that is not widely available between developers.
As a result, there is a need to standardize the interaction with LOSD in a way that hides the LOSD complexity to the developers and o ers a uniform way to access the data. To achieve these objectives we adopt the following methodology: 9 http://www.swirrl.com/ (i) study the related work focusing on APIs that facilitate the interaction with data cubes and statistical data and (ii) collect user requirements from developers that create LOSD applications. 3
Currently, several APIs that standardize the interaction with multi-dimensional statistical datasets have been developed. For example, SDMX has proposed the SDMX-REST API [20] that o er programmatic access to data and metadata disseminated in an SDMX-compliant source. This API is currently used by several organisations including the Eurostat, OECD and World Bank. Eurostat o ers also the \JSON & UNICODE Web Services"10 to allow access to its data. This service is complementary to the SDMX-REST API since it supports di erent output formats (i.e. JSON and UNICODE). Another REST API that enables the accessing of multidimensional databases is o ered by PX-Web 11 internet server application. The API is used by many National Statistics O ces including Finland, Sweden, Estonia and Switzerland. Table 1 presents a summary of the o ered functionality of the above three APIs. The functionality is separated to three main categories, namely \search", \get meta-data" and \get data".
These APIs focus on supplying metadata and data about a speci c dataset or cube. They do not address requirements regarding the combination of data from multiple datasets or cubes. Although two of the APIs in Table 1 support search functionalities, the search provides limited ltering options such as free text search and search based on the category of the indicator. As a result they do not support the discovery of datasets that are structural compatible to integrate.
Additionally, APIs that support advanced OLAP operations on data cubes, such as aggregation, slice and roll-up/drill-down, have been proposed. For example, the Oracle OLAP Java API [19] allows users to select, explore, aggregate and perform analytical tasks on data stored in an Oracle data warehouse. Olap4j12 is another Java API for accessing data cubes stored at OLAP servers. It supports Multidimensional Expressions (MDX) that is the query language for OLAP.
Regarding the output, the APIs support many formats including, SDMXJSON [21], SDMX-ML, JSON-stat13, CSV, Unicode, PC-Axis etc. An interesting JSON extension for encoding linked data is JSON-LD14. JSON-LD is not currently used by existing APIs, however it is a candidate for a JSON-QB API.
Finally, APIs that hide the complexity of SPARQL endpoints have been
proposed. For example OpenPHACTS [
To collect the requirements, we established an ongoing interaction with developers that currently create applications for LOSD. The interaction mainly occurs within the EU funded project OpenGovIntelligence15, which aims to exploit LOSD for improving the public services. To facilitate the collection of requirements we organized a dedicated workshop in Manchester with participation of relevant developers. 15 http://www.opengovintelligence.eu/ 4.1
The LOSD cloud currently contains many data cubes and their number still increases. Thus, applications need to search for cubes based on some criteria. For example, get cubes that measure unemployment, or get cubes for Greece. The search criteria can be even more complex e.g. get cubes about unemployment in Greece after 2010. Thus, the API should provide a exible way to express complex data queries. A parameter that should also be taken into consideration is the support of multiple natural languages, for example in helping to match search terms with concepts that could have multi-lingual labels.
The search functionality can also be extended to support not only user speci c data queries, but also support the \automatic" search of compatible cubes that could be processed together. For example, having a cube at hand search for other cubes that are compatible for combined statistical analysis, for visualisation or for browsing. The compatibility search needs to access both the structure and the data of the cube. However, the compatibility criteria is still an open issue and is out of the scope of this paper. 4.2
Once a cube has been identi ed (e.g. through the search functionality) the processing application (e.g. cube browser) needs to initialize the user interface or the analysis with information related to the cube structure. For example, populate drop-down menus with the cube dimensions and measures. The QB vocabulary clearly identi es the main elements of the structure that should be accessed through the JSON-QB API: { Dataset meta-data. They include information like the label, description, issue date, publisher and license. { Dimensions. They include all the dimension properties of the cube (e.g. reference area, reference period). { Measures. They include all the measure properties of the cube (e.g. unemployment, poverty) { Attributes. They include all the attribute properties of the cube (e.g. unit of measure) { Dimension values. They include all the values of a dimension (e.g. male, female) that appear at the cube. { Dimension levels. In the case of hierarchical data, dimension values are organized to hierarchical levels (e.g. region, district). { Attribute values. They include all the values of an attribute (e.g. euro, dollar) that appear at the cube.
Regarding the last three elements, the QB vocabulary does not o er a way to retrieve the values / levels directly from the structure. Thus the API should iterate over the cube observations, which is a time consuming task. 4.3
There are already methods available for downloading entire data cubes but people often want just small parts. Whole cubes are often too big to be well-suited to interactive applications, and if the data updates frequently, then it's important for people to be able to retrieve up-to-date extracts of the data, rather than keeping their own copies of full datasets up to date. The JSON-QB API should provide a exible way to applications to take exactly the data they need by de ning constrains (i.e. lters). For example it should support many ltering options to the dimension values including: { Single values e.g. refPeriod=2010. { Multiple values e.g. refPeriod=[2010, 2011, 2012] { Ranges e.g refPeriod=[2010 ... 2015] { Greater/smaller than e.g. refPeriod>2010 { Hierarchical data ltering e.g. refArea=\all council areas in Scotland"
In many cases, applications do not need all the requested data at once, because they process them at bunches. For example, a cube browser shows a part of the data allowing the user to navigate to the previous/next page of data. Thus, the JSON-QB API should support paging and ordering of the results. The ordering of the results can be in ascending or descending order based on a dimension. However, in some cases this is a complicated task e.g. ordering based on 2 ore more dimension. Moreover, lexicographical ordering is not always appropriate (e.g. for the days of the week), thus other types of ordering should be applied. 4.4
Linked data o er many bene ts to web developers, including the easy integration on the web. However, linked data technologies (i.e. RDF, SPARQL) are unfamiliar to many developers, thus hindering their adoption. The purpose of the JSON-QB API is to exploit the advantages of linked data through a style of interaction that is familiar to web developers, thus helping them create data visualisations and applications. It is not necessary for the API to be a complete \round-trippable" representation of the data, it is acceptable to lose some information in favour of greater ease of use.
The ease of use of an API is related both to the input and the output. Regarding the input of the API there are mainly two design options: i) use a separate REST parameter for each input and ii) model all the input as a JSON object. The rst option was traditionally used by APIs, while the second is recently becoming popular since it is more exible and enables the creation of a data query language for the API. For example, using JSON objects is easier to express relations other than equality e.g. greater than, while using parameters is more awkward as custom encoding conventions should be used which require extra processing on part of the developer.
Regarding the output of the API, JSON is a popular, easy to use format. Usually, applications and visualizations do not require an n-array/ tabular response (e.g. JSON-stat); an array of observations is su cient and more straightforward. In case that a tabular response is required, then it can easily be constructed from the observations. While JSON-QB API aims in hiding some of the complexity of linked data, responses should include URIs as identi ers of key entities (e.g. JSON-LD), to retain the connection to data on the web and to support reliable combining of data from di erent sources within a data consuming application. 4.5
Currently many LOSD have been published, however a lot of them adopt di erent publishing practices. The JSON-QB API should work on top of any of these data, o ering uniform access to the data. Obviously, this will require separate implementations to comply with the di erent publishing practices.
Ideally, the standardization of the JSON-QB API speci cation will also contribute to the formulation of an application pro le for the QB vocabulary. The pro le will include best practices that can be used by data publishers to provide data in a compatible way, facilitating in this way the development of generic LOSD tools. This will add some constraints on the way that publishers manage their data, however those constraints will lead to greater exploitation of the data. 4.6
The volume of LOSD is big, reaching the magnitude of million triples per cube. Thus, SPARQL queries that iterate over all the observations tend to be slow. For example, a query to get all the dimension values that appear at a cube needs to iterate over all the observations.
The JSON-QB API can improve performance of demanding SPARQL queries through e cient caching of the responses. The caching policy (e.g. Least Recently Used, Least Frequently Used) plays an important role at the performance improvement. Note that caching of API responses is much easier that caching of arbitrary SPARQL queries. Allowing a SPARQL query to run on a collection of data means that if any of the data changes, it is possible that the query response changes. It is complex to analyse which queries touch a particular data cube or particular part of the data, thus making cache clearing di cult. With the API call, most requests will return data from individual data cubes, so it is easier to know which cached responses must be invalidated when data is updated.
Another task that can improve the performance of the API is the precomputation of aggregations: i) across a dimension of the cube e.g. compute the SUM of the sales over time and thus ignore the time dimension of the cube and ii) across a hierarchy e.g. if a cube contains the election results at municipality level, then aggregations can be computed at region and at country level. The pre-computation of the aggregations facilitates the execution of queries, because there in not the need to compute the aggregations on-the- y when requested.
Finally, the performance and network tra c can be improved by returning exactly the data requested. This can be achieved through a exible data query language. In this way web applications can be fast and stable because they control the data they get, not the server. Many organisations have lots of not \fully" open data use cases. For example, ethical and legal restrictions exist to the access of health and tness data [14]. The restrictions may derive from: i) strict regulations that protect personal data, ii) agreements that are speci ed in consent forms and iii) policies of stakeholders owing the data. Thus there is a need for an access control mechanism that ensures the availability of data only to authorized persons and prevent the unauthorized and unintended withholding of data. Finally, the JSON-QB API should be extensible, thus take future growth into consideration minimizing the e ort required for the extension. Extensions can be implemented through the: i) addition of new functionality e.g. while the initial aim is to build an API on top of RDF databases, other kinds of database could be used, ii) modi cation of existing functionality e.g. support modi ed ltering options and iii) the harmonisation with deployed solutions e.g. SDMX-REST. 5
The architecture of the JSON-QB API ( gure 2) is simple and is developed as a middle-ware between LOSD and the applications that consume the data. The API receives the REST calls and translates them to SPARQL queries which are executed at LOSD portals. Then, the returned results are transformed to JSON format that can easily be consumed by applications.
We have developed a proof of concept implementation of the JSON-QB API16 which can be installed on top of existing RDF repositories that store data using the QB vocabulary. Two options are currently examined for the input of the API (see sec. 4.4). The rst option is to use a separate REST parameter for each input and the second to model the input as a JSON object. Results so far show that the second option seems promising since it is more exibile and extensibile. Speci cally, it enables the expression of complex search and ltering data queries limiting the transmitted data to exactly what requested. Towards this direction the implementation uses GraphQL17, which is a data query language proposed by Facebook. Other technologies used by the API include the Jersey framework18 for the implementation of the RESTful services, the Rdf4j19 for processing RDF data and the Gson20 library to serialize Java Objects into their JSON representation.
Table 2 presents an example API call that returns a cube slice by ltering the dimension values. Both options for API input are considered. The example presents also the corresponding SPARQL query and the returned JSON result.
The second option (GraphQL) is more exible e.g. it enables the request of the title and description of the cube except from the ltered observations. However, the GraphQL approach raises some challenges that need to be addressed. For example it does not support namespaces, so all schemas exist in a single global namespace. Thus, it's hard to be sure that an extension added to the schema doesn't con ict with another extension. Another challenge is related with the conversion of URIs into elds whilst retaining uniqueness, since the characterset in GraphQL is very small. Finally, there isn't really a standardised schema serialisation for GraphQL, so you can't detect that an endpoint supports the data types you're looking for. 16 https://github.com/OpenGovIntelligence/json-qb-api-implementation 17 http://graphql.org/ 18 https://jersey.github.io/ 19 http://rdf4j.org/ 20 https://github.com/google/gson
The API is still under development and the existing version implements only a subset of the proposed functionality. A complete list of the currently implemented functionality can be found at [18]. 6
Currently, the LOSD cloud contains many datasets and their number still increases. However, their exploitation remains low due to two reasons. First, skills and tooling for linked data are not widespread among developers and second, existing portals adopt di erent publishing approaches, thus hindering the development of tools that can operate across LOSD datasets.
In this paper we describe the requirements and design criteria of a JSONQB API that standardises the interaction with LOSD aiming at their broader exploitation. Speci cally, the API facilitates the development of LOSD tools through a style of interaction familiar to web developers. It also provides uniform access to LOSD data. However, in order to achieve the uniform data access, either di erent implementation of the API should be created (one for each set of publishing practices) or a set of best practices should be widely adopted by publishers to provide data in a compatible way. We anticipate that the standardization of the JSON-QB API speci cation will contribute towards the formulation of these best practices.
The proof of concept implementation of the JSON-QB API raises many issues that need to be clari ed, including: i) whether GraphQL covers the needs of the API, ii) the part of JSON-LD that will be considered (currently only limited functionality is included), iii) the relation with JSON-stat and whether it can be used as an output format of the API and iv) technical details e.g. content negotiation, status codes, success/error responses. Moreover, there are open issues related to the cube compatibility criteria and the ordering of the API results. Acknowledgments. Part of this work was funded by the European Commission within the H2020 Programme in the context of the OpenGovIntelligence project (http://OpenGovIntelligence.eu) under grant agreement no. 693849. 11. Kalampokis, E., Nikolov, A., Haase, P., Cyganiak, R., Stasiewicz, A., Karamanou, A., Zotou, M., Zeginis, D., Tambouris, E., Tarabanis, K.: Exploiting linked data cubes with OpenCube toolkit. In: ISWC 2014 Posters and Demos Track, vol. 1272.
CEUR-WS (2014) 12. Kalampokis, E., Tambouris, E., Tarabanis, K.: A classi cation scheme for open government data: towards linking decentralised data. Int. J. Web Eng. Technol. 6(3), 266{285 (Jun 2011), http://dx.doi.org/10.1504/IJWET.2011.040725 13. Kalampokis, E., Tambouris, E., Tarabanis, K.: Ict tools for creating, expanding, and exploiting statistical linked open data, statistical. IAOS 33(2), 503{514 (2017) 14. Kamateri, E., Kalampokis, E., Tambouris, E., Tarabanis, K.: The linked medical data access control framework. Journal of Biomedical Informatics 50, 213 { 225 (2014), special Issue on Informatics Methods in Medical Privacy 15. Maali, F., Shukair, G., Loutas, N.: A dynamic faceted browser for data cube statistical data. In: W3C workshop on Using Open Data (2012) 16. Martin, M., Abicht, K., Stadler, C., Ngonga Ngomo, A.C., Soru, T., Auer, S.: Cubeviz: Exploration and visualization of statistical linked data. In: Proceedings of the 24th International Conference on World Wide Web. pp. 219{222 (2015) 17. Miles, A., Bechhofer, S.: SKOS simple knowledge organization system. Tech. rep.,
W3C (August 2009) 18. OpenGovIntelligence: D3.2: Opengovintelligence ict tools - 1st release (2016) 19. Oracle: Oracle olap developer's guide to the olap api, 10g release 2 (10.2) (2006) 20. SDMX: Guidelines for the use of web services (version 2.1) (2013) 21. SDMX: Sdmx-json data message: syntax and documentation (2014) 22. Varga, J., Etcheverry, L., Vaisman, A.A., Romero, O., Pedersen, T.B., Thomsen, C.: Qb2olap: Enabling olap on statistical linked open data. In: 32nd International Conference on Data Engineering (ICDE). pp. 1346{1349. IEEE (May 2016)