In this demonstrator, we will introduce the latest features and capabilities added to DataGraft - a Data-as-a-Service platform for data preparation and knowledge graph generation. DataGraft provides data transformation, publishing and hosting capabilities that aim to simplify the data publishing lifecycle for data workers (i.e., Open Data publishers, Linked Data developers, data scientists). This demonstrator highlights the recent features added to DataGraft by exemplifying data publication of statistical data - going from the raw data published at a public portal to published and accessible Linked Data with the help of the tools and features of the platform.
In the recent years, a large number of public organisations and governments have been increasingly committed to publishing data in reusable formats and under open licenses. Nevertheless, in many cases such organisations choose to prioritise e-government and open government initiatives due, to a large extent, to the high costs and domain-specific expertise required. DataGraft1 started as an initiative with the goal to alleviate such obstacles by providing new tools and services for faster and lower-cost publication of Open Data. The lifecycle for the creation and provisioning of (Linked) Open Data typically involves raw data cleaning, transformation, and preparation (most often from tabular formats), mapping to standard Linked Data vocabularies and generating a semantic RDF graph. The resulting semantic graph is then stored in a triple store, where applications can reliably access and query the data. Conceptually, this process is rather straightforward; however, such an integrated workflow is not commonly implemented. Instead, publishing and consuming (Linked) Open Data remains a tedious task due to a variety of reasons, including: 1 https://datagraft.io/ 1. The technical complexity of preparing Open Data for publication is high – toolkits are poorly integrated and require expert knowledge, particularly for publishing of Linked Data; 2. There is considerable cost for publishing data and providing reliable access to it. In the absence of clear monetisation channels and cost recovery incentives, the relative investment costs can easily become excessively high for many organisations; 3. The poorly maintained and fragmented supply of Open Data reduces the reuse of data: datasets are often provided through disconnected outlets; sequential releases of the same dataset are often inconsistently formatted and structured.
DataGraft features and capabilities have been developed to tackle such Open Data challenges. 2
The DataGraft Platform (beta v2) DataGraft aims to streamline the process of publishing, managing and accessing Open Data and related artefacts in order to support data workers and the users of their data. The beta v2 version of DataGraft supports the following features: • Asset management – metadata, access management (public/private), sharing – for files, SPARQL endpoints, data transformations, queries; • Automatic reliable storage of files and RDF databases; • Data hosting and access services for all data stored on DataGraft; • Interactive design of data transformations and RDF generation; • Interactive visual exploration of RDF data in the SPARQL endpoints published; • REST-based access to all platform assets; • Layered security – encrypted user login information and SSL; asset security using OAuth2, API keys for semantic graph databases.
DataGraft platform
DataGraft portal
Grafterizer
UI User asset management File storage Secure access Interactive data transformations Repeatable data transformations On- and off-line execution RDF mapping Based on the Grafter DSL Ontotext S4 Elastic resource provisioning Reliable storage, Back-up, Geospatial querying Failover mechanisms
Fig. 1. Main components of DataGraft platform The DataGraft platform mainly consists of three components – the DataGraft portal, Grafterizer and a cloud-enabled semantic graph database-as-a-service (as shown on Fig. 1), which is based on a dedicated instance of the Ontotext S4 platform2.
Since the initial release of the DataGraft platform reported in [
Related Work. In the current state-of-the art there are several software tool ecosystem solutions that provide support for publication of Linked Data (data extraction, RDFisation, storage, access). Examples of such are Linked Data Stack 4 , LinDA 5 , and COMSODE 6 projects, which provide software tools and methodologies for Linked Data management. Such tools come functionally close to the features provided by DataGraft, but are not provided "as-a-service", which puts the burden of deploying and managing the infrastructure on the data publisher. OpenRefine7, with its RDF plugin (including powerful RDF mapping features such as RDF reconciliation) comes closest to the data transformation approach implemented in DataGraft. Nevertheless, OpenRefine is unsuitable for usage in a service offering and the data transformation engine is memory intensive with large data volumes. 3
Demo Scenario: Statistical Data Publication and Access The usage scenario will demonstrate the core capabilities of DataGraft using statistical data from Statistics Norway8. Firstly, it will demonstrate how to use the DataGraft dashboard to explore published data and user assets using the new filters and search 2 https://console.s4.ontotext.com 3 https://github.com/guiveg/rdfsurveyor 4 http://stack.linkeddata.org 5 http://linda-project.eu 6 http://www.comsode.eu 7 http://openrefine.org 8 https://www.ssb.no/ functionalities. We will then show the different types of assets that are now supported by the platform along with the features related to browsing assets versions. Furthermore, we will demonstrate how to use the DataGraft wizards to publish Linked Data based on raw tabular data with the improved Grafterizer tool. Grafterizer allows to interactively specify data transformations and mappings of tabular data to RDF. We will demonstrate how transformations can be applied on-the-fly or offline. Then the resulting Linked Data will be published in a dynamically created semantic graph database and registered in the DataGraft catalogue in a SPARQL endpoint page (such as the one shown in Fig. 2). We will demonstrate how the endpoint can be explored using userdefined queries (as free-text or using pre-defined ones) or the new browsing tool integrated in DataGraft. Finally, we will showcase how the new interactive APIs can be directly used by users to access data hosted on the platform. As of September 2017, DataGraft is available via http://datagraft.io and more details can be found in the platform online documentation9.
Acknowledgements. This work is partly funded by the EC H2020 project proDataMarket (Grant number: 644497). 9 https://github.com/datagraft/datagraft-reference/blob/master/documentation.md