Processing RDF Data in Uni edViews

Abstract

Uni edViews is an Extract-Transform-Load (ETL) framework that allows users { publishers, consumers, or analysts { to de ne, execute, monitor, debug, schedule, and share RDF data processing tasks. The data processing tasks may use custom plugins created by users. Uni edViews di ers from other ETL frameworks by natively supporting RDF data and ontologies. The practical demonstration of Uni edViews at the conference will (1) clearly demonstrate how Uni edViews helps RDF/Linked Data users with RDF data processing (2) and show the real instance of Uni edViews with tens of data processing tasks and DPUs motivated by real data processing use cases.

Introduction

There are lots of tools used by the RDF/Linked Data community3, which may support various phases of RDF data processing; e.g., a user may use any23 4 for extraction of non-RDF data and its conversion to RDF data, Virtuoso5 database for storing RDF data and executing SPARQL (Update) queries [1, 2], Silk [4] for RDF data linkage, or Cr-batch6 for RDF data fusion.

Nevertheless, the user who is preparing a data processing task producing a data mart typically has to ( 1 ) con gure every such tool using a di erent congurator, ( 2 ) implement a script for retrieving source data, ( 3 ) write his own script holding the set of SPARQL Update queries re ning the data, ( 4 ) implement custom transformers which, e.g., enrich processed data with the data in his knowledge base, (5) write his own script executing the tools in the required order, so that every tool has all desired inputs when being launched, (6) prepare a scheduling script, which ensures that the task is executed regularly, and (7) extend his script with noti cation capabilities, such as sending an email in case of an error during task execution. Furthermore, in case of errors in data processing, the user has no support for debugging the tasks. ? This work was supported by EU ICT FP7 under No.257943 (LOD2 project) and by projects SVV-2014-260100 and PRVOUK. 3 http://semanticweb.org/wiki/Tools 4 https://any23.apache.org/ 5 http://virtuoso.openlinksw.com/ 6 https://github.com/mifeet/cr-batch To address these problems, we developed Uni edViews, an Extract-TransformLoad (ETL) framework, where the concept of data processing task is a central concept. Another central concept is the native support for RDF data format and ontologies.

A data processing task (or simply task) consists of one or more data processing units. A data processing unit (DPU) encapsulates certain business logic needed when processing data (e.g., one DPU may extract data from a SPARQL endpoint or apply a SPARQL query). Every DPU must de ne its required/optional inputs and produced outputs. Uni edViews supports exchange of RDF data between DPUs. Every tool produced by RDF/Linked Data community can be used in Uni edViews as a DPU, if a simple wrapper is provided7.

Uni edViews allows users to de ne and adjust data processing tasks, using graphical user interface (an excerpt is depicted in Figure 1). Every user may also de ne their custom DPUs, or share DPUs provided by others together with their con gurations. DPUs may be drag&dropped on the canvas where the data processing task is constructed. Data ow between two DPUs is denoted as an edge on the canvas (see Figure 1); a label on the edge clari es which outputs of a DPU are mapped to which inputs of another DPU. Uni edViews natively supports exchange of RDF data between DPUs; apart from that, les and folders may be exchanged between DPUs.

Uni edViews takes care of task schedulling, a user may con gure Uni edViews to get noti cations about errors in the tasks' executions; user may also get daily summaries about the tasks executed. Uni edViews ensures that DPUs are executed in the proper order, so that all DPUs have proper required inputs when being launched. Uni edViews provides users with the debugging capabilities { a user may browse and query (using SPARQL query language) the RDF inputs to and RDF outputs from any DPU. Uni edViews allows users to share DPUs and tasks as needed.

Paper [3] contains discussion about related work and lists projects where Uni edViews is used. The code of Uni edViews is available at GitHub8 under a combination of GPLv3 and LGPLv3 license9. 7 https://grips.semantic-web.at/display/UDDOC/Creation+of+Plugins 8 https://github.com/Uni edViews 9 http://www.gnu.org/licenses/gpl.txt, http://www.gnu.org/licenses/lgpl.txt

Uni edViews - The Demo

The demo of the tool is available at http://odcs.xrg.cz:8080/uni edviews. You can use the account guest/guest to test the framework. When you log in, you can see tasks (menu item Pipelines) and DPUs (menu item DPU Templates) available in the framework. You can, for example, run DBpedia pipeline, which is extracting data about Prague from DBpedia, and supplement the pipeline with SPARQL Transformer DPU executing certain SPARQL (Update) queries on top of the extracted data.

The practical demonstration of Uni edViews at the conference will ( 1 ) clearly demonstrate how Uni edViews helps RDF/Linked Data users with RDF data processing ( 2 ) and show the real instance of Uni edViews with tens of data processing tasks and DPUs motivated by real use cases.

References

1. S. H. Garlik , A. Seaborne , and E. Prud 'hommeaux. SPARQL 1.1 Query Language . W3C Recommendation , 2013 . http://www.w3.org/TR/2013/REC-sparql11-query20130321/, Retrieved 20/03/ 2014 .

2. P. Gearon , A. Passant , and A. Polleres . SPARQL 1.1 Update. Technical report, W3C , 2013 . Published online on March 21st , 2013 at http://www.w3.org/TR/2013/REC-sparql11 -update-20130321/, Retrieved 20 /03/ 2014 .

3. T. Knap , M. Kukhar , B. Machac , P. Skoda , J. Tomes , and J. Vojt . Uni edViews: An ETL Framework for Sustainable RDF Data Processing . In Extended Semantic Web Conference (ESWC 2014 ), Anissaras, Crete, Greece, 2014 . Springer.

4. J. Volz , C. Bizer , M. Gaedke , and G. Kobilarov. Silk - A Link Discovery Framework for the Web of Data . In Proceedings of the WWW2009 Workshop on Linked Data on the Web (LDOW) , Madrid, Spain, 2009 . CEUR-WS.org.