SPARQL, the W3C standard querying language for RDF, provides rich capabilities for slicing and dicing RDF data. The latest version, SPARQL 1.1, added support for aggregation, nested and distributed queries among others. Nevertheless, the purely declarative nature of SPARQL and the lack of support for common programming patterns, such as recursion and iteration, make it challenging to perform complex data processing and analysis in SPARQL. In the database community, similar limitations of SQL resulted in a surge of proposals of analytics languages and frameworks. These languages are carefully designed to run on top of distributed computation platforms. In this paper, we review these e orts of the database community, identify a number of common themes they bear and discuss their applicability in the Semantic Web and Linked Data realm. In particular, design decisions related to the data model, schema restrictions, data transformation and the programming paradigm are examined and a number of related challenges for de ning an RDF analytics language are outlined.
The cost of acquiring and storing data has dropped dramatically in
the last few years. Consequently, petabytes and terabytes datasets
are becoming commonplace, especially in industries such as telecom,
health care, retail, pharmaceutical and nancial services. This
collected data is playing a crucial role in societies, governments and
enterprises. For instance, data science is increasingly utilised in
supporting data-driven decisions and in delivering data products [
In order to process the available massive amount of data, a
number of frameworks were built on top of distributed cluster of
commodity machines. In 2004, Google introduced the MapReduce
framework [
While analytics languages aim to utilise the high scalability of
distributed platforms, they also aim to increase developer
productivity by facilitating exible adhoc data analysis and exploration. It
is increasingly recognised that SQL, the main database query
language, has a number of limitations that restrict its utility in analytics
and complex data processing scenarios [
A close parallel can be drawn in the Semantic Web and Linked
Data realm. The size of available RDF data is increasing and massive
datasets are becoming commonplace. The 1.2 billion triple of
Freebase can be freely downloaded2 and the LOD Cloud grew to 31 billion
RDF triples as of September 2011 3. Furthermore, distributed
execution platforms are being utilised to process RDF data particularly
for building query engines that support (part of) SPARQL [
In this paper, we present lessons that can be learned from existing e orts towards building an analytics language on top of RDF. We review a number of high-level analytics languages proposed in the big data and database communities. We identify ve common themes they bear. For each of these themes, we present our observation, discuss corresponding e orts in the Semantic Web community and present pertinent challenges (section 2). We also discuss some further characteristics of RDF and Linked Data that can prove useful for analytics language (section 3). 2 2.1
Data model4 Observation: Adoption of \relaxed" versions of the relational data model.
A large number of data models that relax the constraints of the
relational data model has been proposed and adopted recently,
particularly in the context of big data. MapReduce [
RDF is the data model underlying Semantic Web data. RDF is
a graph-based data model that consists of a set of of triples. There
has been a number of proposals for a more abstract view of RDF
data. Ding et al. proposed the notion of RDF molecule [
We argue that an RDF analytics language requires de ning a
data model that abstracts the data at a higher level than individual
triples. This introduces a number of challenges and design choices of
whether to support a notion of records or tuples, whether to support
nesting data structures and whether to support collection types such
as sets and arrays. Nested data structures simplify data manipulation
and processing. Additionally, a collection of nested data is easier to
be partitioned and processed in parallel. On the other hand, adopting
a nested data structure on top of RDF requires enforcing the RDF
graph into a set of trees. This approach was adopted by Tabulator
for intuitive presentation of RDF graphs [
Observation: Supporting only a subset of relational algebra, focusing on operators that can be easily executed in a distributed architecture.
There has been a number of proposals to support SQL on top
of MapReduce framework. However, many of those proposals chose
not to support the full relational algebra underlying SQL. HiveQL
for instance supports only equality predicates in a join. Similar
restrictions are included in SCOPE [
Challenge: de ne a subset of SPARQL algebra that is su cient to address most common needs and is amenable to distributed execution. 2.3
Observation: A shift from pure declarative languages towards hybrid ones. 5 http://www.w3.org/TR/json-ld/
Declarative languages are abstract, concise, easier for domain experts and provide opportunities for optimisation. Nevertheless, they are not always the preferred way by programmers. It is often hard to t complex needs in a single query. Imperative scripts also allow programmers to pin down an execution plan to exploit optimisation opportunities that automatic optimisers might miss.
Increasingly, declarative languages are enriched with features from other paradigms such as imperative, functional and logic-based. PIG Latin adopts a hybrid imperative-declarative programming paradigm. Jaql and Cascalog6 adopt features from functional programming.
On the other hand, most of the languages utilised in the
Semantic Web and Linked Data realms are pure declarative languages.
Examples include R2RML7 for mapping relational databases to RDF,
SPIN8 to de ne rules in SPARQL and the languages de ned as part
of the Linked Data Integration Platform (R2R for schema
mapping [
Challenge: Adopt a hybrid declarative-imperative programming paradigm for RDF processing. 2.4
Observation: From rigid schemas to partial or no schemas.
Relational databases require the schema to be designed before any data can be added to the database (a.k.a. schema rst). It is generally not easy to change the schema and data that does not strictly adhere to the schema cannot be added to the database. There is a 6 https://github.com/nathanmarz/cascalog 7 http://www.w3.org/TR/r2rml/ 8 http://www.w3.org/Submission/spin-overview/ 9 http://activerdf.org/ 10 https://code.google.com/p/surfrdf/ number of advantages to schema speci cation, including data validation, transactional consistency guarantees, static type checking and optimisation. Nevertheless, requiring a prede ned rigid schema can be an overkill particularly for ill-de ned ad-hoc analytics tasks. In this context, users want to start working with the data right away in an exploratory read-only manner. Consequently, schema-on-read is increasingly adopted. PIG and Jaql support partial schema de nition and allow schema de nition to evolve as users are interacting with the data.
RDF data is self-describing in the sense that (a signi cant part
of) the schema is explicitly encoded in the data and can be extracted.
However, an essential task in consuming RDF data from di erent
sources is schema mapping [
Challenge: support partial and evolving schema mapping while interacting with RDF data. 2.5
Observation: in-place processing has become an important tool for dealing with data.
The increasing volume of data generated by applications has added constraints on how easily and e ciently it can be processed. Requiring data to be moved before it can be processed, especially with read-only analytics tasks, is not a viable mechanism at extreme scale. Therefore, processing data in-place is more and more supported. In-situ data processing is also re ected by processing data coming from di erent locations and in di erent formats. It is common for analytics language to support plain text, HDFS les, JSON and databases. Most RDF tools require full transformation and materialisation of data into RDF before it can be processed (with R2RML being a notable positive exception).
Challenge: Support in-situ RDF data processing and in-place transformation of non-RDF data. 3
Linked Data has an additional number of characteristics that should be utilised in the design of an analytics language. In the following, we go through some of these characteristics.
HTTP accessibility deploying RDF data as Linked Data makes it available via the Web and interlinked to related information. Support for retrieving data over the Web and following links should be employed by RDF processing languages.
Graph-based nature this introduces opportunity to support graph
traversal and graph algorithms on top of the RDF data. SPARQL
1.1 property path provides rst support in this regards. However,
other languages such as Gremlin11 and Green-Marl [
Inference RDF has a formally de ned semantics that can be used for inferencing. Inferencing allows enriching the data and makes implicit relations and facts explicit. An RDF processing language should include some support for basic inference tasks. However, a trade-o between inference capabilities and performance is inevitable. 4
The Linked Data community has been very successful in publishing large amounts of useful data as evidenced by the growth of the LOD Cloud. Further emphasis is being put on building applications that utilise this data. The interlinked nature of RDF data along with its clearly de ned semantics form a great basis to enable rich analysis and distilling valuable insights from this data. 11 https://github.com/tinkerpop/gremlin/wiki
In this paper we focused on a number of lessons that can be learned from existing e orts on designing analytics language and on identifying some of the challenges ahead. Our current work focuses on de ning use cases where SPARQL and other existing approaches for processing RDF data fall short. These use cases, along with the design clues outlined in this paper, will be used to inform the design, the implementation and the evaluation of an RDF analytics language.
Acknowledgements. Fadi Maali is funded by the Irish Research Council, Embark Postgraduate Scholarship Scheme. The ideas in this paper bene ted from valuable discussions with Aidan Hogan and Marcel Karnstedt and from the material of the \Introduction to Data Science" course on Coursera by Bill Howe. 28. J. Urbani, S. Kotoulas, E. Oren, and F. Harmelen. Scalable Distributed Reasoning Using MapReduce. In Proceedings of the 8th International Semantic Web Conference, ISWC '09. Springer-Verlag, 2009. 29. Y. Yu, M. Isard, D. Fetterly, M. Budiu, U. Erlingsson, P. K. Gunda, and J. Currey.
DryadLINQ: a System for General-purpose Distributed Data-parallel Computing Using a High-level Language. In Proceedings of the 8th USENIX conference on Operating systems design and implementation. USENIX Association, 2008.