On-Line Analytical Processing (OLAP) tools allow querying large multidimensional (MD) databases called data warehouses (DW). OLAP-style data analysis over the semantic web (SW) is gaining momentum, and thus SW technologies will be needed to model, manipulate, and share MD data. To achieve this, the definition of a vocabulary that adequately represents OLAP data is required. Unfortunately, so far, the proposals in this direction have followed different roads. On the one hand, the QB vocabulary (a proposal by the W3C Government Linked Data Working Group) follows a model initially devised for analyzing statistical data, but does not adequately support OLAP multidimensional data. Another recent proposal, the Open Cube vocabulary (OC) follows closely the classic MD models for OLAP and allows implementing OLAP operators as SPARQL queries, but does not provide a mechanism for reusing data already published using QB. In this work, we propose a new vocabulary, denoted QB4OLAP, which extends QB to fully support OLAP models and operators. We show how data already published in QB can be analyzed à la OLAP using the QB4OLAP vocabulary, and vice versa. To this end we provide algorithms that build the structures that allow performing both kinds of analysis, and show that compatibility between QB and QB4OLAP can be achieved at low cost, only adding dimensional information.
Business Intelligence (BI) comprises a collection of techniques used for extracting
and analyzing business data to support decision-making. As part of the BI machinery,
On-Line Analytical Processing (OLAP) [
BI over the SW requires, as a starting point, the definition of a precise vocabulary
allowing to represent OLAP data. Unfortunately, so far, the proposals in this direction
have followed different roads. The RDF Data Cube vocabulary (QB) [
We first define the MD model for OLAP that will be used in our study. Detailed formal
models can be found in [
In OLAP, data are organized as hypercubes whose axes are called dimensions. Each point in this MD space is mapped into one or more spaces of measures, representing facts that are analyzed along the cube’s dimensions. Dimensions are structured in hierarchies that allow analysis at different aggregation levels. The actual values in a dimension level are called members. A Dimension Schema is composed of a non-empty finite set of levels, with a distinguished level denoted All. We denote ‘!’ a partial order on these levels; the reflexive and transitive closure of ‘!’ (‘! ’) has a unique bottom level and a unique top level (the latter denoted All). Levels can have attributes describing them. A Dimension Instance assigns to each dimension level in the dimension schema a set of dimension members. For each pair of levels (lj ; lk) in the dimension schema, such that lj ! lk, a relation (denoted rollup) is defined, associating members from level lj with members of level lk. A Cube Schema contains a set of dimension schemas and a set of measures, where for each measure an aggregate function is specified. A Cube
cou nOtry GOO R UA (a) Schema
5 England i 5 All i
MiltonKeynes
Reading
Bournemouth
Cardiff
Newport (b) An instance Instance, corresponding to a cube schema, is a partial function mapping coordinates from dimension instances into measure values.
Example 1. (Cube schema and instance) We want to analyze the household projection in the UK (a measure) along the dimensions geographic location and date. Let us call these dimensions geoDim and dateDim, respectively. Dimension geoDim is organized in a hierarchy of levels: unitary authority (UA), Government Office Region (GOR) and country; dimension dateDim has only one level: year. The household measure is associated with the SUM aggregate function. Based on these dimensions and measures, we define a cube, denoted householdCS. Figure 1 shows dimension geoDim (its schema and a sample instance), while Figure 2a presents an instance of householdCS cube retrieved from StatsWales3 and Open Data Communities4.The symbol ? means that the value of a measure, corresponding to a set of coordinates, is unknown or undefined.
A well-known set of operations is defined over cubes. We present some of these
operations next. They are based on the recently proposed Cube Algebra [
Roll-Up summarizes data in a cube, along a dimension hierarchy. Given a cube C, a dimension D 2 C, and a dimension level lu 2 D such that ll ! lu with ll the level in dimension D used to represent cube instances in C, Roll-Up(C; D; lu) returns a new cube C0 where measure values are aggregated along D up to the level lu, using the aggregate function defined for each measure. Analogously, Drill-Down disaggregates previously summarized data, and can be considered the inverse of Roll-Up. Note that this requires to store the aggregation path. Slice receives a cube C and a dimension D 2 C, and returns a new cube C0, with the same schema as C except for dimension D. Measure values in the cube are aggregated along dimension D up to level All before removing the dimension. Dice receives a a cube C, and a first order formula over levels and measures in C, and returns a new cube C0 which has the same schema as cube C. Instances in C0 are the instances in C that satisfy .
Example 2 (OLAP Operators). Consider the cube schema of Example 1 and the cube instance in Figure 2a. The result of Slice(householdCS; geoDim) is shown in Figure 2b. Figure 2c shows the result of Dice(householdCS; household >= 60 ^ GOR <> South West). Figure 2d shows Roll-Up(householdCS; geoDim; GOR). 3 Report number 028727 in http://statswales.wales.gov.uk/index.htm 4 Household projections by district, England, 1991-2033 http://opendatacommunities.
org/id/dataset/housing/household-projections year country GOR UA 2006 2007 2008 England South East Milton Keynes 92 94 96
Reading 58 58 60
South West Bournemouth 71 72 73 Wales Wales NCeawrdpiofrft 15382..71 13?4.2 15396..67
(a) An instance of householdCS (b) Slice
The RDF Data Cube vocabulary [
Observations (in OLAP terminology, facts) are described in QB as instances of qb:Observation, representing points in a MD data space indexed by dimensions. They are grouped in datasets (qb:DataSet) by means of the qb:dataSet property. The schema of a dataset is specified by the data structure definition (DSD) (instance of qb:DataStructureDefinition). Although QB can define the structure of a fact (via the DSD), it does not provide a mechanism to represent an OLAP dimension structure (i.e., the dimension levels and the relationships between levels). However, QB allows representing hierarchical relationships between level members in the dimension instances, using the SKOS vocabulary. The qb:Slice construct does not represent an operator over an existent dataset, but the result of the application of a constraint over the dimension values on an existent data cube. Observations that populate the slice are not automatically computed from the observations in the original data cube and must be explicitly added to the RDF graph.We further explain the vocabulary through an example, and refer the reader to the reference for details.
Figure 3 (a) shows a representation using QB of the dimension geoDim. Dimension members are represented using URIs defined in Data.gov.uk5 according to the Ordnance Survey Administrative Geography Ontology6. For the sake of space we only include the data corresponding to the Reading UA and omit prefix declarations. A dimension property is used to define the dimension (line 1). Then, a concept scheme defines the 5 http://statistics.data.gov.uk/def/administrative-geography 6 http://www.ordnancesurvey.co.uk/oswebsite/ontology/v1/ AdministrativeGeography.rdf 1 eg:dateDim a qb:DimensionProperty, 2 qb:CodedProperty. 3 eg:household a qb:MeasureProperty. 4 eg:householdCS a qb:DataStructureDefinition; 5 qb:component [qb:dimension eg:geoDim]; 6 qb:component [qb:dimension eg:dateDim]; 7 qb:component [qb:measure eg:household]; 8 qb:component [qb:attribute 9 sdmx attribute:unitMeasure] . 10 eg:dataset hh a qb:DataSet; 11 rdfs:label "Household in UK"@en; 12 qb:structure eg:householdCS. 13 eg:o1 a qb:Observation; 14 qb:dataSet eg:dataset hh ; 15 eg:geoDim ns0:00mc ; 16 eg:dateDim db:2007; 17 eg:household 58 ; 18 sdmx attribute:unitMeasure db:Thousand. (a) The geoDim dimension structure.
(b) Cube structure and instances. hierarchical relationship between dimension members, which is linked to the dimension property using the qb:codelist property (line 2). The most general concepts within the hierarchy are defined using the skos:topConcept property. In this example there is only one top concept (line 4) and it corresponds to an instance of the Country class (England). Hierarchical relationships among members (from the more general concepts down to more specific ones) are stated using the skos:narrower property (lines 9 and 13). Figure 3 (b) shows a representation, using QB, of the data cube of Figure 2a. The observation depicted represents the household corresponding to Reading in 2007.
The implementation of OLAP operations is not considered in the specification of the QB vocabulary. In spite of this, some OLAP operations can be defined over a structure based on QB, although in a limited way. For example, since dimension levels and aggregate functions for each measure are not modeled, is not possible to implement Roll-Up over QB. The same issues apply to Drill-down and Slice. Finally, a particular case of Dice could be implemented over QB, given that the FO formula can only involve cube measures (again, because of the lack of support of dimension levels).
Kämpgen et al. [
Open Cube (OC) [
The Open Cube vocabulary overcomes the problems of QB to support OLAP
operations. In [
Comparing approaches. Table 1 summarizes the modeling features supported by each
proposal. Note that OC allows representing all the MD concepts in Table 1, however at
the expense of the need of adopting a new vocabulary, which prevents reusing data cubes
already published using QB. Conversely, [
In Section 2 we have shown that QB presents some shortcomings to support BI analysis on the semantic web. On the other hand, OC does not support existing cubes based on QB, that means, data already published using QB (and applications built over this model) cannot be reused. This represents a problem since, as we already mentioned, although QB is not yet a W3C candidate standard, many datasets have already been published using QB. Therefore, to allow BI analysis and, at the same time, supporting existing applications over data published using QB, we propose the QB4OLAP vocabulary7. QB4OLAP allows either publishing and analyzing MD from scratch (similarly to what can be done using OC), or performing OLAP analysis on observations already published using QB. To achieve this, QB4OLAP adds to QB the capability of representing dimension levels, level members, rollup relations between levels and level members, and associating aggregate functions to measures.
Figure 4 presents the QB4OLAP vocabulary, where classes and properties that are added to QB (with prefix qb4o) are depicted in light gray background. The class 7 http://publishing-multidimensional-data.googlecode.com/git/ index.html
Fig. 4: The QB4OLAP Vocabulary qb4o:LevelProperty models dimension levels; the qb4o:parentLevel property represents relations between dimension levels. Level members are represented as instances of the class qb4o:LevelMember, and rollup relations between them can be expressed using the property skos:broader. Level attributes are defined via the qb4o:hasAttribute property. In the DSD, level properties are stated. These levels, like in OC, correspond to the lowest levels in the hierarchy of each dimension in the cube. Also, like in OC, although dimensions are not directly linked to the data structure, they can be reached navigating from any level using the qb4o:inDimension property. The class qb4o:AggregateFunction represents aggregate functions, and the association between measures and aggregate functions in a data cube is represented using the property qb4o:hasAggregateFunction. This property, together with the idea of component sets, allows a given measure to be related to different aggregate functions in different cubes, which is an improvement with respect to OC where each measure could only be related to one aggregate function.
We now show how QB4OLAP can be used to publish MD data from scratch. Later, in Section 5, we present how this vocabulary can also be used to analyze data already published using QB. Figure 5 (a) shows the definition of the geoDim dimension schema and instances using the QB4OLAP vocabulary. Figure 5 (b) shows how the data cube of Figure 2a can be represented using QB4OLAP. Although resembling the definition of the cube in QB presented in Figure 3 (b), there are some relevant differences. The data cube schema is declared in lines 8 through 12, using an instance of qb:DataStructureDefinition. Level properties (qb4o:level) are used to specify the cube schema, instead of dimension properties (qb:dimension). Also, the aggregate function corresponding to measure eg:household is stated using the qb4o:hasAggregateFunction property (line 12). Data cube instances are modeled using the qb:Observation class, and grouped using the qb:DataSet class, which is linked to the corresponding fact schema using the property qb:structure. 1 eg:geoDim a qb:DimensionProperty. 1 eg:dateDim a qb:DimensionProperty. 2 2 3 eg:unitaryAuthority a qb4o:LevelProperty; 3 eg:year a qb4o:LevelProperty; 4 qb4o:inDimension eg:geoDim; 4 skos:closeMatch db:Year; 5 qb4o:parentLevel eg:governmentOfficeRegion. 5 qb4o:inDimension eg:dateDim. 6 eg:governmentOfficeRegion a qb4o:LevelProperty; 6 7 qb4o:inDimension eg:geoDim; 7 eg:household a qb:MeasureProperty. 8 qb4o:parentLevel eg:country. 8 eg:householdCS a qb:DataStructureDefinition; 9 eg:country a qb4o:LevelProperty; 9 qb:component [qb4o:level eg:unitaryAuthority]; 10 qb4o:inDimension eg:geoDim; 10 qb:component [qb4o:level eg:year]; 11 skos:closeMatch adgeo:Country. 11 qb:component [qb:measure eg:household; 12 12 qb4o:hasAggregateFunction qb4o:sum]. 13 ns0:00mc qb4o:inLevel eg:unitaryAuthority; 13 eg:dataset hh a qb:DataSet; 14 rdfs:label "The Borough of Reading@en"; 14 rdfs:label "Household in UK"@en; 15 skos:broader ns1:J. 15 qb:structure eg:householdCS. 16 ns1:J qb4o:inLevel eg:governmentOfficeRegion; 16 eg:o1 a qb:Observation; 17 rdfs:label "South East@en" ; 17 qb:dataSet eg:dataset hh ; 18 skos:broader ns2:921. 18 eg:unitaryAuthority ns0:00mc ; 19 ns2:921 qb4o:inLevel eg:country; 19 eg:year db:2007; 20 rdfs:label "England@en". 20 eg:household 58.
(a) The geoDim dimension structure
(b) Data cube structure and instances
One of the key aspects of OC is that it supports performing OLAP operators presented
in Section 2, directly over the RDF representation of cubes. In [
The QB4OLAP vocabulary is compatible with QB, in the sense that QB4OLAP cube schemas can be built on top of data cube instances (observations) already published using QB. Existing applications, or applications that do not require OLAP style-analysis, can still use the QB schema and instances. Therefore, the cost of adding OLAP capabilities to existing datasets is the cost of building the new schema, in other words, the cost of building the analysis dimensions. Conversely, cubes built over QB4OLAP from scratch can be transformed into QB cubes in order to be exploited by existing applications
HouseholdByGOR schema 1 CONSTRUCT { ?id a qb:Observation . ?id qb:dataSet eg:dataset hh1 . ?id eg:year ?year . 2 ?id eg:governmentOfficeRegion ?gor . ?id eg:household ?sumHhold } 3 WHERE{ { SELECT ?gor ?year (SUM(?hhold) AS ?sumHhold) 4 (iri(fn:concat("http://example.org/hhold#", "hholdGOR","_", 5 fn:substring after(?gor,"http://example.org/hhold#"),"_", 6 fn:substring after(?year,"http://example.org/hhold#"))) AS ?id) 7 WHERE { ?o qb:dataSet eg:dataset hh . ?o eg:year ?year . 8 ?o eg:household ?hhold . ?o eg:unitaryAuthority ?ua . 9 ?ua skos:broader ?gor . ?gor qb4o:inLevel eg:governmentOfficeRegion; 10 }GROUP BY ?gor ?year}}
HouseholdByGOR instances supporting the latter. As in the case above, the cube instances remain untouched. Figure 7 depicts this scenario. One of the main differences between QB4OLAP and QB is the possibility of the former of specifying a dimension hierarchy. On the contrary, QB allows dimension members to be hierarchically organized into a skos:ConceptScheme structure using skos:narrower and skos:broader properties. This skos:ConceptScheme represents a hierarchy of level members instead of a hierarchy of levels. Therefore, generating a QB4OLAP cube from a QB cube requires knowing the hierarchy of levels. Thus, we assume that this hierarchy can be obtained from the hierarchy of level members, either automatically or manually (e.g., by a curator). Implicitly, this also assumes that the association between members and dimension levels can also be obtained. If the dimension does not have hierarchical information, we assume a hierarchy of only one level. Also, generating a QB4OLAP cube from a QB cube requires knowing the association between measures and aggregate functions, which we assume can be obtained automatically or through the intervention of an expert user.
Recall that observations in QB are expressed in terms of dimensions, while QB4OLAP requires observations to be expressed in terms of dimension levels. Therefore, to avoid rewriting observations, we propose to reuse the URIs that represent dimensions in QB, to represent the bottom level of each dimension in the generated QB4OLAP schema (we show this in lines 8 and 9 in Figure 8 (a)).
Algorithm 1 receives a QB cube schema, and produces a QB4OLAP cube schema. The new schema must be linked to the dataset containing the (existing) observations8.The algorithm creates and populates the dimension structure, and creates a new DSD. Algorithm 1 Creating a cube in QB4OLAP from a cube in QB Input: dsd1 is the data structure definition of a data cube c1 in QB, D1 is the set of dimensions in c1, M is the set of pairs (mi; agi) where mi is a measure and agi is its corresponding aggregate function. Output: dsd2 is the data structure definition of a data cube c2 in QB4OLAP that allows to analyze the observations that populated dsd1 1: for all di 2 D1 (di a qb:DimensionProperty) do 2: Create a new dimension dj 2 D2 (dj a qb:DimensionProperty) 3: Obtain a hierarchy of levels hli and a hierarchy of level members hmi for di 4: for all li 2 hli do 5: Add triples (li a qb4o:LevelProperty) and (li qb4o:inDimension dj ) 6: for all lmi 2 hmi such that lmi belongs to level li do 7: Add a triple (lmi qb4o:inLevel li). 98:: end efonrd for 10: for all (lj ; lk) 2 hli such that lj ! lk do 11: Add a triple (lj qb:parentLevel lk) 1123:: iefnldi fisorthe bottom level in hli then 14: Add a triple (dsd2 qb:component [qb4o:levelli]) 15: end if 16: end for 17: for all mi such that (dsd1 qb:component [qb:measure mi]) do 18: Add a triple (dsd2 qb:component [qb:measure mi;qb:hasAggregateFunction agi]) 19: end for Example 3. Figure 8 (a) presents the result of applying Algorithm 1 to the data cube of Figure 3. A new prefix eg1 is used to define new dimensions and the new cube schema. In lines 1 to 10 we show the triples that represent geoDim dimension, lines 12 to 15 show the representation of the dateDim dimension, while lines 19 to 23 present the DSD of the new cube. This DSD has to be added as another structure of the dataset that contains the observations to be analyzed, which is done in line 25. 5.2
Algorithm 2 receives a QB4OLAP cube schema and creates a QB cube schema. To avoid rewriting observations we propose a similar strategy than in Algorithm 1. In this case we reuse the URIs that represent the lowest levels in each dimension in QB4OLAP, 8 We are not interested here in discussing the most efficient way of generating cubes. We are just proving that this generation is feasible. 1 eg1:newGeoDim a ab:DimensionProperty. 1 eg:unitaryAuthority a qb:DimensionProperty; 2 eg1:country a qb4o:LevelProperty; 2 qb:codeList eg2:cl1. 3 qb4o:inDimension eg1:newGeoDim. 3 eg2:cl1 a skos:ConceptScheme; 4 ns2:921 qb4o:inLevel eg1:country. 4 skos:hasTopConcept ns0:00mc. 5 eg1:governmentOfficeRegion a qb4o:LevelProperty; 5 6 qb4o:inDimension eg1:newGeoDim. 6 ns0:00mc skos:inScheme eg2:cl1. 7 ns1:J qb4o:inLevel eg1:governmentOfficeRegion. 7 ns1:J skos:inScheme eg2:cl1. 8 eg:geoDim a qb4o:LevelProperty; 8 ns2:921 skos:inScheme eg2:cl1. 9 qb4o:inDimension eg1:newGeoDim. 9 10 ns0:00mc qb4o:inLevel eg:geoDim. 10 11 11 12 eg1:newDateDim a ab:DimensionProperty. 12 eg:year a qb:DimensionProperty; 13 eg:dateDim a qb4o:LevelProperty; 13 qb:codeList eg2:cl2. 14 qb4o:inDimension eg1:newDateDim. 14 eg2:cl2 a skos:ConceptScheme. 15 db:2007 qb4o:inLevel eg:dateDim. 15 db:2007 skos:inScheme eg2:cl2. 16 16 17 eg1:household a qb:MeasureProperty. 17 eg2:household a qb:MeasureProperty. 18 18 19 eg1:householdCS a qb:DataStructureDefinition; 19 eg2:householdCS a qb:DataStructureDefinition; 20 qb:component [qb4o:level eg:geoDim]; 20 qb:component [qb:dimension eg:unitaryAuthority]; 21 qb:component [qb4o:level eg:dateDim]; 21 qb:component [qb:dimensionl eg:year]; 22 qb:component [qb:measure eg1:household; 22 qb:component [qb:measure eg2:household]. 23 qb:hasAggregateFunction qb4o:sum] . 23 24 24 25 eg:dataset hh qb:structure eg1:householdCS. 25 eg:dataset hh qb:structure eg2:householdCS. (a) Creating a cube in QB4OLAP from a (b) Creating a cube in QB from a cube in cube in QB
QB4OLAP to represent dimensions in the QB schema. Algorithm 2 creates and populates the dimension structure, and produces the DSD. For each dimension di a new dimension is created, identified by the URI corresponding to the lowest level in its hierarchy (lj ). The skos:ConceptScheme csi is created to represent the hierarchy of dimension members, and it is associated with the dimension di through a qb:CodeList property (line 5). Then, for each level li in the dimension, all its level members lmi are added to csi (line 8). We also need to state which level members lmi are top concepts in the concept scheme, to allow traversing the hierarchy via skos:broader relationships (lines 9 to 11). Measures are added to the new DSD in lines 15 to 17. 6
We have shown that the RDF Data Cube Vocabulary (QB) does not suffice for modeling and querying OLAP data cubes. On the other hand, the OC vocabulary, a former proposal by the authors of the present paper, overrides those problems, at the expense of not being compatible with already existing applications based on QB. Thus, we proposed a new vocabulary, which we denoted QB4OLAP, that fills the gap between QB and OC. We also provide algorithms to transform cubes based on QB into equivalent cubes supporting QB4OLAP cubes (and vice versa), showing that compatibility is obtained only at the cost of building the analysis dimensions.
We are currently implementing the OLAP operators using QB4OLAP as http
RESTful services, and working in a query language based (at the conceptual level) on the Cube
Algebra [