A challenging question is how XML can be used to support distributed databases. This leads to the problem of how to obtain a suitable, cost-efficient distribution design for XML documents. In this paper we sketch a heuristic approach to minimise query costs for the case of horizontal fragmentation. The approach is based on a cost model that takes the complex structure of queries on XML documents into account. We show that the minimisation of transportation costs is decisive, and that this can be achieved locally by either accepting or rejecting a horizontal fragmentation with a simple predicate that arises from one of the most frequent queries.
Since a few years XML has attracted a lot of attention as a highly expressive datamodel. In particular, XML enables an integrated view on data at least within an organisation, and it is not unrealistic to set up an enterprise-wide XMLdatabase. As large organisations are located at different places, a consequence of this view is that we have to deal with distributed databases. Thus, the problem arises to provide adequate distribution techniques for XML, and to show that these techniques will enable increased performance of the distributed systems in comparison to using centralised XML documents.
In the context of the relational datamodel (RDM) distribution design mainly addressed the problems of schema fragmentation and allocation of the fragments to the machines in a computer network [11]. Fragmentation is commonly divided into horizontal fragmentation, which splits a relation into disjoint unions, and vertical fragmentation, which projects a relation onto a subset of its attributes. Horizontal and vertical fragmentation have been discussed intensively in [3,4,7]. Several authors have approached the generalisation of fragmentation techniques to object oriented datamodels. For instance, horizontal fragmentation is discussed in [1,5,8], and vertical fragmentation in [2,6,10]. Due to the similarity between object oriented and semi-structured data, some of these techniques have also been adapted to XML [9].
In this paper we continue our work on distribution design for XML-based databases from [9], where we introduced fragmentation operations for XML. Finding a reasonable distribution is usually accompanied by a query performance cost model. The problem is to design fragments and to allocate them in such a way that the overall performance of the distributed database system is better than the one of an equivalent centralised one.
This problem was only briefly sketched in [9] assuming that XML documents are stored in a relational database and that cost optimisation techniques known from the RDM would be applied. In this paper we present an alternative approach that does not assume a relational representation of the XML documents. Instead of this we develop a query cost model for XML. Then we present a heuristic approach to minimise query costs for the case of horizontal fragmentation. We show that the minimisation of transportation costs is decisive, and that this can be achieved locally by either accepting or rejecting a horizontal fragmentation with a simple predicate that arises from one of the most frequent queries.
Let us now look briefly at query processing costs. For this we assume that the queries are expressed by some XML query algebra, so each query give rise to a query tree. We may then make the assumption that these query trees are optimised, which allows us to assume that the leaves of such trees are the input document(s), predecessors of leaves are selection nodes, which themselves have projection nodes as predecessors. In exceptional cases there are no such selection or projection nodes.
Crucial to the query costs are the sizes of documents that have to be built during query execution, as these documents have to be stored at secondary storage, retrieved from there again, and sent between the locations of a network. Therefore, we first approach an estimation of sizes adapting the work in [8].
In order to do so, we look at the elements and attributes a defined in the schema DTD, and estimate the size s( ) = s(e), respectively, depending on the defining regular expression e for , the attributes of , and the types of these attributes. Then, if r is the root element, s(r) is the (estimated) size of a document rooted at r. Let s i be the average size of elements in the set V i -recall that V i = dom(b i ) for a base type b i . In particular, let us assume b 0 = ID, i.e. s 0 is the average size of an identifier. This can be used to determine first the size s(a) of an attribute a, i.e. the average space needed for it in storage. We obtain s(a) = s i , if a is of type b i , s(a) = s 0 , if a is of an identifier or reference type, and s(a) = r • s 0 if a is of mutiple reference type. In the last of these cases r is the average number of elements named that are referenced in . Now let att( ) denote the set of attributes defined on element n. We can proceed inductively to define the size s(e):
-If the defining expression e for is a base type b i , then we obtain s(e) = b i .
-If the defining expression e for is , we obtain s(e) = a∈att( ) s(a).
-If the defining expression e for is an element name n, we obtain s(e) = s(n) + a∈att( ) s(a).
-If the defining expression e for is an iteration (e ) * , we obtain s(e) = q • s(e ) + a∈att( ) s(a), where q is the average number of successor elements with defining expression e .
-If the defining expression e for is a sequence e 1 , . . . , e n , we obtain s(e) = n i=1 s(e i ) + a∈att( ) s(a).
-If the defining expression e for is a choice e 1 | • • • | e n , we obtain s(e) = n i=1 p i • s(e i ) + a∈att( ) s(a), where p i is the probability that the successor of is defined by e i . In particular, we have
Horizontal fragmenentation corresponds to replacing a document x by some union x 1 υ p . . . υ p x n . The path p used herein just leads to a successor of the root element. The new documents x i (i = 1, . . . , n) are all obtained by selections, and we may assume n = 2 by switching to horizontal fragmentation with normal predicates, i.e. satisfiable conjunctions of simple selection formulae.
Another round of query optimisation might shift the selection σ p2,ϕ and the projection π p1,e inside the newly introduced union ν p3 , but the "upper part" of the query tree would not be affected. Therefore, in order to optimise horizontal fragmentation, it is decisive and sufficient to consider the projection-selection subqueries above.
The size of a selection node σ p ,ϕ is p • s, where s is the size of the successor node in the query tree and p is the probability that an element that matches the path p will satisfy ϕ. The size of a projection node π p ,e is (1
where s is the size of the successor node in the query tree, s o is the average size of an element reached by p, s e is the average size of an element defined by e, and p is the probability that two elements x 1 , x 2 matching p coincide on their projection to e, i.e. π e e (x 1 ) = π e e (x 2 ). Fragmentation results in a set of fragments {f 1 , . . . , f n } of average sizes s 1 , . . . , s n . If the network has nodes N 1 , . . . , N k we have to allocate these fragments to the nodes, which gives rise to a mapping λ : {1, . . . , n} → {1, . . . , k}, which we call a location assignment.
However, the fragments only appear on the leaves of query trees. More generally, we must associate a node λ(v) with each node v in each relevant query tree. λ(v) indicates the node in the network, at which the intermediate query result corresponding to v will be stored.
Given a location assignment λ we can compute the total costs of query processing. Let the set of queries be Q m = {Q 1 , . . . , Q m }. Query costs are composed of two parts: storage costs and transportation costs:
The storage costs give a measure for retrieving the data back from secondary storage, which is mainly determined by the size of the data. The transportation costs provide a measure for transporting between two nodes of the network.
The storage costs of a query Q j depend on the size of the involved documents and on the assigned locations, which decide the storage cost factors. It can be expressed as
, where h ranges over the nodes of the query tree for Q j , s(h) are the sizes of the involved documents, and d i indicates the storage cost factor for node N i (i = 1, . . . , k).
The transportation costs of query Q j depend on the sizes of the involved documents and on the assigned locations, which decide the transport cost factor between every pair of sites. It can be expressed by trans
Again the sum ranges over the nodes h of the query tree for Q j , h runs over the predecessors of h in the query tree, and c ij is a transportation cost factor for data transport from node N i to node N j (i, j ∈ {1, . . . , k}).
For each query Q j we get a value for its frequency freq j . The total costs of all the queries in Q m are the sum of the costs of each query multiplied by its frequency. It can be expressed by
We now present a heuristic approach for the optimisation of horizontal fragmentation. The major objective of the approach is to base the fragmentation decision on the efficiency of the most frequent queries. We have already seen that we may concentrate on projection-selection subqueries (PSSs) and that horizontal fragmentation with follow-on algebraic query optimisation will only affect these subqueries. As each such subquery defines a set of simple selection formulae. Obviously, we should only be interested in horizontal fragmentation based on normal predicates that are defined by this set of simple selection predicates.
Let us first look at the effects to query costs of a fragmentation with a single simple predicate. Basically we have to adapt query trees, take into account which effect algebraic optimisation will have on the PSSs, and find a modified location assignment λ that would reduce the query processing costs, provided such a λ exists. We have to distinguish three cases for this scenario.
Case I. Assume that the selection formula ϕ in the PSS has the form ϕ ≡ ψ∧ϕ 0 and that we use ϕ 0 to fragment the document x. So, the leave x would be replaced by the fragment x 1 of x that is determined by ϕ 0 , and its predecessor in the query tree would become σ p,ψ . There are no other changes to the query tree.
Then the size associated with the projection node does not change. Therefore, if the location assignment λ was optimal before the fragmentation, it does not make sense to change it for any node other than the three nodes corresponding to the PSS. Unless the selection node is deleted from the query tree, its size remains unchanged. This implies that there is at most a small change to the storage costs. In particular, as transportation costs are larger anyway, we neglect this effect on the storage costs. The processing of the subquery itself does not require any transport of data. Therefore, we can assume that λ assigns the same network node N i to these nodes in the query tree.
As a consequence, the fragmentation can only require the change of the assigned network node from N i to some N j . This corresponds to the allocation of the fragment x 1 . to a re-allocation of a fragment, whereas the fragmentation with elements in Φ − Ψ does not add changes. Then can be determined by binary search.
In this paper we continued our work on distribution design for XML-based databases from [9]. We introduced a detailed query performance cost model, and then addressed the problem to design fragments and to allocate them in such a way that the overall performance of the distributed system is better than the one of an equivalent centralised one. Our approach does not assume a relational representation of the XML documents. Instead of this we presented a heuristic approach to minimise query costs for the case of horizontal fragmentation. We showed that the minimisation of transportation costs is decisive, and that this can be achieved locally by either accepting or rejecting a horizontal fragmentation with a simple predicate that arises from one of the most frequent queries.
Case II. Assume now that the fragmentation with the selection formula ϕ 0 leads to two fragments x 1 and x 2 . Then the PSS would be replaced by a subquery of the form π p1,e ( p σ p2,ϕ ( p x 1 υ p3 x 2 / ) / ). Algebraic query optimisation will move the selection inside the union.
In this case the size associated with the projection node does not change. As we can assume that the union node is assigned to the same network node as the projection node, we obtain again marginal changes to the storage costs. Thus, we can concentrate on the transportation costs. In addition, the processing of the subquery would require sending the smaller selection result to the location of the larger one.
As a consequence, the fragmentation can only require that the network node assigned to the two new fragments are different and maybe also different from the network node that was assigned to x. Then the fragment with the larger size after selection will determine the location assignment for the union and the projection node.
The assumption for the third case is the same as for case II. However, we now assume that the sizes of σ p2,ϕ (x i ) are almost equal for i = 1, 2, and that the projection has only a small impact on the size of the result of the PSS. In this case it is advantageous to take the predecessor node n of π p1,e in the query tree, and to assign λ(n) to the projection and the union node. This implies that we have to transport both selection results to the same network node λ(n). As in the two other cases, we may neglect the changes to the storage costs. Now consider all the PSSs. Let {q 1 , . . . , q N } be the set of these local queries. Each of these local queries q i has a "target node" T i , i.e. a network node to which the result of the query will be transported. Let s i be the size of the result of q i and let freq i be the frequency of q i (i = 1, . . . , N). Let x be some document fragment. Then for each network node N j the costs of allocating x to the node
We apply the cost minimisation heuristics, which allocates a fragment x to a network node N h such that cost h (x) will be minimal. According to our discussion for the three basic cases of how fragmentation affects the query costs, the allocation of fragments to network nodes following the cost minimisation heuristics, already determines the location assignment, provided that an optimal location assignment for the queries was given prior to the fragmentation. Now we observe that one of the two fragments resulting from horizontal fragmentation with a formula in Φ will reside at the same location as the document before fragmentation, whereas the other fragment will be moved to a new location. In the first case the transportation costs remain the same. In the second case the transportation costs will be reduced.
This suggests to take a total order on the elements of Φ according to their frequency, i.e. let Φ = {ϕ 1 , . . . , ϕ } with freq(ϕ i ) ≥ freq(ϕ j ) iff i ≤ j. Then determine Ψ = {ϕ 1 , . . . , ϕ } such that fragmentation with elements in Ψ leads