<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>A Quantitative Study of Two Matrix Clustering Algorithms</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Alexander Slesarev</string-name>
          <email>alexander.g.slesarev@gmail.com</email>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Viacheslav Galaktionov1, 2 Nikita Bobrov1, 2</string-name>
          <email>nikita.v.bobrov@gmail.com</email>
          <email>viacheslav.galaktionov@gmail.com</email>
          <email>viacheslav.galaktionov@gmail.com nikita.v.bobrov@gmail.com</email>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>George Chernishev1, 2</string-name>
          <email>g.chernyshev@spbu.ru</email>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>1 JetBrains Research, 1 JetBrains Research, 2 Saint-Petersburg State University, Russia 2 Saint-Petersburg State University</institution>
          ,
          <addr-line>Russia, Saint-Petersburg, Russia Saint-Petersburg</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>1 JetBrains Research, 2 Saint-Petersburg State University</institution>
          ,
          <addr-line>Russia, Saint-Petersburg</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Saint-Petersburg University</institution>
          ,
          <addr-line>Russia, Saint-Petersburg</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>-Matrix clustering is a technique which permutes rows and columns of a matrix to form densely packed regions. It originated in the 70's and initially was used for various object grouping problems, such as machine-component grouping. The database community noticed these algorithms and successfully applied them to the vertical partitioning problem. Recently, there has been a resurgence of interest in these algorithms. Nowadays, they are being considered for dynamic (on-line) vertical partitioning and tuning of multistores. In our previous papers we have described our project aimed at studing the applicability of recent matrix clustering algorithms for the vertical partitioning problem. We have presented our evaluation approach and reported results concerning several of these algorithms. Our idea was to evaluate them directly using the PostgreSQL database. Previous studies have found that these algorithms can be of use if they employ the attribute replication strategy. In this paper, we continue our investigation and consider a novel algorithm of this class. Its distinctive feature is that it performs attribute replication during the branch and bound search. We compare it with the best one of the earlier algorithms using both real and synthetic workloads. Our experiments have demonstrated that the novel algorithm produces slightly worse configurations (about 10%), but its run times are significantly better and are almost independent of the cohesion parameter. Index Terms-databases, database tuning, physical design, vertical partitioning, experimentation, matrix clustering, fragmentation.</p>
      </abstract>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>I. INTRODUCTION</title>
      <p>
        Vertical partitioning is a technique used to speed up query
processing in databases. Its core idea is dividing a table into
fragments which contain only a subset of attributes. In order
to ensure that the database will not undergo semantic changes,
the following rules of vertical partitioning are used [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ]:
completeness, reconstruction, and disjointness. Sometimes the
disjointness rule is relaxed. In this case, it is said that vertical
partitioning is performed with attribute replication.
      </p>
      <p>
        The speedup comes from the fact that some queries would
have to read less data. Indeed, suppose that for a given
query all needed attributes are allocated into a single
fragment, and this fragment contains no extra attributes. In this
case, one can roughly estimate that number of rows
extra attributes lengths bytes can be saved during the data
reading phase in case of a slotted page data layout [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ].
      </p>
      <p>However, if there is a query that requires attributes from
two or more fragments, then its performance may suffer due to
the record reconstruction costs. Data modification operations
(inserts, deletes, and updates) complicate things further since
they involve all attributes of a record and thus, all fragments
should be modified. The impact of additional disk seeks on a
hard drive may be so large that it can make the partitioning
scheme impractical.</p>
      <p>Due to all these facts, there is still no support of
fullyautomatic vertical partitioning in industrial database systems.
Moreover, unlike the horizontal, vertical partitioning is not
supported in SQL DDL: e.g., in PostgreSQL it is possible
to define horizontal fragments using the “PARTITION BY”
clause for a “CREATE TABLE” statement.</p>
      <p>
        Nevertheless, there are multiple semi-automatic stand-alone
tools (“advisors”, see surveys [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ], [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ]) for this task. All of
them recommend beneficial vertical partitioning schemes for a
specified workload (queries) and let the database administrator
decide whether to implement them or not.
      </p>
      <p>
        The reason for the limited success of these tools (the
overwhelming majority of them are academic research prototypes
and not industrial products) is that finding an optimal solution
is an NP-hard problem for many different formulations [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]–
[
        <xref ref-type="bibr" rid="ref7">7</xref>
        ]. Another well-known fact is that the number of different
vertical partitioning schemes for a single table is equal to
the N th Bell number, where N is the number of attributes
[
        <xref ref-type="bibr" rid="ref8">8</xref>
        ]. Nevertheless, due to the interest of both industrial and
academic communities, the development of such advisors
continues.
      </p>
      <p>In the core of such a system lies an algorithm that traverses
the partitioning space and selects a beneficial scheme. There
are two classes of algorithms for this task: cost-based and
heuristic. The former employ some kind of a cost-based model
to evaluate the quality of a given partitioning scheme in terms
of query run times, required space, and other metrics. The
latter proposes some kind of procedure to generate a “good”
scheme. Usually, some considerations are presented as to why
it is likely to generate a beneficial partitioning scheme, but not
a strict proof.</p>
      <p>
        The heuristic approach was very popular in the 70’s and
80’s, but later was abandoned in favour of the cost-based
one. Nowadays, there is a resurgence of interest in heuristic
approaches due to the appearance of novel application areas:
dynamization of vertical partitioning [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]–[
        <xref ref-type="bibr" rid="ref12">12</xref>
        ], tuning of
multistores [
        <xref ref-type="bibr" rid="ref13">13</xref>
        ], big data applications or any other cases featuring
limited resources.
      </p>
      <p>
        In our previous studies [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]–[
        <xref ref-type="bibr" rid="ref16">16</xref>
        ] we have described our
project that aims to study the applicability of several
recently developed matrix clustering algorithms. Our project is
motivated by the fact that the authors of these algorithms
have not evaluated their performance (run times, quality)
using a DBMS and a workload. To address this, we have
constructed a framework for evaluating such algorithms that
uses PostgreSQL. Then we have evaluated a number of these
algorithms [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]–[
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] using the TPC-H benchmark. In this
paper, we continue our research and consider the most recent
algorithm of this type [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ].
      </p>
      <p>The rest of this paper is organized as follows. In Section II
we provide a short introduction into the subject and describe
existing types of heuristic approaches. Next, in Section III
we introduce matrix clustering algorithms and provide a
description of the considered algorithm. Section IV describes
our experimental framework, setup, and the experiments. The
results of evaluation are discussed in Section V, threats to
validity of this study are presented in Section VI and Section VIII
concludes this paper.</p>
    </sec>
    <sec id="sec-2">
      <title>II. RELATED WORK</title>
      <p>
        As it was stated in the Introduction, there are two types
of approaches to the vertical partitioning problem —
costbased and heuristic. Since this problem is almost 40 years
old, and a lot of results have been accumulated, we will
only describe studies on heuristic algorithms in this section.
More extensive surveys that examine cost-based approaches
as well can be found in references [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ]. Heuristic vertical
partitioning algorithms can be classified into the following
major groups [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ], [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ]:
      </p>
      <p>
        Attribute affinity and matrix clustering approaches [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]–
[
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]–[
        <xref ref-type="bibr" rid="ref24">24</xref>
        ]. Attribute affinity is a measure which
shows how frequently two attributes are requested
together in a given workload. These approaches use it as
follows:
2) Attribute affinity is calculated for all pairs of
attributes and an Attribute Affinity Matrix (AAM) is
constructed.
3) A special algorithm for row and column permutation
is applied to the AAM. Afterwards, “dense” regions
are extracted and used to define resulting partitions.
Studies employing the matrix clustering approach (and
in particular, the ones considered in our paper) permute
AUMs, but not AAMs.
      </p>
      <p>
        Graph approaches [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ], [
        <xref ref-type="bibr" rid="ref25">25</xref>
        ]–[
        <xref ref-type="bibr" rid="ref28">28</xref>
        ]. Similarly to the
previous type, these approaches start with a workload and
use it to construct an AAM. However, in this case
the AAM is considered as an adjacency matrix of an
undirected weighted graph, where the nodes are attributes
and the edge weights show the affinity for a given pair
of attributes. Finally, this graph is used to search for
special structures which will be used to define resulting
partitions. There are many approaches, e.g. Kruskal-like
algorithms or cutting the Hamiltonian way.
      </p>
      <p>
        Data mining approaches [
        <xref ref-type="bibr" rid="ref29">29</xref>
        ]–[
        <xref ref-type="bibr" rid="ref31">31</xref>
        ]. In this type of
approach, association rule mining is used to derive vertical
fragments. The workload is considered as a transaction
set, and the rules use sets of attributes as items. This
group of vertical partitioning algorithms is relatively new,
so existing algorithms for association rule search are
frequently used. For example, a popular choice is to adapt
Apriori [
        <xref ref-type="bibr" rid="ref32">32</xref>
        ] or FP-Max algorithms.
      </p>
    </sec>
    <sec id="sec-3">
      <title>III. MATRIX CLUSTERING ALGORITHMS</title>
      <sec id="sec-3-1">
        <title>A. Basics</title>
        <p>
          The general scheme of this approach is as follows [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]:
Construct an Attribute Usage Matrix (AUM) from the
workload. The matrix is defined as follows:
        </p>
        <p>Mij =
(1; query i uses attribute j</p>
        <p>0; otherwise
Cluster the AUM by permuting its rows and columns to
obtain a block diagonal matrix.</p>
        <p>Extract these blocks and use them to define the resulting
partitions.</p>
        <p>Some approaches do not operate on a 0-1 matrix. Instead, they
modify matrix values to account for additional information
like query frequency, attribute size and so on. Let us consider
an example. Suppose that there are six queries accessing six
attributes:
q1: SELECT a FROM T WHERE a &gt; 10;
q2: SELECT b, f FROM T;
q3: SELECT a, c FROM T WHERE a = c;
q4: SELECT a FROM T WHERE a &lt; 10;
q5: SELECT e FROM T;
q6: SELECT d, e FROM T WHERE d + e &gt; 0;
1) A workload is used to construct an Attribute Usage
Matrix (AUM), a special way to represent which
attributes are used by each query of a workload.</p>
        <p>The next step is the creation of an AUM using this
workload. The resulting matrix is shown in Figure 1a. After the
application of a matrix clustering algorithm, the reordered
a
1
0
1
1
0
0
b
1
1
1
0
0
0
(a) AUM
AUM (Figure 1b) is acquired. The resulting fragments are the
following: (a; b), (b; f ), (d; e).</p>
        <p>However, not all matrices are fully decomposable. Consider
the matrix presented in Figure 2. The first column obstructs
the perfect decomposition into several clusters. In this case, the
algorithm should produce a decomposition which minimally
harms query processing and results in an overall performance
improvement. Matrix clustering algorithms employ different
strategies to select such a decomposition.</p>
        <p>
          A systematic review of matrix clustering algorithms is
presented in studies [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]. Here, we will consider only
the recent approaches.
        </p>
      </sec>
      <sec id="sec-3-2">
        <title>B. Recent Advances</title>
        <p>
          Within our project, we study a series of works by
ChunHung Cheng et al [
          <xref ref-type="bibr" rid="ref17">17</xref>
          ]–[
          <xref ref-type="bibr" rid="ref20">20</xref>
          ]. These algorithms employ a
branch and bound search that tries to find submatrices that
conform to specific conditions. Their input is the threshold
(target cohesion), which is defined as the share of 1’s in the
resulting matrices.
        </p>
        <p>
          In this study we are interested in two algorithms — A09 [
          <xref ref-type="bibr" rid="ref19">19</xref>
          ]
and A11 [
          <xref ref-type="bibr" rid="ref20">20</xref>
          ].
        </p>
        <p>The A09 algorithm comes with three different strategies
that define the treatment of intersubmatrix attributes (the ones
that were marked as obstacles to decomposition) — nearest,
separate, and replicate. In the first one such attribute goes to
the nearest submatrix, in the second all such attributes are
assigned to a dedicated submatrix, and the last one replicates
the attribute into each submatrix that requires it. Note that the
strategy is applied after the clustering is done.</p>
        <p>The A11 algorithm has a different idea. If during branch
and bound traversal the algorithm encounters such an attribute,
then it replicates it and tries to decompose the matrices further.
(b) Reordered AUM</p>
      </sec>
    </sec>
    <sec id="sec-4">
      <title>IV. EXPERIMENTS</title>
      <sec id="sec-4-1">
        <title>A. Benchmarking</title>
        <p>In our previous works we have developed a special
prototype for experimental evaluation of matrix clustering
algorithms. The idea of our approach is to directly check whether
the generated partitioning schemes help to improve query
performance. For these purposes we employ the PostgreSQL
DBMS and several workloads, both real and synthetic.</p>
        <p>The architecture of our prototype is presented in Figure 3.
It consists of the following modules:</p>
        <p>The parser reads the workload from a file. It extracts the
queries and passes them to the executor, so that their
execution times can be measured. It also constructs the
AUM, which serves as input for the selected algorithm.
The algorithm identifies clusters and passes that
information to the partitioner to create corresponding temporary
tables.</p>
        <p>The query rewriter also receives this information. It
replaces the name of the original table with the ones that
were generated by the partitioner.</p>
        <p>The partitioner generates new names and sends
partitioning commands to the database. The exact commands are
SELECT INTO and ALTER TABLE. The latter lets it
transfer primary keys.</p>
        <p>The executor accepts queries and sends them to
PostgreSQL to measure the time of execution.</p>
      </sec>
      <sec id="sec-4-2">
        <title>B. Experimental Setup and Evaluation Procedure</title>
        <p>In our experiments, we have used the following hardware
and software setup:</p>
        <p>Inspiron 15 7000 Gaming (0798), 8GiB, Intel(R)
Core(TM) i5-7300HQ CPU @ 2.50GHz, TOSHIBA 1TB
MQ02ABD1</p>
        <p>Ubuntu 18.10, PostgreSQL 11.1, gcc 8.2.0</p>
        <p>Data for quality-related graphs was obtained by running
10 invocations of the respective algorithm and averaging the
result. We deemed a single run sufficient for run time graphs,
since even one invocation can require up to two hours.</p>
        <p>In order to ensure maximum quality of experiments, several
measures were taken:
1) We eliminated data caching for both operating
system caches and PostgreSQL caches. For this, we
workload
vpart</p>
        <p>parser
algorithm
executor
partitioner
rewriter</p>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>PostgreSQL</title>
      <p>have restarted PostgreSQL and dropped the
operating system caches before running each query.
Operating system caches were dropped by writing “3” to
/proc/sys/vm/drop_caches.
2) Next, we manually checked plans for each query and
noticed that some queries may have different scan
operator implementations depending on the table.
Frequently, a query on a partitioned table did not have
sequential scan, but rather parallel. To handle this,
we have restricted the query optimizer to use only
sequential scans by issuing the following command set
max_parallel_workers_per_gather to 0;.</p>
      <p>To ensure that no hidden caching or other unaccounted
processes happen, we have designed the following simple
criterion. Suppose that we have a set of queries that involve
only a single table and are essentially scans without complex
data processing. Initially, we run these queries on the original
table and record their run times. Then, for every query we
designate a table that will contain all attributes necessary to
evaluate it. Thus, no joins are needed. At the same time, for
some queries, the tables assigned to them will also contain
extra attributes. Therefore, some tables may serve more than
one query. Then we run each query on corresponding table
and record its run time. Eventually the following two values
should be approximately equal:
1) P
2) Pqi2Queries(size(T )=time(qi))</p>
      <p>qi2Queries(size(table(qi))=time(qi))</p>
      <p>In these equations size(T ) is the size of a table in bytes.
Functions time(qi) and table(qi) return the time it took to run
a query qi and a table that corresponds to query qi.</p>
      <p>In other words, the idea is to check that workload run times
depend solely on the size of the table.</p>
      <p>Having applied all the aforementioned measures, we have
obtained the difference of about 10 15% in these values. We
deemed such a result acceptable and decided to start evaluating
the algorithms.</p>
      <p>
        Finally, we must note that our matrix clustering algorithms
are parallel [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ]. However, in this paper we did not consider
them and instead employed their sequential versions.
      </p>
      <sec id="sec-5-1">
        <title>C. Experiments</title>
        <p>In our study, we have addressed two applicability aspects of
matrix clustering algorithms: quality of generated partitioning
schemes and algorithm run times. Both of them are important
since quality is the primary characteristic of any partitioning
algorithm, and run times determine its suitability for on-line
vertical partitioning.</p>
        <p>
          To evaluate the quality of partitioning, we have compared
algorithm A11 to the best of other matrix clustering algorithms
(according to our previous studies [
          <xref ref-type="bibr" rid="ref14">14</xref>
          ], [
          <xref ref-type="bibr" rid="ref15">15</xref>
          ]) — A09. This
algorithm has three different strategies that were described
earlier. In our experiments we compare the quality of resulting
partitions of all three of them with the ones obtained by A11.
        </p>
        <p>To conduct experiments we have employed the “Star” table
of the SDSS (Sloan Digital Sky Survey) dataset. The SDSS is a
publicly available astronomical database that contains detailed
three-dimensional maps of the Universe. It is frequently used
as a testing dataset in various data partitioning studies. We
have used the following pack: SDSS-IV Data Release 14,
2016. Its “Star” table contains 509 attributes and 492515
records.</p>
        <p>To obtain representative workloads, we have also used the
SDSS dataset. In SDSS, it is possible to see what queries users
have issued via a special website1. Using this website, we have
selected 8 queries from the workload that address solely this
table.</p>
        <p>In our first experiment we have varied the cohesion measure
(a ratio of 1 in the resulting matrices) for three strategies of
A09 and compared it with A11. The results are presented in
Figure 5a. On this chart, each bar represents the performance
of an individual algorithm with the corresponding strategy.
There also are two horizontal lines: not clustered and pinched
not clustered. The first one is the workload run time on
the original, unmodified table. The second is the workload
run time on the cleaned up original table, containing only
30 attributes that are referenced in the workload. In this
experiment we varied the cohesion measure parameter.</p>
        <p>To evaluate algorithm run times we used both SDSS and
synthetic (generated) tests. The results of the SDSS tests are
presented in Figure 5b. Here, we also vary cohesion for the
same four algorithms.</p>
        <p>In the synthetic tests, we have tried to study the scalability
of the A11 algorithm in terms of run times. For this, we
have generated a set of random 0-1 matrices with different
probabilities of having 1 in each position (cohesion). Then,
we have examined the dependency of the run time on the
size of the matrix. The specified threshold was set to 0:9
in all experiments. If the threshold is more than the used
cohesion, then a solution (the original matrix) is found almost
1http://skyserver.sdss.org/log/en/traffic/
immediately. We also set a time limit of 2 hours, after reaching
which the algorithm is stopped.</p>
        <p>We started with square matrices (see Figure 4a), then
separately evaluated the influence of the number of columns
(Figure 4b) and the number of rows (Figure 4c) on the
algorithm run time. In the last two experiments we fixed one
dimension to 20 and increased the other up until the time limit
was reached.</p>
        <p>Finally, we have looked into the storage requirements of
these algorithms (Figure 6). Here, we show the required disk
space for each generated configuration. On top of each bar,
an overall number of fragments is shown. We have also
divided each bar into parts representing the sizes of resulting
fragments. The sizes of original and pinched tables are shown
by horizontal lines.</p>
      </sec>
    </sec>
    <sec id="sec-6">
      <title>V. RESULTS AND DISCUSSION</title>
      <p>All of the algorithms produced partitioning schemes that
provide better performance than the original and pinched
tables, regardless of the cohesion value.</p>
      <p>The quality of produced solutions heavily depends on the
cohesion value. Starting with the cohesion value of 0:8
results of A11 start to rival the results of the best A09
strategies. However, up to this point, the clear winner is
A09 with replication.</p>
      <p>Overall, the best result was produced by a replicating
variant of A09 (3:358, cohesion=0:55), with a separate
variant of A09 being the fourth (3:500, cohesion=0:8),
and A11 being the fifth (3:553, cohesion=0:8).
It is interesting to note that there is some sort of a global
minimum at the 0:7 point. Here, the total time over all
algorithms is minimal in the whole cohesion range.
With the SDSS workload algorithm A11 works almost
ten times faster than A09, regardless of the employed
strategy. Note that increasing the target threshold also
increases run times. For A09, run times increased from
less than 1 second to almost 140 seconds, while A11 took
0:06 and 0:119 seconds respectively.</p>
      <p>
        The scalability of A11 is not as good as desired. However,
two points should be taken into account. Firstly, run
times depend on the number of referenced attributes
in the workload, not on the total number. Secondly, in
our scalability experiments we used an extremely large
threshold of the cohesion — 0:9. Finally, the author [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]
noted that it is possible to interrupt the algorithm earlier
while still obtaining decent results. Therefore, further
studies are needed.
      </p>
      <p>Increasing the number of attributes impacts run times
more than increasing the number of queries. In two hours
time it is possible to process either a 20 25 matrix or
a 205 20 one.</p>
      <p>The solutions produced by all algorithms require from
1:5 to 2 times more disk space than the pinched table.
Increasing the target threshold increases the number of
fragments and the overall required disk space.
Interestingly, for high cohesion values A11 produces more
fragments, but does not help to improve performance.</p>
    </sec>
    <sec id="sec-7">
      <title>VI. THREATS TO VALIDITY</title>
      <p>
        We have identified a number of issues that should be kept
in mind while discussing our results:
1) First of all, the policy of database restarts after each
query may be unfair. In real-life scenarios where these
algorithms will hypothetically be used, database caching
would be present. However, such scenarios are nearly
impossible to simulate since they require hundreds or
thousands of real queries and more important, their
frequencies and arrival patterns.
2) Next, the SDSS dataset is only a single dataset, so
the results may differ on other datasets. Moreover, it
is a scientific dataset used by the astronomy research
community and therefore, its queries and data may not
be comparable to the industrial ones. Nevertheless, it
is popular in the vertical partitioning community (e.g.
see [
        <xref ref-type="bibr" rid="ref33">33</xref>
        ]–[
        <xref ref-type="bibr" rid="ref36">36</xref>
        ]) due to the lack of industrial schema-less
benchmarks.
3) There may be errors in our implementation of these
algorithms. In order to mitigate this threat we have tested
our implementation on example matrices presented in
the considered papers and ensured that the resulting
partitioned matrices are the same. Furthermore, to address
this issue we plan to release the source code on GitHub.
4) Contemporary DBMSes are very complex systems in
which minimal changes to inputs may drastically affect
performance. Therefore, during experimental evaluation
performance may change not due to vertical partitioning,
but due to other events, such as query optimizer selecting
a completely different plan. To counter this we have
carefully checked query execution plans to find and
eliminate any inconsistencies. We have also devised a
criterion that allows to detect such inconsistencies in
simple cases.
5) We have considered a relatively simple workload which
involves only a single table. Having to perform extra
joins in addition to the partitioning-induced ones may
significantly decrease overall performance and thus, the
desirability of vertical partitioning. However, joins with
other tables are extremely rarely considered in
literature [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]: only a handful of studies address them.
      </p>
    </sec>
    <sec id="sec-8">
      <title>VII. ACKNOWLEDGEMENTS</title>
      <p>We would like to thank Anna Smirnova for contributing to
the editing and proofreading process.</p>
    </sec>
    <sec id="sec-9">
      <title>VIII. CONCLUSION</title>
      <p>In this paper we have presented a quantitative study of
two recent matrix clustering algorithms. We have studied their
output quality, run times, and storage requirements using both
synthetic and real datasets.</p>
      <p>Our evaluation has shown that for schema-less data all
algorithms can produce a beneficial configuration, while a
(a) A11 run times on a square matrix
(b) Dependency of A11 run times on matrix width
(c) Dependency of A11 run times on matrix height</p>
      <p>Fig. 4: Run times of the A11 matrix clustering algorithm, synthetic datasets.
(a) Quality of partitioning (b) Algorithm run times
Fig. 5: Performance of the A11 and A09 matrix clustering algorithms, SDSS datasets.
replicating variant of A09 is 10% better than A11. However,
A11 is significantly faster and more importantly, less impacted
by the target threshold parameter.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>M. T.</surname>
          </string-name>
          <article-title>O¨zsu and P. Valduriez, Principles of Distributed Database Systems</article-title>
          , 3rd ed. Springer Publishing Company, Incorporated,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>R.</given-names>
            <surname>Ramakrishnan</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Gehrke</surname>
          </string-name>
          ,
          <source>Database Management Systems</source>
          , 3rd ed. New York, NY, USA:
          <string-name>
            <surname>McGraw-Hill</surname>
          </string-name>
          , Inc.,
          <year>2003</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>G.</given-names>
            <surname>Chernishev</surname>
          </string-name>
          , “
          <article-title>A survey of dbms physical design approaches</article-title>
          ,
          <source>” SPIIRAS Proceedings</source>
          , vol.
          <volume>24</volume>
          , pp.
          <fpage>222</fpage>
          -
          <lpage>276</lpage>
          ,
          <year>2013</year>
          . [Online].
          <source>Available: www.mathnet.ru/trspy580</source>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4] --,
          <source>“Vertical Partitioning in Relational DBMS,” 30 4</source>
          <year>2015</year>
          ,
          <article-title>talk at the Moscow ACM SIGMOD chapter meeting; slides and video</article-title>
          : http://synthesis.ipi.ac.ru/sigmod/seminar/s20150430.html [Accessed:
          <year>2015</year>
          11 09].
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>X.</given-names>
            <surname>Lin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Orlowska</surname>
          </string-name>
          , and
          <string-name>
            <given-names>Y.</given-names>
            <surname>Zhang</surname>
          </string-name>
          , “
          <article-title>A graph based cluster approach for vertical partitioning in database design,” Data &amp; Knowledge Engineering</article-title>
          , vol.
          <volume>11</volume>
          , no.
          <issue>2</issue>
          , pp.
          <fpage>151</fpage>
          -
          <lpage>169</lpage>
          ,
          <year>1993</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>D.</given-names>
            <surname>Sacca</surname>
          </string-name>
          and G. Wiederhold, “
          <article-title>Database partitioning in a cluster of processors,” ACM Trans</article-title>
          . Database Syst., vol.
          <volume>10</volume>
          , pp.
          <fpage>29</fpage>
          -
          <lpage>56</lpage>
          ,
          <year>1985</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>P. M. G.</given-names>
            <surname>Apers</surname>
          </string-name>
          , “
          <article-title>Data allocation in distributed database systems</article-title>
          ,
          <source>” ACM Trans. Database Syst.</source>
          , vol.
          <volume>13</volume>
          , pp.
          <fpage>263</fpage>
          -
          <lpage>304</lpage>
          ,
          <year>1988</year>
          . [Online]. Available: http://doi.acm.
          <source>org/10</source>
          .1145/44498.45063
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>M.</given-names>
            <surname>Hammer</surname>
          </string-name>
          and
          <string-name>
            <given-names>B.</given-names>
            <surname>Niamir</surname>
          </string-name>
          , “
          <article-title>A heuristic approach to attribute partitioning,” in Proceedings of the 1979 ACM SIGMOD international conference on Management of data, ser</article-title>
          .
          <source>SIGMOD '79</source>
          . New York, NY, USA: ACM,
          <year>1979</year>
          , pp.
          <fpage>93</fpage>
          -
          <lpage>101</lpage>
          . [Online]. Available: http://doi.acm.
          <source>org/10</source>
          .1145/582095.582110
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>A.</given-names>
            <surname>Jindal</surname>
          </string-name>
          and
          <string-name>
            <given-names>J.</given-names>
            <surname>Dittrich</surname>
          </string-name>
          , “
          <article-title>Relax and let the database do the partitioning online</article-title>
          ,
          <source>” ser. Lecture Notes in Business Information Processing</source>
          ,
          <year>2012</year>
          , vol.
          <volume>126</volume>
          , pp.
          <fpage>65</fpage>
          -
          <lpage>80</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <given-names>L.</given-names>
            <surname>Li</surname>
          </string-name>
          and
          <string-name>
            <given-names>L.</given-names>
            <surname>Gruenwald</surname>
          </string-name>
          , “
          <article-title>Self-managing online partitioner for databases (smopd): A vertical database partitioning system with a fully automatic online approach,” ser</article-title>
          .
          <source>IDEAS '13</source>
          ,
          <year>2013</year>
          , pp.
          <fpage>168</fpage>
          -
          <lpage>173</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>L.</given-names>
            <surname>Rodr</surname>
          </string-name>
          <article-title>´ıguez and X. Li, “A dynamic vertical partitioning approach for distributed database system,” in Systems</article-title>
          , Man, and
          <string-name>
            <surname>Cybernetics</surname>
          </string-name>
          (SMC),
          <year>2011</year>
          IEEE International Conference on,
          <year>2011</year>
          , pp.
          <fpage>1853</fpage>
          -
          <lpage>1858</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>L.</given-names>
            <surname>Rodr</surname>
          </string-name>
          <article-title>´ıguez, X. Li, and</article-title>
          <string-name>
            <given-names>P.</given-names>
            <surname>Mej</surname>
          </string-name>
          <article-title>´ıa-Alvarez, “An active system for dynamic vertical partitioning of relational databases</article-title>
          ,
          <source>” ser. Lecture Notes in Computer Science</source>
          . Springer Berlin Heidelberg,
          <year>2011</year>
          , vol.
          <volume>7095</volume>
          , pp.
          <fpage>273</fpage>
          -
          <lpage>284</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>J. LeFevre</surname>
            , J. Sankaranarayanan,
            <given-names>H.</given-names>
          </string-name>
          <string-name>
            <surname>Hacigumus</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          <string-name>
            <surname>Tatemura</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          <string-name>
            <surname>Polyzotis</surname>
            , and
            <given-names>M. J.</given-names>
          </string-name>
          <string-name>
            <surname>Carey</surname>
          </string-name>
          , “Miso:
          <article-title>Souping up big data query processing with a multistore system,” ser</article-title>
          .
          <source>SIGMOD '14</source>
          ,
          <year>2014</year>
          , pp.
          <fpage>1591</fpage>
          -
          <lpage>1602</lpage>
          . [Online]. Available: http://doi.acm.
          <source>org/10</source>
          .1145/2588555.2588568
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>V.</given-names>
            <surname>Galaktionov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            <surname>Chernishev</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Novikov</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D.</given-names>
            <surname>Grigoriev</surname>
          </string-name>
          , “
          <article-title>Matrix clustering algorithms for vertical partitioning problem: an initial performance study,” in Selected Papers of the XVIII International Conference on Data Analytics and Management in Data Intensive Domains (DAMDID/RCDL</article-title>
          <year>2016</year>
          ), Ershovo, Moscow Region, Russia,
          <source>October 11-14</source>
          ,
          <year>2016</year>
          .,
          <year>2016</year>
          , pp.
          <fpage>24</fpage>
          -
          <lpage>31</lpage>
          . [Online]. Available: http://ceurws.org/Vol-
          <volume>1752</volume>
          /paper05.pdf
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>V.</given-names>
            <surname>Galaktionov</surname>
          </string-name>
          , G. Chernishev,
          <string-name>
            <given-names>K.</given-names>
            <surname>Smirnov</surname>
          </string-name>
          ,
          <string-name>
            <given-names>B.</given-names>
            <surname>Novikov</surname>
          </string-name>
          , and
          <string-name>
            <given-names>D. A.</given-names>
            <surname>Grigoriev</surname>
          </string-name>
          , “
          <article-title>A study of several matrix-clustering vertical partitioning algorithms in a disk-based environment,” in Data Analytics and Management in Data Intensive Domains</article-title>
          ,
          <string-name>
            <given-names>L.</given-names>
            <surname>Kalinichenko</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. O.</given-names>
            <surname>Kuznetsov</surname>
          </string-name>
          , and Y. Manolopoulos, Eds. Cham: Springer International Publishing,
          <year>2017</year>
          , pp.
          <fpage>163</fpage>
          -
          <lpage>177</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>V.</given-names>
            <surname>Galaktionov</surname>
          </string-name>
          , “
          <article-title>Parallelization of matrix clustering algorithms (accepted</article-title>
          ),
          <source>” in Proceedings of the Sixth International Conference on Informatics Problems (SPISOK</source>
          <year>2016</year>
          ),
          <year>2016</year>
          . [Online]. Available: http://spisok.math.spbu.ru/2016/txt/SPISOK-2016.pdf
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>C</surname>
          </string-name>
          .
          <article-title>-</article-title>
          H. Cheng, “
          <article-title>A branch and bound clustering algorithm</article-title>
          ,
          <source>” Systems, Man and Cybernetics</source>
          , IEEE Transactions on, vol.
          <volume>25</volume>
          , no.
          <issue>5</issue>
          , pp.
          <fpage>895</fpage>
          -
          <lpage>898</lpage>
          ,
          <year>1995</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18] C. Cheng, “
          <article-title>Algorithms for vertical partitioning in database physical design</article-title>
          ,
          <source>” Omega</source>
          , vol.
          <volume>22</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>291</fpage>
          -
          <lpage>303</lpage>
          ,
          <year>1994</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>C</surname>
          </string-name>
          .
          <article-title>-</article-title>
          H. Cheng and J. Motwani, “
          <article-title>An examination of cluster identification-based algorithms for vertical partitions,”</article-title>
          <string-name>
            <given-names>Int. J.</given-names>
            <surname>Bus</surname>
          </string-name>
          . Inf. Syst., vol.
          <volume>4</volume>
          , no.
          <issue>6</issue>
          , pp.
          <fpage>622</fpage>
          -
          <lpage>638</lpage>
          ,
          <year>2009</year>
          . [Online]. Available: http://dx.doi.org/10.1504/IJBIS.
          <year>2009</year>
          .026695
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>C</surname>
          </string-name>
          .
          <article-title>-</article-title>
          H. Cheng,
          <string-name>
            <surname>K.-F. Wong</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          K.
          <string-name>
            <surname>-H. Woo</surname>
          </string-name>
          , “
          <article-title>An improved branchand-bound clustering approach for data partitioning</article-title>
          ,”
          <source>International Transactions in Operational Research</source>
          , vol.
          <volume>18</volume>
          , no.
          <issue>2</issue>
          , pp.
          <fpage>231</fpage>
          -
          <lpage>255</lpage>
          ,
          <year>2011</year>
          . [Online]. Available: http://dx.doi.org/10.1111/j.1475-
          <fpage>3995</fpage>
          .
          <year>2010</year>
          .
          <volume>00781</volume>
          .x
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <given-names>G.</given-names>
            <surname>Chernishev</surname>
          </string-name>
          , “
          <article-title>A survey of DBMS physical design approaches (in Russian)</article-title>
          ,
          <source>” Tr. St. Petersburg Inst. Infor. Avtom. Ross. Akad. Nauk SPIIRAN</source>
          , vol.
          <volume>24</volume>
          , pp.
          <fpage>222</fpage>
          -
          <lpage>276</lpage>
          ,
          <year>2013</year>
          . [Online].
          <source>Available: www.mathnet.ru/trspy580</source>
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <given-names>N.</given-names>
            <surname>Gorla</surname>
          </string-name>
          and
          <string-name>
            <given-names>W. J.</given-names>
            <surname>Boe</surname>
          </string-name>
          , “
          <article-title>Database operating efficiency in fragmented databases in mainframe, mini, and micro system environments,” Data &amp; Knowledge Engineering</article-title>
          , vol.
          <volume>5</volume>
          , no.
          <issue>1</issue>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>19</lpage>
          ,
          <year>1990</year>
          . [Online]. Available: http://www.sciencedirect.com/science/article/pii/0169023X9090030H
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <given-names>J. A.</given-names>
            <surname>Hoffer</surname>
          </string-name>
          and
          <string-name>
            <given-names>D. G.</given-names>
            <surname>Severance</surname>
          </string-name>
          , “
          <article-title>The use of cluster analysis in physical data base design,” ser</article-title>
          .
          <source>VLDB '75</source>
          ,
          <year>1975</year>
          , pp.
          <fpage>69</fpage>
          -
          <lpage>86</lpage>
          . [Online]. Available: http://doi.acm.
          <source>org/10</source>
          .1145/1282480.1282486
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <given-names>N.</given-names>
            <surname>Bobrov</surname>
          </string-name>
          , G. Chernishev, and
          <string-name>
            <given-names>B.</given-names>
            <surname>Novikov</surname>
          </string-name>
          , “
          <article-title>Workload-independent data-driven verticalpartitioning</article-title>
          ,
          <source>” in New Trends in Databases and Information Systems</source>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Kirikova</surname>
          </string-name>
          , K. Nørva˚g,
          <string-name>
            <given-names>G. A.</given-names>
            <surname>Papadopoulos</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Gamper</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Wrembel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Darmont</surname>
          </string-name>
          , and S. Rizzi, Eds. Cham: Springer International Publishing,
          <year>2017</year>
          , pp.
          <fpage>275</fpage>
          -
          <lpage>284</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <surname>C</surname>
          </string-name>
          .
          <article-title>-</article-title>
          H. Cheng, W.-K. Lee, and
          <string-name>
            <surname>K.-F. Wong</surname>
          </string-name>
          , “
          <article-title>A genetic algorithmbased clustering approach for database partitioning</article-title>
          ,
          <source>” Systems, Man, and Cybernetics</source>
          , Part C:
          <article-title>Applications and Reviews, IEEE Transactions on</article-title>
          , vol.
          <volume>32</volume>
          , no.
          <issue>3</issue>
          , pp.
          <fpage>215</fpage>
          -
          <lpage>230</lpage>
          ,
          <year>2002</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <given-names>S. B.</given-names>
            <surname>Navathe</surname>
          </string-name>
          and
          <string-name>
            <given-names>M.</given-names>
            <surname>Ra</surname>
          </string-name>
          , “
          <article-title>Vertical partitioning for database design: a graphical algorithm,” ser</article-title>
          .
          <source>SIGMOD '89</source>
          ,
          <year>1989</year>
          , pp.
          <fpage>440</fpage>
          -
          <lpage>450</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <given-names>J.</given-names>
            <surname>Du</surname>
          </string-name>
          ,
          <string-name>
            <given-names>K.</given-names>
            <surname>Barker</surname>
          </string-name>
          , and
          <string-name>
            <given-names>R.</given-names>
            <surname>Alhajj</surname>
          </string-name>
          , “
          <article-title>Attraction - a global affinity measure for database vertical partitioning,” in ICWI</article-title>
          . IADIS,
          <year>2003</year>
          , pp.
          <fpage>538</fpage>
          -
          <lpage>548</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <given-names>J. H.</given-names>
            <surname>Son and M.-H. Kim</surname>
          </string-name>
          , “
          <article-title>-partitioning algorithm: Vertical partitioning based on the fuzzy graph,” ser</article-title>
          .
          <source>DEXA '01</source>
          ,
          <year>2001</year>
          , pp.
          <fpage>537</fpage>
          -
          <lpage>546</lpage>
          . [Online]. Available: http://dl.acm.org/citation.cfm?id=
          <volume>648314</volume>
          .
          <fpage>755837</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <given-names>M.</given-names>
            <surname>Bouakkaz</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Ouinten</surname>
          </string-name>
          , and
          <string-name>
            <given-names>B.</given-names>
            <surname>Ziani</surname>
          </string-name>
          , “
          <article-title>Vertical fragmentation of data warehouses using the FP-Max algorithm</article-title>
          ,”
          <source>in Innovations in Information Technology (IIT)</source>
          , 2012 International Conference on, march
          <year>2012</year>
          , pp.
          <fpage>273</fpage>
          -
          <lpage>276</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <given-names>N.</given-names>
            <surname>Gorla</surname>
          </string-name>
          and
          <string-name>
            <given-names>B. P. W.</given-names>
            <surname>Yan</surname>
          </string-name>
          , “
          <article-title>Vertical fragmentation in databases using data-mining technique,” in Database Technologies: Concepts, Methodologies, Tools, and</article-title>
          <string-name>
            <surname>Applications</surname>
          </string-name>
          , J. Erickson, Ed.
          <source>IGI Global</source>
          ,
          <year>2009</year>
          , pp.
          <fpage>2543</fpage>
          -
          <lpage>2563</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <given-names>L.</given-names>
            <surname>Rodriguez</surname>
          </string-name>
          and
          <string-name>
            <given-names>X.</given-names>
            <surname>Li</surname>
          </string-name>
          , “
          <article-title>A support-based vertical partitioning method for database design</article-title>
          ,” in
          <source>Electrical Engineering Computing Science and Automatic Control (CCE)</source>
          ,
          <year>2011</year>
          8th International Conference on, oct.
          <year>2011</year>
          , pp.
          <fpage>1</fpage>
          -
          <lpage>6</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <given-names>R.</given-names>
            <surname>Agrawal</surname>
          </string-name>
          and
          <string-name>
            <given-names>R.</given-names>
            <surname>Srikant</surname>
          </string-name>
          , “
          <article-title>Fast algorithms for mining association rules in large databases,” ser</article-title>
          .
          <source>VLDB '94</source>
          ,
          <year>1994</year>
          , pp.
          <fpage>487</fpage>
          -
          <lpage>499</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <given-names>S.</given-names>
            <surname>Papadomanolakis</surname>
          </string-name>
          and
          <string-name>
            <given-names>A.</given-names>
            <surname>Ailamaki</surname>
          </string-name>
          , “
          <article-title>AutoPart: automating schema design for large scientific databases using data partitioning,” in Scientific and Statistical Database Management</article-title>
          ,
          <year>2004</year>
          . Proceedings. 16th International Conference on,
          <source>june 2004</source>
          , pp.
          <fpage>383</fpage>
          -
          <lpage>392</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <given-names>M. G.</given-names>
            <surname>Ivanova</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M. L.</given-names>
            <surname>Kersten</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. J.</given-names>
            <surname>Nes</surname>
          </string-name>
          , and
          <string-name>
            <given-names>R. A.</given-names>
            <surname>Gonc</surname>
          </string-name>
          <article-title>¸alves, “An architecture for recycling intermediates in a column-store,” in Proceedings of the 35th SIGMOD international conference on Management of data, ser</article-title>
          .
          <source>SIGMOD '09</source>
          . New York, NY, USA: ACM,
          <year>2009</year>
          , pp.
          <fpage>309</fpage>
          -
          <lpage>320</lpage>
          . [Online]. Available: http://doi.acm.
          <source>org/10</source>
          .1145/1559845.1559879
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <given-names>T.</given-names>
            <surname>Malik</surname>
          </string-name>
          ,
          <string-name>
            <given-names>X.</given-names>
            <surname>Wang</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Dash</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Chaudhary</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Ailamaki</surname>
          </string-name>
          , and
          <string-name>
            <given-names>R.</given-names>
            <surname>Burns</surname>
          </string-name>
          , “
          <article-title>Adaptive physical design for curated archives</article-title>
          ,”
          <source>in Proceedings of the 21st International Conference on Scientific and Statistical Database Management, ser. SSDBM 2009</source>
          . Berlin, Heidelberg: Springer-Verlag,
          <year>2009</year>
          , pp.
          <fpage>148</fpage>
          -
          <lpage>166</lpage>
          . [Online]. Available: http://dx.doi.
          <source>org/10.1007/978- 3-642-02279-1 11</source>
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <given-names>I.</given-names>
            <surname>Alagiannis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Idreos</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <given-names>A.</given-names>
            <surname>Ailamaki</surname>
          </string-name>
          , “
          <article-title>H2o: A hands-free adaptive store,” in Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, ser</article-title>
          .
          <source>SIGMOD '14</source>
          . New York, NY, USA: ACM,
          <year>2014</year>
          , pp.
          <fpage>1103</fpage>
          -
          <lpage>1114</lpage>
          . [Online]. Available: http://doi.acm.
          <source>org/10</source>
          .1145/2588555.2610502
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>