Document stores like MongoDB are among the most popular NoSQL data stores. Due to the nested document structure and possible redundancy, a large amount of alternative modeling possibilities arise in contrast to relational data stores, even for simple applications. Therefore, designing an optimal schema is a complex task. In [1] we presented a visionary approach to automatic schema migration and optimization for migrating relational data stores to various NoSQL stores. This is essentially based on an in-depth data and workload analysis. A big challenge, however, is to find the optimal compromise between read and write optimization. In this paper, we show the impact of diferent workload profiles (mix of read and write operations) on the schema design using initial measurements. This illustrates the fundamental need to consider realistic workload profiles in the schema design and optimization process.
NoSQL data stores such as MongoDB are gaining in popularity. However, designing a suitable
schema is a major challenge due to the large number of modeling alternatives, which is often
based on a trial and error approach. Thus, there are several works that address automatic
schema optimization and design, respectively [
For the evaluation of automatically generated, but also manually created schemas, it is therefore absolutely necessary to use realistic workload profiles in order to be able to estimate the behavior in a productive environment.
Contribution: First measurements showing the diferent behavior of realistic workload profiles compared to single execution (one by one) of queries.
This paper is organized as follows: Section 2 presents work discussing the evaluation of diferent schema modeling and optimization approaches. Section 3 describes the environment and the results of our first measurements. Section 4 provides the conclusion as well as an outlook to our future work.
In the following, related work on the evaluation of schema modeling and optimization approaches is described. Since the focus of this paper is on the evaluation and the necessity of considering realistic workload profiles, only papers regarding the evaluation of NoSQL schema design methods are considered.
In [
None of the works describes in detail how the measurements were made with respect to the workload. In particular, the number of concurrent threads (clients) is not discussed. Therefore, it is dificult to understand exactly what the benchmark environment looks like. 3. Experimental environment and results The goal of this work is to show the impact of executing queries one by one compared to diferent workload mixes of read and write queries on multiple clients. This is done using two diefrent schema designs. One has been optimized for read-only queries and one for write-only queries.
The schema of a prototypical application1 to manage digital print media is used because it is a real world scenario that is not too complex but has suficiently complex relationships to illustrate the impact on the two schema designs (see Figure 1).
The test data was generated using our test data generator [
Author id first_name last_name birthday pk id name
Publisher pk id
Item
MetaData
Page pk id title release_date pk id content size pk id name
Collection pk
For modeling the physical schemas and defining diferent workload mixes, 3 read and 3 write queries are used: (1) Given a Collection.id, return the Collection, related Items, Metadata, 1https://sammlungen.hebis.de/Handbuecher/ Publisher and Authors. (2) Given an Author.id, return the Author, related Items, Publisher and Collections. (3) Given an Item.id, return the Item related Metadata, Pages, Publisher and Authors. (4) Given an existing Publisher.id and multiple existing Collection.ids, add one or more new Authors and a new Item with corresponding Metadata and Pages. (5) Given multiple existing Item.ids and Collection.ids, add the Items to the Collections. (6) Given an existing Item.id and changed Metadata.title, update the Metadata.
Due to the various relationship modeling possibilities, designing the optimal schema for a given workload in document stores such as MongoDB is a complex task.
Here, optimizations with respect to reading queries result in redundancy in the Collection, Item and Author documents so that all attributes needed for the queries are present in the corresponding documents. This increases the complexity of the write queries (4 to 6) since redundant data must be written in several collections. To avoid inconsistencies, transactions were used for the write operations in the read-optimized schema.
For the optimization of the write queries, redundancy must be avoided, since the performance decreases the more redundant data has to be written. Here, only 5 is realized as a transaction, since several documents are updated, which ensures a rollback in the case of an error.
All measurements were done on a single node of MongoDB version 6.0.9 with 64 GB RAM and
16 CPU cores. For the benchmark, py-tpcc2, a Python implementation of the TPC-C3 benchmark
was extended, which already has a MongoDB driver as part of performance measurements on
multi-document transactions [
Results: First, it can be observed that for sequential execution (single thread) of queries, the read queries in the read-optimized schema and the write queries in the write-optimized schema always have the best performance (see 3 and 6 one_by_one in Figure 2 as an example).
However, this may be diferent for individual queries when running mixed workloads on multiple concurrent clients. For example, query 6 (write) performs better in the read-optimized schema than in the write-optimized schema for mixed workloads (see Figure 2b). . 30,000 inM 20,000 / ise 10,000 r ueQ 0
Finally, Figure 3 shows the behavior of diferent workload mixes on the read and write optimized schemas. Here, as expected, the correspondingly optimized schemas show better performance for a higher amount of read or write operations. For better readability the plot has been split, since the performance is much higher in the read-optimized schema for workload r99_w1. 2https://github.com/apavlo/py-tpcc 3https://tpc.org/tpcc/ r20_w80 r1_w99
Based on initial measurements, it could be shown that when looking at the execution performance of individual queries, significant diferences exist between queries executed sequentially (one by one) or, in a more realistic scenario, on several concurrent clients in a workload mix.
This shows the importance to consider realistic workload profiles (mixes) with concurrent clients when evaluating schema optimization approaches.
There are many dimensions to be taken into account when executing database benchmarks. We therefore plan to perform more extensive measurements. Our current vision is to use a MongoDB cluster with replication and sharding. We also plan to further examine important variables and their efects, such as the diferent number of clients, diferent document sizes, and diferent number of related documents.
This work has been funded by Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) grant #385808805.