OrientDB is a full-function, NoSQL MMDMS (multi-model database management system), addressing the big data variety problem with one single, multi-model store and a SQL-based, multi-model query language. It combines graph and semistructured data management with object-oriented, text and spatial capabilities, and features a variety of deployment and distribution / replication options, transactional / ACID storage and indexing, making it a commercially successful MMDMS. With that, OrientDB is well-suited for novel adaptations of applications like smart logistics, asset management, social data storage and analysis, and other use cases that require multiple perspectives on the data.
tence (→ persisting data)
Modern analytical, business and smart applications (e. g., in the areas of logistics, asset management, and social network analysis) require eficient storage and access to large amounts of multi-model data [1, 2, 3]. Data management platforms addressing the multi-model or in general the big data variety problem could be more generally referred to as multi-model data platforms [1] (e. g., as in the Q3/2021 Forrester wave1 that especially stresses on the importance of such a polyglot persistence model for modern applications).
are commonly diferentiated into polystores (e. g., BigDAWG [4]) and multi-model database management system (MMDMS) [1, 5], according to the number of separate data stores: polystores have multiple, federated data stores and MMDMSs have one single, integrated data store. In practice, there are several (commercial) multimodel data platforms2 that are (a) more on the polystore side, e. g., from Microsoft, Oracle, and SAP, or (b) on the “best-in-class” NoSQL systems like MongoDB (document) and Neo4j (graph) (all in the Forrester Wave). nEvelop-O LGOBE 0000-0001-6146-3365 (D. Ritter) CEUR htp:/ceur-ws.org ISN1613-073 © 2021 Copyright for this paper by its authors. Use permitted under Creative (2) SQL-like, multi-model query / operations, e. g., traversal, matching, lookup (→ querying data) multi-model database side, e. g., from ArangoDB, Couch- like inheritance with semi-structured document (schema
from existing single-model databases that are growing Relational Graph Document OrientDB
into multi-model data platforms (in Tab. 1: MongoDB Table Vertex / Edge Collection Class
misses (2) and partially (3), Postgres and Redis require Row Vertex Document Vertex /
extensions / modules for (
This paper is organised as follows: Sect. 2 covers the document approaches and ofers vertex and document
data model (cf. diferentiator (
link , , logical storage entities
schema 1
1 n
1 n Constraint / rule super class 1 "collection", "table" physical storage 1..n
default, id, selection
1 --- collection as vertex Cluster 11 customer 1 2 ... 382 .C.. lusnter 23 C-R-E-ATiEnsCeLrAtSSdocpeurmseonntsEXTENDS V; [_europe] Cluster 4 INSERT INTO person CONTENT {"name": "Frank"}; default 5 INSERT INTO person CONTENT {"name": "Linda",
6 "likes": "#13:382"} Figure 2: Clusters and record identifiers @rid 7 --- collection as edge
8 CREATE CLASS friend EXTENDS E; identifier and <RP> is the record’s position in that cluster, 9 CREATE EDGE friend FROM #13:382 to #13:37; thus it is a direct pointer to its physical storage location (cf. Ex. 1), similar to CTID in PostgreSQL. Listing 2.: Records / documents in OrientDB Example 1 (Clusters, RIDs). Figure 2 shows a 1 {"@rid": "13:382", "@class": "person", Customer class with three clusters (i. e., identified by 2 "name": "Frank", ... } 11–13) and default cluster 11. A record with RID #13:382 3 can be found in cluster 13 at position 382. ∎ 4 {"@rid": "13:37", "@class": "person", OrientDB has diferent record types with document as the 5 "name": "Linda", "likes": "#13:382", ... } most relevant. A document represents semi-structured The corresponding JSON records in OrientDB are shown in data by a set of JSON field elements or key-value pairs, Listing 2. Notably, the RIDs and class assignments are part /demnioxteindgscphreompear)t.ieUsnolfiktheeortheceorrddo’scuclmasesn(tes.tgo.,refos,r dstorciuct- of the documents and can be used as shown above. ∎ ments in OrientDB can have links to other documents, Sizing / limits While the number of databases within where a link is a set of RIDs. While in a schema-less an OrientDB server is not limited, each database can have setup no properties are required, the creation of an index 215 − 1 clusters. A cluster can store 263 − 1 records, which for a field requires a property. allows up to 278 − 1 records per database. The maximum
From a graph perspective, vertex and edge entities are size of a record / document is 2GB (cf. 16MB in MongoDB documents, thus allowing them to be simple property and 512MB in Redis). There is no limit on the number of graphs with key-value pairs or carrying more complex properties for a schema-less, and 2 × 109 properties per data like JSON documents (cf. Ex. 2). database for schema-full usage.
Example 2 (Class, document, graph). OrientDB’s Summary OrientDB specifies a single, unified data multi-model, SQL DDL, DML allow for the creation of a model with an object-oriented class entity at its core. class that is of type vertex V as shown in Listing 1. The The object-oriented features allow for expressive modelclass extends built-in V to specify that the documents in ing, including key-value or JSON document records and person can be vertices in a graph (line 2). As in SQL, complex graph data at vertices and edges. there are several way to insert data. For simplicity, two JSON documents are inserted into the person class, the second one with a RID, indicating a document link with label likes (lines 4–6). Since the documents are vertices in a graph, they can also be related by creating an edge class friend and specifying an edge using the RIDs of the 2.2. Querying data On top of the unified data model, OrientDB specifies OSQL with SQL-like query capabilities and operations (cf. diferentiator (2)), which we will briefly introduce by example for document, graph, text and geospatial (including object-oriented capabilities).
Document queries OSQL queries on documents are Text and geospatial queries The text and geospatial mostly similar to standard SQL. Notable deviations de- features are based on the data model in Sect. 2.1 and both note the dot-notation for specifying fields in nested JSON require index creation (cf. Ex. 4). objects (e. g., address.city) and working with arrays Example 4 (Graph traversal, pattern matching). (e. g., UNWIND returns the entries of a JSON array as single We recall that for indexed fields, a property has to be added lines). to the class, as shown in Listing 5.
Besides the usual SQL keywords like DISTINCT, object- Listing 5.: Text and spatial indexes oriented extensions are added (e. g., to check the type of a class with INSTANCEOF). 1 TRAVERSE friend FROM #13:37 2 WHILE $depth <= 3 BREADTH_FIRST; 1 --- property, fulltext index 2 CREATE PROPERTY person.name STRING; Graph queries Apart from the support of TinkerPop 3 CREATE INDEX name ON Person(name) /trGavreemrsalilns qanuderpieasttoenrnitmsagtrcahpihnsg, O(cSf.QELx.a3ll)o.ws for graph 45 ---FULGLeToEsXpTatiEaNlGIiNnEdeLxUCENE; Example 3 (Graph traversal, pattern matching). 6 CREATE PROPERTY person.location EMBEDDED An example of graph traversal on the documents of ↪ OPoint; Listing 2 is depicted in Listing 3 and pattern matching is 7 INSERT INTO person SET location = illustrated in Listing 4. ↪ St_GeomFromText(
Listing 3.: Graph traversal: Breadth-first search 8 "POINT (51.498308 -0.176882)") 9 WHERE name = "Frank"; 10 CREATE INDEX person.location ON
↪ person(location) 11 SPATIAL ENGINE LUCENE;
Listing 4.: Graph pattern matching 1 MATCH 2 {class: Person, as: people, where: (name =
↪ 'Linda')} 3 RETURN people For spatial coordinates (e. g., numeric), decimal degree values can be parsed from String using St_GeomFromText (e. g., by updating Frank’s record).
Listing 6.: Text and spatial queries 1 --- text search using SEARCH_CLASS function Queries in Listings 3 and 4 return records #13:382, #13:37, 2 SELECT FROM person respectively. ∎ 3 WHERE SEARCH_CLASS("+name:Fran*") = true Graph traversals are defined using TRAVERSE instead of 4 --- spatial search using NEAR operator SELECT, but traversals can be embedded into queries as 5 SELECT *,$distance FROM person sub-queries. In traversal queries, the FROM-clause can 6 WHERE [location,$spatial] contain classes, clusters, one or more record identifiers 7 NEAR [51.495449,-0.17625,{"maxDistance":1}] and sub-queries. Similar to a WHERE-clause, the WHILE condition limits the traversal, while the result set can The syntax to support spatial arguments (i. e., $spatial) is be limited by LIMIT (not shown). During a traversal, taken from Lucene. ∎ projections help to restrict to fields that should be fol- In contrast to all other supported index types (e. g., hash, lowed. While in Listing 3 vertices of type friend are spec- B-tree / range), text and spatial indexes are created ified, OSQL supports *, ALL(), and ANY(). When spec- in Lucene5, which was therefore embedded into Oriifying a class, polymorphic traversals can be specified. entDB (cf. ENGINE LUCENE, ENGINE SPATIAL ENGINE For instance, when customer is a person, then specifying LUCENE). When creating indexes in Lucene, additional customer.name will also traverse person vertices. OSQL metadata can be passed to the underlying engines to consupports diferent search strategies like BREADTH_FIRST, figure according to their full capabilities (e. g., analyzers, which can be limited by selections on context variables parsers). like $depth (nesting depth) or $path (path traversed). Text queries require function SEARCH_CLASS (line 3) Similarly, linked documents are traversed and traversals to specify complex fulltext searches (e. g., regular exprescan be directed by using keywords like IN() or OUT(). sions) in a query’s WHERE-clause. Further search func
Graph pattern matching requires the MATCH keyword tions like SEARCH_FIELDS (search index for more than with JSON input for a valid target CLASS, an alias for one field) enable the full spectrum of Lucene’s search a node pattern (e. g., people), and a WHERE-clause that capabilities. matches a node in the pattern.
Summary OrientDB defines a SQL-like, multi-model query language that covers all aspects of the unified data model (i. e., including object-oriented, document, graph, Summary OrientDB uses ACID transactions for key-value, text, and geospatial). External engines can be change operations on its single multi-model storage. The added, however, which loose their state, if not persistent. data is queried through extensible index capabilities, while data is accessed through a RID mapping to data pages on disk (i. e., mostly independent of the database size).
ture and deployment / distribution options for horizontal
scaling (cf. diferentiator (
Hash- and SB-Tree [9] for fast lookup and sequential record access. The indexes can be set as UNIQUE (i. e., not allowing for duplicate keys), which is also guaranteed in distributed setups. Composite keys allow for searches in multiple indexes at the same time. In addition, custom indexes can be specified, configured and loaded into the system. For instance, fulltext and geospatial indexes are provided that way through the Lucene search engine.
There can be up to two billion indexes per database, without limitations regarding the number of indexes per class (cf. 64 indexes per collection in MongoDB). 3.1. Component architecture The main deployment unit of OrientDB is the ODB Server shown in Fig. 4. The server accepts local OSQL and remote JDBC and REST calls. In addition, specialized integration like graph / Gremlin is available through modules Transactions, Storage Compared to many other provided by the open source community. The oficial NoSQL databases, OrientDB supports ACID transactions database administration tool is OrientDB studio, which with isolation levels READ COMMITTED (default, dis- uses the REST API. While business applications usually tributed) and REPEATABLE READS (single instance). Dur- use JDBC and REST, other clients use OrientDB’s binary ing the commit of a transaction, records are physically protocol to access the server. Besides data import and mistored using its main storage entity (i. e., cluster). Each gration tools like ETL, Neo4j (community projects) and cluster is split in pages, which contain system informa- teleporter (relational data migration tool). OrientDB’s tion (e. g., checksum for integrity / recovery) and record active open source community contributed several landata (i. e., paginated storage). On disk, the data is stored guage bindings / connectors for Python, .Net, NodeJS that in variable size data files as shown in Fig. 3. The RIDs are complement the standard Java, JDBC, and REST drivers. mapped through a fixed size, append-only cluster position Subsequently, we briefly introduce internal layers that map through cluster pointers (i. e., page identifier, record were recently significantly reworked: multi-model API,
binary protocol
Graph API / Gremlin r e v r e S B D O database, and storage layers. The distribution layer will be discussed in Sect. 3.2.
Multi-model API OrientDB originally started out with
separate document and graph APIs. From version 3.0,
the access to the database layer is managed by a unified,
multi-model API that we introduced in Sect. 2.2. The
new API makes use of the data model from Sect. 2.1 to
give a combined perspective on multi-model data. The
old document and graph APIs as well as access via
TinkerPop 2.x is deprecated and support for TinkerPop 3.x
was added (cf. Graph API / Gremlin).
(
ODB
ODB (3) Replication
ODB
(replica N-2ODB
r/w (replica N-1)
r/w
Figure 5: Distribution and replication
(
ODB
(replica N)
r/w
Database layer The OSQL optimizer and execution (cf. variant (
Regarding security, OrientDB supports authentication A seamless evolution or scaling is possible through a
through Kerberos and LDAP integration, database en- Raft -based [13] auto-discovery mechanism that
identicryption and a sophistiacated security concept reaching ifes OrientDB instances, stores the runtime cluster
confrom clients over server down to record-level. ifguration and synchronizes certain operations between
Storage layer The storage components, introduced in nodes. After nodes have been added to the distributed
Sect. 2.3 are seamlessly integrated into the ODB server to system, OrientDB uses a variant of Fast Paxos [14, 15]
support full ACID transactions with disk-based storage. to support distributed transactions (i. e., fast rounds, but
Despite the exception of Lucene (text, spatial), which are bigger quorums than in classic Paxos). In that way,
Orikept in their native storage, built-in and custom indexes entDB supports multi-leader replication, which allows
operate on the same ODB storage. several nodes to perform change operations (insert,
update, delete). To avoid quorum nodes lagging behind and
3.2. Distribution / Replication requiring expensive quroum rounds to catch up
(espeThe ODB server can be deployed in diferent (distributed) cially after failure), each node records change operations
setups that allow applications to scale OrientDB from per transaction as persistent version counters. In case
small to larger installations, as shown in Fig. 5: (
OrientDB can be embedded into one application (
assessment of OrientDB compared to the databases in Tab. 1, i. e., well-established NoSQL databases (MongoDB, Redis), and one extended RDBMS (Postgres/JSON), using the well-known YCSB benchmark [17]. the results are considered preliminary, but give interesting insights into relative rather than their absolute performance, which we consider valuable. 4.2. Preliminary results The preliminary results of our benchmark runs are shown in Fig. 6 as time per request in seconds (lower bar is better) for MongoDB (short mdb), OrientDB (odb), Postgres/JSON (pqj), and Redis (rds), which we briefly discuss for each workload and concurrent user scaling. 4.1. Setup, Limitations The YCSB benchmark is an open-source, NoSQL database benchmark with a broad coverage of database systems (including all selected databases from Tab. 1). The bench- Workload A: Update-heavy, read The first workload mark features several workloads of JSON documents that is a combination of 50% read and upgrade operations with cover important operations like read, insert, update and a Zipfian record selection. Fig. 6a shows that this keyread-modify-write as well as configuration parameters value store workload is best for Redis, while OrientDB is for scale-factors (e. g., number of records and fields per slightly better for read operations compared to MongoDB record, field length). and Postgres. The update performance is similar for the three non key-value stores.
(A–F) and use the default parameters (e. g., single user Workload B: Mostly read, update Similarly, Redis and record, records with 10 fields of length 100). We set dominates the read-heavy workload with 95% read operthe number of documents per workload to five million ations with Zipfian distribution in Fig. 6b. The focus on records, operation count to 500k and the batch size to read operations leads to better results for OrientDB and 10k. If a workload requires an index, a hash index is set. Postgres, which have slightly slower update compared
Out of the multitude of polystore and multi-model to read operations.
NoSQL systems, we decided to choose MongoDB as one Workload C: Read-only The Zipfian read-only workof the leading document stores, Postgres/JSON as one load underpins the previous observations on read operof the mostly used RDBMS with multiple NoSQL exten- ations for OrientDB and Postgres, which gain ground sions, and Redis, as a representative key-value store (cf. on Redis, shown in Fig. 6c. Hereby, OrientDB might Tab. 1). For all databases we benchmarked the latest profit from its WTinyLFU read cache implementation server versions available, for which we updated the Ori- that keeps a mix of LRU and heavily read older records senctaDnBopderrivaetirotnos vinerwsioornk3lo.1a.d3 Ean6.d added support for the in memory and its physical RID mapping.
All measurements are conducted on two Intel X5650 CPUs with 2.67GHzs (12 cores), 24GB RAM, Windows 10 operating system, and JDK 1.8.
and 5% insert workload for reading the latest inserts shows a similar read performance for all databases. Unlike for the similar update case of workload B, OrientDB Limitations YCSB originally targeted key-value stores, is slightly behind MongoDB and Postgres, while Redis and thus only works with “flat” key-value pair JSON doc- shows similar results to its update performance. uments (i. e., no nesting, arrays). However, even widecolumn and document stores provide YCSB drivers, due Workload E: Short-ranges In this scan operation to the lack of open source benchmark alternatives. Re- dominated workload, the inserts (again only 5%) are simcently, MongoDB tried to adapt their workloads to TPC-C ilar to that of workload D. The scans, however, show [18], but had to admit that neither YCSB nor TPC-C suf- best performance for Postgres and worst for Redis, while ifciently address their workloads. MongoDB and OrientDB are closely left in-between.
Since OrientDB is an MMDMS, one could expect a Workload F: Read-modify-write For the combined suitable multi-model benchmark. In fact, there are poly- r-m-w workload, all databases perform similar to the store benchmarks (e. g., PolyBench [19]) and at least one previous workloads, with Redis on top, then OrientDB multi-model benchmark (UniBench [20]), but none of the with slightly faster read performance than MongoDB and benchmarks is available as open-source (e. g., for database Postgres, but similar update and r-m-w time. driver development).
Due to the lack of more suitable benchmarks and the Concurrent user scaling Since the results for workrather simple nature of YCSB’s data sets and workloads, loads A–F were for a single user, and thus can be considered as baseline performance, we now briefly study workloads B and F for 64 concurrent users in Fig. 7.
MongoDB and Postgres perform slightly better for read
6YCSB OrientDB 3.1. port, visited 11/2021: https://github.com/ operations than OrientDB and Redis in both workloads,
brianfrankcooper/YCSB/pull/1468 while MongoDB shows the fastest updates, followed by
cus). For single user workloads, MongoDB seems to have
additional overhead (i. e., MongoDB’s hash index is
actually a B-tree, cf. [21]) that amortizes for multiple users
in a single database instance (cf. observation (
INSERT READ (d) Read latest
INSERT SCAN (e) Short-range scans READ UPDATE R-M-W (f) Read, modify, write
recently revamped OrientDB, a commercially successful
NoSQL, open source MMDMS. While more and more
READ UPDATE R-M-W database systems strive to become multi-model data
plat(b) Workload F: Read, modify, write forms, OrientDB early on addressed NoSQL multi-model
Figure 7: YCSB workloads B, F (single instance, 64 users) key diferentiators, such as (
Firstly, the slightly better update and insert perfor- Spranz and Dr. Pepke, who made this work possible, and mance of MongoDB and Redis could be explained by we thank the fellow OrientDB contributors, mainly Entheir slightly more relaxed ACID guarantees (e. g., with rico Risa and Colin Leister, for the joint journey through single-document focus, compare-and-set operation fo- the OrientDB 3.0 to 3.2 releases.
Storage 13 (2017) 35:1–35:31. URL: https://doi.org/ io/javacc/documentation/grammar.html
grammar, which is used to generate an OSQL parser that creates a query AST. To speed up the generated JavaCC parser, recurring queries are cached (i. e., logical plan cache). Several AST-level optimizations are applied (e. g., task push-down, index look-ups on link chains) that (partially) rewrite the AST (e. g., JSON path / chain dot notation becomes a set of sub-queries).
An execution planner creates an execution plan from the (optimized) AST, using pre-defined execution steps. The resulting physical plan itself is “executable” in the sense that its steps (e. g., fetch data, project, filter) and is cached (physical plan cache).