In database technologies, some of the new issues increasingly debated are non-conventional applications, including NoSQL (Not only SQL) databases, which were initially created in response to the needs for better scalability, lower latency and higher flexibility in an era of bigdata and cloud computing. These non-functional aspects are the main reason for using NoSQL database. However, currently there are no systematic studies on data modeling for NoSQL databases, especially the document-oriented ones. Therefore, this article proposes a NoSQL data modeling standard in the form of ER diagrams, introducing modeling techniques that can be used on documentoriented databases. On the other hand the purpose of this article is not structure the data using the model proposed, but it does helping with the visualization of data. In addition, to validate the proposed model, a study case was implemented using genomic data.
Huge amounts of data are produced daily. They
are generated by smart phones, social networks,
banks transactions, machines measured by sensors
are part of Internet of Things provide information
that is growing exponentially. The management of
this data is currently performed in most cases by
relational databases that provide centralized
control of data, redundancy control and elimination
of inconsistencies
The purpose, therefore, of NoSQL solutions is
not to replace the Relational Model as a whole,
but only in cases in which there is a need for
scalability and bigdata. In the recent years, a
variety of NoSQL databases has been developed
mainly by practitioners looking to fit their specific
requirements regarding scalability performance,
maintenance and feature-set. Subsequently, there
have been various approaches to classify NoSQL
databases, each with different categories and
subcategories, such as key-value stores,
columnoriented and graph databases, oriented-document.
MongoDB
Nonetheless, data modeling still has an
important role to play in NoSQL environments. The data
modeling process
Addressing this issue, this article proposes a standard for NoSQL data modeling. This proposal uses NoSQL document-oriented databases, aiming to introduce modeling techniques that can be used on databases with document features.
The remainder of the paper is organized as follows: Section II presents related works. Section III explores the concepts of modeling for NoSQL databases based on documents, introducing the different types of relationships and associations. Section IV shows the proposal model in the context of NoSQL databases based on documents. Section V presents the study case to validate the proposal model. Finally in Section VI, presents the conclusion of the research and future works. 2
Katsov
Arora and Aggarwal
Similarly, Banker
Kaur and Rani
An important step in database implementation is
the data modeling, because it facilitates the
understanding of the project through key features
that can prevent programming and operation
errors. For relational databases, the data
modeling uses the Entity-Relationship Model
Basically, the documents are stored in
collections. A parallel is made with relational databases,
the equivalent for a collection is the record
(tuple) and for a document it is the relation (table).
Documents can store completely different sets of
attributes, and can be mapped directly to a file
format that can be easily manipulated by a
programming language. However, it is difficult to abstract
the modeling of documents for the entity
relationship model
The relational model designed for SQL has some important features such as integrity, consistency, type validation, transactional guarantees, schemes and referential integrity. However, some applications do not need all of these features. The elimination of these resources has an important influence on the performance and scalability of data storage, bringing new meaning to data modeling.
Document-oriented databases have some
significant improvements, e.g., index management by
the database itself, flexible layouts and advanced
indexed search engines
This type of relationship stores the data by
including links or references, from one document to
another. Applications can solve these references to
access the related data in the structure of the
document itself
This type of relationship stores in a single
document structure, where the embedded documents
are disposed in a field or an array. These
denormalized data models allow data manipulation in
a single database transaction
In this scenario, the number of objects in a
database increases the abstraction complexity of
the logical relationship between the stored
information, especially when objects have references
to other objects. Currently, there is a lack of
solutions to conceptually represent those associated
with a NoSQL document-oriented database. As
described in
Therefore, this section proposes a standard for document-oriented database viewing. Our proposal has some properties, considering the conceptual representation modeling type, such as: • Ensuring a single way of modeling for the several NoSQL document-oriented databases. • Simplifying and facilitating the understanding of a document-oriented database through its conceptual model, leveraging the abstraction and making the correct decisions about the data storage. • Providing an accurate, unambiguous and concise pattern, so that database administrators have substantial gains in abstraction, understanding. • Presenting different types of relationships between collections defined as References and Embedded documents. • Assisting the recognition and arrangement of the objects, as well as its features and relationships with other objects.
The following subsections present the concepts and graphing to build a conceptual model for NoSQL document-oriented databases. 4.1
Before starting the discussion about the approach of each type of the conceptual modeling representation, it is important to highlight some basic concepts about objects and relationships in a document-oriented database: • A document (or object) describes a set of attributes that have their properties organized in a key-value structure. • Information contained in an document is described by the identifier (key) and the value associated with the key. • Different types of relationships between documents are defined as References and Embedded Documents • Because NoSQL is a non-relational data database, the concepts of normalization, do not apply. • Some concepts of relationships between objects are similar to ER modeling, such as cardinality (one-to-one, one-to-many, many-tomany). The proposed solution for a conceptual modeling to the NoSQL document-oriented databases has two basic concepts: Document and Collections.
As noted previously, a document is usually represented by the structure of a JSON object, and as many fields as needed may be added to the document. For this solution, a document and a collection of documents is represented by Figure 3.
A one-to-many relationship in embedded
documents is represented by the Figure 5. This is the
case when the notation to represent the cardinality
is the same used in UML
The following section presents the definitions of relationship types and degrees for the objects features. 4.3
This section proposes a model that represents the
one-to-one relationship for documents embedded
in another document. In this case, the proposal
is to use the representation of an individual
Document within another element that represents a
Document. In Figure 4, cardinality is also
suggested to specify the one-to-one relationship type.
A many-to-many relationship in embedded
documents is represented by the Figure 6. According to
the cardinality many-to-many the larger document
has a many to many relationship with the
embedded document. The representation of the
cardinality is the same used in UML
A document can reference another, and in this
case, one must use an arrow directed to the
referenced document, as shown in Figure 7. One can
see that the directed arrow makes the left
document references to the right document.
Furthermore, the cardinality of the relationship should be
specified above the arrow. The notation of
cardinality is based on UML
After analyzing the previously mentioned
concepts, we have chosen to create a collection of
documents for each PROV-DM type used to
create a graph node. We also defined a collection
for genomic documents (raw data). The reference
relationship approach was chosen to connect all
PROV-DM components, complementary
information of PROV-DM and genomic documents. Based
on
Figure. 11 shows our document based data representation, explained as follows:
• Project: stores different experiments of one agent. Attributes: Id, name, description, coordinator, start date, end date and observation. • FASTAQ: files used or generated in the activity; Attributes: Id, filename and description. • Activity: represents the execution of a program; Attributes: Id, name, program, version program, command line, function, start date, end date, account ID, used (name FASTQ, local, size), wasGenerateBy (name FASTQ, local, size), and wasAssociatedWith (Agent name). • Account: represents the performance of an experiment; Attributes: Id, name, description, execution place, star date, end date, observation, version and version date and project Id. • Agent: represents the person responsible for a program or a phase in the workflow. Attributes: Id, name, login and password. 5.1
In this case study, we have considered the MongoDB NoSQL database to store provenance and data files. The primary motivation for this choice was MongoDB’s ability to manipulate large volumes of data. MongoDB is an open-source Document-Oriented database designed to store large amounts of data from multiple servers.
It uses JSON- style documents with dynamic
schemas. The number of fields, content and size
of the document can differ from one document to
another. In practice, however, the documents in
a collection share a similar structure
MongoDB documents have a maximum size of
16MB. This feature is important to ensure that
a single document cannot use excessive amounts
of RAM. In order to store files larger than the
maximum size, MongoDB provides a GridFS API
GridFS uses two collections to store the data: fs.files collections, containing metadata about files, and fs.chunks collections, which store the actual 256k data chunks. The collections FS.file contains the name of the FASTQ file. Thus, it was possible to implement the relationship between MongoDB Collection Activity using Reference Document. In other words, we implemented the connection between Level 1 and Level 2 through the File Name attribute that was present in fileprovenance.files and Activity Collection. Figure. 12 illustrates this particular implementation. In contrast to relational database management systems, NoSQL databases are designed to be schemaless and flexible. Therefore, the challenge of this work was to introduce a data modeling standard for NoSQL document-oriented databases, in contrast to the original idea for NoSQL databases. The objective was to build compact, clear and intuitive diagrams for conceptual data modeling for NoSQL databases. While the current studies propose generic techniques and do not define a specific modeling engine to NoSQL database, our idea was to present a graphical model for any NoSQL document-oriented database. Moreover, while other studies describe techniques based on UML Diagram Class and JSON format as a modeling solution, we have a new approach to solve the conceptual data modeling issue for NoSQL document-oriented databases.
Future work includes: verifying our model for other NoSQL database classifications, such as key-value and column.