Internet of Things (IoT) becomes recently a popular research topic and market reality. According to several research companies, in 2025, up to 75 billion devices are estimated to connect internet and generate an enormous number of data. This increases in data cause several difficulties such as the storage cost and processing of such large data. In this paper, we have been studied on performance comparison of relational (MySQL) and nonrelational (MongoDB) database management systems for storing and processing of this large IoT data. Both types of database management systems have been tested. According to comparison of experimental results, the nonrelational database management systems, which we studied and searched, have provided better performance for storing and processing of large data.
The Internet of Things (IoT) is a self-configuring and
adaptive system that consist of sensor networks and smart
objects whose aim is to interconnect all devices/sensors in
daily life [
–both structured and unstructured. The DBMSs basically can be separated into relational and non-relational DBMS.
tables with a high data consistency. The most commonly used open source relational DBMS is MySQL. Nonrelational databases (NOSQL) have arisen as an alternative to relational databases. The aim of the NOSQL is often not to give guaranty the Atomicity, Consistency,
depend on constant table definitions and rigid schemas.
time without exclusive process. Therefore; the number of records in the columns does not have to equal with each other. Data sets in IoT environment can change after setup of the system, so this environment requires a flexible data storage system. There are four different storage formats in
in NOSQL. In such a system, a record is called document and these documents are usually stored in JSON format [3]. There are various implementation of the NOSQL DBMS such as MongoDB [4] [5], CouchDB [6], HBase [7], Cassandra [8], Amazon SimpleDB [9], and Redis [10]. Since MongoDB is open source and commonly used, it has been chosen MongoDB in this study. There are no database schemas or tables in MongoDB. MongoDB uses “collection” instead of a table, and “document” instead of rows to store data. Furthermore, MongoDB uses two different operations instead of the join operation. These are nesting documents inside each other and to store a reference to the other document rather than nesting entire document.
of databases [11] [12] [13] [14]. These studies vary depending on the data size, the variety of data, the differences in databases used, implementation languages, and subjects of the projects. In the study [15], MongoDB, MySQL, CouchDB, and Redis are compared. It is declared that MongoDB is performing better among the comparative database management systems in terms of the "bulk insert" writing performance. However, MySQL and MongoDB have similar performance results for reading operations. Performance parameters between these DBMSs can be negligible (typically less than 1 second) [15]. However, our test results show that
of reading and writing as represented in “Results of
tools such as Apache Mahout [11]. ACID operations on
compare them [12] [13]. According to the results, the use of the MongoDB has been encouraged for large data applications, especially for applications of big data [12].
in terms of the read, write, delete, and instantiate operations. According to results of this study[14], all keys are needed to fetch, MongoDB has better performance than the others.
The aim of the experiments is a comparison of the relational and non-relational DBMSs for utilization an IoT platform. The system includes IoT devices which publish a tremendous number of sensor data where servers store and process them. Performance of reading and writing tests has been done in both MYSQL and MongoDB in this study. The test results which are calculated after the application runs at least three times are compared to find out where we can store data of IoT considering lowest cost in terms of throughput.
sensor data as seen in Figure 1. The sensor data is produced by devices/clients and collected by Data
a DBMS through insert and update operations. Data Query Server (DQS) in the platform provides an interface for processing and reporting by Client Application. The client application sends the data request to the DQS and they are formed as a query for DBMS. The result of the query is delivered to the client application. In our case, enormous data should be stored in the DBMS. Therefore, writing operations are more important than the reading operations. The platform has active-active architecture.
writing transactions. In our platform, our target throughput is forty messages/milliseconds in average for writing. Our target throughput and test results compared with “Results of Experiments” section.
Environment & Methodology” section consists of the information about the environment of the experiments and methodology. “Results of Experiments” section presents the results and graphics from the experiments. “Conclusions” section summarizes and concludes the experiments and gives a recommendation for future of this study.
non-relational databases. Server and clients of chosen
separately in virtual machines with Ubuntu 16.04 Server version. Virtual machines are hosted on a physical machine (i7 6700HQ, 16 GB DDR3 RAM, and SSD disc).
executed on the physical servers as depicted in Figure 2.
not disrupted the tests. Both DBMSs are installed on the computer with SSD for the fastest possible read and write speed and they are executed separately during the test scenarios. Multithreaded Java applications for reading and writing operations on DBMSs are implemented.
number of threads, number of messages, and the size of the string. These constraints are applied for both DBMSs. The application is executed on a virtual machine; therefore, the applications are limited in terms of the CPU core and memory. In this study, since writing operations are more important than reading operations, our tests concentrate on the writing operations. In our tests, data examples are selected as similar to real-time sensor applications.
In this study, two columns are defined including variablelength string type, an integer type. A primary key column is automatically defined in MySQL, but it needs to be defined in MongoDB. The execution time of the tests is calculated in milliseconds. The number of messages is measured dividing the total number of messages into the experiment’s execution time. 80,00 E M I /T60,00 S E G A S40,00 S E M FO20,00 R E BM0,00 U N 5 0 0 0 0 0 1 0 0 0 0 0 0 1 5 0 0 0 0 0 2 0 0 0 0 0 0 4 0 0 0 0 0 0
NUMBER OF MESSAGES 1-10 1-20 1-40 1-80 2-10 2-20 2-40 2-80
Insert tests are executed with a multithreaded Java application. The application sends insert SQL request which contains a string (100 characters) and an integer value to DBMSs. The application is executed in computers based on the parameters (number of computer and threads) as depicted in Figure 3 and Figure 4. Best throughput is 18.21 messages/millisecond in average for MySQL such that is succeeded with two computers each has twenty threads as presented in Figure 3. As it can be seen that forty threads for one and two computers utilization have close results. Best throughput will go up when the number of threads increases. However, when the number of threads reaches eighty, throughput value begins to decrease. Therefore, it has been decided that MySQL can manage forty threads for best results.
in average for MongoDB such that is succeeded with two computers each has forty threads as presented in Figure 4.
computer is also used for MongoDB. Throughput for utilization of the one, two and three computers is 61.22, 70.95, and 61.65 messages/millisecond, respectively. As it can be seen that two computers’ utilization has the best performance. Another result can be obtained from the
M O N G O D B It is also tested the data with different variable string length as depicted in Figure 5 and Figure 6. Since best results are obtained from the two computers, only two computers cases are tested in these tests. As it can be seen that 18.21, 9.33, and 6.13 messages/millisecond in average are obtained for MySQL with 100, 1000, and 2000 string length respectively as depicted in Figure 5.
MySQL has best throughput results. For MongoDB, forty threads utilization has best throughput results; 70.95, 49.04, and 39.85 messages/millisecond in average are obtained with 100, 1000, and string (2000 characters) respectively as depicted in Figure 6. MongoDB DBMS has about four times better results than MYSQL. As it can be seen that length of the message is one of the most important parameters. Such as when the length is doubled, throughput is decreased about twenty percent.
M O N G O D B E20,00 M I T / S E G A SS10,00 E M F O R E BM0,00 U N E80,00 M I T /S60,00 E G A SS40,00 E M F O20,00 R E B UNM0,00 stable as MongoDB. Best throughput is 60.09 messages/millisecond and 44.34 in average for MySQL with one computer has eighty threads. In MongoDB, the best result is 68.68 messages/millisecond and 58.61 messages/millisecond in average is succeeded with three computers each has eighty threads as depicted in Figure 8.
Our IoT platform requires that forty messages/milliseconds in average should be written to the chosen DBMS. Otherwise, the number of messages waiting in the queue for writing will increase and it can cause memory problems. In the IoT platform, activeactive architecture is applied. Therefore, more than one computer can write to DBMS at the same time. Test results show that MongoDB has better performance than the MYSQL in terms of both writing and reading operations. In the IoT Platform, the message payload is varying between 100 bytes and 200 bytes. For these types of messages, MongoDB has 70.95 messages/milliseconds in average and MySQL has 18.21 messages/milliseconds in average for writing. As it can be seen that only
M O N G O D B 1 0 0 . 0 0 0 1-10 2-10 3-10 2 5 0 . 0 0 0 5 0 0 . 0 0 0 1 . 0 0 0 . 0 0 0 2 . 0 0 0 . 0 0 0 NUMBER OF MESSAGES 1-15 1-20 1-40 1-80 2-15 2-20 2-40 2-80 3-15 3-20 3-40 3-80
on the DBMS. Test results show that MongoDB has better throughput than the MySQL. MongoDB has 55.07 messages/millisecond in average and MySQL has 46.66 messages/millisecond in average for two computers.
applied to TUBITAK 1505 University Industry
“MongoDB https://docs.mongodb.com/manual/. .
“Cassandra.” [Online]. http://cassandra.apache.org/. “Amazon SimpleDB.” https://aws.amazon.com/simpledb/.
[Online].
“Redis.” [Online]. Available: https://redis.io/.
Adv. Softw. Eng. Its Appl. ASEA 2015, pp. 41–47, 2016.
MongoDB to an SQL DB,” Proc. 51st ACM Southeast Conf. ACMSE ’13, p. 1, 2013.
and NoSQL databases,” IEEE Pacific RIM Conf. Commun. Comput. Signal Process. - Proc., no. August 2013, pp. 15–19, 2013.