-Blockchain is an emerging technology that would Security is an ever-growing concern in the financial industry. possibly disrupt the existing centralized financial systems lead With the advent of digital financial information systems and to the rise to a new technology era for the financial sector. rich transaction technologies, the operations have become Amdadniatgioenmaellnyt,, deitfcf.erseungtgensetwthuasteBclaoscekschsuacinh haasshmeaulcthhcwaried,eirdaenptpiltiy- faster and the operational activities have spanned largely; at cations. Blockchain is founded on distributed ledger technology the same time, the risk of breaching information has been that ensures trust through consensus between parties in a peer- increased enormously. Although, there are advanced technoloto-peer network instead of the need to a third party or central gies encryption technologies that enable cryptic transmission authority. However, blockchain has several limitations such as of financial data between financial actors (e.g., banks, insider fsocralBabloilciktyc,hlaaitnenbceyi,ngloawdothprteodugbhyputhtewinhdicuhstarriees.thOef malal,inscbalaarbriileirtys intruder), security remain a problem because cryptographic is the most critical limitation of blockchain that needs an algorithms are still weak to many attacks launched by the efficient and effective solution. In this paper, we aim to enhance adversaries. Blockchain was deemed a major breakthrough the scalability of blockchain by designing and implementing technology that would prevent some unsolved security issues. a massively scalable architecture for private blockchain-based Therefore, the huge adoption of Blockchain was forecasted by awpeplciocantdiounctse,dcaslelveedraEl laesxtpiceBrilmoCen.tTsoonevEalluaasttiecBoluorC.coTnhteribruestiuolnts, many such as Gartner within not only the financial industry showed that ElasticBloC is a high-performant architecture that but also the other industries. scales massively. Furthermore, trust is critical when it is related to commuIndex Terms-Blockchain, Performance, Scalability nication between two or more parties. Historically and till now, trust is achieved mostly by the third party like banks or I. INT RO D U CTI O N authorities, that holds our data. In other words, communicating Over the last few years, Blockchain has drawn huge atten- parties rely on a common ledger which is held and managed tion to industry experts, technology evangelists, and academic by this third party. As a typical example, Clearing Households researchers due to its immense potentiality described in a large validates and manages the communication between trading body of literature and other sources such as blogs, forums, parties. So Blockchain technology comes as a potential techetc. Many researchers and industry experts argued that it is nology. Its disruptive aspect is that it eliminates the need for a revolutionary technology like the Internet that will provide intermediaries while performing transactions [7]. Hence, it can a highly efficient way to transact in a secure, immutable, empower groups of parties to agree on events without needing transparent, and auditable manner. In 2016, it was one of the the third party, such as the promise of this new technology [8]. technologies that reached a peak in inflated expectation; since Our study revealed several limitations of Blockchain techthen the interests in blockchain have been soaring to different nologies, as mentioned in the earlier section. However, the types of industries. In particular, the interest of Blockchain major concerns of Blockchain technologies are two-fold: scaltechnology among the financial industry started to grow as ability and performance. Scalability is considered the critical it might be a potential one that would enable them to avoid drawback that stands against blockchain technology. In fact, financial debacles such as the Heartland Payment Systems data it is the first limitation that must be addressed to make breach that had happened in 2008. blockchain an acceptable technology. The reason is obvious. Consider a Walmart payment system that processes more 250 transactions every hour. Since Blockchain technology
replicates blocks in different consensus servers hosted in dif- jointly manage a public Blockchain (BC) that ensures endferent locations to increase trust and guarantee security against to-end privacy and security. The overlay is organized as any odd modification, Blockchain must require a scalable distinct clusters to reduce overheads and the cluster heads are infrastructure not only at the operational level but also at the responsible for managing the public BC. LSB incorporates physical level. Because blockchain technologies are founded several optimizations which include algorithms for lightweight on immutability principle which means that every block cre- consensus, distributed trust and throughput management. To ated cannot be updated or replaced by a new block. All new ensure scalability, the overlay nodes are organized as clusters blocks will be appended only which will require physical and only the cluster heads (CH) are responsible for managing scalability especially for retail companies like Walmart, banks, the public BC. Technically speaking, this is a conventional and high-end manufacturing companies. approach to gain scalability which is very limited. However,
In the existing blockchain protocols such as Ethereum, Bit- it is not possible to gain massive scalability because it is not
coin, Ripple, and Tendermint, each participating node should supported by underlying system-level technologies such as file
process every transaction in the network. In this case, nodes system.
need more storage, bandwidth, and computation power as Zamani et al. [
Realizing the significance of a massively scalable infras- cess (and confirm) more than 7,300 tx/sec with an expected
tructure of Blockchain technology which cannot be addressed confirmation latency of roughly 8.7 seconds in a network of
by existing solutions, in this paper we developed a scalable 4,000 nodes with an overwhelming time-to-failure of more
Blockchain technology in which scalability is strongly corre- than 4,500 years. RapidChain is focused more on performance.
lated with the performance that has an impact on blockchain A limited effort was put on scalability; not to mention that the
efficiency. scalability is logical like the one proposed in [
The remainder of this paper is organized as follows. Section not address the massive scalability limitation.
2 discusses works related to the core issue of this paper. We Eyal et al. [
This research revolves around the scalability issue of limited only by the capacity of the individual nodes and
blockchain technology. In this section, we described a review latency limited only by the propagation time of the network.
of works related to this issue. Ehmke et al. [
To sum up, the research on Blockchain technology is still of this protocol Byzantine Fault Tolerance [
ElasticBloC is composed of several components. Fig. 1 depicts the architecture of ElasticBloC. The components are briefly explained in the following: III. ELASTICBLOC – THE MASSIVELY SCALABLE
BLOCKCHAIN SOLUTION
This section provides a detailed description of the core contribution of this paper. It begins with an overview of ElasticBloC, then a description of the high-level architecture of ElasticBloC is provided followed by a presentation of the solution workflow. The functionalities of ElasticBloC are briefly explained and finally, I explained the implementation of ElasticBloC.
ElasticBloC is a solution for developing blockchain applications. It is a generic solution that aims to support building all types of blockchain-based applications such as asset management, smart contract or notarization in a scalable manner. The users can deploy these applications on ElasticBloC which perform the internal blockchain functions such as generating blocks, adding blocks in the storage, retrieving the blocks, etc. ElasticBloC guarantees scalability at the physical level for blockchain applications.
ElasticBloC relies on a cluster computing paradigm that underpins developing an ecosystem consisting of a massive number of physical nodes (servers). As mentioned earlier, the focus of the solution proposed in this paper is to achieve the scalability of the physical layer instead of the logical layer. The scalability at the logical layer can be achieved in many ways such as the logical partition of blocks and then store the partition in different nodes within the cluster which consists of the limited number of nodes. However, scalability at the physical level needs extensible architecture. ElasticBloC architecture is extensible which enables users to add physical nodes on the fly or offline to enhance the capability to store any number of blocks generated by transaction applications. In order to gain easy extensibility, it reuses an existing distributed file system that simplifies adding new nodes. This file system supports commodity hardware; therefore, building a large cluster using ElasticBloC is cost-effective.
Performance is another issue dealt with by ElasticBloC. The
file system adopted in blockchain support extreme parallelism
as it underlies the MapReduce functional programming model
used by the applications to read and write blocks. Furthermore,
ElasticBloC adopted a technology that avoids computationally
expensive proof of work [48]. Proof-of-work based consensus
protocols are also slow, requiring up to an hour to reasonably
confirm a payment to prevent double-spending. ElasticBloC
• Transaction Gateway: The transaction gateway is a
connection component that enables to discover and connect
with blockchain endpoint. It is also a channel through
which users launch a transaction request.
• Request Receiver: It is an upfront server that receives
the HTTP requests.
• Communication Interface: It is a standard interface
for communication with Python applications. Since
ElasticBloC is developed using Python programming
language, this component is critical in communicating with
Python applications
• Operation Synthesizer: It handles various operations
including scaling up complex applications, object-relational
mapping, validation of a request, authentication checking,
and upload handling, etc.
• Blockchain Engine: It is one of the key components that
perform a multitude of tasks including handling events,
data modeling, and operation orchestration.
• Functional Interface: It is another important component
of ElasticBloC. It allows for Byzantine Fault Tolerant
replication of applications written in any programming
language. The consensus engine communicates with the
application via a socket protocol that satisfies the
functional interface. This interface consists of 3 primary
message types that get delivered from the core to the
application. The application replies with corresponding
response messages. It consists of three message types:
– DeliverTx Messages: Each transaction in the
blockchain is delivered with this message.
– CheckTx Messages: The CheckTx message is similar
to DeliverTx, but it is only for validating transactions.
– Commit Messages: The Commit message is used to
compute a cryptographic commitment to the current
application state, to be placed into the next block
header.
• Broadcasting Interface: It receives HTTP post request
from blockchain engine and broadcast it blockchain
repository.
• Blockchain Repository: It securely and consistently
replicates an application on many nodes. It works even
if up to 1/3 of machines fail in arbitrary ways [
The ability to tolerate machines failing in arbitrary ways,
including becoming malicious, is known as Byzantine
fault tolerance (BFT) [
In fact, it is not only a storage cluster, but the columnoriented database also enables querying and managing blocks.
ElasticBloC enables us to perform different blockchain operations using various methods that are described into two categories: BigchainDB methods that are provided by BigchainDB server and HBase connection methods for establishing a connection with BigchainDB and performing various operations. I implemented all HBase connection methods within the scope of this paper. These methods are presented in the following subsections.
a) BigchainDB Methods
ElasticBloC provides a library that allows the client to perform ElasticBloC functionalities. This library is called “bigchaindb driver”. The table below lists the major methods that a client can use to transact or operate in ElasticBloC. The above table represents the interface of functions that a client can use to communicate with ElasticBloC. This facilitates dealing with such modular architecture. The reason behind this facilitation is that once a client transacts or operates via these functions, the rest of the flow is automated i.e. the operation flows through the required components automatically. So, the client communicates with one component.
BigchainDB driver calls in its method’s implementation the methods of the BigchainDB-HBase connector, that will be explained in the successive section, to perform any operation that accesses HBase, such as retrieving data (transactions, assets, metadata, etc.), checking the preexistence of a newly submitted transaction, or storing of committed block with its details. b) BigchainDB-HBase Connector
BigchainDB-HBase connector is considered the core of our contribution. In this connector, I implemented a group of methods that BigchainDB can expose in order to connect and operate with HBase as a backend database. The following table describes the methods implemented with the connector.
Hence the preceding methods represent the interface of the connector that integrates BigchainDB sever with
ElasticBloC has a unique general workflow. In fact, this
workflow differs in its small parts according to the submitted
transaction mode or the nature of the desired operation. The
diagram below shows the general workflow of ElasticBloC.
Once the client has a valid transaction i.e. the transaction
conforms to the BigchainDB Transactions Specification, he
submits it to one or more ElasticBloC nodes through the
BigchainDB HTTP API [
After that, the HTTP request holding the transaction arrives
at the BigchainDB node at the Gunicorn [
Method connect create tables delete tables store transaction store transactions get transaction get transactions store metadatas get metadata store asset store assets get asset get assets store block get block get spent get latest block get txids filtered get owned ids get spending transactions get block with transaction delete transaction connection, transaction ids connection, metadata connection, transaction ids connection, asset connection, assets connection, asset id connection, asset ids connection, block connection, block id connection, transaction id, output index Connection connection, asset id, operation connection, owner connection, inputs connection, transaction id connection, transaction id delete transactions connection,
transaction ids delete latest block connection store unspent outputs get unspent outputs delete unspent outputs store pre commit state get pre commit state connection, unspent outputs connection, query
*, connection, unspent outputs Connection, state Connection, commit id Parameters backend, host, port, name, connection timeout connection, dbname connection, dbname connection, signed transaction
Description Creates a new connection to the backend database (HBase) Creates tables in HBase to be used by BigchainDB Deletes the created tables in HBase Stores a transaction in
Transactions table connection, Stores a list of transacsigned transactions tions in Transaction table connection, Gets a transaction from transaction id Transactions table
Gets a list of transactions from Transactions table Stores metadata in Metadata table Gets metadata from Metadata table Stores asset in Assets table Stores a list of assets in Assets table Gets an asset from Assets table Gets a list of assets from Assets table Stores a block in Blocks table Gets a block from Blocks table Check if a transaction id was already used as an input. A transaction can be used as an input for another transaction.
Bigchain needs to make sure that a given txid is only used once Gets the spending transaction Gets the latest committed block Gets all transactions for a particular asset id and optional operation Gets a list of transactions ids we can use which has inputs Gets transactions which spend given inputs Gets block holding a specific transaction Deletes a transaction from database and its relevant asset and metadata Deletes transactions from database and their relevant assets and metadata Delete the latest commited block Stores unspent outputs in utxos table Gets unspent outputs Deletes unspent outputs from utxos table Stores pre commit state Gets pre commit state of a commit id
Now, the operations between the local Tendermint instance and BigchainDB are established by the Application Blockchain Interface (ABCI) which is an integral part of Tendermint and implemented also at the BigchainDB server side. In this case, Tendermint uses the broadcast endpoint which is relevant to the initial BigchainDB’s request chosen. For example, if a client sent a transaction through /API/v1/transactions?mode=commit endpoint, Tendermint uses /broadcast tx commit endpoint respectively. Tendermint stores the initial validated transactions in its own mempool (memory pool). When it decides to create a block, Tendermint sends the creation request to BigchainDB by exposing a specific ABCI method. Then, it starts to send initial validated transactions that needed to be grouped in the desired block also using another ABCI method to BigchainDB which rechecks the validity of the transaction before it is added to the block.
The proposed block is then broadcasted to the network by Tendermint. Then it makes sure that all the nodes agree on this block in a Byzantine fault tolerance way. When the network agrees on a new block, Tendermint appends the new block to the blockchain in its local LevelDB, and the BigchainDB server receives a commit message enforcing it to write the new block and the including transactions, assets, and metadata in a separate way to the HBase repository. HBase then writes these data into the Hadoop Distributed File System underlying it. The same process is done at each node in the ElasticBloC network.
As mentioned earlier, my goal is to use existing technologies for building ElasticBloC. The reason is two-fold: avoiding reinventing the same technology that already exists and it would be impractically ambitious to develop a complex architecture like ElasticBloC. I developed ElasticBloC using the most advanced technologies. I briefly described the main technologies in the following: a) Tendermint
Tendermint is a secure state-machine replication
algorithm in the blockchain paradigm. It provides a form
of BFT-ABC (Atomic Broadcast) that is furthermore
accountable - if safety is violated, it is always possible
to verify who acted maliciously [
Tendermint begins with a set of validators, identified by
their public key, where each validator is responsible for
maintaining a full copy of the replicated state, and for
proposing new blocks (batches of transactions), and
voting on them [
The Blockchain Engine of ElasticBloC is implemented
using BigchainDB, which is for database-style
decentralized storage: a blockchain database. BigchainDB
combines the key benefits of distributed DBs and
traditional blockchains, with an emphasis on the scale [
The scalable block storage cluster (See Fig. 1) is implemented using HBase technology. Although BigchainDB aims at increasing scalability, yet massive scalability could not be achieved using BigchainDB.
Therefore, I implemented the scalable block storage
of using HBase. HBase [
The HBase connector which is the primary contribution of this paper was implemented using Python. The first step of building the connector was to indicate the tables that are needed to store the architecture data (blocks, transactions, assets . . . ). The second step was to write a file that opens a connection to Hbase, based on the connection parameters and values given by the BigchainDB configuration file, and return an instance of this connection.
In the next, the schema file was written. The schema file defines and creates the database schema at HBase once BigchainDB is initialized. After that, the required querying methods were implemented. Some of these methods are for retrieving data, others for storing, updating or deleting data. In addition to the above, some web application development tools have been used in developing.
This section describes some experiments that we conducted on ElasticBloC and discusses its results. The goal of the experiments is to evaluate two characteristics of ElasticBloC: scalability and performance. I conducted the following experiments are: Initial loading experiment, functionality experiment, and its result, scalability experiment and its result, and the ElasticBloC performance evaluation.
• Purpose:
Testing the start-up running and initializing of the whole architecture. • Requirements:
The required steps are to run the ElasticBloC components and check the connectivity between these components. The following summarizes these steps: – Run the Hadoop cluster and HBase. – Run the BigchainDB server.
– Run Tendermint instance. • Results:
The architecture components run successfully and the connection between the components is established. Once established, BigchainDB executed the schema file in the implemented connector and created the needed tables in HBase. The following screenshots represent some results of the successful initial start-up.
As we can see above, the components ran successfully and Tendermint opened the required sockets and established the ABCI Handshaking. It compares the application’s highest height and the application hash Fig. 4. BigchainDB Web Interface After Startup.
Fig. 5. Tendermint Start-up. to that stored in HBase in order to confirm that the data is the same. b) Functionality Experiment • Purpose: Test if ElasticBloC performs operations normally. • Required Steps: Testing the functionality of ElasticBloC is done through writing and executing a Python script that gets uses of the bigchaindb driver library. The steps below show the required steps: – Run ElasticBloc components. – Write a Python script that creates a transaction, fulfills it with the sender private key, sends the transaction in a commit mode. The following Figure represents the testing Python script. • Results
The transactions are successfully created and sent. Because the sent transactions are in commit mode, so directly BigchainDB created a block for the transaction. This block was appended to the Tendermint local copy of the blockchain, and the block, including transaction, assets, and metadata are stored in their specific tables at HBase.
Fig. 7 shows the created block in the blockchain stored at Tendermint.
Fig. 8 shows the result of the submitted transaction with its details retrieved from HBase. • Discussion
The experiment is conducted successfully and the architecture is working finely.
One of the positive aspects of the architecture is that creation, fulfillment, sending, and validating of the transaction, with the block creation and appending to the blockchain, and its storage in HBase took around only one second.
One major limitation of the experiment is that these experiments were conducted on a limited blockchain dataset. The reason behind this is that there was neither possibility to access a large blockchain dataset nor time to build our own large dataset. Instead, we take into consideration previous benchmarks and experiments were done using huge bulks of data in which it helps us to evaluate our architecture. c) Scalability Experiment • Purpose
Test if ElasticBloC can scale massively. • Required Steps
Massive scalability means that ElasticBloC has the ability to scale as much as it needs on its physical layer. This could be ensured if we succeeded in adding new data nodes to the ElasticBloC node. To do that we tried to add new Hadoop nodes to the Hadoop cluster either while ElasticBloC is running or when it is offline. • Results
The previous experiment is conducted successfully and
• Discussion
As ElasticBloC scales by adding a new server to the Hadoop cluster, it is feasible to add data nodes either on fly or offline. This means that ElasticBloC has the ability to scales massively. d) Performance Evaluation
Concerning the large blockchain dataset, we were not able to access a large blockchain dataset rather than building it. For that, we could not test ElasticBloC on a massive scale dataset. Accordingly, we rely on this section on some previously conducted experiments and workbenches that give us, theoretically, a clear idea on the performance of the overall architecture.
The overall performance of ElasticBloC is evaluated by its components, in particular, the performance of Tendermint as a consensus engine and HBase as a backend database.
Tendermint acts as a high-performant in a large
distributed environment. According to Cosmos white paper
[
Moreover, based on several benchmarks, such as [
Hence, according to theory and pre-existing experiments, HBase would also enhance the overall performance of ElasticBloC. However, in its worth case, replacing MongoDB with HBase will not downgrade the performance of ElasticBloC.
Blockchain has not been adopted widely until now except for cryptocurrency applications, however, it has been identified as potential technologies for several areas that need trust and security.
It is relatively a new technology that needs various improvements to reach to a maturity level. It has several limitations that are the main barriers to the wider adoption of this technology. Of all, scalability and performance are the major limitations that must be addressed.
This research primarily aims at addressing the scalability problem. There are some solutions that offer techniques methods, and guidelines for scalable blockchain. However, we found that state-of-the-art technologies focus on scalability at the logical level which is an inadequate approach if scalability at the physical level is to be guaranteed. In this paper, we designed and implement a scalable architecture called ElasticBloC which enables users to build a highly scalable blockchain-based ecosystem consisting of tens or more of physical nodes.
In this paper, we proposed a solution for the scalability limitation of blockchain technology. We discussed our solution ElasticBloC which is a scalable architecture for building blockchain-based applications such as payment system, notarization, the smart contract can be implemented by ensuring scalability. ElasticBloC is a built on cluster computing paradigm that building infrastructure with a massive number of nodes. We presented the components with a detailed description. We discussed the workflow of ElasticBloc. We discussed technologies that we used in implementing the proposed solution.
We tested ElasticBloc to evaluate the scalability. Our experiment shows that ElasticBloc has the ability to scale up; it is flexible for adding as many servers. We discussed the results of our experiments in this paper. Additionally, we provided some previous workbenches to provide a comparative view of the abilities of ElasticBloc.
Several works are lined up for a future extension of ElasticBloC. However, to the best of our knowledge, the imminent critical task that must be accomplished in the near future is extending the functional capabilities of our solution. We planned to enhance add new modules to ElasticBloC to enable users to develop permission-less blockchain-based applications or for permission blockchain-based applications.