Nowadays, building an e cient, reliable and scalable decentralized data storage architecture is an actual problem in both corporate and academic communities since decentralization requires new approaches that adapt existing models of distributed systems. The aim of this research is to form a model for the decentralized storage system to distribute data over available storage nodes, e ciently reorganize data when nodes are added or removed and enforce exible constraints on object replica placement that maximize data safety in the presence of coincident or correlated failures. The model of distributed decentralized object storage integrated with blockchain payment system is proposed in this paper. A novel approach based on representing a network map as a graph and creating extended ltering mechanism for deterministic subgraph de nition for object placement is introduced. The concept of a container, de ning a network map subgraph, and a placement function for obtaining a container subgraph (based on the network map graph) or an object placement subgraph (based on the container subgraph) are presented. A modi cation of the CRUSH method is proposed for data placement.
The development of blockchain technology has recently moved not so much towards global public permissionless blockchains as toward the implementation of corporate- and state use to solve speci c internal tasks. It is logical to expect the development of Dapp projects in this direction. At the same time, DApps are supposed to store data in the decentralized storage systems to preserve their advantages over classic applications.
Currently, most projects [
Building an e cient, reliable and scalable decentralized data storage architecture is an actual problem in both corporate and academic communities since decentralization requires new approaches that adapt existing models of distributed systems.
The aim of this research is to propose a model for the decentralized
faulttolerant [
The proposed model contains the following components: network structure, network map, data placement method and the concept of a container. These components are discussed in this paper. 2
2.1
Network Structure
The Distributed Decentralized Storage Platform (DDSP) p2p network is
comprised of two types of nodes: Inner Ring nodes are responsible for maintaining
information about network topology, accounting and data audit; Outer Ring
nodes are responsible for storing data and ensuring its integrity and availability.
Information about all active Outer Ring storage nodes and their properties forms
a Network Map. Inner Ring is responsible for keeping Network Map up to date
and distributes changes over all network peers. At the same time, Inner Ring
nodes have to maintain all operations in the conditions of total distrust [
Network Map
DDSP keeps the network map up to date and distributes it over nodes. It
contains information about groups of nodes, their location in the network and main
parameters necessary for correct data search and placement. The network map
is represented as a graph [
Placement Function and Placement Rule Instead of maintaining and querying database with information about each object's location, the proposed DDSP uses consistent placement function with the following arguments: { storage Policy de ned with Placement Rules, { network map (or the network map subgraph), { rendezvous hashing salt.
The one can de ne a storage policy that is applied to the stored object. The placement rules consist of a set of SELECT() or FILTER() operations applied to the network map. The result of these operations is a subgraph of the network map where data can be placed. The SELECT() operation is applied to a tree-like subset of Network Map. It takes as inputs a replication factor and a bucket type. Multiple operations in the storage policy are enqueued and each subsequent operation is recursively applied to the previous operation's output. The FILTER() operation is applied to the graph. The operation inputs are the bucket type, bucket value and comparison operation. For text values, the operations eq and ne are available. For numeric values, gt, ge, lt, le are additionally available.
A set of operations on the graph (in the placement rule) can be grouped by
using AND, OR, NOT. The placement function is performed recursively, with
the operation of the next step being performed for all the nodes retrieved at
the previous step. Buckets and placement nodes in the bucket are selected using
Rendezvous hashing algorithm. With it, each node or bucket has an individual
hash number for an individual item, and a bucket or node having the largest hash
number are chosen. Data in this algorithm is distributed uniformly and small
number of data movement is required when nodes are added or removed [
The placement function is executed recursively with the operation of the next step being applied for all the nodes retrieved at the previous step.
This network map (or network map subgarph) (see Fig. 1) consists of the bucket with types Loc (country), City (city), Storage (storage disk type) and the nodes of the corresponding buckets. The storage policy may be de ned as follows: { it is stored in 2 di erent countries, { it has 3 copies per each country, { it should be stored on SSD drives only.
This informal description should be represented as sets of SELECT(r, type) and FILTER(type:value, op) operations. The total replication factor is obtained by multiplying all the factors of each SELECT(r, type) replication. The nal statement must always be SELECT(X, Node) (see Table 1). FILTER ( [ Eu ] \ [ Storage:SSD ] ) [ (Storage:SSD, equal) ( [ Ru ] \ [ Storage:SSD ] )
[[ Node 3, Node 4, Node 5, Node 6 ], [ Node 7, Node 8, Node 9, Node 10 ]] [[ Node 4, Node 5, Node 6 ], [ Node 7, Node 8,
Node 9, Node 10 ]] ( [N1; N2; N3] 2 ( [ EU ] \ [ Storage:SSD ] ) ) [ [[ Node 4, Node 5, Node 6 ], ( [N4; N5; N6] 2 [ Node 8, Node 9, Node 10 ]] ( [ Ru ] \ [ Storage:SSD ] ) )
Thus, the subgraph of the network map and the 6 nodes have been obtained (see Fig. 2), where the object should be placed on to meet the placement policy requirements. The result of the placement function operation is a subgraph of the network map { the Placement group { leaves of which are a deterministic and consistent list of the storage nodes. If the storage policy, salt and network map are known the Placement group can be retrieved without referring to a third party or storing \object-storage node" pairs. 2.4
Container In order to place objects in the system a user needs to de ne a container. The container has the following elds: storage policy (placement rule and redundancy factor), uuid, owner and maximum capacity. The container de nes the subgraph of the network map. The approach, where a complete network map, which operates with all the connected nodes, plays the role of an initial graph, can impose additional costs as the network increases. The allocation of limited space within the network map (subgraph), de ned by the container, allows to do the following: root Loc: EU
Storage: SSD
Loc: RU
City: Milan
To form the container`s subgraph, the replication factor in each SELECT (r, type) call is increased by the redundancy factor (Kr). The container uuid is used as salt for selection in the placement function.
For ease of presentation, consider the example, where Kr = 2 see Table 2.
SELECT(2 Kr, node) [[NN59;; NN610;;NN71;1N; N8],12],
[N13; N14; N15; N16]]
As a result, a number of nodes is obtained that can be represented as a subgraph of the network map. When forming the container, it is possible to approximately estimate its capacity. To do so, the weights of the nodes might be used, depending on their capacity. The weight of the selected nodes has to
correspond to the container capacity declared. The object is placed into the container by using the placement function, where the hash of the object being placed is used as salt (see Table 3).
Network map
Container root Loc: US
Loc: EU
Loc: Ca Loc: RU
Loc: Ja
The highlighted buckets are the Placement group subgraph for the object. (see Fig. 3). 3
The paper proposes a novel approach to scalable data placement in decentralized distributed storage systems. Using a subset of network map, storage policy rules for object placement, and a Rendezvous hashing for a node selection allow to control an object location and a minimal data movement in case of storage nodes failures. Metrics comparison with the existing models are the subjects of further research.