The mathematical model “Peer-to-peer distributed system for securable information storage and processing in enterprise networks” is described hereinafter. It is a versatile distributed operating system designed for the protection of distributed computing and insulation of private networks without restricting the possibilities of effective interactions, cryptographic security, protection from unauthorized access with the application of biometry and an innovative protocol of data exchange for topology control based on distributed ledger technology. The modeling was performed with the purpose of evaluation of performance of the system depending on productivity of the hardware of its nodes and the network's telecommunications equipment.
Mainstream systems for data storage and data protection are built, for the most part, using centralized
architecture or they have operation centers, the control over which may be intercepted via hardware- or
software vulnerabilities or by way of planting a mole in the technical staff. Also normally for data
calculation and data storage they do not use advanced laptops and personal computers installed on user
workplaces (workstations) featuring significant computing resources (terabytes of ROM [
The peer-to-peer distributed system for securable information storage and processing (“the system”) is first and foremost designed for removal of cybersecurity threats to servers, central computation and subscriber nodes within a network. It is a distributed operating system, in which each node gives away its computing resources (storage subsystem, central processor and graphic processor, and randomaccess memory) to the system and has no independent meaning. However, nodes vary in terms of their functional purposes: subscriber node (it can simultaneously serve as a storage node or metadata node), storage node, and metadata node. This functional purpose is assigned at the stage of implementation of the system and can be changed automatically later. The assignment of a role to a node occurs under the control of an automatic maintenance subsystem without participation of the owner of a specific node. Data storage in the system is performed in such a manner that any user data block (a file) is divided into N identical packets, then it is encrypted and submitted for storage to ∙ redundancy coefficient) in encrypted form. The list of block storage nodes is placed in the metadata block, which is, in turn, encrypted and placed in metadata nodes. No one, including the owner, knows in which nodes, at a particular time, parts of the file are stored, except for the maintenance subsystem network nodes (K – that has access to metadata.
When designing the system it is necessary to bear in mind that at any discrete instant of time significant volumes of information must not only be passed between nodes of the system with high speed, but they must also undergo encryption and decryption procedures. It is also necessary to consider additional dataflow of service blocks of the distributed ledger (metadata) which contain data about writing/reading nodes for data and information of the data access control system. If the data processing speed turns out to be insufficiently high, the system would not be able to deliver comfortable user experience, which would, in turn, put into question the possibility of application of this system in a reallife enterprise network. To assess the possibility of implementing the system with sufficient data access speed let’s build its mathematical model. For clarity, sufficient data access speed means the speed of reading/writing operation comparable with the average user data access speed in existing data storage network systems.
The system is a queueing network (QN) which may be shown as an entire graph [
The matrix describes the speed of data transfer between nodes of the system: - speed of data frame (1) , , … , , , … , ,
, , … , , , … , , ⎥⎥, ⎡ ⎢ ⎢ ⎢ ⎣ , , … , , , … , , ⎦ ⎤ ⎥ … … , values of local maximum speeds of connection of system’s nodes. where , –connection speed, - number of nodes of the system, , connection speeds of –ith and th nodes of the system, correspondingly, – countable set of possible - local maximum
Nodes are independent queueing systems (QS) with confined queue [
, / ; ∑ 1⁄ , ∙ ;
∑ ∙ Ϸ ∙ , number of nodes within QN:
. the subsystem for data encoding and data mixing (SDEM), as well as a full-fledged type 2 node. Thus, the QN can be subdivided into 2 sub-QN’s: a data processing network comprising and third type nodes, and metadata processing network comprising first type nodes. The total second type nodes
Although the hardware of type 3 nodes is oftentimes less powerful than the hardware of type 2 nodes, where specialized server equipment is normally used, in reality the following formula is executed: ≪ , while the cumulative computing resource of type 3 nodes is substantially greater than the relevant indicator for types 1 and 2 nodes. Therefore the application of type 3 nodes as data centers for QN network in distributed computing systems is more than substantiated.
On top of QN on nodes ∈
of type 3 there is the functional network of JSG, which even though consumes the resources of nodes within QN network it functions in an absolutely independent and isolated manner, and it acts as an external source of jobs in relation to QN network. The input source of jobs in the system is VM of the nodes, each of them generating an ordinary random flow of , , , , … , , , … ; ⃗ , , , , … , , , … ; ⃗ , , , , … , , , … ; 1, ;
is generated with the intensity and the statistical expectation mutually independent.
In which at random times , random size jobs , are generated of one of the two types: 1. job for reading data from the system , 0 ; 2. job for recording data into the system ,
are generated in sequence, not more than one per any specific time , . The values , , , and , are ∙
The system provides for simultaneous processing of class D1 jobs generated by SDEM of one , in the amount of ,
, / , and it strictly prohibits to record two packets from one , into one node of processing classes 1 or 2. I.e. the average number of class D1 jobs as generated by SDEM th source must satisfy the inequality , where Ϸ – coefficient that characterizes mean-square deviation of the value , from its statistical expectation , which defines the requirement to assigning the size
of the system:
initial jobs :
where
1, ∞; , ∈ 0, 1 .
value , [
Then the stream is transferred to SDEM, where a distributed ledger transaction is opened, the job of , is converted into a set of standard data blocks (jobs) of the system: 1. Metadata blocks with the size bits containing the status of the distributed ledger transaction of the system, and are processed by type 1 nodes; 2. Data blocks with the size of bits are processed by type 2 and 3 nodes. Thus, 3 class – job for reading the data block
; M class – job for processing the metadata block classes of standard jobs are generated in the system: D1 class – job for recoding the data block ; D0 .
Let’s define the average time for transfer of jobs between the node of the system and the node as arithmetic mean of the matrix (1) for the packets and , correspondingly: max ; It is admissible to determine the size of the block The average time spent by SDEM to process one job from the stream : in an arbitrary way.
(2) of the (3) (4) (5) (6) ; ; , , ∈ ;
1, Ϸ 1⁄ , ∙ ; ;
∙ ;
;
are independent, and none of them can be compared in terms of capacity with the
cumulative stream, therefore in accordance with Khinchin theorem [
→ ∞ in the network the cumulative stream
will work for the simplest one [
Let’s determine common determinate parameters of the node of QS (job processing centers) which depend on hardware parameters of the node, namely: – the queue size, established the processing node receives jobs with a fixed length of ∈ , hardware and identification parameters of the node which are significant for the modeling: – the bandwidth of the node’s QS, , – the job processing time. In accordance with the classification bits. The vector of – the storage size, will be the function of resources and statistical expectation correspondingly:
represented by the node for working with SDEM, intensity , which characterizes the average value of a job in bits.
The total intensities of generation of the jobs of classes D0, D1 and M in the simulated QN are, ℎ , ℎ , ,
∈ ; 1, , – the variable for storage of the new node number; ℎ – the attribute of availability of free RAM space (slots in the node’s queue); – the unique identifier of the node in the system network; speed, maximum available RAM capacity, number of processors, number of cores per processor, and processing power of the processor’s core, correspondingly, ∈ , 1, , , 0, jobs sent to the node; – the number of nodes in the system. 0
1,7; – the confined countable set, members of which are included in the master data by equipment manufacturers; – the node set of the system; (t) – the instantaneous number of unserved
Types 1 and 2 nodes function as processing centers. On them, the node’s software consumes a certain fixed part, the size of which is determined by the value of the parameter ℎ : 0; ℎ ; 2 3 where , ℎ
0; ℎ
Type 3 nodes combine the function of type 1 processing center which consumes the resources as per (13) and the function of JSG as part of VM and SDEM which consume: , … , , 0 , и , … , , ,
1, , correspondingly. The vector of resource consumption in JSG: 0, ℎ , ℎ 3 3 1 1 Let’s determine the resources available to the data processing center taking into account (1), (11) ℎ 0; ℎ 0; In accordance with (15) let’s determine the parameters , , and , of th node: Let’s define the coefficients taking into account (9) and (21) using the following equation: ⟹ 2 ; ℎ , , , ∏ , ℎ , ℎ ; 2 ∙
Let’s define as: additional indices ,
: ℎ , ℎ , , , |ℎ
∈ 2, 3 ∈ ; ordered set: : the subsets : ⟶
∈ In accordance with the system’s operation logic in QN let’s select ∈ – the subset of nodes in the independent QN that processes jobs of class M, and let’s divide the subset of ∈ of the nodes , which generate the network for processing jobs of classes D0, D1 into the subnetworks (subsets) , 1, in such a way that the subnetwork receives the nodes with the highest bandwidth of the node’s QS , and in with the smallest one for this. Let’s enter in the ℎ , ℎ , , , |ℎ 1 ∈ ; 1, ;
1, , and apply to the function of sorting to get the . As a resulting set of the nodes let’s determine ∈ | ; ∆ ; ∆ ; ∆ ; 1, ; (20) – the attribute of the node belonging to a specific type based on the classification introduced. After this, we will be analyzing the networks: , . The network is an open QN with the intensity of stream from the outer source
(9) and one class (M) of jobs. When processing each job from (2) 2 jobs are generated with class M, where of them one at a time enters each ∈ , while the remaining jobs are distributed by the nodes , depending on the capacity and size of the node’s queue. The intensity of the input stream, without taking into account the stream of resent jobs in the nodes : (15)
(16) (17) (18) (19) (21) (22)
, where ⎧ ⎪ ⎨ ⎪ ⎩
, , ,
0 ⋀ , ∑
as: , ∙
, ,
,
, , ∙ , , , 0 max m ; max
min 2 ∙ m ∙
min
Determination of the coefficients in the form (23) not only satisfies the equation (22), but it also
QS of G|G|1|
redistributes the load to the most productive nodes of the network . Each node of the network
is a
type as per Kendall’s classification with the queueing discipline FCFS [
, 0 0 0 ; 0 0
0 1, . . Solution of the system (24) is, in general, ;
(23) ; with the starting condition , 0
1, the normalization requirement ∑ intensity of output stream ,
possible using numerical techniques [
, based on the equation: ∙
0 ⋀
0 ⋀ 0 0 0
(1) 0
, ; max
∈ : ℎ max
∈ : ℎ 1 ∙ 1 1 ; 1 ; , , with the initial condition , 0 1, the normalization requirement ∑ 1, and intensity of the output stream jobs in the queue of the node ,
. Let’s define distribution of the average number of , ∑ ∙ , . Then distribution of the virtual time for execution of the job taking into account (4), (19): , , generates the streams of denials of classes D0 and D1 with the intensities: , ∙ , and , , , , . The jobs denied must be submitted for processing to nodes of the network Here the stream of resent jobs sent to the nodes which are available based on the size of the queue taking into account the function of availability (12) . The node , , will be as follows: , ,
, , ∙ , , , ∙ ∑ , , , ,
where ∑ , 1, ∑ , , , , , ℎ and ∑ d ∑ , , , , ℎ ℎ
The network from the external source the nature of the networks the nodes of the networks ∑
; ∑ is divided by the nodes:
of the network system’s processing speed is defined by the speed of reading, and capacity of the nodes , let’s define as: ⁄∑ , ; ∑ , input stream of the node
without taking into account the resent jobs: and
, divided into the subnetworks arranged by capacity of the nodes with the intensities:
Taking into account the intensity of the stream of denials it would be fair to record (21) as: 1; ∑ : ∙ , , , , , ,
ℎ 147 (25) (26) (27) 0 is the , а
(28)
distribution ,
equations [
Considering the functional model of the network (21) – (25) let’s define the concluding virtual distribution of time for execution of the jobs , by the network :
, is an open QN with two classes (D1, D0) of jobs and intensity of the streams The node of the network ,
, , ; is a QS of G|G|1| condition where the number of jobs in the queue and at any specific time t it can be in the condition , , 1 means that the node is overloaded and the servicing of the job is denied. The probability of the conditions ,
was defined by the system of Kolmogorov differential ∙ , , , ; ∙
(20). Taking into account let’s divide the stream D1 between
, ∙ 1, 1, / , where ,
0. The stream D0 . As subjective estimation of a is the multitude arranged by 1. Then the intensity of the type with the intensity of the input steam
, system on the basis of (7), (28), (41): 4. Some Modeling Results
, , max , t , , t ; ℎ
Together with the intensity of the stream of denials it will be fair to record (27) as: Taking into account the functional model of the network (20), (27) – (29) and concurrent processing of the jobs of classes D0 and D1 as generated by SDEM from the job , , let’s define the final virtual distribution of time for execution of the jobs , by the network : ∑ ℎ
∙ ℎ , , Then the virtual distribution of time for execution of the jobs , as generated by the node of the , ; ∑ (29) (30) (31) each hardware type. The JSG network comprises 500 nodes.
Based on the model (31) and using Python programming language [
4 subsystems, 50 nodes each. Actually utilized are 50 nodes per
1 2 3 4
50 50 350 50
packet was within the range of parameters of the job stream ∈ 100 , 2000 ; the size of metadata packet was 2 ; for two experiments (Table 2). Parameters of the first experiment simulate peak activity in the test net when processing multimedia data (opening, editing, and reporting video-, audio- or any other graphic data) by real users. As part of the second experiment an extreme situation is simulated, where each node of the network at any specific time reads out from distributed storage or transfers to distributed storage super large amounts of data in automatic mode, for instance, it executes a queue of jobs for copying multimedia data or graphic data. The generalized stream of traffic coming from JSG network is determined by the sum (10).
(jobs/sec.) (kB)
1 300 ; 2000 148
1 3000 ; 20000
In order to estimate the outgoing job stream (the results of execution of ) the following statistical
values were used [
Based on the results of operation in Experiment 1 mode (Figure 1:) it is apparent that grows purely and linearly as grows, and it practically does not depend on the size of the packet . Certain correlation of the value , with can be simultaneously observed. We did not demonstrate the chart as there were no service denials in Experiment 1 mode. This behavior indicates sufficiency and even certain redundancy in terms of capacity of the distributed data storage system built using the model proposed for traffic with the input parameters, equaling the values of Experiment 1 mode (Table 2).
(a)
(b)
Based on the results of operation in Experiment 2 mode (Figure 2:) it is apparent that grows purely and linearly as grows to the value 7000 , starting from which, we observe a clearly-defined productivity dip, which testifies to the initial stage of the system overload and accumulation of queues at the nodes of the data processing network, and there can be observed a meaningful dependency on the size of the packet . Starting with the values 7000 the variance begins to grow substantially. Despite the obvious overload, the denials are not present, except for the area ∈ 100 , 500 ; ∈ 11000 ; 20000 . Thus, the distributed data storage system built using the model proposed for traffic with the input parameters equaling the values in Experiment 1 mode (Table 2) demonstrates acceptable productivity, except for the values ∈ 100 , 500 ; ∈ 11000 ; 20000 .
(а) (b) Figure 2: Simulation Results in Experiment 2 Mode (c)
Testing of the model proposed in two fairly hard operating modes, at peak loads, was performed. The results of testing suggest that the productivity of the distributed data storage systems built as per the mathematical model proposed, despite the absence of high-end server hardware in the network and rather mediocre hardware parameters of its nodes, is sufficiently high and comparable with the productivity of operation of centralized data storage systems designed and built with high-end and expensive server hardware.