The solution of the problem of resource management in distributed computing systems of processing stream data in safety systems of distributed objects is considered. The tasks of streaming data processing in a multi-level multi-agent evacuation system in an infrastructure object are considered. The features of the mathematical model of a distributed stream data processing system are discussed.
An infrastructure of socio-technical systems includes various complexes of buildings that have a large
area and the possible presence of a large crowd of people. The goal of risk management with the
security system of an infrastructure is minimizing collective risk in the event of a critical situation in
the cyber-physical system or in the context of infrastructure degradation. For the analysis of the state
of these objects, various systems for collecting and processing information are used: sensors for the
state of physical parameters, video surveillance systems and telecommunication systems. It should be
noted that the part of the information is transmitted and processed as streaming data. In the event of a
critical situation information about the state of the information field can be lost due to factors affecting
the elements of the computer system. So we can lose the accuracy of measurements of the physical
parameters of the internal space of the infrastructure [
So, let the observable information field be the state of the internal physical and information space
of the considered infrastructure. In this case, the task of reallocating computing system resources in a
technical safety system, in which information about the state of an infrastructure object is collected
from various sensors in real time, becomes extremely important. This system should be considered in
the class of self-learning control systems for distributed objects that have a variable structure [
The state of the information field may change depending on internal and external factors affecting the physical energy flow, which, in turn, affects the distribution of risk levels in the event that this energy goes out of control. In this case, the solution to the problem of risk management can be based on the application of self-learning elements of a multi-level management system at its planning and coordination levels. The formation of scenarios for the development of a critical situation and possible ways to minimize risks is possible with the use of a self-learning multi-agent data collection system.
The considered multi-agent system can be implemented in various ways: we can use a multi-agent
software platform, such as JADE, or implement software agents based on different programming
languages: Python, Java, etc. [
Although the databases can be implemented on the basis of various modern approaches, it is
necessary to solve the tasks of reallocating computing power in real time in order to ensure the
efficiency of solving problems of collecting and processing, including streaming data, in the safety
systems of the distributed infrastructure facility [
As it is known, modern airports and other similar structures are complex socio-technical systems. For example, the system for ensuring the process of evacuating passengers in critical situations includes technical means of detecting and responding to the alarms, various sensors, computing, telecommunications and human resources of engineering services and building security systems. In addition, it is necessary to take into account the many uncertainties that affect the effectiveness of decision-making procedures in the evacuation process. Currently, the solution to the problem of reducing the risk of evacuation is still an important scientific and practical task.
Information support in critical situations based on a decision support system can serve as a risk management tool. In this regard, it is advisable to develop an intelligent decision support system (DSS) based on self-learning technologies and multi-agent modeling using streaming information processing.
It is quite difficult to ensure effective decision making in critical situations for large socio-technical systems under the conditions described above, using a centralized control scheme. One of the solutions is the use of a three-level decision support system, which allows managing the complexity of the decision-making problem and, as a result, reduce risk levels (figure 1). Upstream information from sensors I1 from the control level of sensors and actuators is transmitted to the level of online solutions. In the opposite direction, control actions are transferred to actuators and technical equipment that support evacuation management processes I2.
A database storing information transmitted from sensors and actuators receives a stream of data Q3 on the state of the information field and elements of the executive level. Information R3 is transmitted in the opposite direction, which is necessary for the effective functioning of sensors and instruments.
Decision making at the level of online decisions in real time is based on a set of specified logical rules R2 received in response to Q2 requests. At the same time, the database stores information about the effectiveness of decisions made on the basis of Q2 rules. The flow of data on the state of the critical situation I3 is transmitted to the multiagent modeling level, which, in turn, is looking for effective solutions for transmitting information I4 to the operational management level.
The data on the simulation results are transferred to the database of simulation results. If necessary, they can serve as input for the next stage of the simulation.
The risk of inefficient evacuation was adopted as a criterion for the effectiveness of an intelligent decision support system:
QE = 1 − PE , (1) where PE is the probability of a successful evacuation, for example, in the event of a fire.
Probability of escape PE is defined in accordance with standard methods of fire risks evaluation in accordance with the requirements of GOST 12.1.004-91 “Fire safety. General Requirements” as follows:
PE = 0,8 ⋅ tbl − test , test < 0,8 ⋅ tbl ,
0,8 ⋅ tbl where tbl is the time of blocking the building exits, it depends on fire conditions, building configuration, etc. Also, as it is given in GOST 12.1.004-91 “Fire safety. General Requirements”, tbl = 288 sec. in considered case; test is the estimated evacuation time determined for each of the best evacuation route. Estimated evacuation time test depends on information flows presented on figure 1 and, therefore, on the adopted solutions in an emergency evacuation. It also can be obtained on fire and evacuation integrated modeling results:
test = f (I1, I2 , I3 , I4 , R1, R2 , R3 , Q1, Q2 , Q3 ).
It should be noted that figure 1 shows only a generalized information structure of the hierarchical decision support system. In real techno-social systems, every single information flow is more powerful, all of them are transmitted in real time, and each of them by itself has a very complex information structure. With this in mind, the practical implementation of intelligent DSS requires high performance computing technologies and smart integration of hardware and software.
A set of agents of the coordination level use the information received by the system planner to solve the task of minimizing risks through interaction mechanisms in multi-agent systems. At the level of coordination, among other things, the problem of ensuring the integration of information on the state of the control object using multi-agent sensor systems must be solved, which makes it possible to minimize errors of the first kind and of the second kind due to the effect of self-organization of different sensors. At this level, the problem of developing control actions on the ventilation system, fire protection system, information support for evacuation must be also solved.
The architecture of multiagent system of cyber physical system safety is presented in figure 2. The structure of this system is based on hierarchical architecture.
In the sense of Big Data concept here we have different flows of information and different formats of data. We need to gather information from different types of sensors and store these data in Databases: DB_Video – Data Base of Video Sensors Data, DB_Sensor1 – Data Base of Infrastructure Sensors, DB_Sensor2 – Data Base of Dynamic Objects, DB_BD – Data Base of Mix Content, DB_M – Data Base of Simulation Results Data.
Multigent system of safety of cyber-physical system includes: Agent_Image_An – agent of Video Stream Flow Analyzer, Agent_Dy_Objects – agent of Dynamic Objects trajectory Identification, Agent_Inf_An – agent of Infrastructure State Identification, Agent_Integ_Date – agent of Integration of different flows from sensors, Agent_Mod – Agent of Simulation of the different Scenarios of Critical Situations, Agent_Integ_Dm – Agent of Analysis the state of Cyber-Physical System, Agent_R – agent of risk field identification, Agent_F – agent of integrated risk identification, Agent_U – agent of control and management of minimizing risk, Agent_U_I – agent of control signal generation for actuators, Agent_U_S – agent of management decisions realization, C&MC – implementation of control and management system. (2) (3) Agent_Image_An
DB Video Agent_Inf_An
DB2 Sensors Agent_U_I
C&MS e tr u c u tr s a lfIr n a c iitr C
Agent_Integ_Data
Agent_Dy_Objects
DB1 Sensors
So, here we have the massive application of different streaming data flows and we need to solve the resource allocation problem in case of degradation of the information system.
The model of the processing and storing of streaming data (PSSD) based on the computer network includes resource model of the computer network N, model of data processing system (DPS) and the relation D describing system deployment in the network.
IS = N , DPS, D . (4) Here: • N– the computational network, • DPS – the PSSD model, • D – the deployment relation that shows ∀p ∈ P on what node it physically deployed.
D = {(w, p) : w executes p; w ∈W , p ∈ P} .
Let B :W × R → R + – function that shows ∀w ∈W amount of the involved resource. Values of the function can be calculated from resource capacities of data processing model and network model as follows.
Bcpu (w) = Bmem (w) =
V ( p) ∑ ∀p∈D(w) T ( p)
∑ M (a) . ∀a∈(DC)(w)
.
Bninet (w) = Bnoeutt (w) =
∑ ∀p∈D(w)
∑ ∀p∈D(w)
∑ M (a) ∀a∈I ( p)∩ A\(DC )(w) .
T ( p)
∑ M (a) ∀a∈I ( p)∩(DC)(w)
T ( p)
The introduced notation makes it possible to formulate the problem of optimal allocation of computational resources in PSSD as the problem of combinatorial optimization. It is necessary to find such deployment relation D*, that gives the maximum to predefined efficiency function F. (5) (6) (7) (8) (9) (10) (11) (12) The following constraints apply
rem(w) = Let rem(w) – the unitless function, that shows average free resources for every node as 1 Sr (w) − Br (w) D* = arg
max D⊂W ×P
F (IS D ) .
Br (w) ≤ S r (w) .
∑ R ∀r∈R and the average total of free resources as • στ2 [ξ (τ )] denotes the dispersion of the stochastic process on the interval τ ∈ (0, t) , • δ (D) denotes the domain of relation D – the set of nodes where system is deployed.
To create flexible, robust and scalable software that implements a multi-level security system, an agent-based approach is proposed, which involves using a software agent as a basic software object. This allows us to further talk about the software architecture based on agents and implies the presence of some additional elements in the system, such as: • agent platform is a set of software components that support the life cycle of target agents, the interaction between them in a single namespace and the model of system events; • agent container is a software environment that provides access to the resources of each compute node and runs software agents.
In accordance with the proposed mathematical models, security system architecture has been developed. It is assumed that the software consists of the following components of the structure: • set of target agents, each of which is a software object that implements a procedure for processing streaming data; • set of processing attributes implemented as an in-memory database that provides distributed storage of streaming data and access to them with low latency.
A unified storage space for streaming data is provided by a distributed cached in-memory storage that provides fast and transparent access to attribute values regardless of their physical location. The agent platform is a unified distributed execution space for data processing procedures.
The target agent is the basic software entity in the implementation of the data processing model within the proposed architecture (figure 3). It includes the following components: • data processing procedure; • local cache of used attributes; • local cache of calculated attributes; • mechanism for the management messages receiving and executing; • mechanism for the agent initialization and self-destruction; • agent's state.
The activity diagram of the target agent shows the process of the agent's functioning after creation. In the process of distributed data processing, the target agent can be in the following states: • “Created” is the initial state in the agent's lifecycle. • “Initialized” is the calculated attributes are set to the initial values.
• “Running” is the agent's operating state when the attributes are recalculating in cyclic mode. • “Holding” is the agent's operating state when recalculating of attributes is temporary suspended. • “Completion” is the final state in the agent's lifecycle (preparation for self-destruction, completion of all recording processes).
A change of the target agent’s state occurs either according to the stage of the lifecycle, or under the influence of management commands.
The system is implemented as a simulation model and has a graphical user environment and allows you to use the Java language to develop models. Figure 4 shows the main interface of the program with controls that provide access to each node of the required number of CPUs and Memory for the system to work; flag "Enable/disable crashes"; a slider that sets a threshold at which tasks will be redirected to a more free node; input window required resources to perform tasks; a slider designed to select a node; a flag that allows you to switch between node selection modes; graphics processor and memory usage at each node.
For the convenience of the user, the ability to create and assign tasks has been added. The user must specify the required amount of resource that the task will request from the node on which the tasks will be performed, then you must click the “create” button. The dependence of the expenditure of CPU resource (%) on time for a computer system of two nodes (sec) is presented on figure 5.
The simulation was carried out on the capacity of the computer network, which consisted of ten nodes. The number of tasks in the system is 25 (the number of required CPUs and Memory resources is from 5-30% of the maximum). If on the node the amount of free resource is less than the threshold value (in this case, less than 20%), then the random task will be reassigned to the node with the most resources. Also, periodically, a random failure occurs, which leads to a decrease in free resources by half. After a certain time (in this case, 20 minutes), on a random node (in this example, node number 4), the amount of free resource decreased by half. Task number 13 was reassigned from the first node to the second.
The task of the risk management in cyber-physical systems on the basis of multilevel management system is considered. The principals of designing and architecture of multilevel self-learning multiagent system for risk management are discussed. Self-learning ability of the proposed multiagent system is based on application of Big Data algorithms an machine learning algorithms.
As a result of testing software errors were not detected. To assess the quality of the developed software, failures of computing nodes were simulated. One of the handler nodes was forcibly reduced the amount of computing resources by half. At the same time, the system retained its performance; failure did not affect the functioning of other nodes.