The paper presents the information technology for performance assessment of complex multilevel systems in managing technogenic objects, which complements the theory and methods for solving tasks of ensuring system reliability and survivability, based on the interaction of a set of operability and human factor indicators in control and decision-making at each of its hierarchical levels. Risk conditions of the influence on system performance are determined and the classification and evaluation of the impact degree of the factors on decision-makers in the system's three-level hierarchical structure under fuzzy risk are carried out. A review, systematization, and generalization of publications on the issues of analysis and assessment of the systems for managing technogenic objects were carried out, based on which it was established that, in addition to the parameters of the system, and the influence of the external and industrial environment on the human factor, the efficiency indicator also depends on its hierarchical structure, the nature of the connections between the components, and on the regularities of the system's functioning, being poorly amenable to formalized description and evaluation. To assess the performance of the system, a Bayesian network was built, through which, based on the knowledge of experts, the probability of its performance at each hierarchical level was assessed, taking into account the influence of external and internal workplace factors on the cognitive component of a decision-maker under the fuzzy risk of taking irrelevant decisions. For practical substantiation of the obtained results, an experiment was conducted, the results of which confirmed the practical value of the information technology, which can be used to assess the performance of complex multilevel systems in managing technogenic objects.
Currently, in the creation and operation of complex multilevel systems (CMS) for managing complex technogenic objects (CTO), the main task is to increase efficiency, which is associated with increasing their technical and software complexity. As a result, the requirements for both the sustainability of the components that make up the system, and the reliability and performance of the decision-maker are increasing. To solve this problem, it is necessary to be able to assess the levels of reliability of all components and their contribution to the level of performance of the entire system.
Modern CMSs for managing CTO are developed, as a rule, based on computer
networks, which in combination with software and users are designed to increase
efficiency, especially that of human management. Currently, the issue of assessing the
performance of CMS is given insufficient attention; there is no single conceptual
approach to the study of such systems. The reliability of the operator and the
performance of software packages have also been insufficiently studied. Elements of a
modern CMS can adapt to the operating conditions, i.e. they are adaptive. For such a
system, it is difficult to formulate the concept of failure [
The issues of the performance assessment related to the emergence of risk situations in the implementation of management decisions at each hierarchical level of the system due to the imperfection of the used mathematical, statistical, and intellectual tools are understudied.
In this regard, solving the above problems is relevant. 2
The solution to this problem is reflected in the results of the following scientific studies.
The papers [
In [
In [
In [
In [
In this regard, for the development of the theory of evaluation of the efficiency of distributed man-machine systems and based on the above analysis of the literature, the information technology to assess the performance of complex multilevel systems in managing technogenic objects is proposed in this paper. 3
Review, systematization, and generalization of the publications on analysis, assessment, and management in CMS show that in addition to the parameters of the system, the impact of the external and industrial environment on the human factor, the performance indicator also depends on the system architecture (its hierarchical structure, the nature of the connections between components), as well as on the regularities of the system functioning, poorly amenable to formalized description and assessment.
The functioning of CMS is characterized mainly by the following hierarchical levels: ─ the level of local control and automation of technological sections of a complex organizational and technical object (COTO), based on digital control, the tasks of controlling individual units, and technology departments are performed. The purpose of the system functioning at this level is to conduct the technological process to the upper control levels in real-time; ─ the level of dispatch control, automatic collection of real-time data, calculation of complex indicators, as well as the upbuilding of the object history; ─ analytics level, based on an analytical server, performing tasks on analyzing production data and the efficiency of technological processes. The efficient performance of CMS at each level is characterized by the following components: the level of the technological process status - TP; software status level PZ; hardware status level - TZ; the level of the state of functional sustainability of decision-maker, which characterizes risk conditions of decision-making – FSOPR; the level of the state of functional sustainability of CMS - FSCMS; the level of monitoring and adaptation of the system - MAC; the level of the operational formation state of adapted alternatives to decision-making support – FA PPR (Fig. 1).
If the functioning of CMS occurs under uncertainty and fuzzy risk, the effectiveness of management is ensured by the quality of the performance of decision-makers performance, its psychological state, the cognitive component, and working conditions, which can be characterized as functional sustainability (FS) of decision-maker, which impacts the degree of risk in critical project management.
As shown in [
In [
The group of influencing factors related to the user's current psychological and cognitive state includes a) the level and frequency of information load I0; b) the degree of user's fatigue F; d) the efficiency of real-time decision-making Tp; e) the degree of psychological tension TS; f) concentration A.
The first group of factors determines the level of the external and production environment condition - Sc, the second group of factors determines the level of the psychological and cognitive state of decision-maker - Sp. Formally, this means that ∆P Sc = f1 I N , IV , E,T , f ,
∆t S p = f 2 (I 0 , F ,Tp ,TS , A)
FSOPRvr = G(S p v , Scv The magnitude of risk (risk) RF of making irrelevant decisions depends on the functional sustainability of the decision-maker, which in turn depends on the relevant factors following (3). Then, given (1-3), it can be written that
RF = ϕ (G(S pv , Scv )) =ψ (S pv , Scv ) =
∆P =ψ I N , IV , E,T , f , , I0 , F ,Tp ,TS , A ∆t (1) (2) (3) (4) Even though in the literature there are significantly different interpretations of the concept of "risk", they have in common that risk includes uncertainty whether an undesirable event will occur that can lead to adverse consequences [Mushik, Muller].
When a decision is made by the DM in conditions of uncertainty, under the
influence of such factors as ambiguity, inaccuracy, and fuzziness [
Therefore, in this paper, the fuzzy risk RF (or risk level) is understood as the probability of making an irrelevant decision by the DM under the influence of the abovementioned factors, i.e. RF=Pr.
Therefore, the main objectives of the work performed are to determine the functional dependence of the efficiency of the 3-level CMS performance on production factors and the fuzzy risk of making irrelevant decisions by the DM, as well as the dependence of fuzzy risk on external and cognitive factors.
Herewith, it is not possible to obtain analytical formulas for the above functional dependencies.
Most managerial decisions at each level of CMS are made under fuzzy risk, which
is determined by the lack of complete information, the presence of opposing trends,
elements of randomness, when possible outcomes can be described using a certain
probability distribution, for the construction of which it is necessary to have statistical
data or expert evaluations [
The solution of the task under uncertainty, when the initial information is incomplete, inaccurate, non-quantitative, and a type of the formal display is either too complicated or not known, then in such cases, the expert knowledge is involved. For its representation and processing, various methods of applied decision-making theory and artificial intelligence methods are used.
This paper considers the solution to the above problems using the mathematical apparatus of the Bayesian network (BN).
BNs are graphical models of events and processes based on combining some
inferences of probability theory and graph theory [
The total probability of BN is calculated by the following formula:
N PB ( X (1) ,..., X (N ) ) = ∏ PB ( X (i) PA( X (i) ))
i=1 where: A − is a set of environment states; N = (x1,..., xN ) − vector of parameters of their distributions.
The conditional probability of an event is determined by the following ratio: p(E H k ) = p(E ∩ H k )
p(H k ) n Ei = Ω i=1 where: E and H k − are interrelated variables. This dependence allows determining what the probability of the event E will be if a particular event Hk occurs.
Mutually exclusive events form an exhaustive set if
p(E ∩ H k ) = p(E | H k )⋅ p(H k ) = p(H k | E )⋅ p(E ) whence it follows that: (5) (6) (7) (8) (9) Two variables do not intersect if they do not have equal values. Bayesian network theory is based on the assumption that events are exhaustive and do not intersect. In this case, the probability of the event E can be calculated using conditional probabilities:
n n p(E ) = ∑ p(E ∩ Hi ) = ∑ p(E | Hi )⋅ p(Hi )
i=1 i=1 Using the formula (6), the probability of the intersection of the events E and H can be expressed as follows: Considering the formula (7) and (10), it can be represented as follows: p(E | H k )⋅ p(H k ) p(H k | E ) =
p(E ) p(E | H k )⋅ p(H k ) p(H k | E ) = n ∑ p(E | Hi )⋅ p(Hi ) i=1 (11) BN (Fig. 3) was used to assess the CMS efficiency, where for considering the influence on its performance efficiency (node "Efficiency") of such factors as the technological process, software, hardware, functional sustainability of CMS, the support of the system monitoring and adaptation, the operative formation of the adapted alternatives to decision-making support the nodes TP, PZ, TZ, FC, MAC, FA PPR respectively are introduced. They are nodes of the "Decision" type and can take three values: "normal", "workable" and "critical" depending on the states that the above factors take.
Since the CMS efficiency is significantly influenced by the risk of adopting irrelevant decisions by the DM, the probability Pr is taken to assess the degree of influence of the fuzzy risk. Thus, the risk is set on the BN by a node Chance - NoisyMax type, which is called “Risk”. Since the DM risk of making a decision depends on the state of the environment and the cognitive state of the decision-maker, the Risk node has two parent nodes “External Environment” and “Cognitive State”.
The “External Environment” is the node of the Chance type and characterizes the
degree of influence of the environment on the decision-maker. This influence is
determined to a varying degree at each hierarchical level of the system by the factors
following Fig. 3. The standard values of these factors are given in Table 1.
To answer this question and to estimate the probability Pn , a system is created for
predicting the values of probabilities based on fuzzy inference according to the
Mamdani algorithm, in which the values of the input variables IN; IV; E; T; f; ∆P ∆t
and the output variable Pn are given by fuzzy sets. For this inference, a fuzzy
knowledge base was proposed [
As a result of expert evaluation, a correspondence is established, presented in Table 2, between the interval of values of cognitive factors and the value (˝low˝, ˝middle˝ and high˝) of the corresponding BN node.
The assignment of the Chance - Noisy Max type to the nodes "Cognitive State", "Risk", "Efficiency" is conditioned by the following considerations: firstly, these nodes have a large number of parent connections, which greatly complicates the filling of the table of conditional probabilities; secondly, it is much easier for experts to estimate the value of the unconditional probability than the conditional one.
To describe the noisy nodes "Cognitive State" and "Risk", and to describe the node "Efficiency", the experts-specialists and experts-administrators correspondingly were requested to evaluate the conditional probabilities of possible states of the indicated nodes. The results of the expert evaluation are presented in tables 3-5. 4
Further, it is assumed that only one factor negatively affects the efficiency of CMS performance to some extent: the technological process - TP, i.e. TP node takes the value "workable". In this case, the nodes PZ, TZ, FC, MAC, FA PPR take the value "normal". The result of calculating the probability Pef of efficient performance of CMS according to the above-considered BN without taking into account the risk ( Pr = 0 ) of making an irrelevant decision by the decision-maker is shown in Fig. 4.
The figure shows that the efficiency of the CMS performance, when the risk Pr = 0 , is 90% ( Pef = 0,9 ).
To demonstrate how the risk of irrelevant decision making by the DM depends on his cognitive state and environmental factors and how it affects the efficiency of the CMS performance.
The lower level of the system is considered below.
At this level, cognitive factors practically do not have a noticeable effect on decision-makers. Therefore, it is assumed that the nodes F; Tp; TS; A take on the values "low", and the node Io takes on the value "high". At the same time, environmental factors with the probability Pn = 0,3 negatively affect decision-makers.
At this case, the calculation according to the above-considered BN shows that the risk at the lower level is minor ( Pr = 0,09 ), which leads to a decrease in the efficiency of the CMS performance by only 5% ( Pef = 0,85 ) in comparison with the case when the risk is not taken into account.
At the middle level, cognitive factors have a minor effect on decision-makers. Therefore, it is assumed that F, TS, and A nodes take the value "middle", and the nodes Tp and I0 take the values "low" and "high", respectively. At the same time, environmental factors at the middle level negatively affect decision-makers to a lesser extent than at the lower level. It is assumed that Pn = 0,2 . In this case, the calculation according to the above-considered BN shows that the risk at the middle level significantly increases compared to the lower level and amounts to Pr = 0,28 , which leads to a decrease in the efficiency of the CMS performance by 15% ( Pef = 0,75 ) as compared to the case when the risk is not taken into account.
At the upper level, cognitive factors have a significant impact on decision-makers. Therefore, it is assumed that the nodes F; TS; A take on the value "high", and the nodes Tp and I0 take on the values "low" and "high", respectively. At the same time, environmental factors at the upper level negatively affect decision-makers to an even lesser extent than at the middle level. It is assumed that Pn = 0,1 .
The result of calculating the risk Pr and the probability Pef of the CMS efficient performance at the upper level shows that the risk at the upper level is much greater than the risk at the lower and middle levels Pr = 0,57 , which leads to a significant decrease in the efficiency of the CMS performance to the value Pef = 0,59 .
In this work, the information technology for performance assessment of complex multilevel systems under fuzzy risk is developed, which complements the theory and methods of solving issues of ensuring system reliability and survivability, based on the interaction of a set of operability and human factor indicators in control and decision-making at each of its hierarchical levels.
Risk conditions of influence on system performance are determined and the classification and evaluation of the impact degree of the factors on the DM in the system's three-level hierarchical structure under fuzzy risk are carried out. Fuzzy risk is proposed to mean the probability of making an irrelevant decision by the decision-maker.
To determine a quantitative assessment of the dependence of the 3-level CMS efficiency on production factors and the fuzzy risk of making irrelevant decisions by the DM, as well as the dependence of fuzzy risk on external and cognitive factors, taking into account the knowledge of experts, a BN was developed.
For practical substantiation of the obtained results, an experiment was carried out, where, taking into account the characteri6stic values of external and cognitive factors at each of the 3 system's levels, the fuzzy risk of making an irrelevant decision by DM and the CMS efficiency was calculated. It is shown that the magnitude of the risk grows from the lower to the upper level. Herewith, the efficiency of the CMS is correspondingly reduced from an acceptable value at the lower level to a critical value at the upper level. The calculation results are in good agreement with the experimental results of the CMS operation.
The results of the experiment confirmed the practical value of information technology, which can be used to assess the performance of complex multilevel systems under the fuzzy risk of decision-making in managing technogenic objects.