<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Risks Prediction for Artificial Intelligence Systems Using Monitoring Data</article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Andrey Kostogryzov</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Federal Research Center “Computer Science and Control” of the Russian Academy of Sciences Moscow</institution>
          ,
          <country country="RU">Russia</country>
        </aff>
      </contrib-group>
      <fpage>29</fpage>
      <lpage>33</lpage>
      <abstract>
        <p>-The approach for cognitive processing of monitored data is proposed. An application for artificial intelligence systems (AIS) allows to use the possibilities of probabilistic modeling. There are described the models and methods for prediction a probability of “success” and/or a risk of “failure”, software tools to support them, techniques for solving the problems of rationale preventive measures against threats and effective risk control. The approach means practically a proactive commitment to excellence in uncertainty conditions. A suitability of the proposed models and methods is demonstrated by some practical examples.</p>
      </abstract>
      <kwd-group>
        <kwd>analysis</kwd>
        <kwd>control</kwd>
        <kwd>model</kwd>
        <kwd>probability</kwd>
        <kwd>risk</kwd>
        <kwd>safety</kwd>
        <kwd>system</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>-</title>
      <p>INTRODUCTION</p>
      <p>
        Today global trends in the development of modern systems
for various functional purposes indicate the need for a radical
turn from "manual" control of certain types of safety, based on
the implementation of established instructions and expert
estimations of emerging situations, to the implementation of
proactive measures based on prediction. This allows, on the
basis of a prognostic look ahead, to proactively take effective
managing actions. This idea runs like a red line through all
world concepts and the latest standards of systems engineering.
But how to do it remains behind the scenes. There is no
universal approach to the implementation of this idea yet. In
search – all the leading countries of the world. Special hopes
are connected with the use of AIS. Here AIS are understood as
systems, operating in uncertainty conditions by logic reasoning
on the base of processing the monitored data. In AIS practice
there are often used subjective expert estimations (for AIS
training), a regression analysis of collected data, a simulation
of processes [
        <xref ref-type="bibr" rid="ref1 ref2 ref3 ref4 ref5 ref6 ref7 ref8 ref9">1-9</xref>
        ]. It means, that search of new methods for
rationale AIS operation is very important.
      </p>
      <p>Note. System is combination of interacting elements organized to achieve
one or more stated purposes (according to ISO/IEC/IEEE 15288).</p>
      <p>This paper focuses on applications that are critical from the
point of view of safety depending on the structural complexity
of systems, formal conditions of uncertainty, implemented
methods of countering threats, as well as the conditions of
elements operation. Available probabilistic methods in
cognitive processing of monitored data are proposed. As a
result the predicted probability of "success" or risk of "failure"
is produced. The inputs for modeling are monitored data from
current and previous states of compound elements.</p>
      <p>
        The proposed ideas, models and methods are designed to
implement feedback to rationale requirements and conditions
that guarantee the non-exceeding of the specified acceptable
risks. The proposed probabilistic approach develops the
established probabilistic approaches [
        <xref ref-type="bibr" rid="ref10 ref11 ref12 ref13 ref14 ref15 ref16 ref17 ref18 ref19 ref20 ref21 ref22 ref23 ref24 ref25 ref26 ref27 ref28 ref29 ref30 ref31 ref32 ref33 ref34 ref35 ref36">10-36</xref>
        ], applicable where
there is some similar repeatability of events.
      </p>
      <p>II.</p>
    </sec>
    <sec id="sec-2">
      <title>THE ESSENCE OF THE PROPOSED APPROACH</title>
      <p>The monitored system may itself be a system of interest for
analysis (for example, a dispatching intelligence center) or may
be part of another, more comprehensive system of interest (for
example, a complex of functionally oriented robots). To
perform the functions of the system, current information is
collected and processed. It is proposed to carry out
probabilistic prediction of critical processes in order not only to
act, but also to compare predictions and their coincidence with
subsequent realities, to accumulate and use this knowledge.
The cognitive decisions for AIS using monitoring data is in the
accumulation, analysis and the use of emerging knowledge
about the possible integrity of the system in the future.</p>
      <p>When the system is operating in the conditions of
heterogeneous threats, the degree of acceptability of events is
proposed to be assessed by the probability of "success" and/or
"failure" taking into account the consequences (risk of
"failure") during a given period of prediction. In each case of
modeling, the concept of "success" must be defined in terms of
the acceptable state of the system concerned to perform the
given or expected functions. The concept of "failure" means no
"success".</p>
      <p>
        It is proposed to carry out analytical prediction of risks on
the basis of probabilistic modeling. For practical application,
methods and models [
        <xref ref-type="bibr" rid="ref10 ref11 ref12 ref13 ref14 ref15 ref16 ref17 ref18 ref19 ref20 ref21 ref22 ref23 ref24 ref25 ref26 ref27 ref28 ref29 ref30 ref31 ref32 ref33 ref34 ref35 ref36">10-36</xref>
        ] are recommended (not an
exhaustive list of adequate ones), where subjective weight
coefficients are excluded. The proposed models are based on a
classically constructed probabilistic space (Ω, B, P) [
        <xref ref-type="bibr" rid="ref37 ref38 ref39 ref40">37-40</xref>
        ],
where: Ω - is a limited space of elementary events; B – a class
of all subspace of Ω-space, satisfied to the properties of
σalgebra; P – is a probability measure on a space of elementary
events Ω. Because, Ω={ωk} is limited, there is enough to
establish a reflection ωk→pk =P(ωk) like that pk≥0 and
∑ pk = 1.
k
      </p>
      <p>A complex system is decomposed to compound elements to
solve problems with respect to each of the elements and
subsystems with the possibility of integrating them into the
system as a whole. Each of the elements is represented as a
"black box", and various probabilistic models can be applied to
it to calculate and construct the desired probability distribution
function (PDF) for time between neighboring integrity losses,
taking into account heterogeneous threats, the measures taken
to control, monitor and integrity recovery. Below are some
generalized models "black box" for the risk prediction.</p>
      <p>III.</p>
      <p>THE MODELS PROPOSED</p>
      <p>In general case successful system operation (not only AIS)
is connected with system counteraction against various
dangerous influences on system integrity - these may be
counteractions against failures, defects events, “human
factors” events, etc. There are proposed the formalization for
two general technologies of providing counteraction against
threats: periodical diagnostics of system integrity (technology
1, without monitoring between diagnostics) and additionally
monitoring between diagnostics (technology 2). As a rule
these technologies are implemented by AIS.</p>
      <p>Assumptions: for all time characteristic the PDF exists. It
is supposed for technologies 1 and 2 that the used diagnostic
tools allow to provide necessary system integrity recovery
after revealing danger sources penetration into a system or
consequences of influences.</p>
      <p>The probability of system operation with required safety
within the given prognostic period (i.e. probability of
“success”) may be estimated as a result of modeling. Risk to
lose integrity (R) is an addition to 1 for probability of correct
system operation (P), i.e. R=1-P considering consequences.
A. Model for the periodical diagnostics of system integrity</p>
      <p>The model considers periodical diagnostics of system
integrity, that is carried out to detect danger sources penetration
into a system or consequences of negative influences (see
Figure 1).</p>
      <p>Dangerous influence on system is acted step-by step: at first
a danger source penetrates into a system and then after its
activation begins to influence. System integrity can’t be lost
before a penetrated danger source is activated. A danger is
considered to be realized only after a danger source has
influenced on a system. The lost system integrity (after an
accident event) can be detected only as a result of diagnostics.
System recovery is started after detection.</p>
      <p>There are possible the next variants for technologies 1 and
2: variant 1 – the given prognostic period Treq is less than
established period between neighboring diagnostics
(Treq &lt; Tbetw.+Tdiag); variant 2 – the prognostic period Treq is
more than or equals to established period between neighboring
diagnostics (Treq ≥ Tbetw.+Tdiag). Here Tbetw. – is the time
between the end of diagnostic and the beginning of the next
diagnostic, Tdiag – is the diagnostic time.</p>
      <p>
        The next formulas for PDF of time between the losses of
system integrity are proposed [
        <xref ref-type="bibr" rid="ref11 ref13 ref15 ref29 ref30">11, 13, 15, 29, 30</xref>
        ].
      </p>
      <p>PDF for the model of technology 1, variant 1: Under the
condition of independence for characteristics the probability of
providing system integrity for variant 1 is equal to</p>
      <p>P(1)(Treq) = 1 - Ωpenetr ∗ Ωactiv(Treq),
(1)
where Ωpenetr(t) – is the PDF of time between neighboring
penetrations of dangers; Ωactiv(t) – is the PDF of activation time
of penetrated danger. For different dangers a frequency of
dangers for these PDF is the sum of frequencies of every kind
of dangers.</p>
      <p>PDF for the model of technology 1 , variant 2. Under the
condition of independence for characteristics the probability of
providing system integrity for variant 2 is equal to</p>
      <p>P(2) (Treq) = N((Tbetw +Tdiag)/Treq) P(1)N(Tbetw +Tdiag) +
+(Trmn/Treq) P(1)(Trmn), (2)
where N=[ Тreq./(Тbetw.+ Тdiag.)] – may be real (for PDF) or
the integer part (for estimation of deviations), Trmn = Treq
N(Tbetw +Tdiag)). The probability of providing system integrity
within the given time P(1)(Tgiven) is defined by (1).</p>
      <p>B. Model for continuous monitoring between the periodical
diagnostics of system integrity</p>
      <p>Technology 2, unlike the previous one, implies that system
integrity is continuously monitored between diagnostics by
operator (operator functions may be performed by a man or
special AIS component or their combination). In case of
detecting a danger source an operator recovers system integrity.
The ways of integrity recovering are analogous to the ways of
technology 1.</p>
      <p>Faultless operator’s actions provide a neutralization of a
danger source trying to penetrate into a system. A penetration
of a danger source is possible only if an operator makes an
error but a dangerous influence occurs if the danger is activated
before the next diagnostic. Otherwise the source will be
detected and neutralized during the next diagnostic.</p>
      <p>
        The next formulas for PDF of time between the losses of
system integrity are proposed [
        <xref ref-type="bibr" rid="ref11 ref13 ref15 ref29 ref30">11, 13, 15, 29, 30</xref>
        ].
      </p>
      <p>PDF for the model of technology 2, variant 1. Under the
condition of independence for characteristics the probability of
providing system integrity is equal to
.</p>
      <p>Here A(τ) is the PDF of time between operator’s error.
(3)</p>
      <p>PDF for the model of technology 2, variant 2. Under the
condition of independence of characteristics the probability of
providing system integrity is equal to</p>
      <p>P(2) (Treq) = N((Tbetw +Tdiag)/Treq) P(1)N(Tbetw +Tdiag) +
+(Trmn/Treq) P(1)(Trmn), (4)
where the probability of providing system integrity
within the given time P(1)(Treq.) is defined by (3).</p>
      <p>The final clear analytical formulas for modeling are
received by Lebesque-integration of (3) expression.
C. About a generation of probabilistic models for complex
system</p>
      <p>
        The basic ideas of correct integration of probability
metrics are based on a combination and development of
models. For a complex systems with parallel or serial
structure described there are proposed the method to generate
adequate probabilistic models described in [
        <xref ref-type="bibr" rid="ref11 ref13 ref15 ref29 ref30">11, 13, 15, 29,
30</xref>
        ]. Considering the importance to rationale the generation of
new probabilistic models for complex system, the approach is
described below. Let's consider the elementary structure from
two independent parallel or series elements. Let’s PDF of time
between losses of i-th element integrity is Вi(t) =Р (τi≤ t),
then:
      </p>
      <p>1) time between losses of integrity for system combined
from series connected independent elements is equal to a
minimum from two times τi: failure of 1st or 2nd elements (i.e.
the system goes into a state of lost integrity when either 1st, or
2nd element integrity is lost). For this case the PDF of time
between losses of system integrity is defined by expression
В(t) = Р[min (τ1,τ2)≤t]=1- Р[min (τ1,τ2)&gt;t]=1-Р(τ1&gt;t)Р(τ2 &gt;
t)= 1 – [1-В1(t)] [1- В2(t)], (4)
2) time between losses of integrity for system combined
from parallel connected independent elements (hot
reservation) is equal to a maximum from two times τi: failure
of 1st and 2nd elements (i.e. the system goes into a state of
lost integrity when both 1st and 2nd elements have lost
integrity). For this case the PDF of time between losses of
system integrity is defined by expression</p>
      <p>В(t)=Р[max(τ1,τ2)≤t]=Р(τ1≤t)Р(τ2≤t)=В1(t)В2(t). (5)
Applying recurrently expressions (4) – (5), it is possible to
build PDF of time between losses of integrity for any
complex system with parallel and/or series structure and theirs
combinations.</p>
      <p>
        Analytical modeling of complex systems is supported by
the software tools “Mathematical modeling of system life
cycle processes” – “know how” (registered by Rospatent
№2004610858), “Complex for evaluating quality of
production processes” (registered by Rospatent
№2010614145) and others [
        <xref ref-type="bibr" rid="ref31 ref32 ref33 ref34 ref35 ref36">31-36</xref>
        ].
      </p>
      <p>IV MODELING TO THE RATIONALE OF PREVENTIVE MEASURES.</p>
      <p>EXAMPLES</p>
      <p>The proposed practical way to forming input for modeling
is explained in application to a parameter conditions.</p>
      <p>Example 1. For each critical parameter (for which
prognostic estimations are needed to do actions) the ranges of
acceptable conditions can be established. The traced
conditions of monitored parameters are data about a condition
before and on the current moment of time. For example, the
ranges of possible values of conditions may be established:
“Working range inside of norm”, “Out of working range, but
inside of norm”, “Abnormality” for each separate critical
parameter. If the parameter ranges of acceptable conditions
are not established in explicit form than for modeling purpose
the may be implied and can be expressed in the form of
average time value. These time values are used as input for
probabilistic modeling. For example, for coal mine some of
many dozens heterogeneous parameters are: for ventilation
equipment - temperature of rotor and engine bearings, a
current on phases and voltage of stator; for modular
decontamination equipment - vacuum in the pipeline, the
expense and temperature of a metano-air mix in the pipeline
before equipment, pressure in system of compressed air, etc. It
may be interpreted similarly by light signals – "green",
"yellow", "red" - see Fig.2 and following Example 2.</p>
      <p>Example 2. For avoiding the possible crossing a border of
“Abnormality” a prediction of residual time, which is available
for preventive measures, according to gathered data about
parameter condition fluctuations considering ranges should be
carried out. For prediction it is proposed: 1) a choice of
probabilistic models for construction PDF of time before the
next abnormality for one element (“black box”), 2)
development of the algorithm of generation PDF of time before
the next abnormality for complex system, 3) formalization of
calculative methods of estimating the mean residual time
before the next parameters abnormalities for monitored critical
system.</p>
      <p>The method allows to estimate residual time before the
next parameter abnormality state (i.e. time before first next
coming into “red” range) Tresid(1) for a given admissible risk
Radm.(Treq) to lose integrity. The estimated Tresid(1) is the
solution t0 of equation:</p>
      <p>R(Tpenetr, t, Tbetw, Tdiag, Тerr., Treq.) = Radm.(Treq) (6)
concerning of unknown parameter t, i.e. Tresid(1) = t0.</p>
      <p>Here R(Tpenetr, t, Tbetw, Tdiag, Тerr., Treq.) is risk to lose
integrity, it is addition to 1 for probability P(Treq) of providing
system integrity (“probability of success”), for calculations the
formulas (1)–(3), (6) are used. Tpenetr is the mathematical
expectation of PDF Ωpenetr (τ ), it is defined by parameter
statistics of transition from “green” into “yellow” range (see
Fig.2). The others parameters Tbetw, Tdiag in (6) are known.
The main practical questions are: what about Treq. and what
about a given admissible risk Radm.(Treq)? For answering we
can use the properties of function R(Tpenetr, t, Tbetw, Tdiag, Тerr.,</p>
      <p>- if parameter t increases from 0 to ∞ for the same another
parameters, the function R(…, t, …) is monotonously
decreasing from 1 to 0, i.e. if the mean activation time of
occurred danger (threat - from the 1-st input at the “yellow”
range to the 1-st input in the “red” range) is bigger to lose
integrity is less;</p>
      <p>
        - if parameter Treq increases from 0 to ∞ for the same
another parameters, the function R(…,Treq) is monotonously
increasing from 0 to 1, i.e. for large Treq risk approaches to 1.
It means the such maximal x exists when t=x and Treq.=x and
0&lt;R(Tpenetr, x, Tbetw, Tdiag, Тerr., x)&lt;1. The residual time before
the next parameter abnormality (i.e. time before first next
coming into “red” range) is equal to defined x with confidence
level of admissible risk R(Tpenetr, x, Tbetw, Tdiag, Тerr., x). The
implementation see on Fig. 3 [
        <xref ref-type="bibr" rid="ref20 ref29">20, 29</xref>
        ].
      </p>
    </sec>
    <sec id="sec-3">
      <title>V THE POSSIBLE PRAGMATIC EFFECTS</title>
      <p>Author of this article took part in creation of the Complex
of supporting technogenic safety on the systems of oil&amp;gas
transportation and distribution and have been awarded for it by
the Award of the Government of the Russian Federation in the
field of a science and technics. The AIS is a part of the created
peripheral posts are equipped additionally by means of
Complex to feel vibration, a fire, the flooding, unauthorized
access, hurricane, and also intellectual means of the reaction,
capable to recognize, identify and predict a development of
extreme situations – see engineering decisions on Fig. 4.</p>
      <p>
        The applications of this Complex for 200 objects in several
regions of Russia during the period 2009-2014 have already
provided economy about 8,5 Billions of Roubles. The
economy is reached at the expense of effective
implementation of the functions of risks prediction and
processes optimization [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ].
      </p>
      <p>The proposed approach for cognitive processing of
monitored data develops the existing approaches to risk
prediction, ensuring and improving the safety of systems with
AIS. Probabilistic models and methods that allow predicting
the probability of "success" and/or the risk of “failure”,
supporting their software tools, methods for solving practical
problems are presented. Application of the proposed approach
allows to counteract threats by rationale preventive actions. A
suitability of the approach is illustrated by practical examples.</p>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>Alan</given-names>
            <surname>Turing</surname>
          </string-name>
          ,
          <source>Computing Machinery and Intelligence, Mind</source>
          , vol. LIX, no.
          <issue>236</issue>
          ,
          <year>October 1950</year>
          , pp.
          <fpage>433</fpage>
          -
          <lpage>460</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>Experimental</given-names>
            <surname>Robotics</surname>
          </string-name>
          . Springer,
          <year>2016</year>
          , 913 p.
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>Robotics</given-names>
            <surname>Research</surname>
          </string-name>
          . Springer,
          <year>2017</year>
          , 646 p.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          <source>[4] Cybernetics Approaches in Intelligent Systems. Computational Methods in Systems and Software</source>
          , vol.
          <volume>1</volume>
          , Springer,
          <year>2017</year>
          , 405 p.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          <source>[5] Applied Computational Intelligence and Mathematical Methods. Computational Methods in Systems and Software</source>
          , vol.
          <volume>2</volume>
          , Springer,
          <year>2017</year>
          ,
          <year>393p</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>R.</given-names>
            <surname>Valencia</surname>
          </string-name>
          ,
          <string-name>
            <given-names>J.</given-names>
            <surname>Andrade-Cetto</surname>
          </string-name>
          , Mapping, Planning and
          <string-name>
            <surname>Exploration with Pose</surname>
            <given-names>SLAM</given-names>
          </string-name>
          , Springer,
          <year>2018</year>
          , 124 p.
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>The</given-names>
            <surname>DARPA Robotics Challenge</surname>
          </string-name>
          <string-name>
            <surname>Finals</surname>
          </string-name>
          : Humanoid Robots To The Rescue, Springer,
          <year>2018</year>
          , 692 p.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          <source>[8] Cognitive Reasoning for Compliant Robot Manipulation</source>
          , Springer,
          <year>2019</year>
          ,
          <year>190p</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>S.</given-names>
            <surname>Tadokoro</surname>
          </string-name>
          , Disaster Robotics, Springer,
          <year>2019</year>
          , 532 p.
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Kostogryzov</surname>
            ,
            <given-names>A.I.</given-names>
          </string-name>
          <article-title>Software Tools Complex for Evaluation of Information Systems Operation Quality (CEISOQ)</article-title>
          .
          <source>Proceedings of the 34-th Annual Event of the Government Electronics and Information Association (GEIA)</source>
          ,
          <source>Engineering and Technical Management Symposium</source>
          , USA, Dallas,
          <year>2000</year>
          , pp.
          <fpage>63</fpage>
          -
          <lpage>70</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          :
          <article-title>Standardization, mathematical modelling, rational management and certification in the field of system and software engineering</article-title>
          .
          <source>Armament.Policy.Conversion</source>
          , Moscow,
          <year>2004</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Zio</surname>
            <given-names>En.</given-names>
          </string-name>
          :
          <article-title>An Introduction to the Basics of Reliability and Risk Analysis</article-title>
          ,
          <source>World Scientific Publishing Co.Pte.Ltd.</source>
          ,
          <year>2006</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stepanov</surname>
            <given-names>P.V.</given-names>
          </string-name>
          :
          <article-title>Innovative management of quality and risks in systems life cycle (modern standards and ideas of system engineering, mathematical models, methods, techniques and software tools complexes for system analysis, including modelling through Internet, 100 examples with an explanation of logic of the reached results, useful practical recommendations)</article-title>
          .
          <source>Armament.Policy.Conversion</source>
          , Moscow,
          <year>2008</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Kolowrocki</surname>
            <given-names>K.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Soszynska-Budny</surname>
            <given-names>J</given-names>
          </string-name>
          .:
          <source>Reliability and Safety of Complex Technical Systems and Processes</source>
          , Springer-Verlag London Ltd.,
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          and
          <article-title>Nistratov A.: Some Applicable Methods to Analyze and Optimize System Processes in Quality Management</article-title>
          .
          <source>Total Quality Management and Six Sigma, InTech</source>
          ,
          <year>2012</year>
          :
          <fpage>127</fpage>
          -
          <lpage>196</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Grigoriev</surname>
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Guseinov</surname>
            <given-names>Ch.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kershenbaum</surname>
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A</given-names>
          </string-name>
          .
          <article-title>The methodological approach, based on the risks analysis and optimization, to research variants for developing hydrocarbon deposits of Arctic regions</article-title>
          .
          <source>Journal of Polish Safety and Reliability Association. Summer Safety and Reliability Seminars</source>
          , Volume
          <volume>5</volume>
          ,
          <fpage>Number1</fpage>
          -
          <lpage>2</lpage>
          ,
          <year>2014</year>
          :
          <fpage>71</fpage>
          -
          <lpage>78</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>Akimov</surname>
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          , Mahutov N. at al.
          <source>Security of Russia. Legal</source>
          , Social&amp;Economic and
          <string-name>
            <given-names>Scientific&amp;Engineering</given-names>
            <surname>Aspects</surname>
          </string-name>
          .
          <article-title>The Scientific Foundations of Technogenic Safety. Under the editorship of Mahutov N.A</article-title>
          .
          <string-name>
            <surname>Znanie</surname>
          </string-name>
          , Moscow,
          <year>2015</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zubarev</surname>
            <given-names>I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stepanov</surname>
            <given-names>P.</given-names>
          </string-name>
          , Grigoriev L.
          <article-title>About accuracy of risks prediction and importance of increasing adequacy of used adequacy of used probabilistic models</article-title>
          .
          <source>Journal of Polish Safety and Reliability Association. Summer Safety and Reliability Seminars</source>
          , Volume
          <volume>6</volume>
          ,
          <string-name>
            <surname>Numbers</surname>
            <given-names>2</given-names>
          </string-name>
          ,
          <year>2015</year>
          :
          <fpage>71</fpage>
          -
          <lpage>80</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Eid</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Rosato</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          <article-title>Critical Infrastructure Disruption Scenarios Analyses via Simulation. Managing the Complexity of Critical Infrastructures. A Modelling and Simulation Approach</article-title>
          , SpringerOpen,
          <year>2016</year>
          :
          <fpage>43</fpage>
          -
          <lpage>62</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Artemyev</surname>
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>Rudenko</given-names>
            <surname>Ju</surname>
          </string-name>
          .,
          <string-name>
            <surname>Kurpatov</surname>
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Probabilistic methods of estimating the mean residual time before the next parameters abnormalities for monitored critical systems</article-title>
          .
          <source>In: Proceedings of the 2nd International Conference on System Reliability and Safety (ICSRS)</source>
          , Milan, Italy,
          <year>2017</year>
          :
          <fpage>368</fpage>
          -
          <lpage>373</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stepanov</surname>
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Klimov</surname>
            <given-names>S.</given-names>
          </string-name>
          , Grigoriev L.:
          <article-title>The method of rational dispatching a sequence of heterogeneous repair works</article-title>
          .
          <source>Energetica</source>
          . Vol.
          <volume>63</volume>
          , No 4,
          <year>2017</year>
          :
          <fpage>154</fpage>
          -
          <lpage>162</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stepanov</surname>
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Atakishchev</surname>
            <given-names>O.</given-names>
          </string-name>
          :
          <article-title>About Probabilistic Risks Analysis During Longtime Grain Storage</article-title>
          .
          <source>In: Proceedings of the 2nd Internationale Conference on the Social Science and Teaching Research (ACSS-SSTR)</source>
          ,
          <source>Volume 18 of Advances in Social and Behavioral Science</source>
          . Edited by Harry Zhang.
          <source>Singapore Management and Sports Science Institute</source>
          , PTE.Ltd. ,
          <year>2017</year>
          :
          <fpage>3</fpage>
          -
          <lpage>8</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stepanov</surname>
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Grigoriev</surname>
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Atakishchev</surname>
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          :
          <article-title>Improvement of Existing Risks Control Concept for Complex Systems by the Automatic Combination and Generation of Probabilistic Models and Forming the Storehouse of Risks Predictions Knowledge</article-title>
          .
          <source>In: Proceedings of the 2nd International Conference on Applied Mathematics, Simulation and Modelling (AMSM)</source>
          , Phuket, Thailand.
          <source>DEStech Publications</source>
          , Inc.,
          <year>2017</year>
          :
          <fpage>279</fpage>
          -
          <lpage>283</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Atakishchev</surname>
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stepanov</surname>
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          , Grigoriev L.:
          <article-title>Probabilistic modelling processes of mutual monitoring operators actions for transport systems</article-title>
          .
          <source>In: Proceedings of the 4th International Conference on Transportation Information and Safety (ICTIS)</source>
          , Canada, Banff,
          <year>2017</year>
          :
          <fpage>865</fpage>
          -
          <lpage>871</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref25">
        <mixed-citation>
          [25]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Panov</surname>
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Stepanov</surname>
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Grigoriev</surname>
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          :
          <article-title>Optimization of sequence of performing heterogeneous repair work for transport systems by criteria of timeliness</article-title>
          .
          <source>In: Proceedings of the 4th International Conference on Transportation Information and Safety (ICTIS)</source>
          , Canada, Banff,
          <year>2017</year>
          :
          <fpage>872</fpage>
          -
          <lpage>876</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref26">
        <mixed-citation>
          [26]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Atakishchev</surname>
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Golovin</surname>
            <given-names>S.</given-names>
          </string-name>
          , Grigoriev L.:
          <article-title>The probabilistic analysis of the possibilities to keep “organism integrity” by continuous monitoring</article-title>
          .
          <source>In: Proceedings of the International Conference on Mathematics, Modelling, Simulation and Algorithms (MMSA)</source>
          , Chengdu, China. Atlantis Press,
          <source>Advances in Intelligent Systems Research</source>
          , volume
          <volume>159</volume>
          ,
          <year>2018</year>
          :
          <fpage>432</fpage>
          -
          <lpage>435</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref27">
        <mixed-citation>
          [27]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Grigoriev</surname>
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Golovin</surname>
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Klimov</surname>
            <given-names>S.</given-names>
          </string-name>
          :
          <article-title>Probabilistic Modeling of Robotic and Automated Systems Operating in Cosmic Space</article-title>
          .
          <source>In: Proceedings of the International Conference on Communication, Network and Artificial Intelligence (CNAI)</source>
          , Beijing, China.
          <source>DEStech Publications</source>
          , Inc.,
          <year>2018</year>
          :
          <fpage>298</fpage>
          -
          <lpage>303</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref28">
        <mixed-citation>
          [28]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Grigoriev</surname>
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kanygin</surname>
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Golovin</surname>
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          :
          <article-title>The Experience of Probabilistic Modeling and Optimization of a Centralized Heat Supply System Which is an Object for Modernization</article-title>
          .
          <source>International Conference on Physics, Computing and Mathematical Modeling (PCMM)</source>
          , Shanghai, DEStech Publications, Inc.,
          <year>2018</year>
          :
          <fpage>93</fpage>
          -
          <lpage>97</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref29">
        <mixed-citation>
          [29]
          <string-name>
            <surname>Artemyev</surname>
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <given-names>Rudenko</given-names>
            <surname>Ju</surname>
          </string-name>
          .,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.</given-names>
          </string-name>
          :
          <article-title>Probabilistic modeling in system engineering. Probabilistic methods and technologies of risks prediction and rationale of preventive measures by using “smart systems”. Applications to coal branch for increasing Industrial safety of enterprises</article-title>
          . Edited by Andrey Kostogryzov, IntechOpen,
          <year>2018</year>
          :
          <fpage>23</fpage>
          -
          <lpage>51</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref30">
        <mixed-citation>
          [30]
          <string-name>
            <surname>Kershenbaum</surname>
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Grigoriev</surname>
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kanygin</surname>
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Probabilistic modeling in system engineering. Probabilistic modeling processes for oil and gas systems</article-title>
          . Edited by Andrey Kostogryzov, IntechOpen,
          <year>2018</year>
          :
          <fpage>55</fpage>
          -
          <lpage>79</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref31">
        <mixed-citation>
          [31]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bezkorovainy</surname>
            <given-names>M.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Lvov</surname>
            <given-names>V.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratova</surname>
            <given-names>E.N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bezkorovainaya</surname>
            <given-names>I.V.</given-names>
          </string-name>
          <article-title>Complex for Evaluation of Information Systems Operation Quality - “know-how” (CEISOQ), registered by Rospatent №2000610272.</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref32">
        <mixed-citation>
          [32]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratova</surname>
            <given-names>E.N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A</given-names>
          </string-name>
          .A.
          <article-title>Mathematical modeling of system life cycle processes - “know-how”</article-title>
          , registered by Rospatent №
          <volume>2004610858</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref33">
        <mixed-citation>
          [33]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratova</surname>
            <given-names>E.N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.A.</given-names>
          </string-name>
          <article-title>Complex for evaluating quality of production processes, registered by Rospatent №2010614145</article-title>
        </mixed-citation>
      </ref>
      <ref id="ref34">
        <mixed-citation>
          [34]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.A.</given-names>
          </string-name>
          <article-title>Remote analytical support of informing about the probabilistic and time measures of operating system and its elements for risk-based approach</article-title>
          , registered by Rospatent №
          <volume>2018617949</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref35">
        <mixed-citation>
          [35]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.A.</given-names>
          </string-name>
          <article-title>Remote rationale of requirements to means and conditions for providing "smart" systems operation quality</article-title>
          , registered by Rospatent №
          <volume>2018618572</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref36">
        <mixed-citation>
          [36]
          <string-name>
            <surname>Kostogryzov</surname>
            <given-names>A.I.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>G.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Nistratov</surname>
            <given-names>A.A.</given-names>
          </string-name>
          <article-title>Remote probabilistic prediction of informatized systems operation quality</article-title>
          , registered by Rospatent №
          <volume>2018618686</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref37">
        <mixed-citation>
          [37]
          <string-name>
            <surname>Feller</surname>
            <given-names>W. An</given-names>
          </string-name>
          <article-title>Introduction to Probability Theory and Its Applications</article-title>
          . Vol. II,
          <string-name>
            <surname>Willy</surname>
          </string-name>
          ,
          <year>1971</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref38">
        <mixed-citation>
          [38]
          <string-name>
            <surname>Martin</surname>
            <given-names>J.</given-names>
          </string-name>
          (
          <year>1972</year>
          )
          <article-title>System Analysis for Data Transmission. V. II, IBM System Research Institute</article-title>
          . Prentice Hall, Inc.,
          <string-name>
            <surname>Englewood</surname>
            <given-names>Cliffs</given-names>
          </string-name>
          , New Jersey,
          <year>1972</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref39">
        <mixed-citation>
          [39]
          <string-name>
            <surname>Gnedenko</surname>
            <given-names>B.V.</given-names>
          </string-name>
          et al.
          <article-title>Priority queueing systems</article-title>
          , МSU, Moscow,
          <year>448p</year>
          .,
          <year>1973</year>
        </mixed-citation>
      </ref>
      <ref id="ref40">
        <mixed-citation>
          [40] Probabilistic Modeling in System Engineering / by ed. A.
          <string-name>
            <surname>Kostogryzov</surname>
          </string-name>
          . - London: IntechOpen,
          <year>2018</year>
          . 278 p. DOI:
          <volume>10</volume>
          .5772/intechopen.71396.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>