<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Quantitative Assessment of the Criticality Level of  Organizational System Elements </article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Hryhorii Hnatiienko</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Tetiana Babenko</string-name>
          <xref ref-type="aff" rid="aff0">0</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>Taras Shevchenko National University of Kyiv</institution>
          ,
          <addr-line>Volodymyrs'ka str. 64/13, Kyiv, 01601</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
      </contrib-group>
      <fpage>62</fpage>
      <lpage>81</lpage>
      <abstract>
        <p>   This paper discusses various aspects of determining quantitative indicators of the criticality level for elements of a complex organizational system. The concept of the criticality level of organizational system elements is introduced and formalized. The problem statement of determining the criticality level of elements is defined. Approaches to determining the limit values for critical elements of the organizational system are suggested. Features of relationships between functions performed by elements of the organizational system are described. The main aspects of the system activity and the measurement scale for determining the criticality level of elements are reviewed. Both resources used by the elements of the system to ensure its vital activity and flows that characterize the activity of the system are indicated. Approaches to the assessment of aspects of the system and features of using the quantitative scales while calculating the relative criticality level of system elements are described. Formulae are also given for values normalization of elements functioning parameters as well as approaches to the objective aggregation of various values of the parameters into a single integral indicator for each element of the system. Additional information has been studied that can be used for a comprehensive and objective study of the criticality level of the organizational system elements and an adequate determination of a integral indicator of the criticality level for each element.</p>
      </abstract>
      <kwd-group>
        <kwd>1  Critically important elements</kwd>
        <kwd>criteria</kwd>
        <kwd>quantitative indicators</kwd>
        <kwd>criticality level</kwd>
        <kwd>decisionmaking</kwd>
        <kwd>normalization of parameter values</kwd>
        <kwd>data aggregation</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction </title>
      <p>
        Functional stability (FS) of a system is an important characteristic of operating of a complex
organizational system (COS), along with reliability, survivability, and fault tolerance. This property
indicates the level of sustainable performance of COS purpose and the system security from external
threats. FS testifies to the ability of COS to maintain a given level of performance quality of the
functions, they are the purpose of developing the system, and the system is designed to perform them
[
        <xref ref-type="bibr" rid="ref1 ref3">1, 3</xref>
        ]. These functions must be performed under any conditions, despite the damage, various
extraneous influences, management errors, failure of some COS subsystems, equipment failures, and
various environmental disturbances during the system operation [
        <xref ref-type="bibr" rid="ref4 ref5">4, 5</xref>
        ]. Reasonable and adequate
approaches to assessing the quality of system functioning allow for formalizing the task of
maintaining FS and responding to external influences in a timely manner [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>
        Critical elements (CE) are extremely important factors influencing the functioning of COS in
general [
        <xref ref-type="bibr" rid="ref7">7, 8</xref>
        ]. The definition of such elements and the features of their influence on the system will be
given below. The main characteristics of CE with a significant impact on the activities of COS will
also be investigated, and approaches to the quantitative determination of the performance indicators of
the elements and subsystems of the organizational system will be proposed. Thus, the patterns of
providing FS of COS will be studied more deeply. This, of course, will entail the need to generate
new approaches to the provision of FS and affect the development of appropriate strategies for
managing COS.
      </p>
      <p>The definitions of criticality are the main features for defining CE of COS:
 by functionality, not by job titles, level in the hierarchy, or competencies of employees;
 with a mandatory adjustment for the authority of the employee in the performance of
functions;
 for the impact on the activities of COS in general (image, brand, goodwill, etc.);
 by direct or indirect influence on the external manifestations of COS (PR, sales level,
legend sustainability, etc.);
 according to the criticality of the main flows in COS (goods, finance, information, etc.);
 by influence on internal relationships (mission, vision, corporate culture).</p>
      <p>The purpose of this work is to analyze and quantify CE based on direct and indirect data:
 set of persons to whom CE is subordinate in this or that kind;
 set of persons subordinate to CE;
 flows passing through CE;
 functional duties and responsibilities of CE;
 CE with the authority and the availability of means for exercising these powers;
 dependence of the functions performed by other elements of COS on the quality of the
functioning of CE;
 time and expenses of COS for the training and adaptation of new employees for this
position;
 difficulty in replacing CE if necessary.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Problem statement </title>
      <p>
        Let some set of indices of functions that should be provided by COS be given. Let some set of
indices of functions that should be provided by COS be given. We will assume that there will be
such functions. Let us denote the set of all functions performed as A  a1,...an, J  1,..., n. A
complex system may perform hundreds or thousands of functions that are not duplicated, that is,
n   ni – each function in the system is unique: Ai1  Ai2  , i1 , i2  J , where is an empty
iJ
set [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>There are CE among the elements of COS. In this work, the criticality levels of elements will be
proposed and studied in order to ensure the qualitative performance of their functions, if necessary,
their motivation, etc.</p>
      <p>
        The relationship between functions and the sequence of their execution is given by a binary
relation , which is a subset of the Cartesian product A  A. The binary relation is built in
accordance with the consistency of some organizational system and reflects an approach to solving
issues faced by the system elements [9, 10]. Moreover, the performance of each function of the
system is provided by some element of the system e i , i  I  1,..., k and can be performed by some
other element e j , i  j, i, j  I , – in the general case with different degrees of quality – that is,
redundancy is embedded in the system [
        <xref ref-type="bibr" rid="ref3 ref7">3, 7</xref>
        ].
3. Some aspects of COS functioning and critical elements of COS 
      </p>
      <p>It is necessary to monitor characteristics grouped by some features to analyze data and decide on
the criticality level of elements [11-13]. It is advisable to consider the following aspects of the
elements of a complex semi-structured system, which are the nodes of a graph that models a specific
system in a certain subject area [14, 15]:
 1  impact on system resources;
 2  flows that the element controls;
 3  impact on decision-making in the system;
 4  a set of managerial influences on the system elements;
 5  frequency or participation in responding to multiple requests processed by the system;
 6  access control to significant features of the system;
 7  a set of graph edges providing information to the vertices;
 8  a set of graph edges modeling the reconciliation procedures for decision-making in the
system.</p>
      <p>Modern organizational systems are characterized by incompleteness, inaccuracy, and irrelevance
of information [16-18]. This is due to various reasons:
 irregular monitoring of the organization's activities;
 confidentiality of information;
 unavailability of data;
 insufficient qualification of the personnel providing the information system;
 different reference coordinates in data arrays;
 variability of operating conditions of organizational systems;
 dynamic changes in the environment that the organizational system interacts with.</p>
      <p>If the status and health of system elements significantly impact the system or its major parts, such
elements are considered critical parts (element, node, layer, subsystem) of the system. Relations
between elements can be represented as a multi-connected graph. If some vertices of the graph in case
of failure of related elements affect the decrease in FS or the loss of connectivity of a significant
number of graph nodes, such vertices are considered critical nodes. Therefore, all other elements of
the system are not critical and will be named linear.</p>
      <p>Critical elements play an important role at all stages of providing FS of COS and in all aspects of
decision-making on the management of FS. To determine the level of influence of the CE on the
functioning of the system, it is necessary to introduce the concept of criticality level (CL). CL is a
numerical indicator that reflects the integral influence of the CE on the quality of the functioning of
the system as a whole.</p>
      <p>The main goals of determining the quantitative CL of system elements are:
 increased danger for the system due to failures of CE functioning;
 the significant impact of CE on the quality of functioning of the organizational system;
 the indispensability of CE or issues with the quality performance of the functions
performed by CE.</p>
      <p>The main features and key characteristics of CE in COS are:
 have the appropriate level of access and authority for exceptional actions;
 have resources by orders of magnitude greater than ordinary elements and the authority to
manage them;
 their activity costs significantly higher than other elements;
 have a significant impact both on the elements of the lower level of management and on
the elements of the upper levels of the COS hierarchy;
 have a significant indirect impact on the system, subsystems, and other elements that could
be higher than direct impact;
 manage elements with high CL;
 in case of failure, their subordinate elements bear significant losses, as well as the system
itself;
 they have a significant number of various connections with other elements and, as a rule,
act a nodal role.</p>
      <p>Expert decision-making technologies are used to quantitatively determine the criticality level of
elements. Quantitative characteristics for graph vertices could be determined by means of incident
matrix also known as the Kirchhoff (Laplace) matrix. Some integral values that reflect the number,
importance and complexity of documents passing through administrative and other flows can be
considered as the weight of the edge.</p>
    </sec>
    <sec id="sec-3">
      <title>4. Resources and flows in COS </title>
      <p>
        Let us consider in more detail parameters [19, 20], which can be useful in some aspects [10, 21].
An in-depth analysis of all available system resource should be conducted to determine the influence
by a system element on its resources [
        <xref ref-type="bibr" rid="ref6">6, 8</xref>
        ]:
      </p>
      <p> 11  human (labor) resources – number and quality of personnel, staffing, the perfection of the
organizational structure, the quality of corporate culture;</p>
      <p>12  financial resources – currency, securities, percentage of liquid resources in the investment
portfolio;</p>
      <p> 13  material resources – equipment with fixed assets, the level of physical and economic
security;
14  intangible resources – goodwill, modern technologies, brand;
 15  information resources – availability of external information, orderliness, security of internal
information, computing power, and reliability of communication tools.</p>
      <p>The more critical element will be the one that has a greater impact on the specified resources
regardless of the level of use (operational, tactical, strategic, etc.). In context of the first aspect CL of
the element is calculated by the formula f1  1i1,..., 1i5  , where i, i  I  are the indices of the
system elements, parameters could be given tabular or analytically.</p>
      <p>The elements of the system are controlled by the flows that have a significant impact on them, that
is why the flows should be analyzed too [22, 23]:
 21  financial flows;
 22  material flows;
 23  information flows;
 24  service flows;
 25  managerial influences.</p>
      <p>The characteristics of a function may include various types of flows that need to be operated to
perform this function.
5. The structure of COS and the complexity of replacing CE as a criterion for 
the CL 
The following components should be considered when simulating COS:
 place of CE in the COS hierarchy (for example, the management level of the element,
direct subordination to the top-level element, subordination to sets of elements from lower
levels, functional subordination, basic functions, the quality level of functions, adjacent
functions, the quality level of adjacent functions);
 establishment of hierarchical connections between elements of the system and
determination of effect levels for one item on another or no effect;
 effect level on the subdivision that contains the element;
 effect level of the subdivision leading element on other subdivisions;
 effect level of the subdivision leading element on the whole system;
 interactions between subdivisions are affected significantly by functional subordination of
elements;
 effect level according to the staffing table with double and triple subordination;
 interaction between elements in accordance with functional interactions between
subdivisions (for example, cross-functional or mixed business process).</p>
      <p>One of the most important criteria of CL is the difficulty of replacing CE. This is because each
COS position has an estimate of the cost of replacing its incumbent, expressed as an estimate of the
sum of costs from different types of sources:
 difficulty in finding a competent specialist;
 the recruitment duration for the position of CE;
 probability of mistakes in recruiting;
 duration of the period of adaptation of a new employee;
 the degree of performance degradation during the adaptation period;
 demotivation of employees during the period of personnel rotation;
 the decreased discipline of personnel in a situation of staff turnover;
 a corporate culture that does not promote synergistic effects from the work of the
organization;
 the income level of an employee in the position of CE;
 securing the position with compensation and social packages;
 model of administrative and functional subordination.</p>
      <p>An important factor in assessing CL of CE is the features of the management system implemented
in COS. The controllability rate can be one of the important criteria for determining the criticality
level of system elements [24, 25]. In this case, it is necessary to take into account not only
administrative management but also functional. We will schematically consider the model of
functional control. There are two types of management in COS:
 administrative, when the head of the unit is responsible for the overall organization of the
activities of the unit's employees, for the quality performance of functional duties, for the
volume of tasks assigned to subordinates and their results, as well as for the fulfillment by
the unit's employees of all formal requirements for personnel;
 functional, when the manager organizes the activities of an employee who performs a
function in the area of responsibility of this manager, and the resource of the employee's
working time is managed by the administrative manager.</p>
      <p>
        To formalize this fundamental aspect of the functioning of COS, we present a model of the
functional management of COS. It should be noted that in most of the current COS the corporate
structure does not coincide with the legal one due to a number of objective and subjective reasons.
Therefore, in system organizations, a scheme is established for the distribution of power and
interaction between administrative management and functional management [
        <xref ref-type="bibr" rid="ref8">26, 27</xref>
        ].
      </p>
      <p>The scheme of power sharing and interaction between the administrative and functional managers
is represented in Table 1.</p>
      <p>Regarding CE, it should be noted that the functional manager must agree in advance with the
administrative manager on the hiring, dismissal, and vacation of employees with key functions who
are directly subordinate to the administrative manager.
6. Approaches to determining the criticality level and measurement scale </p>
      <p>At least three approaches can be applied to the definition of CL for the elements of COS, which
determine the numerical indicators of the influence of the elements on the quality of functioning and
the FS indicators of COS as a whole:
 expert approach;
 analytical (criteria) approach;
 expert-analytical.
1. Participation of the </p>
      <p>manager in the 
employee's career 
1.1. Coordination of the 
introduction of new </p>
      <p>positions 
1.2. Recruitment 
1.3. Adaptation </p>
      <p>procedure 
1.4. Transfer to another 
position within the scope 
of official duties 
1.5. Vacation 
1.6. Dismissal 
2. The manager's 
influence on the salary </p>
      <p>level 
3. Bonuses for 
subordinates 
4. Degree of control and </p>
      <p>interaction 
5. Providing functional </p>
      <p>subordinates with 
resources (information, </p>
      <p>labor, material) 
6. Using budgets by 
subordinate employees 
7. Duties of the manager </p>
      <p>to control the 
professional activities of 
the employee </p>
      <p> 
Initiates 
Agrees 
Initiates </p>
      <p>Initiates </p>
      <p>May initiate and 
approve the initiative of 
the administrative </p>
      <p>manager 
Regarding performance 
of functions in the area 
of responsibility 
Provides support </p>
      <p>Monitoring the 
implementation of the 
approved budgets </p>
      <p>accordingly 
On professional issues 
in the context of COS </p>
      <p>activities 
8. Business trip </p>
      <p>Initiates, agrees 
9. Bonus cancellation </p>
      <p>Initiates, agrees </p>
      <p>Administrative 
manager </p>
      <p> 
Agrees, signs the </p>
      <p>order 
Agrees, signs the </p>
      <p>order 
Approves, signs the </p>
      <p>order 
May initiate and 
approve the order 
Agrees, signs the </p>
      <p>order </p>
      <p>Initiates 
Regarding labor 
discipline and 
performance of basic </p>
      <p>functions </p>
      <p>According to the 
standards adopted 
for all COS 
employees 
Control over working </p>
      <p>hours </p>
      <p>On general official 
issues of work in COS 
Signs the order after </p>
      <p>approval by the 
functional manager 
May initiate, signs 
the order after 
approval by the 
functional manager </p>
      <p>
        The analytical approach provides a parametrization of the CE concept and describes this indicator
in an analytical form [
        <xref ref-type="bibr" rid="ref9">28</xref>
        ]. CL is based on the number of subordinates and the impact on financial and
information flows. The advantage of this approach is the clarity and transparency of the CL
calculation result. Among the shortcomings, it should be noted the possible incompleteness of the
mathematical model of CE.
      </p>
      <p>The expert approach is the opposite of the analytical one. You require the participation of qualified
experts who understand the specifics of the device industry, and the specific COS. But this is
compensated by the fact that experts can make a more comprehensive and detailed assessment. Due to
the participation of a large number of experts, the human factor is almost completely eliminated, since
it is unlikely that a large number of experts will make the same estimation error. Among the risks, it
should be noted that when creating questions posed to experts, emphasis can also be deliberately or
spontaneously placed on some tendentious aspects of COS.</p>
      <p>In turn, the expert approach can be based on the application of:
 ordinal scales;
 cardinal scales of expert evaluation.</p>
      <p>For ordinal scales, advantage ratios can be given in the form:
 ranking of alternatives (strict or non-strict);
 matrices of pairwise comparisons of alternatives in a qualitative form (strict or non-strict);
 multiple comparisons of alternatives (strict or non-strict);
 incomplete rankings of alternatives (strict or non-strict).</p>
      <p>For cardinal scales, advantage ratios can be given in the form:
 metricized matrices of pairwise comparisons;
 scoring of alternatives;
 relative weights of alternatives.</p>
      <p>In turn, the weighting coefficients can be:
 normalized;
 centralized;
 idealized;
 in interval form;
 in the form of membership functions to a fuzzy set.</p>
      <p>The expert-analytical approach is a combination of expert and analytical approaches. The
computational experiment described in this paper was conducted on its basis.</p>
      <p>A feature of modeling the criticality of elements of a complex system is that the data used to
analyze and compare criticality are measured or evaluated on different scales. Therefore, combined
data into aspects for each system element could be aggregated in various forms:
 defining discrete criticality levels;
 evaluating the membership function of element criticality to a fuzzy set;
 determining the intervals of criticality values for each element;
 calculating the intervals of criticality values for each element in metricized scales.
7. Diagram of precedents in determining the criticality level </p>
      <p>To illustrate the processes carried out in the organization when determining the CL, it is advisable
to give a diagram of precedents that reflects the relationship between actors in the process of
determining the CE and quantifying the LC. Such a diagram is presented in Figure 1.</p>
      <p>Based on this diagram, the researcher could formally assess various aspects of the CE activity and
quantify the CL.
    Definition of access levels and   
authorizations for exclusive </p>
      <p>actions 
       
    Determining the amount of   
resources authorized to </p>
      <p>manage 
       
    Determining the amount of   </p>
      <p>costs for the operation 
       
    Number of administrative   </p>
      <p>subordinates 
       
The head of the    Number of functional   </p>
      <p>organization  subordinates 
       
    Assessment of the influence   </p>
      <p>level on elements of the 
lower level of management 
       
    Assessment of the influence   </p>
      <p>level on elements of the 
higher level of management 
       
    Estimation of system losses   </p>
      <p>when elements fail 
       
    The number of connections of   
system elements with other </p>
      <p>elements 
Figure 1: A use case diagram for identifying critical elements and evaluating CL 
8. Additional information to clarify the criticality level 
 
 
 
 
 
 
 
 </p>
      <p>Financial 
Directorate </p>
      <p> 
Personnel 
Directorate 
 
 
 
IT Directorate </p>
      <p>Security 
Marketing 
Directorate </p>
      <p>Additional information can be used to improve the accuracy of determining CL: the responsibility
matrix and the authority matrix.</p>
      <p>The RACI (Responsible, Accountable, Consulted, Informed) matrix can be a special aspect that
characterizes the system elements represented by the graph nodes because this matrix provides a
description and coordination of the structure of responsibility for the implementation of work
packages in projects and business processes performed by the system elements. The RACI matrix is a
form of describing the distribution of responsibility for the job implementation on a business process
project, indicating the role of each element of the system in its implementation. The RACI matrix is a
convenient tool for allocating responsibilities between the elements of the system: Responsible,
Accountable, Consulted and/or Informed. Based on the analysis of these roles of elements in the
operation of the system, a conclusion can be drawn regarding the criticality of each element.</p>
      <p>An important tool for determining the CL is also the authority matrix (AM), which is developed
and approved in each system COS. AM describes the authority levels of key managers in all major
areas of COS activity. Therefore, the local CL indicators for this matrix can be easily folded and
digitized.
9. Methods  for  metrization  of  rankings  of  alternatives  specified  in  ordinal 
scales </p>
      <p>
        The issue of determining the complete order of a set of objects is a frequent task of expert
evaluation and has many practical applications [
        <xref ref-type="bibr" rid="ref10 ref11 ref12">29-31</xref>
        ]. But often the information about the
advantages in the ordinal scale on the set of alternatives is incomplete and insufficient, and therefore
cannot satisfy the researcher in all cases [
        <xref ref-type="bibr" rid="ref13 ref14">32, 33</xref>
        ]. In addition, it is not always possible to adequately
apply the entire arsenal of mathematical methods using expert data on the ratio of advantages in this
form of advantage aspect [
        <xref ref-type="bibr" rid="ref15 ref16">34, 35</xref>
        ].
      </p>
      <p>
        To date, a significant number of complex procedures for the metrization of features specified in an
ordinal scale have been developed [
        <xref ref-type="bibr" rid="ref17 ref18">36, 37</xref>
        ]. But the metrization procedures have a number of
disadvantages:
 they are usually too time-consuming for an expert;
 sometimes the expert is forced to change the advantages in the process of refining his
judgments, since the requirement to provide a numerical assessment of the still
insufficiently formalized phenomenon under study may turn out to be too strict;
 during metrization, some information about the structure of the expert's advantages may be
lost due to the approximation of a whole range of possible relationships by vectors of
numerical values.
      </p>
      <p>Let us describe a group of methods developed by the authors that allow us to carry out a consistent
metrization of the advantages given by an expert on an ordinal scale, making these transformations
automatically, without the participation of an expert. Quantitative data on benefits are presented, as a
rule, in interval form. Fixed values of weight coefficients of objects are also calculated, which can be
further used in methods that do not allow the use of interval estimates.</p>
      <p>
        Among the common ways to represent the values of weight coefficients, real numbers are most
often used, taking into account the normalization condition [
        <xref ref-type="bibr" rid="ref19 ref20">38, 39</xref>
        ]:
  i  1,   (1) 
iI
 i  0, i  I  1,..., n, (2) 
where n  number of alternatives.
9.1.
      </p>
    </sec>
    <sec id="sec-4">
      <title>Finding the gravity center </title>
      <p>Without diminishing the generality, we assume that the expert has established the advantage relation on
the set of objects in the form of a strict ranking as follows: a1  a2  ...  an . Then the simplex
corresponding to this relation is given as the intersection of such a system of half-spaces, taking into
account condition (2):
The
coordinates
of
the
vertices</p>
      <p>of
x j   x1j ,..., x nj  , j  1,..., n  1, calculated as follows:</p>
      <p>x j  1/ j, 1/ j,..., 0,..., 0 , j  1,..., n  1.</p>
      <p>The number of non-zero elements of the vector x j is equal to j  n, and the number of zero
components  n  1  j . At the same time, the  n  1 vertex of the simplex is at the origin of coordinates:
xn1  0,..., 0. It is obvious that the point   1,...,  n  lies on the hyperplane described by equation
(1) and, thus, satisfies the condition of normalization of weight coefficients.
9.2. Normalization of the midpoints of the intervals of changes in weighting 
coefficients </p>
      <p>Let all inequalities of the system (3) be strict. Then, considering the components of the vector of
weight coefficients of objects to be independent and taking into account inequalities of the form (3)
and restrictions on the normalizability of weight coefficients (1), (2), it is easy to verify that the
intervals of variation of the vector components  are as follows:
this
simplex,
which
are
denoted
by
1  1/ n, 1,...,  i  0, 1/ i , i  2,..., n.</p>
      <p>If the first relations between the alternatives are equalities, that is,</p>
      <p>1  1  ...   S 1  ...   n  0,
Then
the
corresponding
weight
coefficients
belong
to
the
intervals
1  1/ n,1/  s  1 ,...,  S 1  1/ n,1/  s  1.</p>
      <p>Thus, the intervals of changes in the weight coefficients of objects, that is, the hyperparallelepiped
of weight coefficients (HWC), in this case, are as follows</p>
      <p>1  1/ n,1 , if a1  a2 ,
 i  1/ n,1/  s  1, i  1,..., s, if a1 ~ a2 ~ ... ~ aS  aS 1,</p>
      <p> i  0,1/ i  , i  2,..., n, if a1  a2  ...  an .</p>
      <p>Heuristic E2. To approximate the obtained HWC by a normalized vector, its center is selected.
That is, the average value of the HWC, calculated by applying the proposed procedure
 iC   iH   iB  / 2, i  I ,
where iH ,iB i  I ,  respectively, the lower and upper limits of the change in the relative importance of
the i  th object.</p>
      <p>At the last stage of the procedure, the average values of the HWC should be normalized
 i   iC /   iC , i  I .</p>
      <p>jI
9.3. Calculation  of  weight  coefficients  based  on  the  average  value  of  the 
largest coefficient </p>
      <p>Let conditions (2)-(4) be satisfied. Let's introduce such a heuristic.</p>
      <p>Heuristic E3. The values of the weight coefficient of the "best" object are selected from the interval
1/ n,1.</p>
      <p>The value of the largest weight coefficient will be chosen as the midpoint of the interval introduced by
the E3 heuristic:</p>
      <p>1C  1/ n  1 / 2   n  1 / 2n.</p>
      <p>The selected value determines the interval for selecting the next weighting factor, i.e.,
2C 0,min1/ 2,1C  and so on. If i  1 coefficients are chosen, then the next average value of the
(5) 
(6) 
interval is calculated according to the formula (5), in which  iH  0,  iB  1/ i,  iC1  , due to the system of
restrictions (3), we have the inequality  i   i1.</p>
      <p>If some restrictions of the system (3) are equations  i   i1, 1  i  n 1, then the boundaries of the
intervals are calculated by the formulae  iH  0, iB 1/ i  s 1,iC1 , j  i,...,i  s  1, where s  is
the number of equations. Thus, the parallelepiped of possible values of the weight coefficients of objects is
as follows:</p>
      <p>1H  1/ n, 1B  1,
 iH  0,  iB  1/ i,  iC1  , if a1  a2  ...  an ,
 iH  0,  iB  1/ i  s  1,  iC1  , j  i,..., i  s  1,</p>
      <p>if a j ~ a j1 ~ ... ~ a jS , 1  i  n  s,
where the values  iC ,i  I , are calculated by the formula (5).
9.4. Calculation  of  weight  coefficients  based  on  the  average  value  of  the 
smallest coefficient </p>
      <p>This method differs from the previous one in that it is based on the introduction of a different
heuristic.</p>
      <p>Heuristic E4. The value of the weight coefficient of the "worst" object is selected from the interval
 n  0,1/ n .</p>
      <p>The value of the smallest weight coefficient is set equal to the value  nC  1/ 2n. The values of the
lower bounds of the i -th interval are determined by formulae  iH  min 1/ n,  iC1  due to the
inequality  i   i1 of system (3), and the values of the upper bounds are assumed to be equal
 iB  1/ i, i  2,..., n. Interval midpoints  iC , i  I , are calculated by formula (5). After that, the values
 iC ,i  I , are normalized by formula (6).
9.5.</p>
    </sec>
    <sec id="sec-5">
      <title>Method of equal intervals </title>
      <p>To calculate the boundaries of the intervals of changes of weight coefficients, we first use their
point estimates. Let  nH  , where   0  any positive number. We introduce the following
heuristic.</p>
      <p>Heuristic E5. The following assumptions are correct:
 if the objects are equal ai ~ ai1, then  iH  iH1,  iB   iB1;
 if ai  ai1, then  iH  iH1,  iB1  iB  / 2;  iB  iH  ;
 if ai  ai1, then  iB  iH  ,  iH  iH1.</p>
      <p>To calculate the boundaries of changes in the weight coefficients of objects, we normalize the estimates
obtained using the E5 heuristic according to the formulae
 iH  iH /   Hj   , i  I ,</p>
      <p>
 jI 
 iB  iB /   Bj   , i  I.</p>
      <p>
 jI </p>
      <p>The given methods of metrization of the advantages specified in the ordinal scale have a number of
positive properties:
 the expert is required to specify only the facts of the advantage between the objects because the
metrization of the advantages is performed automatically and does not require the use of
complex procedures for determining the intensity of the advantages;
 the obtained numerical values adequately reflect the expert's advantages, and do not change the
structure of the advantages on the set of objects;
 calculated as a result of the application of HWC methods, better reflect the psychological
characteristics of a person who is characterized by vague, inaccurate expert assessments.</p>
      <p>These methods can also be used as a first approximation to apply more complex benefit metric
procedures. At the same time, in the case of calculating the HWC, information about the advantages of an
expert, expressed on an ordinal scale, is not lost.
10.Determining the aggregate value of the criticality level </p>
      <p>
        The accumulated experience of expert assessment in various fields of human activity convincingly
indicates that any statistical calculations become more useful and justified while decreasing the
number of features to be analyzed [
        <xref ref-type="bibr" rid="ref21 ref22">40, 41</xref>
        ]. Therefore, there is an opportunity for expert assessment to
aggregate features of objects into a smaller number of constructed factors, aspects, etc. [
        <xref ref-type="bibr" rid="ref23 ref24">42, 43</xref>
        ].
10.1. Metrics used in expert evaluation issues 
      </p>
      <p>The following metrics could be used to determine distance between rankings of alternatives:
 Cook's metrics of mismatch of ranks (places, positions) of alternatives
d R j , Rl    ri j  ril ,</p>
      <p>iI
where ril  the rank of i  th alternative by l  th expert, Rl , l  L, 1  ril  n,</p>
      <p>Heming’s metrics;
Euclid's metrics;
elements of a vector of advantages, are the number of alternatives preceding each
alternative in the ranking.
10.2. Generalized quality criteria </p>
      <p>
        Constructing a convolution (a generalized, aggregating, integral criterion for the quality of an
object) means supplementing a partial order on a set of objects to a complete one [
        <xref ref-type="bibr" rid="ref10 ref11">29, 30</xref>
        ]. This can be
done in many ways, and necessarily involves an element of subjectivity [
        <xref ref-type="bibr" rid="ref22">41</xref>
        ]. Therefore, the
convolution method must be justified only to the following extent: the total order generated by the
convolution must be consistent with the natural partial order. Some of the most common convolution
families are:
 linear convolution
      </p>
      <p>Qi1    j ij , i  I ,</p>
      <p>jJ</p>
      <p>Qi3  mjaJx  j ij , i  I ,
multiplicative convolution</p>
      <p>Qi2   j ij , i  I ,</p>
      <p>jJ
a generalized convolution of indicators, which is sometimes also called the "bottleneck"
principle





where  ij , i  I , j  J ,  the normalized values of the parameters of objects aij , i  I , j  J , are
determined by transformations (1.6) - (1.13). Sometimes a convolution of the following form is also
used</p>
      <p>Qi4  jJ  j   ij  mjiJn ij  / mjiJn ij  , i  I .</p>
      <p>In some papers, such an approach is proposed to determine the complete order on a set of objects.
The set of parameter sets corresponds to a certain "cloud" of points in a multidimensional space. If it
is assumed that each object has a unit mass, then the Karhunen-Loev transformation makes it possible
to find some axis passing through the midpoint (the gravity center of objects), around which the
moment of inertia of the system of points is the smallest. After centering the object parameter values
 ijЦ   1n iI  ij , i  I , j  J ,
the gravity center of objects is at the origin of coordinates and the straight line V passes through it. Then
the issue is reduced to finding such a straight line V for which the following functional reaches a
minimum
where  iЦ   row vector of the normalized matrix of parameter values, prV iЦ   row vector of the
object projection  iЦ  on the axis V ,   distance between points  iЦ  and prV iЦ  . To
determine the quality of a set of parameters, the distances from the projections of objects prV iЦ  to
indicators Qij6 for j  J , i  I , better than the corresponding values of other objects and best
coincides in direction with the hypothetical ideal vector Qij6 , j  J , the value of all elements of which
are equal to one. Sometimes convolution is used to aggregate information across objects.</p>
      <p>Q7  1   ijЦ  2 ,</p>
      <p>m jJ
1/ 2
where  ijЦ    ij  Cj  /  1n iI  ij  Cj   ,i  I , j  J ,  Cj  1n iI  ij .</p>
      <p>It should be noted that with non-linear functions of indicators, it is difficult for an expert to
interpret his wishes in terms of the weight of the indicators, which significantly reduces the value of
this approach.
11.</p>
    </sec>
    <sec id="sec-6">
      <title>Computational experiment </title>
      <p>To determine the feasibility of using the approaches described in this paper, a computational
experiment was conducted. The experiment was carried out based on one of the largest domestic
system companies. The object of the study was the management company of the holding, which
employs more than a hundred employees. The holding staff is several thousand people. Volumes of
financial, material, service flows, etc. constitute a trade secret and are not subject to disclosure. The
name of the company, positions of experts, and positions of CE also do not need to be disclosed.</p>
      <p>To determine the compromise solutions to the issue of multi-attribute choice that arises in the
analysis of expert information on the ranking of CE, two criteria were applied: minimax and additive.
Moreover, if the values of the aggregated variant by these criteria turned out to be non-dominated, all
of them were included in the set of effective solutions for further analysis and metrics.
11.1. Conditions of the experiment </p>
      <p>To conduct a computational experiment, it was decided to use a combined approach to determining
the CL.</p>
      <p>At the first stage of the computational experiment, 6 experts were selected who are best oriented in
the situation regarding the activities of the organization under study and have the most inclusive
information about the managers of this organization, about the management of resources and flows of
the organization, etc.</p>
      <p>In the second stage, the normalized coefficients of relative competence of experts were determined
using the methods of self-assessment and mutual assessment.</p>
      <p>In the third stage, five employees were identified by the expert method among all the employees of
the company. According to the experts, these employees are the CE of the organization. Through the
introduction of expert technologies, it is necessary to find the quantitative characteristics of the CL of
these CE.</p>
      <p>Since it is a difficult task for a person to assign cardinal relationships between alternatives, it was
decided to receive information from experts in ordinal scales - in the form of rankings on a set of CE
selected by experts.</p>
      <p>The main features of the definition of CE of COS are the definitions of CL:
 by functionality;
 by authority level;
 by the impact on the operation of the system as a whole;
 by influence on the external manifestations of the system;
 by influence on the main flows in the system;
 by the impact on system resources.
11.2. Determining the compromise rankings on the set of CE </p>
      <p>When determining the order of importance of CE by functionality, expert rankings of CE by this
indicator are summarized in Table 2.</p>
      <sec id="sec-6-1">
        <title>Table 2 </title>
        <p>Determining the ranking of CL of CE by their functionality 
Expert rankings of CE by the level of their powers are given in Table 3.
The value of  The value of 
the minimax  the additive 
criterion  criterion 
   
 
 
 
 
 
8 
 
 
 
 
 
30 </p>
      </sec>
      <sec id="sec-6-2">
        <title>Table 3 </title>
        <p>Determining the ranking of CL of CE by the level of their authority 
Expert  Expert  The rank of CE given by 
competence  the expert 
coefficient 
Expert 1  0,3 
Expert 2  0,2 
The value of 
the minimax 
criterion 
 
 </p>
        <sec id="sec-6-2-1">
          <title>Expert 1 </title>
        </sec>
        <sec id="sec-6-2-2">
          <title>Expert 2 </title>
        </sec>
        <sec id="sec-6-2-3">
          <title>Expert 3 </title>
        </sec>
        <sec id="sec-6-2-4">
          <title>Expert 4 </title>
        </sec>
        <sec id="sec-6-2-5">
          <title>Expert 5 </title>
        </sec>
        <sec id="sec-6-2-6">
          <title>Expert 6 </title>
          <p>Compromise </p>
          <p>ranking 1 
Compromise </p>
          <p>ranking 2 
Compromise 
ranking 3 
 
 
 
a5  a3  a4  a1  a2  </p>
          <p>Compromise rankings of CL of CE according to various aspects of the impacts on COS are
summarized in Table 8.</p>
          <p>Expert rankings of CE by the impact on system resources are presented in Table 7.
Expert rankings of CE by the impact on the main flows in the system are given in Table 6.</p>
        </sec>
      </sec>
      <sec id="sec-6-3">
        <title>Table 8 </title>
        <p>Compromise rankings of CL of CE according to various aspects of impacts on COS 
Compromise ranking of CL of CE  The rank of CE given by  The value of 
the expert  the minimax 
criterion 
By the functionality </p>
        <p>By authority level 
By the impact on the operation of </p>
        <p>the system as a whole 
By influence on the external 
manifestations of the system 
By influencing the main flows in the </p>
        <p>system 
By impact on system resources 1 
By impact on system resources 2 
By impact on system resources 3 
a5  a4  a3  a1  a2  
a5  a4  a2  a3  a1  
a4  a5  a3  a2  a1  
11.3. Calculation of aggregated quantitative CL for selected CE </p>
        <p>The method of calculating weighting coefficients based on the average value of the smallest
coefficient, described in paragraph 9.4, was applied to the compromise rankings of CL of CE
presented in Table 8. The calculation results and averaged normalized CL values are presented in
Table 9.</p>
      </sec>
      <sec id="sec-6-4">
        <title>Table 9 </title>
        <p>Compromise rankings of CL of CE according to various aspects of the impacts on COS 
Compromise ranking of CL of CE  The rank of CE given by  1    2    3  
the expert 
 4  
 5  
By the functionality </p>
        <p>By authority level 
By the impact on the operation of the </p>
        <p>system as a whole 
By influence on the external 
manifestations of the system 
By influencing the main flows in the </p>
        <p>system 
By impact on system resources 1 
By impact on system resources 2 
By impact on system resources 3 
The total value of the coefficients </p>
        <p>Normalized average value 
a5  a4  a3  a1  a2   0,186  0,124  0,217  0,233  0,24 
a5  a4  a2  a3  a1   0,124  0,217  0,186  0,233  0,24 
a4  a5  a3  a2  a1   0,124  0,186  0,217  0,24  0,233 
a5  a3  a4  a2  a1   0,124  0,186  0,233  0,217  0,24 
a5  a4  a3  a1  a2   0,186  0,124  0,217  0,233  0,24 
a5  a3  a4  a1  a2   0,186  0,124  0,233  0,217  0,24 
a5  a2  a4  a1  a3   0,186  0,233  0,124  0,217  0,24 
a5  a2  a4  a3  a1   0,124  0,233  0,186  0,217  0,24 
  1,24  1,43  1,61  1,81  1,91 
  65%  75%  84%  94%  100% </p>
        <p>Thus, the highest CL in quantitative terms has CE #5 – 100%, and the lowest CL has CE #1 –
65%.
12.Results </p>
        <p>Thus, the CL of the system elements are being digitized and we can use the obtained values for
additional motivation of critical elements, providing them with special funding, implementing
security procedures, etc.</p>
        <p>Considering the model of the form (1) we can settle that the influence of CE on FS of COS must
be at least one or more orders greater than the effect of linear (non-critical) elements. This fact is
confirmed by corresponding computational experiments.</p>
        <p>The study proposes a model for determining CE of COS. It is also justified:
 model is eligible for defining critical parts;
 expert assessment is acceptable to evaluate CL of certain aspects of system elements;
 the proposed approach tested positive to determine the integral CL of elements of COS.</p>
        <p>The conducted research suggests that in order to ensure the functional stability of a complex
organizational system, it is important and necessary to identify and maintain the critical elements of
such a system.</p>
        <p>This given model can be applied to the various needs of any organization, as well as adjusted to
other subject areas with hierarchies and interactions. The model assumes further improvement and can
be focused on handling with fuzzy data.</p>
        <p>This work proposed a mathematical model for the issue of determining critical nodes in complex
organizational systems. It is also substantiated using graphs for modeling the relationships between
the elements of the system. We considered the features of the phenomenon of criticality concerning
elements, relations of elements and subsystems within a complex system. We outlined the main
aspects of characteristics of organizational systems, which are essential to maintain coexistence of
criticality and functional stability. It is considered the possibility of expert assessment in determining
the criticality level of system elements in some aspects. The application of well-known approaches is
justified for the allocation of critical elements of the system, in particular, the responsibility matrix.
13.References 
[8] Babenko, T., Hnatiienko, H., Vialkova, V., Iavich, M.: Modeling of Critical Nodes in Complex
Poorly Structured Organizational Systems. In: Proceedings of the 26th International Conference
on Information Society and University Studies, CEUR Workshop Proceedings, vol. 2915, 2021,
pp. 92-101.
[9] Voloshyn, O., Mashcenko, S.: Decision-Making Models and Methods: Teaching Manual for
Students Higher Education Closing. 3rd ed., revision. Kyiv: Lyudmila Publishing House, 2018,
292 pages.
[10] Makarov, I., Vinogradskaya, T., Rubchinskiy, A., Sokolov, V.: Theory of Choice and Decision</p>
        <p>Making. М.: Nauka, 1982, 330 pages.
[11] Ivanyuta, P., Luhivska, O.: Management of Resources and Costs: Textbook. Way. 2nd Type. For</p>
        <p>Order. Ivanyuty SM Kyiv: Center for Educational Literature, 2011, 320 pages.
[12] Kalivoshko, O., Kraevsky, V., Burdeha, K., Lyutyy, I., Kiktev, N.: "The Role of Innovation in
Economic Growth: Information and Analytical Aspect." In: 2021 IEEE 8th International
Conference on Problems of Infocommunications, Science and Technology (PIC S&amp;T), Kharkiv,
Ukraine, 2021, pp. 120-124. doi: 10.1109/PICST54195.2021.9772201.
[13] Lengnick-Hall, C., Lengnick-Hall, M., Neely, A., Bonner, R.: Something Old, Something New:
Reframing the Integration of Social Capital into Strategic HRM Research. In: Academy of
Management Perspectives. Vol. 35, No. 3, 2021, pp. 535-556.
[14] Hnatiienko, H., Kudin, V., Onyshchenko, A., Snytyuk, V., Kruhlov, A.: "Greenhouse Gas
Emission Determination Based on the Pseudo-Base Matrix Method for Environmental Pollution
Quotas Between Countries Allocation Problem." In: IEEE 2nd International Conference on
System Analysis &amp; Intelligent Computing (SAIC), Kyiv, Ukraine, 2020, pp. 150-157. doi:
10.1109/SAIC51296.2020.9239125.
[15] Hnatiienko, H., Kiktev, N., Babenko, T., Desiatko, A., Myrutenko, L.: Prioritizing Cybersecurity
Measures with Decision Support Methods Using Incomplete Data. In: Selected Papers of the
XXI International Scientific and Practical Conference "Information Technologies and Security",
CEUR Workshop Proceedings, vol. 3241, 2021, pp. 169-180.
[16] Belan, S., Belan, N.: Use of Cellular Automata to Create an Artificial System of Image
Classification and Recognition. In: Lecture Notes in Computer Science (including subseries
Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7495 LNCS,
2012, pp. 483-493.
[17] Belan, S., Belan, N.: Use of Cellular Automata to Create an Artificial System of Image
Classification and Recognition. In: Lecture Notes in Computer Science (including subseries
Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). Vol. 7495 LNCS,
2012, pp. 483-493.
[18] Kovalenko, I., Shved, A., Davydenko, Y.: The basic concepts of the normative theory of the
synthesis of information technologies for decision support. In Proceedings of the 15th
International Conference on Computer Sciences and Information Technologies (CSIT 2020),
Zbarazh, Ukraine, September 23-26, 2020, volume 332-335. doi:
10.1109/CSIT49958.2020.9321882.
[19] H. Ansoff. Strategic Management. Springer, p. 251 (2007)
[20] Root, G.: Organisational Objectives in Strategic Planning. Hearts Newspapers, LLC Texas,
Demand Media, Available:
[http://smallbusiness.chron.com/organisationalobjectives-strategicplanning-10034.html.] (2014)
[21] David, F.: Strategic management: concepts and cases. 13th ed. FT Prentice Hall, p.694 (2011)
[22] Bagheri, J.: Overlaps between human resource’ startegic planning and strategic management
tools in public organizations. Social and Behavioral Sciences, 230:430-438,
doi:10.1016/j.sbspro.2016.09.054 (2016)
[23] Dess, G., Lumpkin, G. and Eisner, A.: Strategic Management: Creating Competitive Advantages.</p>
        <p>4th edn, McGraw-Hill, New York (2010)
[24] Lengnick-Hall, M., Lengnick-Hall, C., and Rigsbee, C.: Strategic Human Resource Management
and Supply Chain Orientation. Human Resource Management, 23(4):366-377 (2013)
[25] Jackson, S., Schuler, R., Jiang, K.: An aspirational framework for strategic human resource
management. The Academy of Management Annals, 8(1):1–56 (2014).
[26] Palko, D. et al. Cyber Security Risk Modeling in Distributed Information Systems. Applied
Sciences (Switzerland) 2023, 13, 2393. https://doi.org/10.3390/app13042393</p>
      </sec>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Mashkov</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Zapara</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>The Concept of Functional Stability of the On-board Information Control Complex</article-title>
          .
          <source>In: Equipment for Aircraft. Kyiv</source>
          ,
          <year>1993</year>
          , pp.
          <fpage>25</fpage>
          -
          <lpage>29</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Sobchuk</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Koval</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Musienko</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Matsko</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          :
          <article-title>The Method of Diagnostics of Primed Images in the Information System on the Basis of Storing the Courtyard System and Securing the Functional Stability</article-title>
          .
          <source>In: Science Magazine "Telecommunications and Information Technologies." Kyiv</source>
          , No.
          <volume>1</volume>
          (
          <issue>62</issue>
          ),
          <year>2019</year>
          , pp.
          <fpage>22</fpage>
          -
          <lpage>31</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Mashkov</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Barabash</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          :
          <article-title>Synthesis of the Structure of an Automated System According to the Criterion of Maximum Functional Stability</article-title>
          .
          <source>In: Aerokosmіchnі Systems and Monitoring and Keruvannya. Kyiv</source>
          ,
          <year>2003</year>
          , pp.
          <fpage>193</fpage>
          -
          <lpage>196</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Kravchenko</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vialkova</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          :
          <article-title>The Problem of Providing Functional Stability Properties of Information Security Systems</article-title>
          . In: Modern Problems of Radio Engineering, Telecommunications and Computer Science,
          <source>Proceedings of the 13th International Conference on TCSET 2016</source>
          .
          <year>2016</year>
          , pp.
          <fpage>526</fpage>
          -
          <lpage>530</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Mashkov</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Chumakevich</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mamchur</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kosenko</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          :
          <article-title>The Method of Inverse Problems of Dynamics for the Synthesis of a System of Stabilization of the Movement of a Dynamic Object on Operatively Programmable Trajectories</article-title>
          .
          <source>In: Mathematical Modeling and Computing</source>
          . Vol.
          <volume>7</volume>
          , No.
          <volume>1</volume>
          ,
          <issue>2020</issue>
          , pp.
          <fpage>29</fpage>
          -
          <lpage>38</lpage>
          . doi:
          <volume>10</volume>
          .23939/mmc2020.
          <fpage>01</fpage>
          .029.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Babenko</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hnatiienko</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Vialkova</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          :
          <article-title>Modeling of Information Security System and Automated Assessment of the Integrated Quality of the Impact of Controls on the Functional Stability of the Organizational System</article-title>
          .
          <source>In: Selected Papers of the XX International Scientific and Practical Conference "Information Technologies and Security" (ITS</source>
          <year>2020</year>
          ),
          <source>CEUR Workshop Proceedings</source>
          , vol.
          <volume>2859</volume>
          ,
          <year>2021</year>
          , pp.
          <fpage>188</fpage>
          -
          <lpage>198</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Barabash</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kozelkov</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Mashkov</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          :
          <article-title>Understandable Apparatus of Functional Efficiency of Information-Critical Systems</article-title>
          . In:
          <article-title>Zbirnik naukovykh prats NTs VPS ZS Ukraine</article-title>
          . Kyiv,
          <source>No. 7</source>
          ,
          <issue>2005</issue>
          , pp.
          <fpage>87</fpage>
          -
          <lpage>95</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [27]
          <string-name>
            <surname>Palko</surname>
            ,
            <given-names>D.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Myrutenko</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Babenko</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bigdan</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          "
          <source>Model of Information Security Critical Incident Risk Assessment." 2020 IEEE International Conference on Problems of Infocommunications Science and Technology, PIC S and T 2020 - Proceedings</source>
          ,
          <year>2021</year>
          , pp.
          <fpage>157</fpage>
          -
          <lpage>161</lpage>
          ,
          <fpage>9468107</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [28]
          <string-name>
            <surname>Barabash</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shevchenko</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Dakhno</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kravchenko</surname>
            ,
            <given-names>Y.</given-names>
          </string-name>
          and
          <string-name>
            <surname>Leshchenko</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          "
          <source>Effectiveness of Targeting Informational Technology Application." In IEEE International Conference on System Analysis &amp; Intelligent Computing, SAIC'</source>
          <year>2020</year>
          , Proceedings, p.
          <fpage>193</fpage>
          -
          <lpage>196</lpage>
          . doi:
          <volume>10</volume>
          .1109/SAIC51296.
          <year>2020</year>
          .
          <volume>9239154</volume>
          (
          <year>2020</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [29]
          <string-name>
            <surname>Kemeny</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Snell</surname>
            ,
            <given-names>J.: Cybernetic</given-names>
          </string-name>
          <string-name>
            <surname>Modeling: Some Applications</surname>
          </string-name>
          . Soviet radio. р.
          <volume>192</volume>
          (
          <year>1972</year>
          )
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [30]
          <string-name>
            <surname>Cook</surname>
            ,
            <given-names>W.</given-names>
          </string-name>
          :
          <article-title>Distance-based and adhoc consensus models in ordinal preference ranking</article-title>
          .
          <source>European Journal of Operational Research</source>
          , No.
          <volume>172</volume>
          , p.
          <fpage>369</fpage>
          -
          <lpage>385</lpage>
          (
          <year>2006</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [31]
          <string-name>
            <surname>Elkind</surname>
            ,
            <given-names>E.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Faliszewski</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Slinko</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Distance Rationalization of Voting Rules</article-title>
          .
          <source>Social Choice and Welfare</source>
          ,
          <volume>45</volume>
          (
          <issue>2</issue>
          ),
          <fpage>345</fpage>
          -
          <lpage>377</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [32]
          <string-name>
            <surname>Amodio</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>D'Ambrosio</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Siciliano</surname>
          </string-name>
          , R.:
          <article-title>Accurate Algorithms for Identifying the Median Ranking when Dealing with Weak and Partial Rankings under the Kemeny Axiomatic Approach</article-title>
          .
          <source>European Journal of Operational Research</source>
          , No.
          <volume>249</volume>
          ,
          <fpage>667</fpage>
          -
          <lpage>676</lpage>
          (
          <year>2015</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [33]
          <string-name>
            <surname>Hnatiienko</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tmienova</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kruglov</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          :
          <article-title>Methods for Determining the Group Ranking of Alternatives for Incomplete Expert Rankings</article-title>
          . In: S. Shkarlet,
          <string-name>
            <given-names>A.</given-names>
            <surname>Morozov</surname>
          </string-name>
          ,
          <article-title>and</article-title>
          <string-name>
            <surname>A</surname>
          </string-name>
          . Palagin (eds),
          <source>Mathematical Modeling and Simulation of Systems (MODS'2020)</source>
          , volume
          <volume>1265</volume>
          <source>of Advances in Intelligent Systems and Computing</source>
          ,
          <volume>217</volume>
          -
          <fpage>226</fpage>
          , Springer, Cham,
          <year>2020</year>
          . doi:
          <volume>10</volume>
          .1007/978-3-
          <fpage>030</fpage>
          - 58124-4_
          <fpage>21</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [34]
          <string-name>
            <surname>Wu</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Jin</surname>
            ,
            <given-names>B.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fujita</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Xu</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          :
          <article-title>Consensus Analysis for AHP Multiplicative Preference Relations Based on Consistency Control: A Heuristic Approach</article-title>
          .
          <source>Knowledge-Based Systems</source>
          ,
          <volume>191</volume>
          ,
          <fpage>105317</fpage>
          ,
          <year>2020</year>
          . doi:
          <volume>10</volume>
          .1016/j.knosys.
          <year>2019</year>
          .
          <volume>105317</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [35]
          <string-name>
            <surname>Hnatiienko</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Suprun</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          :
          <article-title>Fuzzy Set Objects Clustering Method Using Evolution Technologies</article-title>
          . In: V.
          <string-name>
            <surname>Skala</surname>
            ,
            <given-names>M. O. R.</given-names>
          </string-name>
          <string-name>
            <surname>Prasetya</surname>
            , and
            <given-names>Z. A. A</given-names>
          </string-name>
          . Baarak, editors,
          <source>Proceedings of the XVIII International Scientific and Practical Conference "Information Technologies and Security" (ITS</source>
          <year>2018</year>
          ),
          <fpage>330</fpage>
          -
          <lpage>337</lpage>
          , Kyiv, Ukraine, November
          <volume>27</volume>
          ,
          <year>2018</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [36]
          <string-name>
            <surname>Voloshin</surname>
            ,
            <given-names>A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Gnatienko</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Drobot</surname>
          </string-name>
          , E.:
          <article-title>A Method of Indirect Determination of Intervals of Weight Coefficients of Parameters for Metricized Relations between Objects</article-title>
          .
          <source>Journal of Automation and Information Sciences</source>
          ,
          <volume>35</volume>
          (
          <issue>1-4</issue>
          ),
          <fpage>41</fpage>
          -
          <lpage>51</lpage>
          (
          <year>2003</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [37]
          <string-name>
            <surname>Bozoki</surname>
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.:</given-names>
          </string-name>
          <article-title>The (Logarithmic) Least Squares Optimality of the Arithmetic (Geometric) Mean of Weight Vectors Calculated from All Spanning Trees for Incomplete Additive (Multiplicative) Pairwise Comparison Matrices</article-title>
          .
          <source>International Journal of General Systems</source>
          ,
          <volume>48</volume>
          (
          <issue>4</issue>
          ),
          <fpage>362</fpage>
          -
          <lpage>381</lpage>
          (
          <year>2019</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [38]
          <string-name>
            <surname>Saaty</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Pairwise Comparisons and Their Contribution to Understanding Consciousness</article-title>
          .
          <source>International Journal of the Analytic Hierarchy Process</source>
          ,
          <volume>8</volume>
          (
          <issue>1</issue>
          ),
          <year>2016</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [39]
          <string-name>
            <surname>Safarzadeh</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Khansefid</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Rasti-Barzoki</surname>
            ,
            <given-names>M.:</given-names>
          </string-name>
          <article-title>A Group Multi-Criteria Decision-Making Based on Best-Worst Method</article-title>
          .
          <source>In: Proceedings of the 7th International Conference "Information Technology and Interactions" (IT and I</source>
          <year>2020</year>
          ), volume
          <volume>2833</volume>
          <source>of CEUR Workshop Proceedings</source>
          ,
          <fpage>164</fpage>
          -
          <lpage>176</lpage>
          , Kyiv, Ukraine, December 2-
          <issue>3</issue>
          (
          <year>2020</year>
          ).
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [40]
          <string-name>
            <surname>Bilan</surname>
            ,
            <given-names>S.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bilan</surname>
            ,
            <given-names>M.M.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Motornyuk</surname>
            ,
            <given-names>R.L.</given-names>
          </string-name>
          :
          <article-title>New Methods and Paradigms for Modeling Dynamic Processes Based on Cellular Automata</article-title>
          .
          <source>IGI-Global</source>
          ,
          <year>2020</year>
          ,
          <volume>200</volume>
          pages.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [41]
          <string-name>
            <surname>Hnatiienko</surname>
            <given-names>H.</given-names>
          </string-name>
          (
          <year>2019</year>
          ).
          <article-title>Choice Manipulation in Multicriteria Optimization Problems</article-title>
          .
          <source>Selected Papers of the XIX International Scientific and Practical Conference "Information Technologies and Security" (ITS</source>
          <year>2019</year>
          ), pp.
          <fpage>234</fpage>
          -
          <lpage>245</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [42]
          <string-name>
            <surname>Saaty</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          :
          <article-title>Aligning the Measurement of Tangibles with Intangibles and Not the Converse</article-title>
          .
          <source>International Journal of the Analytic Hierarchy Process</source>
          ,
          <volume>3</volume>
          (
          <issue>1</issue>
          ),
          <year>2011</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [43]
          <string-name>
            <surname>Hnatiienko</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Snytyuk</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tmienova</surname>
            ,
            <given-names>N.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Voloshyn</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          :
          <article-title>Determining the Effectiveness of Scientific Research of Universities Staff</article-title>
          . In: V.
          <string-name>
            <surname>Skala</surname>
            ,
            <given-names>M. O. R.</given-names>
          </string-name>
          <string-name>
            <surname>Prasetya</surname>
            , and
            <given-names>Z. A. A</given-names>
          </string-name>
          . Baarak, editors,
          <source>Proceedings of the 7th International Conference "Information Technology and Interactions" (IT and I</source>
          <year>2020</year>
          ), volume
          <volume>2833</volume>
          <source>of CEUR Workshop Proceedings</source>
          ,
          <fpage>164</fpage>
          -
          <lpage>176</lpage>
          , Kyiv, Ukraine, December 2-
          <issue>3</issue>
          (
          <year>2020</year>
          ).
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>