<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Knowledge Transfer Platform Toolkit for Strategic Planning </article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>Institute for Information Recording of National Academy of Sciences of Ukraine</institution>
          ,
          <addr-line>Mykoly Shpaka str. 2, Kyiv, 03113</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"</institution>
          ,
          <addr-line>Peremogy ave., 37, Kyiv, 03056</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>Taras Shevchenko National University of Kyiv</institution>
          ,
          <addr-line>Volodymyrs'ka str. 64/13, Kyiv, 01601</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
      </contrib-group>
      <fpage>21</fpage>
      <lpage>31</lpage>
      <abstract>
        <p>   The research aims to help transfer knowledge from people who have it to people who need it to solve practical problems in specific domains. Applying this theoretical background is intended for strategic planning in various areas, especially in weakly structured subject domains. The knowledge transfer platform's technology and the knowledge transfer platform toolkit for strategic planning are proposed.</p>
      </abstract>
      <kwd-group>
        <kwd> 1  knowledge transfer</kwd>
        <kwd>strategic planning</kwd>
        <kwd>goal dynamic evaluation of alternatives</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction </title>
      <p>Using existing knowledge effectively is vital to humankind's growth, progress, and
development. It is a same-level problem as a problem of accelerating the acquisition of
knowledge in various fields.</p>
      <p>The relevance of the work is determined by a significant amount of knowledge in weakly
structured domains that are unregistered and unavailable for use at this moment. Moreover, a
significant percentage of informal knowledge is owned only by certain specialists due to their
unique experience and intuition. When building knowledge bases, it is advisable to use all
available knowledge, both explicit (formalized) and implicit (which can be obtained in
information networks and when contacting experts).</p>
      <p>Convenient knowledge provision mechanisms are created based on the knowledge bases
built by decomposing and structuring everyday problems using advanced decision support
methods. There is a need to process knowledge of a different nature obtained from different
sources: general verification for reliability, completeness, balance, coherence, the possibility
of systematization, and generalization. There have yet to be fully developed methods and
means to increase the application level of such knowledge. Therefore, the subject of the
research is to create new methods, models, and software tools (knowledge transfer platform
tools). They are used to obtain knowledge about a specific weakly structured subject domain,
to process knowledge of a different nature, to formalize it, and to form a mechanism for using
this knowledge by decision-makers in various fields.</p>
    </sec>
    <sec id="sec-2">
      <title>2. Research Methodology </title>
      <p>
        The technology of knowledge transfer is based on the group construction of a goal-oriented
model of the system (subject domain) [
        <xref ref-type="bibr" rid="ref1 ref2">1, 2</xref>
        ], which is created by decomposing the main goal
and considering the time and resource properties of the system components and the relations
between them. The software toolkit allows knowledge engineers and experts to provide
knowledge about the subject domain to create a model of the system.
      </p>
      <p>
        The knowledge transfer platform can solve the problem of strategic planning [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ] in several
steps:
1. Define the main strategic goal.
2. Decompose the goal into sub-goals that affect the main goal in this subject domain.
3. Define non-decomposable goals within the competence of the decision-maker (DM).
4. Identify the required resources to implement each project.
5. Determine the duration and possible delay of each project.
      </p>
      <p>6. Determine optimal resource allocation between the projects, which will maximize the
efficiency of achieving the main strategic goal with a given financing.</p>
      <p>7. Allow corrections to the strategic plan.</p>
    </sec>
    <sec id="sec-3">
      <title>3. Strategic Planning </title>
      <p>Since strategy is a way to achieve a specific goal, the concepts of strategy and goals are
inseparable. Therefore, when building strategic plans, it is proposed to use the so-called
goaloriented approach. It is an approach to model the subject domain as a complex weekly
structured system in the form of interconnected components or goals that affect one another.
3.1.</p>
    </sec>
    <sec id="sec-4">
      <title>Goal‐Oriented Approach to Strategic Planning </title>
      <p>When applying the goal-oriented approach, first, it is necessary to define the main goal
that must be achieved during the implementation of the strategy. This strategic goal is usually
formulated by decision-makers (DMs), such as state leaders, politicians, and businesspeople.</p>
      <p>It is worth noting that the goal-oriented approach involves determining the characteristics
of the main goal through the characteristics of other components of the system — goals that
have a direct or indirect impact on the achievement of the primary goal.
3.2.</p>
    </sec>
    <sec id="sec-5">
      <title>Subject Domain Model </title>
      <p>In strategic planning, assessing the goal's achievement level over time is essential.
Therefore, the model of the subject domain is created considering this requirement.</p>
      <p>The subject domain model is a directed graph of the hierarchy of goals formed due to the
decomposition of the primary goal. Arbitrary influences (connections) between goals can be
added to the graph model to increase the model's adequacy. Although the resulting graph has
a hierarchical tree structure, it is generally a network.</p>
      <p>The model has the following components: goals — vertices (nodes) of the graph, projects
— vertices (leaves of the tree), and influences — arcs of the directed graph. All the listed
components can be of different types and have different properties.
3.2.1. Goal Model </p>
      <p>The main component in the system model is the goal. Goals are represented as graph
vertices in the general model of the system. Goals are formed as a decomposition of the main
goal and are essentially its components.</p>
      <p>Two types of processes lead to goal achievement. Linear goal — when any progress in
achieving the goal causes a change in the impact of this goal on other goals. Threshold goal —
when the goal does not impact other goals until the degree of its achievement exceeds a certain
threshold.</p>
      <p>In addition, goals could be presented as quantitative (when the degree of achievement can
be defined in specific units of measurement) or qualitative. The main goal in strategic planning
should be defined as a linear qualitative goal.</p>
    </sec>
    <sec id="sec-6">
      <title>3.2.2. Project Model </title>
      <p>A project is a goal associated (when achieved) with implementing specific actions. Such
components allow for assessing analytically or expertly the duration of implementation (time)
and resources (finances) necessary to achieve the project's goal. These two properties
distinguish projects from goals, as projects, unlike goals, are not subject to decomposition.</p>
      <p>To increase the adequacy of the model and to rationalize the distribution of available
financial resources, the project model considers the dependence of the degree of project
implementation on its funding. A piecewise continuous linear function is used, as shown in</p>
    </sec>
    <sec id="sec-7">
      <title>3.2.3. Properties of Influences </title>
      <p>Goals, as components of the system, are interconnected: some goals influence others
within the model. Influences are established during the decomposition of a goal simultaneously
with the formulation of the components of this goal. The components influence the goal,
usually called sub-goals, while the goal influenced by the sub-goals is sometimes called the
super-goal.
influenced).
their super-goal.</p>
      <p>In the goal-oriented graph model, the influences between goals are represented as arcs in
the directed graph. If a particular goal directly influences the other, then an arc from the graph's
vertex represents the first goal to the second vertex (which represents the goal being</p>
      <p>Influences have several properties. One of the main ones is the relative indicator, the
socalled partial coefficient of influence (PCI). PCI is an indicator of sub-goals' direct impact on</p>
      <p>It is possible to provide alternative ways to achieve each goal. A set of sub-goals
compatible with each other represents each way of achieving the goal. The sub-goals are
compatible if the achievement of one sub-goal does not prevent one from achieving another.
Such groups of compatible goals are defined during decomposition by providing information
on the compatibility of each pair of sub-goals.</p>
      <p>PCIs are normalized, and for each k-th group of compatible sub-goals:

or negative.</p>
      <p>K is the number of compatible sub-goals in the k-th group.</p>
      <p>
        is the PCI of the j-th sub-goal of the i-th goal in the k-th group of compatible sub-goals;
The modulus (absolute value) is used in (1) because influences could be either positive
The value of PCI before normalization (1) is defined as the relative contribution of a
subgoal to achieving a specific goal in two ways. Suppose there is reliable information about
achieving a sub-goal. In that case, PCI is calculated as a ratio of achieving the sub-goal to the
required resources needed to achieve the super-goal, measured in the same units as the effect.
If there is no reliable information about the effect of achieving a sub-goal or when the sub-goal
is qualitative, expert evaluation methods [
        <xref ref-type="bibr" rid="ref2 ref4 ref5 ref6 ref7 ref8">2, 4-8</xref>
        ], in particular, group expert evaluation
methods [
        <xref ref-type="bibr" rid="ref10 ref9">9, 10</xref>
        ], are used to determine PCI.
      </p>
      <p>
        Pairwise comparison methods using verbal scales of varying detail are practical and
highly reliable, especially when conducting group examinations [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ]. It is advisable to apply
methods with feedback from experts [
        <xref ref-type="bibr" rid="ref12">12</xref>
        ] and with mandatory consideration of their
competence in the issue under consideration [
        <xref ref-type="bibr" rid="ref13 ref2">2, 13</xref>
        ], which allows for achieving a sufficient
level of consistency of expert assessments for the legitimacy of their further aggregation [2,
      </p>
      <p>Before applying expert evaluation methods to determine PCI in each group of mutually
compatible goals, the formulation of sub-goals with a negative impact is replaced by a
formulation with a logical negation.
3.3.</p>
    </sec>
    <sec id="sec-8">
      <title>Group Decomposition </title>
      <p>According to systems thinking, scientific modeling is used to decompose complex
problems into less complex ones. It is followed by analyzing these smaller problems and
synthesizing a solution to the general problem based on the solutions of partial problems. In
the goal-oriented approach, the main goal of the problem, which is a strategic goal, is
decomposed.</p>
    </sec>
    <sec id="sec-9">
      <title>3.3.1. Decomposition Principles in Modeling  </title>
      <p>Any modeling involves simplification and neglecting some properties of the modeled
object. In the goal-oriented approach, this simplification does not consider insignificant
connections between the components of the system. When decomposing a specific goal, only
goals that cause a significant impact on the achievement of the decomposed goal are
considered.</p>
      <p>
        What influences should be considered significant so that they must be included in a model?
The rule is that influences are considered negligible if the relative value of achieving a specific
goal does not exceed 10% of the total influence. By following this rule, requirements for the
reliability of results will be satisfied. The weights of compared alternatives in a pair must
belong to the same order of magnitude [
        <xref ref-type="bibr" rid="ref15">15</xref>
        ], and the number of alternatives should not exceed
7±2 [
        <xref ref-type="bibr" rid="ref16">16</xref>
        ].
      </p>
    </sec>
    <sec id="sec-10">
      <title>3.3.2. Group Decomposition Technology </title>
      <p>Since one person, however qualified, can only endow some of the knowledge, it is
advisable to use expert groups when building models.</p>
      <p>Group decomposition is a part of a model-building process. It allows a group of people
endowed with knowledge to decompose a specific goal and coordinate their ideas on the
necessary conditions for its achievement. Knowledge engineers usually decide to perform
group decomposition if there is a need for more knowledge on goal achievement conditions.</p>
      <p>Knowledge engineers manage the decomposition process. They form an expert group,
knowledgeable in the subject domain. Group decomposition consists of the following main
stages:
– An expert defines a list of sub-goals. At this stage, each of the involved experts defines a
list of goals directly affecting the decomposed goal. This list includes goals (sub-goals) that
significantly influence the goal. The expert first analyzes the list of currently available goals.
Then they define and add to the list the remaining goals (that cause a significant impact on the
decomposing goal). Knowledge engineer terminates this process when a sufficient number of
experts have defined their lists of sub-goals.
– Lists of goals are formulated under the knowledge engineer's supervision. This stage is
semi-automatic: the knowledge engineer uses Natural Language Processing tools. As a result
of this stage, there are lists of goals with identical content combined into groups.
– Group consensus on decomposition. Experts are asked to choose the best wording from each
group. The "None" option is added to the list of goals. When selecting the "None" option, the
expert chooses not to include a certain sub-goal in the hierarchy. When the expert consensus is
reached, the knowledge engineer terminates the process. Then the experts' selections are
aggregated. A list of sub-goals with influences on the goal is the result of this stage.</p>
      <p>The resulting decomposition is presented as a subgraph of the goals hierarchy graph.
3.4.</p>
    </sec>
    <sec id="sec-11">
      <title>Structure of the Subject Domain Model </title>
      <p>Consecutive decompositions form the subject domain model's structure under a knowledge
engineer's supervision. This process begins with the decomposition of the main strategic goal
and continues with the decomposition of sub-goals until no goals are left for decomposition.
Non-decomposable goals become projects.</p>
      <p>When the model is built, the hierarchy of goals is presented to the knowledge engineer as
a graph (Figure 2).</p>
      <p>Figure 2: Screenshot of goal hierarchy in "Consensus‐2" software 
3.5.</p>
    </sec>
    <sec id="sec-12">
      <title>Model Parameters Selection </title>
      <p>After the model structure is built, it is necessary to set its parameters. The knowledge
engineer sets some parameters since their values are known. The rest of the parameters are set
by an expert group.</p>
      <p>The knowledge engineer determines if the goal is quantitative or qualitative, linear or
threshold; they also determine the compatibility of sub-goals and whether influences are
positive or negative. An expert group determines PCIs and time delays.</p>
      <p>
        "Consensus 2" is a web-oriented software for distributed group expert sessions that
implements the technology of group construction of the domain model [
        <xref ref-type="bibr" rid="ref17">17</xref>
        ]. When the subject
domain model is built, it is possible to solve many problems of decision support, forecasting and
analytics. The decision support system "Solon 3" can solve such complex tasks, including
constructing strategic plans [
        <xref ref-type="bibr" rid="ref18">18</xref>
        ].
4. The Method of Goal Dynamic Evaluation of Alternatives 
      </p>
      <p>
        The Method of Goal Dynamic Evaluation of Alternatives (MGDEA) was developed to
evaluate alternative solutions based on a goal-oriented hierarchical model [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]. It was improved to
evaluate alternatives in long-term planning [
        <xref ref-type="bibr" rid="ref20">20</xref>
        ]. MGDEA is primarily used to evaluate alternative
solutions, options, and projects in decision support systems (DSSs). Evaluation is based on the
subject domain model made by an expert.
      </p>
      <p>
        Unlike other methods (for example, multiple criteria methods [
        <xref ref-type="bibr" rid="ref21">21</xref>
        ] that use corresponding
optimization methods [
        <xref ref-type="bibr" rid="ref22">22</xref>
        ]), MGDEA allows the evaluation of heterogeneous projects without a
other [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ]. Then  
following expression:
      </p>
      <p>single set of criteria. Moreover, MGDEA does not require an expert to solve the whole problem. It
allows a group of experts to contribute to the modeling process. MGDEA can be seen as one of the
fundamental methods for expert decision-making support.</p>
      <p>MGDEA is a generalized procedure for determining the degree of achievement of a goal in a
hierarchy at a given time t. When determining the degree of achievement of a specific goal, it is
necessary to analyze its sub-goals for each alternative subset of sub-goals compatible with each
(the degree of achievement of the i-th goal at time t) is defined by the
⎧
⎪
⎪
⎨
⎪
⎪
⎩

0,
 ,
1,
 ,
if  
if
if
if
1 
 
 

1 




1
,
where  
  ;  is the threshold for achieving the i-th goal; 

function of achievement degree of the i-th goal at time t; 
is the PCI of the j-th goal in the
kth group of compatible goals, which has a negative influence on the i-th goal.</p>
      <p>Decision variant rating of the l-th goal of hierarchy at time t is a result of subtracting the main
for comparison 
goal's achievement degree 

1 ,  ∈ ,  

when all goals are achieved within the decision variants intended
. . 
from 

1 ,  ∈ \
 ,  
0 . Rating of
alternative (decision variant) is the result of subtracting the main goal's achievement degree with
the influence of this alternative on the main goal from the main goal's achievement degree without
the influence of this alternative.</p>
      <p>It was proposed to improve the method to calculate the rating of alternatives to achieve the
main goal as well as any chosen goal to expand the possibilities of applying MGDEA.</p>
      <p>The process of calculating</p>
      <p>
        (the chosen i-th goal's achievement degree at time t) consists
of the following steps. In the hierarchy of goals graph, there is a search for goals that do not affect
other goals of this hierarchy — the set of vertices that do not include any arc of the graph.
Calculations of goals' achievement degrees start from this set of goals. The initial values of the
goals' achievement degree from the set are assigned equal to 1 or 0 (although it is possible to use
values from [
        <xref ref-type="bibr" rid="ref1">0,1</xref>
        ] interval if a project is incomplete at a given time).
      </p>
      <p>
        In general, the graph may not have any vertices that do not include any arc. Although this is
unlikely and such a case was not considered in [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ], the "Other factors" goal should be included in
the hierarchy. This goal affects all those goals that could not be achieved by the set available in the
hierarchy. If this recommendation is followed, the initial set of goals will not be empty because it
will include the "Other factors" goal.
      </p>
      <p>Subsequently, a set of goals is created that can be achieved directly with the goals from the set
created in the previous step. All the goals directly influenced by the goals from the previous set are
included in the new set. This set may also include goals from the previous set.</p>
      <p>The achievement degree at time t is determined for each goal of the created set. When
determining the goals' achievement degrees, there is a propagation through the hierarchy graph
from goals of the lower level to goals of the upper levels and, finally, to the main goal. Suppose the
graph has feedback loops (arcs heading from the vertices of higher levels to the vertices of lower
levels). In that case, the iterative process that determines the goals' achievement degree is
3
is a
moment for calculating the goals' achievement degree:

,</p>
      <p>is the value of the delays of the goals' influences in the hierarchy, which contains n goals. It is
proposed to move from lower to upper-level goals, calculating all possible delays of the goals'
influences in the hierarchy. The process is organized simultaneously with determining the main
goals' achievement degree and continues until condition four is met. The list of goals' influence
delays is formed together with the calculation of the mentioned recommended planning period,
which corresponds to the maximum influence delay.</p>
      <p>.</p>
      <p />
      <p>terminated when the modulus of the difference between the calculated values of the selected goal
achievement degree during iterations (x) and (x+1) is not greater than the specified accuracy  :</p>
      <p>The specified accuracy  and planning period are set up as input parameters. Based on the task
that is solved with the DSS, the minimum unit of time is one day. By default, the form suggests a
recommended planning period. This period is calculated from the hierarchy graph heading from
the lower to the upper level (similar to calculating the goals' achievement degrees). The sum of the
propagation delays affects the formation of the maximum period duration. Beyond this period, the
results of calculating the relative project ratings no longer occur.</p>
      <p>
        MGDEA allows calculating the relative ratings of projects at any time point from the start of
their implementation. Since the calculated values of the ratings change only in the so-called
reference points of the time axis, these points are proposed to be determined in advance (and not
before each iteration). In contrast to the iterative method proposed in [
        <xref ref-type="bibr" rid="ref19">19</xref>
        ] for determining the next
      </p>
    </sec>
    <sec id="sec-13">
      <title>5. Resource Allocation </title>
      <p>At the final stage of building strategic plans, it is essential to determine a list of projects with
their financial support, which will make it possible to achieve the strategic goal in a given time
interval with known financial limitations for this period. MGDEA allows calculating the main
goal's achievement degree at a certain point in time, based on the model of the subject domain,
considering the degree of projects' implementation. So, the problem statement is as follows:
1. A set of projects</p>
      <p>,  1, 
2. For each project  , the dependence function 
implementation 
on financing</p>
      <p>(the function is shown in Fig. 1).
3. The algorithm for calculating the main goal's achievement degree, which corresponds to
of the degree of the project's
the vector  of degrees of project implementation: 
Find: vector  , when 
program funding.</p>
      <p>→ 
, and ∑

.
, where 
is the total volume of</p>
      <p>
        This solution is proposed in [
        <xref ref-type="bibr" rid="ref23">23</xref>
        ]. Optimal resource allocation problems are usually solved using
various optimization methods, such as mathematical programming, but this problem has specific
features:
      </p>
      <p>When dealing with a weakly structured system model, the goal function cannot be
presented analytically; there is only an algorithmic representation (for example, calculating
the main goal's achievement degree).</p>
      <p>Since input data for building the model consists of subjective expert assessments (that are
not strict and accurate enough), the requirements for the accuracy of resource allocation are
also low. In other words, it is enough to have a non-optimal yet good enough rational
version of the solution.</p>
      <p>From the practical point of view, it is advisable to move to a discrete domain from a continuous
scale. For this purpose, it is proposed to specify the resource allocation accuracy as input data. It
represents a discretization unit of the financial resource.</p>
      <p>Evolutionary algorithms (a subset of evolutionary computations) are essentially variants of
targeted random search. Therefore, evolutionary algorithms can solve problems with mentioned
features.</p>
      <p>
        It is proposed to use a genetic algorithm (GA) modification, first proposed by Holland [
        <xref ref-type="bibr" rid="ref24">24</xref>
        ].
GA belongs to the larger class of evolutionary algorithms. It is an algorithm that is used to find a
solution to complex problems by sequentially selecting and combining the desired parameters
using mechanisms that are based on biological evolution.
      </p>
      <p>GA uses a set of individuals (population), which are basically strings, and each of that strings
encodes one of the solutions. Therefore, GA differs from most optimization algorithms, which can
only operate with one solution variant at a specific time. Among all individuals in the population,
the following are selected using the fitness function:
- Fitter solutions (solutions that have the maximum fitness function values) that can generate
new offspring.
- Bad solutions (with low fitness) are removed.</p>
      <p>Thus, the adaptability of the new generation is, on average, higher than the previous one.</p>
      <p>One of GA's main advantages is its versatility. Fitness function and solutions' coding are the
only parameters that depend on a specific problem (other steps are performed similarly). Therefore,
these parameters should be the main focus of the resource allocation problem.</p>
      <p>The function of the main goal's achievement degree at given levels of project implementation
is used as a fitness function. This function has already been implemented and used in many DSSs,
so there is no need to reinvent it. It can be used as a fitness function of the GA.</p>
      <p>The resource specified for further distribution between projects is subject to discretization
(division into elementary (indivisible) parts). The possible solution to this problem is a vector with
the number of elementary parts of the resource allocated to a specific project as the vector's
elements.</p>
      <p>It is necessary to pre-calculate the implementation degree of each project with the specified
funding to calculate the fitness of individuals (the main goal's achievement degree). Therefore,
finding the project implementation degree corresponding to the decision vector element is
necessary. Funding parameters for each project are pre-entered by business plan authors.</p>
      <p>It is vital to choose the GA's operators correctly. The following GA operators were used for
this implementation: tournament selection with a set of two individuals, one-point crossover,
mutation, and elitism.</p>
      <p>It is proposed to use the following experimentally chosen input parameters by default:
- 50 individuals in the population,
- 0.05 mutation probability,
- 50 generations with the same result as a stop criterion.</p>
      <p>It is possible to change these parameters by selecting more suitable ones to obtain a result for
a given model effectively.</p>
      <p>So, in this research, the problem of rational distribution of limited resources between projects
was solved. The work results were confirmed with the correct parameters' settings and matched
with the brute force search results. Verification was performed using samples with a limited
number of projects and a small number of given elementary units out of the total amount of
resources.</p>
    </sec>
    <sec id="sec-14">
      <title>6. Conclusions </title>
      <p>Theoretical foundations and methods were proposed for the reliable acquisition and use of
collective knowledge in different areas, which made it possible to develop the knowledge
transfer platform toolkit for strategic planning in various domains.</p>
      <p>The strategy, in this context, is a list of selected measures to achieve the system's main goal
in a specified time using limited resources.</p>
    </sec>
    <sec id="sec-15">
      <title>References </title>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Saaty</surname>
            ,
            <given-names>Thomas L.</given-names>
          </string-name>
          (
          <year>2009</year>
          ).
          <article-title>Mathematical Principles of Decision Making (Principia Mathematica Decernendi)</article-title>
          .
          <article-title>Comprehensive coverage of the AHP, its successor the ANP, and further developments of their underlying concepts</article-title>
          ,
          <source>Pittsburgh.</source>
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Totsenko</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2002</year>
          ).
          <article-title>Methods and Systems for Decision-making Support. Algorithmic aspect</article-title>
          .
          <source>Naukova dumka, Kyiv</source>
          . (in Russian)
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Mintzberg</surname>
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Quinn</surname>
            <given-names>J. B.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Ghoshal</surname>
            <given-names>S.</given-names>
          </string-name>
          (
          <year>1998</year>
          ).
          <article-title>The strategy process (Rev</article-title>
          . European).
          <source>Prentice Hall.</source>
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Saaty</surname>
            ,
            <given-names>T.</given-names>
          </string-name>
          (
          <year>1996</year>
          ).
          <article-title>Decision Making with Dependence and Feedback: The analytic Network Process</article-title>
          .
          <source>RWS Publications</source>
          , Pittsburgh.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Bozóki</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fülöp</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Rónyai</surname>
            ,
            <given-names>L.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>On Optimal Completion of Incomplete Pairwise Comparison Matrices</article-title>
          .
          <source>Mathematical and Computer Modelling</source>
          .
          <volume>52</volume>
          (
          <issue>1-2</issue>
          ),
          <fpage>318</fpage>
          -
          <lpage>333</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Hnatiienko</surname>
            ,
            <given-names>H.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Snytyuk</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Suprun</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Application of decision-making methods for evaluation of complex information system functioning quality</article-title>
          <source>CEUR Workshop Proceedings</source>
          ,
          <volume>2318</volume>
          ,
          <fpage>25</fpage>
          -
          <lpage>34</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Kadenko</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Szadoczki</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Bozoki</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          (
          <year>2021</year>
          ).
          <article-title>An update on combinatorial method for aggregation of expert judgments in AHP</article-title>
          . Production,
          <volume>31</volume>
          ,
          <fpage>1</fpage>
          -
          <lpage>17</lpage>
          . DOI:
          <volume>10</volume>
          .1590/
          <fpage>0103</fpage>
          -
          <lpage>6513</lpage>
          .
          <fpage>20210045</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Szádoczki</surname>
            ,
            <given-names>Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Bozóki</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Juhász</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kadenko</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2022</year>
          ).
          <article-title>Incomplete pairwise comparison matrices based on graphs with average degree approximately 3</article-title>
          .
          <source>Annals of Operations Research</source>
          , DOI: 10.1007/s10479-022-04819-9.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Zgurovsky</surname>
            ,
            <given-names>M.Z.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Totsenko</surname>
            ,
            <given-names>V.G.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.V.</given-names>
          </string-name>
          (
          <year>2004</year>
          ).
          <article-title>Group incomplete paired comparisons with account of expert competence</article-title>
          .
          <source>Mathematical and Computer Modelling</source>
          ,
          <volume>39</volume>
          (
          <issue>4-5</issue>
          ),
          <fpage>349</fpage>
          -
          <lpage>361</lpage>
          . DOI:
          <volume>10</volume>
          .1016/S0895-
          <volume>7177</volume>
          (
          <issue>04</issue>
          )
          <fpage>90511</fpage>
          -
          <lpage>0</lpage>
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kadenko</surname>
            ,
            <given-names>S.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Andriichuk</surname>
            ,
            <given-names>O.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Roik</surname>
            ,
            <given-names>P.</given-names>
          </string-name>
          (
          <year>2018</year>
          ).
          <article-title>Combinatorial Method for Aggregation of Incomplete Group Judgments</article-title>
          .
          <source>In: Proceedings of 2018 IEEE First International Conference on System Analysis &amp; Intelligent Computing (SAIC)</source>
          , Kyiv, Ukraine,
          <fpage>25</fpage>
          -
          <lpage>30</lpage>
          . DOI:
          <volume>10</volume>
          .1109/SAIC.
          <year>2018</year>
          .8516768
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kadenko</surname>
            ,
            <given-names>S.V.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Andriichuk</surname>
            ,
            <given-names>O.V.</given-names>
          </string-name>
          (
          <year>2015</year>
          ).
          <article-title>Using different pair-wise comparison scales for developing industrial strategies</article-title>
          .
          <source>International Journal of Management and Decision Making</source>
          ,
          <volume>14</volume>
          (
          <issue>3</issue>
          ),
          <fpage>224</fpage>
          -
          <lpage>250</lpage>
          . DOI:
          <volume>10</volume>
          .1504/IJMDM.
          <year>2015</year>
          .070760
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <surname>Totsenko</surname>
            ,
            <given-names>V.G.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.V.</given-names>
          </string-name>
          (
          <year>1999</year>
          ).
          <article-title>Method of paired comparisons using feedback with expert</article-title>
          .
          <source>Journal of Automation and Information Sciences</source>
          ,
          <volume>31</volume>
          (
          <issue>7-9</issue>
          ),
          <fpage>86</fpage>
          -
          <lpage>96</lpage>
          . DOI:
          <volume>10</volume>
          .1615/JAutomatInfScien.v31.
          <source>i7-9</source>
          .
          <fpage>480</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <surname>Astakhov</surname>
            ,
            <given-names>A.K.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.V.</given-names>
          </string-name>
          (
          <year>2022</year>
          ).
          <article-title>Technology for Determining the Competence of Experts during Knowledge Transfer. / Data registration</article-title>
          ,
          <source>storage and processing: coll. of science papers based on the materials of the Annual Final Scientific Conference on September 27-28</source>
          ,
          <year>2022</year>
          / National Academy of Sciences of Ukraine. Institute for Information Recording. - Kyiv: IPRI,
          <fpage>77</fpage>
          -
          <lpage>78</lpage>
          . (in Ukrainian)
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2013</year>
          ).
          <article-title>Providing sufficient strict individual rankings' consistency level while group decision-making with feedback</article-title>
          .
          <source>Journal of Modelling in Management</source>
          ,
          <volume>8</volume>
          (
          <issue>3</issue>
          ),
          <fpage>339</fpage>
          -
          <lpage>347</lpage>
          . DOI:
          <volume>10</volume>
          .1108/JM2-11-2010-0050
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <surname>Saaty</surname>
            ,
            <given-names>T.L.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Shang</surname>
            ,
            <given-names>J.S.</given-names>
          </string-name>
          (
          <year>2011</year>
          ).
          <article-title>An innovative orders-of-magnitude approach to AHPbased mutli-criteria decision making: Prioritizing divergent intangible humane acts</article-title>
          .
          <source>European Journal of Operational Research</source>
          ,
          <volume>214</volume>
          (
          <issue>3</issue>
          ),
          <fpage>703</fpage>
          -
          <lpage>715</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <surname>Miller</surname>
            ,
            <given-names>G.A.</given-names>
          </string-name>
          (
          <year>1956</year>
          ).
          <article-title>The magical number seven, plus or minus two: some limits on our capacity for processing information</article-title>
          .
          <source>Psychological Review</source>
          ,
          <volume>63</volume>
          (
          <issue>2</issue>
          ),
          <fpage>81</fpage>
          -
          <lpage>97</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17] Consensus-2
          <string-name>
            <surname>Certificate</surname>
          </string-name>
          (
          <year>2017</year>
          ).
          <article-title>Certificate of copyright registration for work № 75023. Computer program "A system for distributed collection and processing of expert information for decision support systems - "Consensus-2""</article-title>
          , Ukraine, registered on 27/11/2017. Accessed: http://www.dss-lab.org.ua/Consensus-2.htm.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18] Solon-3
          <string-name>
            <surname>Certificate</surname>
          </string-name>
          (
          <year>2003</year>
          ).
          <article-title>Certificate of state registration of copyright to work № 8669. Computer program "Solon-3 decision support system" (Solon-3 DSS)</article-title>
          ,
          <source>Ministry of Education and Science of Ukraine State Department of Intellectual Property, registered on 31/10/2003</source>
          . Accessed: http://www.dss-lab.org.ua/Solon-3.htm.
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>Totsenko</surname>
            ,
            <given-names>V.G.</given-names>
          </string-name>
          (
          <year>2001</year>
          ).
          <article-title>On One Approach to the Decision Making Support while Planning Research and Development. Part II. The Method of Goal Dynamic Evaluation of Alternatives</article-title>
          .
          <source>Journal of Automation and Information Sciences</source>
          .
          <volume>33</volume>
          (
          <issue>4</issue>
          ),
          <fpage>82</fpage>
          -
          <lpage>90</lpage>
          . DOI:
          <volume>10</volume>
          .1615/JAutomatInfScien.v33.
          <year>i4</year>
          .
          <fpage>100</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2013</year>
          ).
          <article-title>Improvement of the method of goal dynamic evaluation of alternatives and features of its application</article-title>
          .
          <source>Data Recording, Storage &amp; Processing</source>
          .
          <volume>15</volume>
          (
          <issue>1</issue>
          ),
          <fpage>90</fpage>
          -
          <lpage>99</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref21">
        <mixed-citation>
          [21]
          <string-name>
            <surname>Figueira</surname>
            ,
            <given-names>J.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Salvatore</surname>
            ,
            <given-names>G.</given-names>
          </string-name>
          , &amp;
          <string-name>
            <surname>Ehrgott</surname>
            ,
            <given-names>M.</given-names>
          </string-name>
          (
          <year>2005</year>
          ).
          <article-title>Multiple Criteria Decision Analysis: State of the Art Surveys</article-title>
          ; New York: Springer, 1048 p.
        </mixed-citation>
      </ref>
      <ref id="ref22">
        <mixed-citation>
          [22]
          <string-name>
            <surname>Steuer</surname>
            ,
            <given-names>R.</given-names>
          </string-name>
          (
          <year>1992</year>
          )
          <article-title>Multicriteria optimization</article-title>
          .
          <article-title>Theory, calculations and applications</article-title>
          . M.: Radio and communication, 504 p.
        </mixed-citation>
      </ref>
      <ref id="ref23">
        <mixed-citation>
          [23]
          <string-name>
            <surname>Tsyganok</surname>
            ,
            <given-names>V.</given-names>
          </string-name>
          (
          <year>2010</year>
          ).
          <article-title>The problem of resource allocation as an expansion of the capabilities of the decision support system</article-title>
          .
          <source>Data Recording, Storage &amp; Processing</source>
          .
          <volume>12</volume>
          (
          <issue>2</issue>
          ),
          <fpage>232</fpage>
          -
          <lpage>237</lpage>
          . (in Ukrainian)
        </mixed-citation>
      </ref>
      <ref id="ref24">
        <mixed-citation>
          [24]
          <string-name>
            <surname>Holland</surname>
            ,
            <given-names>J. H.</given-names>
          </string-name>
          (
          <year>1992</year>
          ).
          <article-title>Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control,</article-title>
          and
          <string-name>
            <given-names>Artificial</given-names>
            <surname>Intelligence</surname>
          </string-name>
          . The MIT Press, DOI: 10.7551/mitpress/1090.001.0001.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>