<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta>
      <journal-title-group>
        <journal-title>November</journal-title>
      </journal-title-group>
    </journal-meta>
    <article-meta>
      <title-group>
        <article-title>Semantic Approach to Decision Making in Comparison of  Complex Objects </article-title>
      </title-group>
      <contrib-group>
        <contrib contrib-type="author">
          <string-name>Julia Rogushina</string-name>
          <xref ref-type="aff" rid="aff1">1</xref>
        </contrib>
        <contrib contrib-type="author">
          <string-name>Anatoly Gladun</string-name>
          <xref ref-type="aff" rid="aff2">2</xref>
        </contrib>
        <aff id="aff0">
          <label>0</label>
          <institution>, Ave Glushkov</institution>
          ,
          <addr-line>Kyiv, 03680 GSP</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Institute of Software Systems of the National Academy of Sciences of Ukraine</institution>
          ,
          <addr-line>40, Ave Glushkov, Kyiv, 03181</addr-line>
          ,
          <country country="UA">Ukraine</country>
        </aff>
        <aff id="aff2">
          <label>2</label>
          <institution>International Research and Training Center for Information Technologies and Systems under NAS and MES of Ukraine</institution>
        </aff>
      </contrib-group>
      <pub-date>
        <year>2022</year>
      </pub-date>
      <volume>16</volume>
      <issue>2022</issue>
      <fpage>0000</fpage>
      <lpage>0001</lpage>
      <abstract>
        <p>   We analyze models and methods that can be used to compare complex information objects as components of decision-making in intelligent information systems. The proposed approach is based on the use of knowledge acquired from domain ontologies and metrics to determine the semantic proximity of ontology components. We propose an algorithm for the semantic comparison of objects with a similar structure defined by the common ontology. This allows for the generation of a finite set of comparison criteria that depend on the definition of the user task and domain ontology. The significance of all criteria for the current situation is evaluated dynamically by expert statements about their hierarchy. The proposed method considers an integrated approach to multi-criteria decision-making in conditions where the set of criteria itself is determined by the requirements of the task and domain knowledge, but at different points of time the relative importance of particular criteria can change dynamically. Practical use of proposed approach is demonstrated on task of the formation of unmanned aerial vehicles groups.</p>
      </abstract>
      <kwd-group>
        <kwd> 1  Complex information object</kwd>
        <kwd>decision-making</kwd>
        <kwd>ontology</kwd>
        <kwd>semantic similarity</kwd>
        <kwd>multi-level hierarchical structure</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction </title>
      <p>One of the key issues in the development of complex systems is to increase the efficiency of
decision-making in problematic situations, such as risk management, management of complex
projects and the allocation of human resources. Decision Support Systems (DSSs) are a specific
subclass of intelligent information systems (IISs) that help specialists to form and choose the right
alternative among a set of acceptable options for making responsible decisions, and often combine
mathematical approaches of decision-making with logical and linguistic models that use
knowledgebased methods and heuristics acquires from accumulated experience of domain experts.</p>
      <p>In such systems, decision-making problems concern the selection of the most informative features
used to compare potential solutions. Processing of complex solutions with big number of connected
sub-elements is complicated by the need to analyze a significant number of parameters of these
objects contained into potential solutions, and the necessity of unification of their structure. In an
open environment, the lack of time for processing of information and its rapid changes limits the
possibility of using traditional decision-making methods that ensure finding the optimal solution. One
of the directions for solving this problem is the application of elements of artificial intelligence (AI)
and use of domain knowledge from external sources.</p>
      <p>
        Currently, IISs implement knowledge based on ontological analysis [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ], which is a special case of
semantic analysis [
        <xref ref-type="bibr" rid="ref2">2</xref>
        ]. Ontologies provide interoperability and unequivocal interpretation of the
domain knowledge, and therefore they can be considered as external sources based on ontological
knowledge representation of information for different IIS, including the decision support subsystems.
      </p>
      <p>
        Decision-making in an open environment requires special means to receive up-to-date information
about changes in the environment, including both the objects of the decision and the criteria that
influence the decision-making process. In many IISs a decision is a structured set of elements
(objects), and these elements have their own structure and can be connected to other elements by
various semantic relations. Decision requirements deal not only with such elements, but also with the
relations between them. In this case, we have to consider such decisions as complex information
objects (CIO) [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ].
      </p>
      <p>Each CIO is represented as an ordered collection of more than one information object (IO) that
are related to each other by relations with defined semantics and meet the requirements for the
structure and property values of the IOs defined by the user's task. In each specific case, the CIO
structure is determined by the domain knowledge and the specifics of the task. The use of ontologies
has become a standard for formalizing domain knowledge in Web-oriented IISs. Therefore, we apply
the elements of ontological analysis to describe the model and methods for comparing CIOs.</p>
      <p>From the point of view of ontological analysis, any IO is a class instance of domain ontology.
Corresponding class of ontology characterizes the IO structure as a set of properties and their
characteristics, as well as acceptable relations with other class instances of this ontology. The class
instance of ontology associated with an IO also defines values (all or some) of these properties, and
some of these properties can define relations with other ontology instances.</p>
      <p>Therefore, every CIO can be considered a subset of the domain ontology that is distinguished
according to the user's task.</p>
      <p>The CIO structure is a set of ontology classes and the relations between them that define the
structure of available CIOs and restrictions on the available values of class instances, but it does not
contain exact instances.</p>
      <p>
        For example, CIOs that can be generated based on the organizational ontology of a research
institute include the team and equipment used to carry out a research project or hold a scientific
conference. In this case, decision-making requires the selection of such subset of the institute
personnel that can effectively implement the project and has all necessary competencies. Examples of
CIOs based on the ontology of an educational institution include the choice of a specialty or the
construction of a curriculum to obtain a certain set of competencies. Decision-making in this case is
related to obtaining the necessary competencies in the shortest time or with the greatest completeness
that can be provided by the selected institution [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ].
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Comparison of complex information objects </title>
      <p>In this work we do not consider the problems of decision generation as a sequence of certain
actions in a dynamic information environment, but concentrate on other important component of
decision-making that deals with comparison of possible solutions represented as CIOs. Our main
focus is on conducting a multi-criteria comparison of these CIOs and defining the sources and
estimates of these criteria.</p>
      <p>A specific feature of our proposed approach is that we only need to compare a relatively small
number of CIOs: we analyze not all theoretically possible CIOs that can fit within CIO structure, but
only those ones that can be constructed from existing IOs in the current situation.</p>
      <p>Therefore, the problem is not finding the optimal (according to certain criteria) solution, but rather
choosing an acceptable solution from the set of available options. For example, to carry out a research
project, you need to select a group of employees from a specific department, rather than from all
individuals in general. Therefore for some situations all possible solutions are unsatisfactory and can
change this situation only for the worse. For example, the execution of project by insufficiently
competent employees can lead to a loss of time and resources without obtaining of the desired result.
At the same time, the significance of the criteria can be changed over time due to changes in the
dynamic information environment, and these changes can cause obtaining of the available result on
base of the same IOs. The most common example of changing priorities is the cost of work and the
speed of obtaining results. For example, the use of some energy sources (such as diesel generators) is
not acceptable for routine activity by ecological criterion, but people can use them during blackout
because health reasons and safety criteria become more significant.</p>
      <p>Comparison of CIO has to provide justification for information search at the meaningful level.
Semantic search differs from traditional one by use domain concepts and relations instead of
keywords to describe user needs. CIO structure can be considered as such requests. Results of
semantic search can be represented not only by list of documents or other IOs but also as structured
sets of such IOs that satisfy requirements for components of this structure. Examples of semantic
search: to find a group of people with certain qualifications working in the same organization that
meets the conditions of the competition; to determine the countries where scientific research on a
certain topic was carry out with publishing of the results in a selected set of journals for a certain
period of time.</p>
      <p>While semantic search, usually, results in a set of IOs of one class (for example, people or goods)
or a group of classes with some similar properties (for example, research and educational
organizations), where search requirements are used only for selection of acceptable options of IO
individuals, CIO comparison is a more complex problem where every result is represented by the set
of IOs of different types linked by specified relations. For example, for some task we need a specified
set of equipment, qualified personal for operating this equipment, appropriate means for
transportation of this equipment and personal to the desired place in a certain time.</p>
      <p>
        Some CIOs can contain several levels of the hierarchy of IOs. For these tasks, we need to generate
a criteria hierarchy that determines what criteria should be used for each level of CIO components,
and which of them are applicable to IOs of all levels. An example of such task is Hierarchical
Aggregate Assessment (HAA) that provides a comparison of teams with several levels of the
hierarchy by analyzing individual and collective test results [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. In HAA, the evaluation process is
based on the use of specific tests for each level of the team hierarchy, where each member has an
individual hierarchical position. The process of grouping team members into different hierarchical
levels and defining their positions within the hierarchy is called aggregation. Therefore, this problem
is referred to as aggregated evaluation of the team.
      </p>
      <p>HAA provides assessments of IOs of various levels simultaneously in one assessment session, by
interpreting test results differently for each level.</p>
    </sec>
    <sec id="sec-3">
      <title>3. Stages of CIO comparison </title>
      <p>In the general case, the task of comparing CIOs that have different structures and are based on
different ontologies consists of three subtasks:
 alignment of base ontologies and generation of some common ontology;
 analysis of similarities between their structural elements and retrieval of sub-CIOs that can
be matched;
 matching of retrieved sub-CIOs with similar structure based on common ontology.</p>
      <p>
        In this work, we consider the last subtask where all compared CIOs are based on a single ontology
and have a similar structure. This subtask requires the following stages:
 creation of the CIO reference model of that reflects user requirements (CIO structure);
 generation of a set of existing CIOs that structurally correspond to the reference model,
based on information about the current state of the environment and semantic similarity
between CIO elements [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ];
 choosing among the available CIOs those ones that do not contradict the user's
requirements;
 search for CIO evaluation criteria with use of knowledge from the domain ontology;
 determination of the level of individual significance of each criterion at the current
moment by the comparison of expert evaluations and domain heuristics with use of the
analytic hierarchy process;
 determination of the quantitative assessment of each CIO based on the selected set of
criteria and coefficients of their significance.
      </p>
      <p>
        Comparing CIOs can be seen as a special case of a multi-criteria decision-making (MCDM)
problem in recommendation systems [
        <xref ref-type="bibr" rid="ref6">6</xref>
        ].
      </p>
      <p>
        The main steps of the MCDM methodology proposed by Roy [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ] can be adapted for the specifics
of CIO comparison:
 Defining the decision object involves identifying the CIO structure, which is represented
as a set of IOs with relations between them and individuals who require this structure for
decision-making purposes.
 Defining a consistent set of criteria involves identifying and specifying a set of functions
based on CIO characteristics that represent user preferences among various alternatives.
 Developing a global preference model involves determining the relative significance of
partial preferences to define general evaluations of CIOs based on the set of criteria.
 Selecting the decision support process involves choosing the methods of using the selected
      </p>
      <p>CIO for a particular task (this step is not analyzed in our proposed work).
3.1. Creation  of  the  CIO  reference  model  of  that  reflects  user 
requirements  </p>
      <p>
        We propose using knowledge from the domain ontology to define the structure and properties of
CIO components that reflect user requirements for the decision in question. This structure can serve
as a reference model for evaluating available solutions. This process consists of the following steps:
 selecting the domain ontology relevant to the user's task and required solutions;
 analyzing the natural language (NL) description of the user's task and matching it with the
domain ontology through various semantic similarity evaluations [
        <xref ref-type="bibr" rid="ref8">8</xref>
        ];
 selecting a non-empty subset of ontology classes that correspond with the components of
the required solution (we can use approach proposed for generation of the task thesaurus
by its NL description [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ]);
 selecting a non-empty subset of ontology relations that link the components of the required
solution based on task specifics (it is important that such relations have to link all IOs by
some chain of relations) according to task specifics that can be made by requests to
domain ontology or manually on base of its visual representation in ontology editor
Protégé [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ];
 describing the properties and their values for CIO components that are important for the
solution;
 fixing the CIO structure as an OWL ontology [
        <xref ref-type="bibr" rid="ref11">11</xref>
        ].
3.2.
      </p>
      <p>Generation  and  check  of  the  set  of  CIOs  that  are  relevant  to  CIO 
scheme </p>
      <p>This subtask involves using combinatorial analysis to first identify all individuals belonging to
CIO scheme classes, and then generate all possible combinations of these individuals based on the
reference model. For example, we analyze the joint publications of researchers working in the same
organization and build the CIO structure of such co-authorship. This structure consists of two
components of the class «Person», three of the class «Publications», and one of the class
«Organization». We then generate all possible 6-element sequences, where the first and second
elements are individuals of the class «Person», the third, fourth, and fifth elements are individuals of
the class «Publications», and the sixth element is an individual of the class «Organization». Place of
every individual in generated combination is significant, and combinations with the same elements for
different positions are considered as different CIOs.</p>
      <p>Each individual in the CIO is checked against the user's requirements, which deal with the values
of IO properties and relations with other components of the CIO defined by the structure. In example
proposed above we search for presence of publication for person and for coauthors of this publication
that work in the same organization. We propose sequences that satisfy all these requirements for
future analysis. The selected CIOs can be used as solutions to the user's problem, but we must choose
only one of them to be implemented. Therefore, we try to find the CIO that is most similar to the
reference model described by the CIO structure. For this purpose, we need a method that proposes
evaluations of this similarity, taking into account different criteria.</p>
      <p>3.3. Generation  of  the  CIOs  comparison  criteria  set  with  use  of 
knowledge from the domain ontology </p>
      <p>We consider comparison criteria as a means to find CIOs that are most semantically similar to the
reference model proposed by the user. This model must represent all the main aspects that are
important for the current user task. The review of methods for determining the semantic similarity
between domain concepts shows that the parameters for determining the semantic proximity between
two ontology class individuals are defined by:
 the semantic distance between their classes (defined by a subset of ontological relations
with certain characteristics, such as hierarchical and synonymous ones);
 the semantic proximity between the property values of the same attributes.</p>
      <p>Matching instances of the same class greatly simplifies the task. In such cases, the first group of
parameters can be ignored, and the second one does not require aligning the properties of different
classes by analyzing their semantics (for example, a property of the type 'Year' can characterize both
the year of birth of a person and the year of the start of education).The problem of comparing CIOs
with a different structure is more complex and requires additional stages of information gathering. In
this work, we consider the situation of matching CIOs with the same or similar structure, and
therefore alignment can be reduced to searching for sub-classes and super- classes for analyzed IOs.</p>
      <p>We propose the following algorithm for the preliminary generation of the criteria set for matching
CIOs with a similar structure:
 each IO t i , i  1, p based on the formal model (2) defines a set of its properties K0i divided
in data properties T _ data0i  {t _ dataim , m  0, yi} (their values of ontology class
attributes t _ d im , i  1, p, m  0, yi are constants of various types – number, text, date,
etc.),
and
object
properties</p>
      <p>T _ o 0i  {t _ oik , m  0, z k }
(their
values
t _ oik , i  1, p, m  0, z k are individuals of different classes of ontology O that are used as
attribute
values
of
other individuals in
domain
ontology
for IO
class):
K 0i  T _ d 0  T _ o0 .
for data properties, the analysis ends here, and for object properties, if necessary, it can be
repeated recursively for each IO class that can be a property value for t i for replenishment
of the set K 0i with the corresponding properties considered as additional similarity criteria
for CIO matching;
K0 is generated by of combining the information from the sets K 0i with clear fixation of
IOs used as evaluation criteria: K 0  {(t, i), t  K 0i , i  1, p};
 The user analyzes the constructed set K 0 and can explicitly remove some criteria that
she/he considers irrelevant for current task.</p>
      <p>At this point, the work of the algorithm can be completed or continued for construction of the
criteria set K sem enriched by other classes of the domain O ontology selected with use of various
measures of semantic closeness and semantic similarity.</p>
      <p>The set Ksem is constructed as follows:
 a subset TC _ cl of classes of ontology О that contains the appropriate criterion in K 0 is



defined: TC _ cl  TC , TC _ cl  K 0 ;
a set S j of semantically close or semantically similar concepts of domain ontology
(according to the selected measure f (t a , t b ) and constant L) for each element
t j  TC _ cl , t  1, x , is defined: S j  {t  TC _ cl , f (t, t j )  L} ;
the criteria set of K semi for each set S j , j  1, x , is built by the same algorithm as sets K 0i
are built;
the sets K semi are combined into single set K sem (in the same way as K 0 is built) where is
also
clearly
fixed
what IOs
of</p>
      <p>CIO
are
used
for the
evaluation
criteria:</p>
      <p>K sem  {(t, i), t  K semi , j  1, x} .</p>
      <p>Then the user analyzes the criteria set Ksem and, if it necessary, explicitly removes from it those
criteria that she/he considers insignificant for current task. Additionally, the user can manually add
criteria that they consider important, even if they are not represented in the domain ontology or were
not included in Ksem by the algorithm proposed above. Certain criteria may be lost due to
unsuccessfully chosen ontology, inaccurate estimation of semantic closeness, or insufficient number
of algorithm iterations.</p>
      <p>Another reason of manual editing of K sem can be the user’s expert knowledge about analyzed
domain: it is easier for her/him to clearly indicate the important criteria than to look for them in the
ontology structure. But it should be assumed that quite often such expert knowledge relates only to
some particular aspects of the problem, and the use of the proposed algorithm ensures that other
elements of domain knowledge are taken into account.</p>
      <p>The next step of CIO matching involves creating a hierarchy of criteria from K sem that represents
the user's current needs and their relative importance. This hierarchy can be determined by analyzing
the semantic proximity estimates between pairs of CIOs, including those between the evaluated CIOs
and the reference CIO, which is built based on the user's task description. Values of these estimates
can be defined by user (or group of users) and external domain experts or acquired from pertinent
knowledge bases. Selection of the methods used depends on task specifics, user qualifications, and the
dynamics of user preferences.</p>
    </sec>
    <sec id="sec-4">
      <title>4. Analytic hierarchy process and ontologies </title>
      <p>
        Methods of the multi-criteria problem solving are supported by various approaches (see Figure 1).
[
        <xref ref-type="bibr" rid="ref12 ref13">12, 13</xref>
        ]. One of them is an analytic hierarchy process (AHP) that can be improved by integration
with use of external domain knowledge. Some researchers that process knowledge on base of
ontological analysis try to combine AHP with acquisition of task knowledge from domain ontologies,
but mostly they are aimed at creating an ontology that characterizes the main elements of AHP
variation or at building an ontological representation of the obtained results. The construction of such
AHP ontology ensures the construction of an ontological model of the results of the AHP work: a
glossary of basic terms (concepts and their instances, attributes, actions, etc.), a classification tree of
concepts and their binary relations.
      </p>
      <p>In proposed here approach we consider elements of domain ontology relevant to solved task that
can be used on various stages of AHP application.</p>
      <p>
        The most known method aimed at analyzing hierarchies was proposed by Saati [
        <xref ref-type="bibr" rid="ref14">14</xref>
        ]. A large
number of improvements to this method and other approaches that achieve similar goals for other
variants of the problem statement are now in use. More information with multiple levels of criteria
could potentially help to better understand user preferences. Here, we focus on the task of analyzing
hierarchies for matching CIOs, which utilizes a domain ontology as a source of knowledge about CIO
structure and the relationships between their concepts.
      </p>
      <p>The main task in the construction of the hierarchy is the assessment of higher levels that is based
on the interaction of different levels of the hierarchy, and not only on the direct dependences between
the elements at these levels. Exact methods of hierarchical construction are used in the natural and
social sciences, particularly in the problems of general systems theory related to planning and
constructing social systems.
Methods of the multi-criteria
task solving</p>
      <sec id="sec-4-1">
        <title>Interactive methods</title>
      </sec>
      <sec id="sec-4-2">
        <title>Reducing to singlecriterion methods</title>
      </sec>
      <sec id="sec-4-3">
        <title>Lexicographic optimization</title>
      </sec>
      <sec id="sec-4-4">
        <title>Method of concessions</title>
      </sec>
      <sec id="sec-4-5">
        <title>Effective set method</title>
      </sec>
      <sec id="sec-4-6">
        <title>Analytic hierarchy process</title>
      </sec>
      <sec id="sec-4-7">
        <title>Target programming method</title>
      </sec>
      <sec id="sec-4-8">
        <title>Convolution method</title>
      </sec>
      <sec id="sec-4-9">
        <title>Main criterion method</title>
        <p>Each element of a hierarchy can functionally belong to several other different hierarchies. For
example, the same person may be considered together with other employees of an organization,
education participants or patients of a hospital. The same element can belong to high level of one
hierarchy as a controlling component, but at another hierarchy simply it can be an element of lower
level. For example, a person can be the head of a certain organization as the most authoritative
specialist and occupy the highest level of this organizational hierarchy, and in his/her family the same
person occupies the middle level of the hierarchy compared to older relatives. Information on the
principles of establishing the level of an element in a certain hierarchy is contained into the domain
ontology as a set of relations of transitive relations between elements of the corresponding class.</p>
        <p>The interpretation of the main elements of AHP depends on the complex tasks it solves and the
elements of those tasks. The task of multi-criteria choice (decision-making) for some subject area that
is described by the domain ontology is an example of such task, and its interpretation has to take into
account the structure of required decision, component of this decision (its elements and relations
between them) and more general rules and restrictions of domain on collateral execution of such
decisions.</p>
        <p>A solution is an instance of a class of ontology P, for which a set of semantic properties
C1, C2 ,..., Cn is defined in this ontology. These properties can be both object properties (with values
from instances of other classes) and data properties (with quantitative, qualitative constant values of
various types). Additional information from P can be used in C1, C2 ,..., Cn by considering of
semantically similar (from the task point of view) domain concepts.</p>
        <p>C1, C2 ,..., Cn is a subset of the domain ontology concepts of the that are considered as criteria for
evaluating the compliance of instances of class P with the defined goal. Therefore values of all
properties from C1, C2 ,..., Cn should be evaluated in the range from "completely unacceptable" to
"very successful".</p>
        <p>In this case, the AHP is used to determine the hierarchy of individual decision-making criteria,
which are compared (evaluated) in pairs by one or more domain experts.</p>
        <p>Decision-making consists in selection of the most suitable solution among a limited non-empty set
X instances of class P: X  P .</p>
        <p>The set X is created based on external criteria that are outside the scope of the AHP approach, such
as purchase price, availability of access, or time interval of use. It should be taken into account that in
most practical problems of CIO matching set X is a very small subset of P, and therefore there is no
need to search for generalized rules for classifying instances of X by AHP categories “completely
unacceptable”, “acceptable”, “very successful”, etc., as is done in various methods of machine
learning (ML).</p>
        <p>Rather than a general classification, we aim to solve a problem with lower computational
complexity by using a traductive rule “from the individual to the individual” instead of an inductive
rule “from the individual to the general”.</p>
        <p>In addition, unlike MN, where instances of P are classified according to an arbitrary set of
categories without formalized semantic relations between them (for example, a recommended
position or a probable diagnosis), instances of X in AHP belong to an ordered set of categories with a
fixed number of values and can be connected by various relations. The main differences between ML
and AHP are represented in the table 1.</p>
        <p>Table 1 </p>
        <sec id="sec-4-9-1">
          <title>Comparison of the main characteristics of AHP and MN </title>
          <p> </p>
        </sec>
        <sec id="sec-4-9-2">
          <title>The object of  analysis  </title>
        </sec>
        <sec id="sec-4-9-3">
          <title>Data for analysis  </title>
        </sec>
        <sec id="sec-4-9-4">
          <title>Result of analysis  </title>
        </sec>
        <sec id="sec-4-9-5">
          <title>Representation of </title>
          <p>analysis results  </p>
        </sec>
        <sec id="sec-4-9-6">
          <title>Range of  definition </title>
        </sec>
        <sec id="sec-4-9-7">
          <title>Computational </title>
          <p>complexity  </p>
        </sec>
        <sec id="sec-4-9-8">
          <title>Complexity of the  decision space  </title>
        </sec>
        <sec id="sec-4-9-9">
          <title>The role of  domain ontology   AHP </title>
        </sec>
        <sec id="sec-4-9-10">
          <title>Properties of the domain objects  and their meaning  </title>
        </sec>
        <sec id="sec-4-9-11">
          <title>Expert evaluations of properties  </title>
        </sec>
        <sec id="sec-4-9-12">
          <title>Hierarchy of significance of the </title>
          <p>properties of the domain objects  </p>
        </sec>
        <sec id="sec-4-9-13">
          <title>Weight vector of dimension n   MN </title>
        </sec>
        <sec id="sec-4-9-14">
          <title>Properties of the domain objects and  their meaning </title>
        </sec>
        <sec id="sec-4-9-15">
          <title>Information on instances of domain  objects of a certain class and their  characteristics </title>
        </sec>
        <sec id="sec-4-9-16">
          <title>Classification rule for instances of the </title>
          <p>domain objects of the selected class </p>
        </sec>
        <sec id="sec-4-9-17">
          <title>Decision tree, linear or non‐linear </title>
          <p>expression, neural network, etc. </p>
        </sec>
        <sec id="sec-4-9-18">
          <title>Attribution of all instances of objects of  the selected class to one of the  categories </title>
        </sec>
        <sec id="sec-4-9-19">
          <title>Ordering of a fixed subset of  instances of objects of the selected  class according to their compliance  with the goal  </title>
        </sec>
        <sec id="sec-4-9-20">
          <title>Depends on the dimension of C and  Depends on the dimension of C and the  the volume of X  volume of P </title>
        </sec>
        <sec id="sec-4-9-21">
          <title>An ordered set of evaluations for  An arbitrary set of non‐overlapping  fixed decision class   categories to which instances of the  decision class can be assigned </title>
        </sec>
        <sec id="sec-4-9-22">
          <title>Source of C and structure of class P   Source of C and structure of class P,  their characteristics </title>
          <p>Suppose a group of experts considers n types of actions or objects and has two goals: 1) to make
statements about the relative significance of these objects, and 2) to develop a process of statement
obtaining that allows for quantitative evaluations of these statements for all objects.</p>
          <p>It is clear that achieving the second goal requires developing a suitable method for obtaining a set
of weights associated with individual objects from the quantitative judgments of the expert group,
which are derived from the relative values associated with pairs of objects.</p>
          <p>This approach transforms qualitative information into a quantitative form that is more convenient
for processing without losing any information.</p>
          <p>Let C1, C2 ,..., Cn is a set of objects (or possible actions). Quantitative statements about pairs of
objects (Ci , C j),i  1, n, j  1, n are represented by the matrix A of n  n size: A  (aij),i  1, n, j  1, n .</p>
          <p>The elements of matrix A are determined according to the following rules:
1. if aij  x and x  0 , then a ji  1 x ;
2. if Ci and C j have the same relative importance, then aij  a ji  1 (, i  1, n : aii  1 ).
Thus, this matrix A has the following form:
 a11  1
 ...</p>
          <p>A   ai1  1 a1i</p>
          <p>...


a n1  1 a1n
...
...
...
...
...
aii  1</p>
          <p>...
... a ni  1 ain</p>
          <p>After obtaining quantitative estimates for all pairs (Ci , C j ) , the task is reduced to matching n
possible actions C1, C2 ,..., Cn with a set of n quantitative estimates w1, w 2 ,..., w n . For this, it is
necessary to more clearly formalize the problem in terms of an abstract mathematical structure.</p>
          <p>Therefore it is desirable to describe the main stages of the process of task formulation and describe
each stage in more detail, so that the potential user can assess the feasibility of its application to a
certain practical problem. We have to define clearly how the weights wi , i  1, n depend on expert
opinions aij, i, j  1, n . We propose to separate three steps of task formalization process, from the
simplest individual case to the most general situation.</p>
          <p>Step 1. If statement aij, i, j  1, n are the results of a comparison of exact measurements
w1, w 2 ,..., w n , then the estimate aij  wi w j , a ji  w j wi , , , i.e.</p>
          <p> w1 w1
 ...</p>
          <p>A   w1 w1

 ...
w n w1
...
...
...
...
...</p>
          <p>w1 w
...</p>
          <p>...
wi wi
w n wi ...</p>
          <p>...
...
...
...</p>
          <p>w1 w n </p>
          <p>... 
wi w n </p>
          <p>
... 
w n w n </p>
          <p>But for most practical problems, such a situation is unattainable, because even in the presence of
direct measurements of the values these values are not accurate and have certain deviations, which are
determined by the statistical dispersion of the measurement results.</p>
          <p>Step 2. Determination of deviations in values can be defined as the average value for all
measurements, that is:</p>
          <p>1 n
wi   w jaij, i  1, n .</p>
          <p>n j1</p>
          <p>But even such relaxation of conditions raises the questions about the existence of a unique solution
to the definition of w1, w 2 ,..., w n for the given aij, i, j  1, n .</p>
          <p>As it follows from the above remarks, the proximity of a ij is determined by the value of n. Thus,
for a more generalized case, it is necessary to estimate such a dependence.
1 n
Step 3. We define by lmax the largest value of n for which wi   w jaij, i  1, n
lmax j1
have a unique solution (eigenvalue problem).</p>
          <p>Then we can find aij, i, j  1, n for this value. If analyzed data contains more then lmax compared
elements we can consider the found solution is the unique one.</p>
          <p>To obtain a quantitative assessment of matched objects, the matrix A must be multiplied by the
transposed vector of the characteristic weights. The sum of the assessments for each of the
eigenvectors obtained above is then determined, taking into account the priority of the corresponding
characteristic.</p>
          <p>The main difference between the proposed variant of AHP and traditional ones is the use of
external knowledge about the domain. This knowledge does not replace expert assessments but
provides them with a more structured initial set of information. For example, experts do not need to
manually enter the basic elements of the domain (concepts, relations, etc.) that can be obtained from
the corresponding ontology. Generating a set of examples that satisfy certain user requirements is
easier with semantic queries to the ontology than having experts select them directly from existing
examples. It is important to note that ontological information alone is not the basis for making
decisions regarding the hierarchy of criteria. This information only becomes relevant after experts
confirm its relevance and pertinence to the task.
5. Practical example of the multi‐criteria comparison of CIOs  </p>
          <p>
            Proposed approach to matching of CIOs can be used for generation of specialized subgroups of
heterogeneous unmanned aerial vehicles (UAVs). Today, UAVs, or drones are used in various fields
of application – both military (surveillance and attack) and civilian, as they are suitable for solving a
large the number of tasks, and the reduction of their cost, various technical solutions and the
expansion of the management functionality make their use expedient. UAVs are effective in
dynamically uncertain environments with hard- to-reach areas. However, in such cases, they usually
require special sensors to facilitate the task. Managing the coordination of drones is a complex task
that requires integrating research from multiple fields [
            <xref ref-type="bibr" rid="ref15">15</xref>
            ].
          </p>
          <p>
            Use of UAVs is very urgent and increases interest in both military, industrial and research fields
due to the large number of scenarios and applications they can support and execute in
dynamic/uncertain environments. One of the great challenges of today is the application of UAV
swarms to take advantage of the benefits that the coordinated actions of drones [
            <xref ref-type="bibr" rid="ref16">16</xref>
            ]. The concept of
"swarm of drones", which the American Institute of Modern Warfare associates with the technology
of swarms of mass destruction originally developed for ballistic missiles, can significantly increase
the efficiency of UAVs. But management of such complex system need in it division of such devices
on subparts with selected set of functions. Each UAV can be considered as a separate intelligent agent
(IA) [
            <xref ref-type="bibr" rid="ref17">17</xref>
            ] according to approach of agent-oriented programming (AOP) [
            <xref ref-type="bibr" rid="ref18">18</xref>
            ] that has specific goals,
resources and strategies. Action planning and environmental assessment by drones can use elements
of artificial intelligence (AI). We can analyze the joint activity of a group of UAVs as a multi-agent
system (MAS) [
            <xref ref-type="bibr" rid="ref19">19</xref>
            ] where individual rational agents of individual drones can exchange information
and perform a joint task, optimally allocating goals.
          </p>
          <p>MASs solve problems actual for organization of the drone swarm working together:
 coordination: agents have to make joint decisions to cooperate effectively and achieve a
common goal;
 communication: agents need in information exchange to learn about the state of the
environment and make decisions;
 trust: often agents do not have complete information about the state of the environment
and have to use information provided by other agents;
 ambiguity: agents often have to make decisions under conditions of ambiguity and
unpredictability;
 achieving a common goal: common goal can contradict local goals of individual agents,
and need in means of their harmonization.</p>
          <p>However, there are various methods and technologies that help solve these problems and improve
the performance of multi-agent systems. For example, coordination techniques such as joint planning
can be used to help agents coordinate their actions and make decisions together. Communication
methods, such as formalized information exchange protocols, can also be used to improve the transfer
of information between agents. To deal with ambiguity, techniques such as modeling and uncertainty
assessment can be used to give agents a better understanding of the state of the environment.</p>
          <p>For example, as a result of such information exchange, two drones can attack different objects to
inflict maximum damage, or vice versa, jointly attack one object for its guaranteed damage. At the
same time, the selection of objects is carried out dynamically based on built-in rules and analysis of
the current state.</p>
          <p>To facilitate the cooperation of drones into the swarm, they need to share information with an
unambiguous understanding of it, but the heterogeneity of the information representation used by
different UAVs is a major obstacle. Therefore, we propose to use ontology of pertinent domain to
solve information heterogeneity and allow UAVs to use information equally in the interaction
process.</p>
          <p>In some cases, the use of a single UAV is not enough for task that need in joint use of several
drones with different functionality or in different places. The use of UAV swarms that group of
individual vehicles in a group with limited human intervention for common goal can be used for
division and paralleling of task.</p>
          <p>The use of UAV groups provides a number of advantages: 1) the total cost of acquiring and
maintaining several small commercial UAVs is lower than the total cost of one large unit; 2)
scalability is a key feature of UAV swarms; 3) increased fault tolerance, since the malfunction of one
drone has a limited effect on the swarm; 4) faster operations thanks to the distribution of work. But
the organization of the interaction of UAV groups requires the creation of appropriate theoretical
models of their communication and interaction and the software implementation of algorithms that
ensure decision-making by individual UAVs based on the available information and taking into
account the common goal.</p>
          <p>First step of such organization requires generation of UAV teams as subgroups from existing ones
according to actual goals that are represented as a set of requirements for all drones into group, for at
least one (or other number) of group elements and for combination or disjointness of some properties.
Such problem definition is very similar to matching of CIOs with reference one.</p>
          <p>Using drone teams for a specific task requires effective communication and coordination between
groups of UAVs and individual drones.</p>
          <p>Drones are grouped both by purpose (and existing competencies) and by low-level subordination
into territorially close groups focused on the execution of separate subtasks.</p>
          <p>Personal agents of devices can be classified by the hierarchical set of criteria: for example, by
functionality, by manufacturers or with a similar value of domain-specific parameters (flight range,
weight, availability of certain devices, functions or weapons). Another criterion for the construction of
subgroups deal with cooperation where the subgroup has to contain some fixed set of functions
provided by different devices: for example, one drone has advanced surveillance tools and several
another ones can carry larger load.</p>
          <p>For more effective UAV management, it is advisable to support several levels of group hierarchy
that provide (by analogy with the reference model of open systems) interaction of drones either with
devices of their own level of hierarchy (members of their micro-group), with devices one level higher
(direct commanders or coordinators) or with devices one level below (direct subordinates). Elements
of AHP can be used to choose the most successful structure of the MAS.</p>
          <p>
            In order to determine the functionality of individual devices (both the drones themselves and their
equipment and weapons), it is advisable to develop an ontological model of their competencies, where
various potentially acceptable elements of functionality are characterized both qualitatively
(availability, type of image perception, etc.) and quantitatively (speed, weight, range, etc.). In
addition, this ontology should reflect the semantics of interaction between individual types of drones,
operators and external equipment. The apparatus of atomic competences developed for the assessment
of learning outcomes and education planning [
            <xref ref-type="bibr" rid="ref3">3</xref>
            ] can be easily adapted to describe the available
technical characteristics of independently operating drones and directions for their improvement (see
Table 2).
 
Table 2 
          </p>
        </sec>
        <sec id="sec-4-9-23">
          <title>Correspondences of competence approach with UAV management parameters </title>
        </sec>
        <sec id="sec-4-9-24">
          <title>Competence approach  </title>
        </sec>
        <sec id="sec-4-9-25">
          <title>Potential employees </title>
        </sec>
        <sec id="sec-4-9-26">
          <title>Available staff  </title>
        </sec>
        <sec id="sec-4-9-27">
          <title>Project team </title>
        </sec>
        <sec id="sec-4-9-28">
          <title>Employee competencies </title>
        </sec>
        <sec id="sec-4-9-29">
          <title>Ability to learn additional competencies </title>
        </sec>
        <sec id="sec-4-9-30">
          <title>Set of vacancies for the implementation of the  project </title>
        </sec>
        <sec id="sec-4-9-31">
          <title>Atomic competency </title>
        </sec>
        <sec id="sec-4-9-32">
          <title>Matching of competencies  </title>
        </sec>
        <sec id="sec-4-9-33">
          <title>UAV management </title>
        </sec>
        <sec id="sec-4-9-34">
          <title>Individual drones  </title>
        </sec>
        <sec id="sec-4-9-35">
          <title>UAV swarm </title>
        </sec>
        <sec id="sec-4-9-36">
          <title>UAV subgroup </title>
        </sec>
        <sec id="sec-4-9-37">
          <title>Drone equipment  </title>
        </sec>
        <sec id="sec-4-9-38">
          <title>Possibility of completion with other equipment   task to be performed by a group of drones </title>
        </sec>
        <sec id="sec-4-9-39">
          <title>UAV equipment property </title>
        </sec>
        <sec id="sec-4-9-40">
          <title>Comparison of UAV subgroups </title>
          <p>This approach provides possibility of evaluation and comparison of different UAV sets and their
compliance with actual tasks and environment parameters.</p>
          <p>UAV swarm tasks often require multi-criteria decision-making.</p>
          <p>For example of such task is organization of defense of a critical infrastructure object with the help
of UAVs (drones) against enemy air raids. There is a network of drones with various functions, such
as air defense equipment and monitoring equipment for the airspace and ground area around the
object to detect mobile enemy objects, communication means. We need to find a solution for selecting
the minimum number of drones that can ensure an appropriate defense level for the object.</p>
          <p>Here we don’t consider planning of drone actions and distribution of function because we analyze
the previous stage where we have to generate this swarm subset that would be able to solve practical
task, and elements of this task are considers as criteria of group generation. Examples of these criteria
are protection of different types of dangers for the object of critical infrastructure, potential damages
of enemy (live military forces, mobile equipment, weapons, etc.) and own losses of UAV. However,
simply listing the criteria is not sufficient. We also need to evaluate their relative significance. For
example, safety of drones is important and necessary condition for future defense, but some objects
(such as nuclear power plants) are so critical that their safety is much more significant.</p>
          <p>Therefore, we are dealing with a multi-criteria problem that requires the simultaneous optimization
of conflicting objectives. It is impossible to find a solution that would be the best for all criteria at the
same time, because in general the improvement of the values of some criteria subset leads to the
deterioration of the values of the other ones and solution can only be a compromise solution</p>
        </sec>
      </sec>
    </sec>
    <sec id="sec-5">
      <title>6. Conclusion and prospects for further work </title>
      <p>Comparing CIOs with similar structures is a necessary component of the more general problem of
analyzing and comparing CIOs built on different ontologies and having different structures. When
analyzing CIOs with heterogeneously defined semantics, it is necessary to: 1. align the basic domain
ontologies and identify correspondences between their concepts and relations; 2. identify subsets of
CIOs with similar structures in the compared CIOs; 3. compare these subsets using the algorithm
discussed above.</p>
      <p>The approach based on hierarchical aggregate evaluation of CIOs focuses on integrated
multicriteria decision-making in conditions where the set of criteria depends on the specifics of the task and
can be created based on domain knowledge. However, the relative importance of these criteria can
change dynamically at different points in time. We intend to expanded this approach in the future with
the use of tools and methods such as knowledge management, intelligent data analysis, and machine
learning, which can be employed to obtain competent knowledge.</p>
      <p>
        The theoretical models and methods proposed in this work can be used to support tasks relevant to
the state of war, such as risk management, rapid adjustment of industry for the production of essential
products, restorative construction, and dynamic adaptation of teams with a multi- level hierarchical
structure (military units, expert commissions, rapid response medical teams) to perform critical
operational tasks in the absence of sufficient competencies, skills, and experience [
        <xref ref-type="bibr" rid="ref5">5</xref>
        ]. This approach
enables the identification of strengths and weaknesses of various teams and their members for specific
tasks, facilitating adjustments to their composition or additional training as needed.
      </p>
      <p>In the future, we propose to use matching of complex objects with a hierarchy of criteria as a
scaling instrument. This approach transforms complex tasks into a set of simpler subtasks linked by
various hierarchical relations. Existing standards and domain ontologies used in the semantic structure
require additional research on alignment methods.</p>
    </sec>
    <sec id="sec-6">
      <title>7. References </title>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <given-names>N.</given-names>
            <surname>Guarino</surname>
          </string-name>
          ,
          <string-name>
            <given-names>P.</given-names>
            <surname>Giaretta</surname>
          </string-name>
          ,
          <string-name>
            <surname>P.</surname>
          </string-name>
          <article-title>Ontologies and knowledge bases</article-title>
          .
          <source>In: Towards very large knowledge bases, 1-2</source>
          ,
          <year>1995</year>
          .
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <given-names>P.</given-names>
            <surname>Fitsilis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>V.</given-names>
            <surname>Gerogiannis</surname>
          </string-name>
          , L. Anthopoulos,
          <article-title>Ontologies for Software Project Management</article-title>
          .
          <source>In: Journal of Software Engineering and Applications</source>
          . No.
          <issue>7</issue>
          ,
          <issue>2014</issue>
          , pp.
          <fpage>1096</fpage>
          -
          <lpage>1110</lpage>
          . doi.org/10.4236/jsea.
          <year>2014</year>
          .
          <volume>713097</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <given-names>J.</given-names>
            <surname>Rogushina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S.</given-names>
            <surname>Pryima</surname>
          </string-name>
          ,
          <article-title>Ontological approach to qualification matching based on competences: model and methods</article-title>
          .
          <source>In: Naukovyi visnyk NGU. No. 6</source>
          ,
          <issue>2017</issue>
          , pp.
          <fpage>162</fpage>
          -
          <lpage>168</lpage>
          . URL: http://elar.tsatu.edu.ua/bitstream/123456789/3194/1/4.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <given-names>J.</given-names>
            <surname>Rogushina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gladun</surname>
          </string-name>
          ,
          <article-title>Development of domain thesaurus as a set of ontology concepts with use of semantic similarity and elements of combinatorial optimization Journal of Problems in programming</article-title>
          .
          <source>No. 2</source>
          ,
          <issue>2021</issue>
          , pp.
          <fpage>03</fpage>
          -
          <lpage>15</lpage>
          . doi.org/10.15407/pp2021.
          <fpage>02</fpage>
          .003/.
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <given-names>M.</given-names>
            <surname>Lesage</surname>
          </string-name>
          ,
          <string-name>
            <given-names>G.</given-names>
            ,
            <surname>Raîche</surname>
          </string-name>
          ,
          <string-name>
            <given-names>M.</given-names>
            <surname>Riopel</surname>
          </string-name>
          ,
          <string-name>
            <given-names>F.</given-names>
            <surname>Fortin</surname>
          </string-name>
          ,
          <string-name>
            <given-names>D.</given-names>
            <surname>Sebkhi</surname>
          </string-name>
          ,
          <article-title>An E-assessment website to implement hierarchical aggregate assessment</article-title>
          .
          <source>In: World Academy of Science</source>
          , Engineering and Technology,
          <source>International Science Index</source>
          <volume>86</volume>
          ,
          <year>2014</year>
          ,
          <volume>8</volume>
          (
          <issue>2</issue>
          ), pp.
          <fpage>925</fpage>
          -
          <lpage>933</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <given-names>G.</given-names>
            <surname>Adomavicius</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N.</given-names>
            <surname>Manouselis</surname>
          </string-name>
          ,
          <string-name>
            <given-names>Y.</given-names>
            <surname>Kwon</surname>
          </string-name>
          <article-title>. Multi-criteria recommender systems</article-title>
          .
          <source>In: Recommender systems handbook, 2011</source>
          , Springer, Boston, MA. pp.
          <fpage>769</fpage>
          -
          <lpage>803</lpage>
          . URL: https://www.academia.edu/download/32978074/Recommender_systems_handbook.pdf#page =774. doi.org/10.1007/978-0-
          <fpage>387</fpage>
          -85820-3_
          <fpage>24</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <given-names>B.</given-names>
            <surname>Roy</surname>
          </string-name>
          ,
          <string-name>
            <given-names>R.</given-names>
            <surname>Słowiński</surname>
          </string-name>
          ,
          <article-title>Questions guiding the choice of a multicriteria decision aiding method</article-title>
          .
          <source>In: EURO Journal on Decision Processes</source>
          ,
          <year>2013</year>
          ,
          <volume>1</volume>
          (
          <issue>1-2</issue>
          ), pp.
          <fpage>69</fpage>
          -
          <lpage>97</lpage>
          . doi.org/10.1007/s40070- 013-0004-7.
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <given-names>P.</given-names>
            <surname>Resnik</surname>
          </string-name>
          ,
          <article-title>Using information content to evaluate semantic similarity in a taxonomy</article-title>
          .
          <source>In: Proc. of the14th international joint conference on Artificial intelligence,</source>
          volume
          <volume>1</volume>
          ,
          <year>1995</year>
          , pp.
          <fpage>448</fpage>
          -
          <lpage>453</lpage>
          . doi.org/10.48550/arXiv.cmp-lg/9511007.
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <given-names>J.</given-names>
            <surname>Rogushina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gladun</surname>
          </string-name>
          ,
          <article-title>Task Thesaurus as a Tool for Modeling of User Information Needs</article-title>
          .
          <source>In: New Perspectives on Enterprise Decision-Making Applying Artificial Intelligence Techniques</source>
          , Springer, Cham.
          <year>2021</year>
          , pp.
          <fpage>385</fpage>
          -
          <lpage>403</lpage>
          . doi.org/10.1007/978-3-
          <fpage>030</fpage>
          -71115-3_
          <fpage>17</fpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <article-title>A free, open-source ontology editor and framework for building intelligent systems</article-title>
          . URL: protege.stanford.edu.
        </mixed-citation>
      </ref>
      <ref id="ref11">
        <mixed-citation>
          [11]
          <string-name>
            <given-names>OWL</given-names>
            <surname>Web Ontology</surname>
          </string-name>
          <article-title>Language. Overview. W3C Recommendation. URL: www</article-title>
          .w3.org/TR/owl-features/.
        </mixed-citation>
      </ref>
      <ref id="ref12">
        <mixed-citation>
          [12]
          <string-name>
            <given-names>R.E.</given-names>
            <surname>Steuer</surname>
          </string-name>
          , Multiple Criteria Optimization: Theory, Computations, and
          <string-name>
            <surname>Application</surname>
          </string-name>
          . New York: John Wiley &amp; Sons, Inc.,
          <year>1986</year>
          . ISBN 047188846X. doi.org/10.1002/oca.4660100109.
        </mixed-citation>
      </ref>
      <ref id="ref13">
        <mixed-citation>
          [13]
          <string-name>
            <given-names>O. B.</given-names>
            <surname>Vasyliev</surname>
          </string-name>
          ,
          <string-name>
            <given-names>N. S.</given-names>
            <surname>Vasylieva</surname>
          </string-name>
          ,
          <string-name>
            <given-names>O. D.</given-names>
            <surname>Kichmarenko</surname>
          </string-name>
          ,
          <article-title>Methods of solving multi-criteria optimization problems: tutorial</article-title>
          .
          <source>Odesa: Odesa National University named after I. I. Mechnikov</source>
          ,
          <year>2017</year>
          . 48 p. URL: http://dspace.onu.edu.ua:8080/bitstream/123456789/22586/1/VecOpt.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref14">
        <mixed-citation>
          [14]
          <string-name>
            <given-names>T. L.</given-names>
            <surname>Saaty</surname>
          </string-name>
          ,
          <article-title>Decision making with the analytic hierarchy process</article-title>
          .
          <source>In: International journal of services sciences</source>
          ,
          <year>2008</year>
          ,
          <volume>1</volume>
          (
          <issue>1</issue>
          ), pp.
          <fpage>83</fpage>
          -
          <lpage>98</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref15">
        <mixed-citation>
          [15]
          <string-name>
            <given-names>M. W.</given-names>
            <surname>Mueller</surname>
          </string-name>
          ,
          <string-name>
            <given-names>S. J.</given-names>
            <surname>Lee</surname>
          </string-name>
          ,
          <string-name>
            <surname>R.</surname>
          </string-name>
          <article-title>D'Andrea, Design and control of drones</article-title>
          .
          <source>In: Annual Review of Control, Robotics, and Autonomous Systems</source>
          ,
          <volume>5</volume>
          ,
          <year>2022</year>
          , pp.
          <fpage>161</fpage>
          -
          <lpage>177</lpage>
          . doi.org/10.1146/annurevcontrol-042920-012045.
        </mixed-citation>
      </ref>
      <ref id="ref16">
        <mixed-citation>
          [16]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Shi</surname>
          </string-name>
          ,
          <article-title>Particle swarm optimization</article-title>
          . In: IEEE connections,
          <year>2004</year>
          ,
          <volume>2</volume>
          (
          <issue>1</issue>
          ), pp.
          <fpage>8</fpage>
          -
          <lpage>13</lpage>
          . doi.org/10.1109/CEC.
          <year>2001</year>
          .
          <volume>934374</volume>
          .
        </mixed-citation>
      </ref>
      <ref id="ref17">
        <mixed-citation>
          [17]
          <string-name>
            <surname>M. L. Griss</surname>
          </string-name>
          ,
          <article-title>Software agents as next generation software components</article-title>
          .
          <source>In: Component-based software engineering</source>
          ,
          <year>2001</year>
          , pp.
          <fpage>641</fpage>
          -
          <lpage>657</lpage>
          . URL: https://pdfs.semanticscholar.org/839e/b7a89ed502b637fc6ad5a46acd4dca0cad40.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref18">
        <mixed-citation>
          [18]
          <string-name>
            <given-names>Y.</given-names>
            <surname>Shoham</surname>
          </string-name>
          ,
          <article-title>Agent-oriented programming</article-title>
          .
          <source>In: Artificial Intelligence. N 60</source>
          .
          <year>1993</year>
          , pp.
          <fpage>51</fpage>
          -
          <lpage>92</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref19">
        <mixed-citation>
          [19]
          <string-name>
            <surname>M. L. Griss</surname>
          </string-name>
          ,
          <article-title>Software agents as next generation software components</article-title>
          .
          <source>Component-based software engineering</source>
          ,
          <year>2001</year>
          , pp.
          <fpage>641</fpage>
          -
          <lpage>657</lpage>
          . URL: https://pdfs.semanticscholar.org/839e/b7a89ed502b637fc6ad5a46acd4dca0cad40.pdf.
        </mixed-citation>
      </ref>
      <ref id="ref20">
        <mixed-citation>
          [20]
          <string-name>
            <given-names>J.</given-names>
            <surname>Rogushina</surname>
          </string-name>
          ,
          <string-name>
            <given-names>A.</given-names>
            <surname>Gladun</surname>
          </string-name>
          ,
          <article-title>Ontology-based competency analyses in new research domains</article-title>
          .
          <source>Journal of Computing and Information Technology</source>
          ,
          <year>2012</year>
          .
          <volume>20</volume>
          (
          <issue>4</issue>
          ), pp.
          <fpage>277</fpage>
          -
          <lpage>291</lpage>
          . URL: http://www.scopus.com/inward/record.url?eid=
          <fpage>2</fpage>
          -
          <lpage>s2</lpage>
          .
          <fpage>0</fpage>
          -
          <lpage>84878940246</lpage>
          &amp;partnerID=
          <volume>40</volume>
          &amp;md5=38cda0f803a838713e8d3d88262604c8. doi.org/10.2498/cit.1002034.
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>