<!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Archiving and Interchange DTD v1.0 20120330//EN" "JATS-archivearticle1.dtd">
<article xmlns:xlink="http://www.w3.org/1999/xlink">
  <front>
    <journal-meta />
    <article-meta>
      <title-group>
        <article-title>Assessing the Quality of Visualization Metaphor of Fuzzy Cognitive Maps on the Basis of Formalized Cognitive Clarity Criteria</article-title>
      </title-group>
      <contrib-group>
        <aff id="aff0">
          <label>0</label>
          <institution>A.G. Podvesovskii</institution>
        </aff>
        <aff id="aff1">
          <label>1</label>
          <institution>Bryansk State Technical University</institution>
          ,
          <addr-line>Bryansk</addr-line>
          ,
          <country country="RU">Russia</country>
        </aff>
      </contrib-group>
      <abstract>
        <p>The paper presents continuation of research in the field of constructing a visualization metaphor of cognitive models based on fuzzy cognitive maps. The focus is on the spatial metaphor as the basis for representation metaphor formation. A method is proposed for quality assessment of a spatial metaphor of a fuzzy cognitive map based on formalized cognitive clarity criteria defined in the previous part of the study. To this end, methods have been developed to formalize several nontrivial criteria of cognitive clarity. An example is given that confirms correctness of the proposed method for assessing the quality of a visualization metaphor.</p>
      </abstract>
      <kwd-group>
        <kwd>fuzzy cognitive map</kwd>
        <kwd>graph visualization</kwd>
        <kwd>cognitive clarity</kwd>
        <kwd>visualization metaphor</kwd>
      </kwd-group>
    </article-meta>
  </front>
  <body>
    <sec id="sec-1">
      <title>1. Introduction</title>
      <p>
        This paper continues a series of publications of authors’
research materials in the field of visualization of cognitive
models based on fuzzy cognitive maps (FCM). A FCM reflects
researcher’s subjective idea of a system in the form of a set of
semantic categories (called factors or concepts) and a set of
causal relationships between them [
        <xref ref-type="bibr" rid="ref2 ref8">2, 8</xref>
        ]. Thus, a FCM can be
graphically represented in the form of a weighted directed
graph, the vertices of which correspond to concepts, and edges
– to cause-and-effect relationships.
      </p>
      <p>
        One of the conditions for effective work with a cognitive
model is to ensure its visual representation. In [
        <xref ref-type="bibr" rid="ref4">4</xref>
        ], the authors
proposed an approach to FCM visualization based on reducing
this problem to a graph visualization problem. Later in [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], this
approach was expanded by using the concept of visualization
metaphor and its two components – spatial metaphor and
representation metaphor [
        <xref ref-type="bibr" rid="ref10">10</xref>
        ]. FCM visualization metaphor is
based on graph visualization algorithms [
        <xref ref-type="bibr" rid="ref5 ref6">5, 6</xref>
        ] and cognitive
clarity concept, which characterizes the ease of intuitive
understanding of information [
        <xref ref-type="bibr" rid="ref1">1</xref>
        ] and takes into account the
problem of human’s limited cognitive abilities when reading
graphs (a detailed analysis of this problem can be found, for
example, in [
        <xref ref-type="bibr" rid="ref3">3</xref>
        ]). Thus, a link has been discovered between the
quality of FCM visualization metaphor and the level of
cognitive clarity of the resulting visual image: the higher the
level of cognitive clarity provided by the visualization
metaphor, the simpler is the process of expert understanding of
the cognitive model in its visual analysis. To assess the level of
cognitive clarity, a set of criteria is proposed. It is concluded
that cognitive clarity criteria are the means of the most natural
evaluation of visualization metaphor quality. The present work
is devoted to the development of a method for such an
assessment. The previous part of the study focused on the
representation metaphor, whereas this work focuses on the
spatial metaphor, which is the basic component of the
visualization metaphor and serves as the foundation for the
subsequent formation of the representation metaphor.
      </p>
    </sec>
    <sec id="sec-2">
      <title>2. Methods</title>
      <p>criteria
formalizing
cognitive
clarity</p>
      <p>
        As noted in [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], the result of applying the spatial metaphor
of FCM visualization is the location of the cognitive graph (i.e.,
coordinates of its vertices and edges) on the plane, which is
optimal in the sense of cognitive clarity criteria, that is,
contributing to the quality increase of the resulting visual
image. Note that the cognitive clarity criteria are formulated at a
qualitative level using natural language. At the same time, the
possibility of spatial metaphor automated quality assessment is
of interest. In this connection, the verification of the resulting
visual image for compliance with the criteria of cognitive
clarity must be implemented algorithmically. To this end,
formalized representation of the cognitive clarity criteria is
required.
      </p>
      <p>By formalizing a certain cognitive clarity criterion, we
mean developing methods, techniques and algorithms that allow
determining a numerical score for a visual image of an arbitrary
cognitive map characterizing the extent to which this image
complies with the selected criterion. Formalization of some
criteria (minimizing the length of edges, minimizing the number
of edge crossings, minimizing the number of curved edges) is
trivial, and its description is of no interest. Let us consider
possible ways of formalizing several nontrivial cognitive clarity
criteria.
2.1</p>
      <sec id="sec-2-1">
        <title>Optimizing edge directions</title>
        <p>This criterion is based on the observation that laying out
edges in the directions “from top to bottom” and “from left to
right” helps to accelerate “reading” of a FCM in comparison
with the orientation of edges in the opposite directions. We
shall call the directions that facilitate faster “reading” of FCMs
as well as edges having such directions convenient. As an
example, we can compare two visual images of a cognitive
graph in Figure 1.</p>
        <p>Fig. 1. Examples of visual images with convenient (a)
and not convenient (b) directions of edges</p>
        <p>Apparently, convenient directions coincide with the
direction of reading, adopted in a particular language culture.
Therefore, other conditions being equal, preference should be
given to visual images containing a greater number of
convenient edges. It should be borne in mind that the described
property is inherently fuzzy. So, edge orientation “from top to
bottom” and “from right to left” can be considered partially
convenient, since one of the usual directions of reading is
preserved. Therefore, the mathematical apparatus of the fuzzy
set theory can be used to formalize the criterion in question.</p>
        <p>Let A be a fuzzy set formalizing the concept of a
“convenient edge direction”. In order to set its membership
function, let us define the edge direction as angle α between the
vector drawn from the beginning of the edge to its end and the
positive direction of the horizontal axis OX. Then membership
function  A( ) must satisfy the following requirements:
1)  A( ) 1 when </p>
        <p>   0;
2)  A( )  0 when</p>
        <p>    ;
3) 0   A( )  1 when 0   

2

2

2</p>
        <p>
и      ;
2

4) increases monotonically on the interval      ;
2

.
5) decreases monotonically on the interval 0   
2
Given these requirements,  A( )  cos( ) can be accepted

2
on the interval 0   
, and  A( )  sin( ) can be

accepted on the interval     
.
2</p>
        <p>Having determined for each FCM edge the degree of its
membership to set A, we can obtain a value characterizing the
overall score of the entire visual image by this criterion – for
example, as the average value of membership degrees of all
edges.</p>
        <p>Influence intensities should also be taken into account in the
final assessment, since providing convenient directions for
more significant influences is more important than for less
significant ones. Therefore, absolute values of influence
intensities can be used as weighting coefficients and
membership values of the corresponding edges can be
multiplied by them when calculating the average value.
2.2 Maximizing unidirectionality of consecutive
edges</p>
        <p>This criterion is based on the idea that "reading" a FCM will
be faster if gaze direction has to be changed as little as possible
during the process of viewing paths and cycles of a graph.</p>
        <p>We will call two edges consecutive if one of them enters the
vertex from which the other one starts. Thus, any path and cycle
of a graph consists of pairs of consecutive edges. Therefore, in
accordance with this criterion, preference should be given to
visual images with a greater number of pairs of consecutive
edges depicted unidirectionally. For example, let us compare
two visual images of a fragment of some FCM (Fig. 2).</p>
        <p>Obviously, the unidirectional property is fuzzy. Suppose B
is a fuzzy set formalizing the concept of unidirectional edges.
The membership degree of a pair of consecutive edges to set B
is determined by angle  [0, ] between these edges. We
assume that changing gaze direction by 90 degrees or more
slows down the process of viewing the path in the graph
significantly. Accordingly, the following requirements are
imposed on the membership function B ( ) :
1) B ( )  1 when   0;
2) B ( )  0 when  

2</p>
        <p>;
3) 0  B ( )  1 when 0   

2
;
4) decreases monotonically on the interval 0   

2
.</p>
        <p>
2
we</p>
        <p>Given these requirements, on the interval 0   
accept B ( )  cos( ).</p>
        <p>By analogy with the previous criterion, the score of the
entire visual image by the criterion under study can be found as
the average value of membership degrees of all pairs of
consecutive edges to set B. Influence intensities can also be
taken into account in a similar way.
2.3</p>
      </sec>
      <sec id="sec-2-2">
        <title>Maximizing graph symmetry</title>
        <p>Due to the fact that FCM structure reflects the structure of a
simulated system it is important to ensure the symmetry of
FCM visual image to increase its cognitive clarity. Thus,
symmetries of a graph image help to detect symmetries inherent
in the system itself.</p>
        <p>Let us consider various aspects of determining degree of
image symmetry in relation to FCM visual image.</p>
        <p>Firstly, the following types of symmetries are the simplest
to perceive and, therefore, of greatest practical interest:
1) axial symmetry with respect to the horizontal axis of an
image;
2) axial symmetry with respect to the vertical axis of an image;
3) central symmetry with respect to the geometric center of an
image.</p>
        <p>Secondly, in the case of an FCM, as well as any digraph,
the following levels of symmetry can be distinguished (Fig. 3
considers the case of symmetry about the vertical axis):
1) lack of symmetry at the level of any elements of the graph
(Fig. 3, a);
2) at the level of vertices excluding edges (Fig. 3, b);
3) at the level of edges excluding their directions (Fig. 3, c);
4) at the level of edges including their directions (Fig. 3, d).</p>
        <p>The above example allows for the conclusion that symmetry
at the vertex level does not bring any tangible effect to
increasing cognitive clarity of an FCM visual image. Thus, only
symmetry at the level of edges is of practical interest.</p>
        <p>Thirdly, it is obvious that in addition to strict symmetry
(Fig. 4, a), we can also speak of approximate symmetry (Fig. 4,
b), which can be represented as a certain deviation from the
strict one.</p>
        <p>With this in mind, the degree of symmetry of an FCM
visual image can be defined as a measure of its proximity to a
strictly symmetric image. Thus, it is necessary to develop an
algorithm that is able to determine the degree of symmetry for
an arbitrary image taking into account a given type and level of
symmetry.</p>
        <p>The main idea of the proposed algorithm is as follows. For
each element of an FCM visual image, the position of its
“reflection” relative to a given axis or center is calculated. Next,
for each of the “reflections”, the element closest to it (in the
sense of the chosen metric, for example, Euclidean distance) is
selected from among all the elements of the image. Distances
(in the selected metric) from all “reflections” to their nearest
elements are added up. The resulting value characterizes the
degree of symmetry of the visual image in question and has the
following properties:
1) it is equal to 0 if the image has strict symmetry of a given
(or stronger) level and type;
2) it is greater than 0 in all other cases;
3) it increases as the image becomes less and less symmetrical;
4) it does not have an upper bound since there is no
“maximally asymmetric” image.</p>
      </sec>
    </sec>
    <sec id="sec-3">
      <title>3. Defining spatial metaphor quality assessment</title>
      <p>
        In [
        <xref ref-type="bibr" rid="ref7">7</xref>
        ], it was noted that many of cognitive clarity criteria
contradict each other and it is impossible in the general case to
ensure that FCM visual image meets all the criteria at the same
time from an algorithmic point of view. Thus, it is necessary to
develop decision rules modelling various forms of compromise
among the criteria.
      </p>
      <p>Of primary interest is the class of rules based on various
types of criteria aggregations, primarily, sum and product ones.
At the same time, there is reason to believe that the structure of
relationships among cognitive clarity criteria is quite complex
and is characterized by the following features:
1) criteria may exist that determine quality of a metaphor not
separately but in combination with some other criteria;
2) in the whole set of criteria, there may be several “bundles of
criteria” affecting metaphor quality independently of each
other.</p>
      <p>
        To formalize the described assumption, we shall accept that
set of criteria K  {k1, ,kn} can be divided into disjoint
subsets G , ,Gm . Further, we will also assume that FCM
1
visual image scores by all criteria take their values in the
interval [
        <xref ref-type="bibr" rid="ref1">0, 1</xref>
        ].
      </p>
      <p>
        For each criteria subset Gi , we shall introduce value
gi [
        <xref ref-type="bibr" rid="ref1">0, 1</xref>
        ] – visual image score for this subset. We shall specify
the following requirements for such a score:
1) if the image score by at least one criterion from subset Gi
is 0, then gi  0 ;
2) gi  1 if and only if the image score by all criteria from
subset Gi is 1;
3) if the image score according to all criteria from subset Gi is
a, then gi  a (idempotency).
      </p>
      <p>One of the operations meeting the specified requirements is
a weighted product aggregation:</p>
      <p>l
gi   mjwj ,</p>
      <p>
        j1
where mj is image score by the j-th criterion from
Gi ,
wj [
        <xref ref-type="bibr" rid="ref1">0,1</xref>
        ] is relative importance of the j-th criterion within Gi
(w1   wl  1) , l is power of Gi .
      </p>
      <p>To obtain final score, we shall apply weighted sum
aggregation to the scores obtained for all subsets:</p>
      <p>m
F   wgi gi ,</p>
      <p>
        i1
where wgi [
        <xref ref-type="bibr" rid="ref1">0,1</xref>
        ] – is relative importance of subset Gi
(wg1   wgm  1) .
      </p>
      <p>Moreover, value wgi can be interpreted as FCM visual
image score that fully satisfies the subset of criteria Gi and
does not completely satisfy other subsets of criteria.</p>
      <p>
        Thus, for the final score F, the following properties are
guaranteed:
1) F [
        <xref ref-type="bibr" rid="ref1">0,1</xref>
        ] ;
2) F  1 if scores for all criteria are 1;
3) F  0 if scores for all subsets of criteria G , ,Gm are 0
1
(i.e., at least one criterion scored with 0 is present in each
subset).
      </p>
    </sec>
    <sec id="sec-4">
      <title>4. An example</title>
      <p>assessment
of
spatial
metaphor
quality</p>
      <p>Let us conduct experimental verification of the proposed
method for quality assessment of FCM spatial metaphor. In
view of this, we shall consider three different visual images of a
certain FCM presented in Fig. 5-7.</p>
      <p>
        Assessment of the degree of visual images compliance with
cognitive clarity criteria has been performed using the proposed
methods of formalizing these criteria. Table 1 shows criteria
scores of images normalized to the range [
        <xref ref-type="bibr" rid="ref1">0, 1</xref>
        ]. When
normalizing the scores, the initial need to minimize a number of
criteria was taken into account. Thus, after normalization, all
criteria must be maximized.
      </p>
      <p>Let us demonstrate the use of various decision rules to
obtain final scores for cognitive clarity levels of FCM visual
images.</p>
      <p>First, we shall apply a simple sum aggregation of criteria.
Suppose the criteria priorities are set as follows: w1 = 0.2;
w2 = w3 = w6 = w9 = 0,05; w4 = w7 = w8 = 0.1; w5 = 0.3. In this
case, we obtain the following scores of cognitive clarity levels
of visual images: F1 = 0.53; F2 = 0.84; F3 = 0.25. Accordingly,
the metaphor forming image 2 should be recognized most
qualitative.</p>
      <p>Now we shall adjust the criteria priorities increasing relative
significance of the criteria of optimizing edge direction and
maximizing unidirectionality of consecutive edges:
w1 = w2 = w3 = w6 = w8 = w9 = 0,05; w4 = w7 = 0.2; w5 = 0.3.
Scores of cognitive clarity levels will change and take values
F1 = 0.73; F2 = 0.68; F3 = 0.11. In this case, the metaphor
forming image 1, which meets the highest priority criteria more
than image 2, proves to be most qualitative.</p>
      <p>Now, let us apply the decision rule proposed in this paper.
Suppose that, based on some expert considerations, the set of
criteria has been divided into two subsets: G  {k1, k2, k3, k8}
1
and G2  {k4,k5,k6,k7,k9} , whose priorities were initially set to
0.6 and 0.4 respectively. For simplicity, we will assume that the
criteria priorities within the corresponding subsets are
distributed evenly, that is 0.25 and 0.2 for the criteria from G1
and G2, respectively. We will obtain the following scores of
cognitive clarity levels of visual images: F1 = 0.23; F2 = 0.6;
F3 = 0.25. The metaphor forming image 2, which fully satisfies
all the criteria from the higher priority subset of G1, is
recognized as the most qualitative.</p>
      <p>Now let us set priorities G1 and G2 equal to 0.3 and 0.7,
respectively. In this case, scores of cognitive clarity levels will
take values F1 = 0.41; F2 = 0.3; F3 = 0.13. The metaphor
forming image 1 is now considered the most high-quality, since
this image better than others satisfies a higher priority criteria
subset G2.</p>
      <p>Thus, the use of the proposed decision rule allows for
setting relative importance of a certain set of criteria at once. At
the same time, by regulating criteria priorities within the
framework of corresponding subsets, it is possible to provide a
finer consideration of their influence on the score of these
subsets.</p>
    </sec>
    <sec id="sec-5">
      <title>5. Conclusion</title>
      <p>The paper presents the development of an approach to
constructing a visualization metaphor of fuzzy cognitive maps.
A method for assessing the quality of FCM spatial metaphor
based on formalized cognitive clarity criteria is proposed and
methods for formalizing several such criteria, which are
nontrivial, are described. An example is given confirming the
correctness of the proposed metaphor quality assessment
method: visual images with a higher level of cognitive clarity
get higher scores. In addition, application of two different
decision rules has been demonstrated, allowing for quality
evaluation of a metaphor based on its scores by groups of
cognitive clarity criteria as well as set priorities of these groups.</p>
      <p>Let us indicate directions for further research.</p>
      <p>
        The first one is developing methodology for automatic
selection of the optimal spatial metaphor of a FCM taking into
account set priorities according to the cognitive clarity criteria;
also implementation of the corresponding opportunity in IGLA
decision support system, developed with the participation of the
authors [
        <xref ref-type="bibr" rid="ref9">9</xref>
        ].
      </p>
      <p>The second one is development of new decision rules for
quality assessment of FCM metaphor allowing for more flexible
consideration of user's specific preferences and acceptable
forms of compromise among cognitive clarity criteria.</p>
    </sec>
    <sec id="sec-6">
      <title>6. Acknowledgments</title>
      <p>The reported study was funded by RFBR, project number
19-07-00844.</p>
    </sec>
    <sec id="sec-7">
      <title>7. References</title>
    </sec>
  </body>
  <back>
    <ref-list>
      <ref id="ref1">
        <mixed-citation>
          [1]
          <string-name>
            <surname>Abramova</surname>
            <given-names>N.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Voronina</surname>
            <given-names>T.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Portsev</surname>
            <given-names>R.Y.</given-names>
          </string-name>
          <article-title>O metodah podderzhki postroeniya i verifikacii kognitivnyh kart s primeneniem idej kognitivnoj grafiki [Ideas of Cognitive Graphics to Support Verification of Cognitive Maps] // Upravlenie bol'shimi sistemami</article-title>
          .
          <source>Special issue 30</source>
          .
          <article-title>1 “Setevye modeli v upravlenii”</article-title>
          . - Moscow: ICS RAS,
          <year>2010</year>
          . - pp.
          <fpage>411</fpage>
          -
          <lpage>430</lpage>
          [in Russian].
        </mixed-citation>
      </ref>
      <ref id="ref2">
        <mixed-citation>
          [2]
          <string-name>
            <surname>Borisov</surname>
            <given-names>V.V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Kruglov</surname>
            <given-names>V.V.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Fedulov</surname>
            <given-names>A.S.</given-names>
          </string-name>
          <article-title>Nechetkie modeli i seti [Fuzzy Models</article-title>
          and Networks].
          <string-name>
            <surname>- M.: Goryachaya Liniya - Telekom</surname>
          </string-name>
          ,
          <year>2012</year>
          . - 284 p. [in Russian]
        </mixed-citation>
      </ref>
      <ref id="ref3">
        <mixed-citation>
          [3]
          <string-name>
            <surname>Huang</surname>
            <given-names>W.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Hong</surname>
            <given-names>S.H.</given-names>
          </string-name>
          , Eades P.
          <source>Predicting Graph Reading Performance: A Cognitive Approach // Proc. Asia Pacific Symposium on Information Visualization (APVIS2006)</source>
          . - Tokyo, Japan,
          <year>2006</year>
          . - pp.
          <fpage>207</fpage>
          -
          <lpage>216</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref4">
        <mixed-citation>
          [4]
          <string-name>
            <surname>Isaev</surname>
            <given-names>R.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Podvesovskii</surname>
            <given-names>A.G.</given-names>
          </string-name>
          <article-title>Sovershenstvovanie mekhanizma vizualizacii nechetkih kognitivnyh modelej v sisteme podderzhki prinyatiya reshenij «IGLA» [Improvement of the Mechanism of Fuzzy Cognitive Models Visualization in Decision Support System «</article-title>
          IGLA»] // Proceedings of the International Conference “
          <article-title>Sovremennye tekhnologii v nauke i obrazovanii - STNO2017” [Up-to-date Technologies in Science and Education]</article-title>
          , Vol.
          <volume>1</volume>
          . - Ryazan: Ryazan State Radioengineering University,
          <year>2017</year>
          . - pp.
          <fpage>138</fpage>
          -
          <lpage>142</lpage>
          [in Russian].
        </mixed-citation>
      </ref>
      <ref id="ref5">
        <mixed-citation>
          [5]
          <string-name>
            <surname>Meyer</surname>
            <given-names>B.</given-names>
          </string-name>
          <string-name>
            <surname>Self-Organizing Graphs - A Neural Network Perspective</surname>
          </string-name>
          of Graph Layout // Whitesides S.H. (
          <article-title>eds) Graph Drawing</article-title>
          .
          <source>GD 1998. Lecture Notes in Computer Science</source>
          , vol
          <volume>1547</volume>
          . Springer, Berlin, Heidelberg.
        </mixed-citation>
      </ref>
      <ref id="ref6">
        <mixed-citation>
          [6]
          <string-name>
            <surname>Noack</surname>
            <given-names>A.</given-names>
          </string-name>
          <article-title>An energy model for visual graph clustering //</article-title>
          <source>Proceedings of the 11th International Symposium on Graph Drawing</source>
          . Springer-Verlag,
          <year>2004</year>
          . - pp.
          <fpage>425</fpage>
          -
          <lpage>436</lpage>
          .
        </mixed-citation>
      </ref>
      <ref id="ref7">
        <mixed-citation>
          [7]
          <string-name>
            <surname>Podvesovskii</surname>
            <given-names>A.G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Isaev</surname>
            <given-names>R.A</given-names>
          </string-name>
          . Visualization Metaphors for Fuzzy Cognitive Maps // Scientific Visualization,
          <year>2018</year>
          , Vol.
          <volume>10</volume>
          ,
          <string-name>
            <surname>Num</surname>
          </string-name>
          . 4, pp.
          <fpage>13</fpage>
          -
          <lpage>29</lpage>
          . - doi: 10.26583/sv.10.4.
          <fpage>02</fpage>
        </mixed-citation>
      </ref>
      <ref id="ref8">
        <mixed-citation>
          [8]
          <string-name>
            <surname>Silov</surname>
            <given-names>V.B.</given-names>
          </string-name>
          <article-title>Prinyatie strategicheskih reshenij v nechetkoj obstanovke [Strategical Decision Making in Fuzzy Environment] - Moscow: INPRO-</article-title>
          <string-name>
            <surname>RES</surname>
          </string-name>
          ,
          <year>1995</year>
          . - 228 p. [in Russian]
        </mixed-citation>
      </ref>
      <ref id="ref9">
        <mixed-citation>
          [9]
          <string-name>
            <surname>Zakharova</surname>
            <given-names>A.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Podvesovskii</surname>
            <given-names>A.G.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Isaev</surname>
            <given-names>R.A. Matematicheskoe</given-names>
          </string-name>
          <article-title>i programmnoe obespechenie podderzhki kognitivnogo modelirovanija slabostrukturirovannyh organizacionno-tehnicheskih system [Mathematical and Software Support for Cognitive Modeling of Semi-structured Organizational</article-title>
          and Technical Systems] // International conference CPT2019.
          <article-title>- Nizhniy Novgorod: Pub. NNGASU and SRCIPT</article-title>
          .
          <year>2019</year>
          . - pp.
          <fpage>131</fpage>
          -
          <lpage>141</lpage>
          [in Russian].
        </mixed-citation>
      </ref>
      <ref id="ref10">
        <mixed-citation>
          [10]
          <string-name>
            <surname>Zakharova</surname>
            <given-names>A.A.</given-names>
          </string-name>
          ,
          <string-name>
            <surname>Shklyar</surname>
            <given-names>A.V.</given-names>
          </string-name>
          <string-name>
            <surname>Visualization</surname>
          </string-name>
          Metaphors // Scientific Visualization,
          <year>2013</year>
          , Vol.
          <volume>5</volume>
          ,
          <issue>Num</issue>
          . 2, pp.
          <fpage>16</fpage>
          -
          <lpage>24</lpage>
          .
        </mixed-citation>
      </ref>
    </ref-list>
  </back>
</article>