=Paper=
{{Paper
|id=Vol-2258/paper37
|storemode=property
|title=Conceptual-Visual Metalanguage of Hybrid Intelligent Systems
|pdfUrl=https://ceur-ws.org/Vol-2258/paper37.pdf
|volume=Vol-2258
|authors=Alexander Kolesnikov,Sergey Listopad,Fedor Maitakov
}}
==Conceptual-Visual Metalanguage of Hybrid Intelligent Systems==
https://ceur-ws.org/Vol-2258/paper37.pdf
Conceptual-visual metalanguage of hybrid intelligent systems
A V Kolesnikov1,2, S V Listopad2 and F G Maitakov1
1
Institute of Physical and Mathematical Sciences and Information Technology, Immanuel Kant
Baltic Federal University, Kaliningrad 236041, Russian Federation
2
Kaliningrad Branch of the Federal Research Center “Computer Science and Control” of the
Russian Academy of Sciences, Kaliningrad 236022, Russian Federation
Abstract. The urgency of the metalanguage is caused by the development of visually-shaped
representations and reasoning in hybrid and synergetic intelligent systems. The absence of
formalisms determines the high science intensity of special environments for manipulating and
processing visual models. The formalization of the metalanguage is a condition for the
development of hybrid intelligent systems with a heterogeneous visual field; it ensures
cooperation, relativity, the complementarity of collective natural and artificial intelligence
capable of visual thinking and speaking in verbal-symbolic languages.
1. Introduction
The long experience of the Kaliningrad school of hybrid artificial intelligence confirms not only the
advantages of functional hybrid intelligence systems (FHIS), but also contrasts their shortcomings, for
example, interaction with users and specialists only through symbolic-logical methods of representing
information that practically does not activate the visually-shaped, right hemisphere reasoning of the
decision-maker (DM). Visualization and, in particular, schematization provides understanding and
explanation of problems and their solutions by FHIS, activates the mechanisms of intuitive, insightful
thinking, which is especially important in the context of diversity of opinions. Visualization of
complex structure and dynamics of changing problem situations, the principle of "seeing the problem
at a glance" will allow to act promptly.
Visually-shaped representations and cognitive modeling were considered in papers by D.A.
Pospelov, A.A. Zenkin, G.P. Shchedrovitsky, Yu.R. Val'kman, B.A. Kobrinsky, O.P. Kuznetsov,
G.S. Osipov, V.B. Tarasov, I.B. Fominykh, Т.А. Gavrilova, A.E. Yankovskaya, etc. Visual languages
are developed for functional programming, programming by examples, finite automata, data flows and
other domains [1]. The implementation of these languages requires considerable effort, development
in each case of special environments for creation, manipulation and processing of visual models. To
reduce them, the formalized model of visual language based on the principles of system theory and
system analysis is proposed. It should be considered within scope of I.B. Fominykh’s opinion:
"Solving problems of representation of right hemispheric mechanisms by left hemispheric means,
research and modeling of the interaction of figurative and symbolic-logical thinking, the forming
methods of figurative thinking computer support are of considerable interest".
2. Modeling of visually-shaped representations as semiotic system
The concept of the semiotic system D.A. Pospelov, G.S. Osipov [2] is applied for modeling of
reasoning on visually-shaped schematic images [2 - 6]:
vl VT ,VS ,VA,VP, υτ, υσ, υα, υπ , (1)
305
where VT , VS , VA are sets of basic symbols, syntactic rules and axiom-knowledge about the subject
domain (semantic rules) respectively; VP is a set of rules for inference of solutions (pragmatic rules);
υτ , υσ , υα , υπ are sets of rules for changing sets VT , VS , VA , VP respectively. The sets
VT ,VS,VA,VP,υτ,υσ,υα,υπ in equation (1) are defined by the expressions:
VT P, D,VR ,
VS VT ,VN , PRU ,
VA DO, GRES , GACT , GPR , GR ,
DO RES , ACT , PR, R , G RES : RES P , G ACT : ACT P , G PR : PR D , G R : R VR ,
VP { AG, act , M ,W } ,
υτ P, D, VR ,
υσ υτ, VN , PRU ,
υα DO, GΔRES , GΔPR , GΔR ,
DO RES , ACT , PR, R ,
GΔRES : RES P , G ΔCT : ACT P , GΔPR : PR D , GΔR : R VR ,
υπ { AG, act , M , W } ,
where, in addition to the previously introduced notation P is a set of visual primitives; D is a set of
visual dimensions characterizing visual primitives; VR is a set of visual relations between one or more
primitives [4]; VN is a dictionary of nonterminal symbols; PRU is a set of production rules; RES ,
ACT , PR , R are sets of concepts of resources, actions, properties and relations, respectively; AG is
a set of models of native speakers (experts, elements, agents), to which the behavior norm is addressed
(various social prohibitions and restrictions imposed by the community on a separate speaker);
act ACT is the action, defined on the set of actions ACT , the object of normative regulation (the
content of the norm); M is a set of systems of modalities associated with the action, for example, the
system of norms expressed by deontic modalities: M N {ma, al , in, proh} , where ma is "mandatory",
al is "allowed", in is "indifferent", proh is "prohibited"; W is a set of models of worlds in which
the norm is applicable (the conditions of application, the circumstances in which the action should or
should not be performed) [7]; P , D , VR , VN , PRU , RES , ACT , PR , R , AG ,
M , W are the sets of admissible changes of the sets P , D VR , VN , PRU , RES , ACT , PR ,
R , AG , M , W respectively; act is a set of permissible changes in the content of the norm act .
As shown in [8], the languages of professional activity are poly-languages [9]. This is due to the
inherent structure of the external world and the specificity of the asymmetry of verbal-sign and
visually-shaped representations of resources, properties, actions, structures, situations, states, the
behavior of the control object and, taking into account the subject of management activities, of goals,
tasks, plans, and assessments. In this paper, visually-shaped representations are considered as a
multilayered hierarchy of semiotic systems.
3. The multilayered model of conceptual-visual metalanguage of visually-shaped representations
In [9] the family of verbal-symbolic languages for description of resources, operations, structures,
situations, states, behavior of the control object, as well as goals, plans and problems is represented. In
[10] eight levels of visual languages are allocated for the implementation of automated reasoning in
intelligent systems: 1) conceptual and visual basis vl1 ; 2) resources, actions and properties vl 2 ; 3)
hierarchies of resources, actions and properties vl 3 ; 4) spatial and production structures vl 4 ; 5) states,
situations and events vl 5 ; 6) tasks and problems vl 6 ; 7) experts’ reasoning models vl 7 ; 8) integrated
models of collective intelligence reasoning vl 8 . In this case, the developer has a set of components for
constructing conceptual-visual metalanguage, describing the complex problem solving by combining
several interrelated processes of reasoning in different languages. Depending on the requirements of
the problem some levels could be missed. Thus, the conceptual-visual metalanguage can be formally
represented by the expression:
306
mvl vl1 , vl 2 , vl 3 , vl 4 , vl 5 , vl 6 , vl 7 , vl 8 ,VLR ,
where VLR is a set of relations between language elements vl k , k Ґ , k [1, 8] .
The metalanguage is visualized by the "layered pie" (Figure 1) of semiotic systems in which the
image signs (below the images) reflect reality at two image reflection levels of three distinguished in
[11]: the level of representations (secondary images of objects) and verbal-logical (symbolic-logical)
thinking. At the bottom of the "layered pie" are dictionaries of concepts and relations, a conceptual-
visual basis on which the family of descriptive languages arranged in levels is built.
Figure 1. The multilayered metalanguage model.
Each layer of image representation (Figure 1) is associated with a semiotic system containing: 1)
the core of the base signs (terminal alphabet) VT k ; 2) derived signs VN k , formed according to the
rules PRU k . The visual core of the higher layer languages can contain the signs of the core of the
lower layer VT k VT k 1 and signs formed outside the core of the lower layer language
VT k 1 VN k , k Ґ , k [1, 7] .
Let's consider interrelations of the images formed by syntactic rules, at various layers of
metalanguage.
The conceptual-visual basis of the language is in the first layer. It’s a set of basic concepts and
forms needed to build images on higher layers.
The first layer language vl1 uses heuristic rules PRU 1 for constructing of images of derivative
(composite) relations vr1 VR1 VT 1 from P1 , D1 and VR1 :
vl1 ( P1 , D1 ,VR1 , PRU 1 ) {vr1}.
At the second layer language vl 2 , heuristics PRU 2 are used to generate images of resources
res 2 RES 2 VT 2 , actions act 2 ACT 2 VT 2 and properties pr 2 PR2 VT 2 without regard for
their hierarchy using the definition relations VR11 VR1 :
vl 2 ( P1 , D1 ,VR11 , PRU 2 ) RES 2 PR2 ACT 2 .
At the third layer, inclusion relations VR51 VR1 and heuristics PRU 3 formalize the hierarchies of
resources res 3 RES 3 VT 3 , actions act 3 ACT 3 VT 3 and properties pr 3 PR3 VT 3 :
307
vl 3 ( P1 , D1 , RES 2 , PR2 , ACT 2 ,VR51 , PRU 3 ) RES 3 PR3 ACT 3 .
The fourth layer formalizes the spatial str14 STR14 VT 4 , operational and technological
str34 STR34 VT 4 structures on the basis of images defined at previous layers using temporal
VR31 VR1 , spatial VR41 VR1 and cause-effect VR61 VR1 relations, as well as heuristics PRU 4 :
vl 4 ( P1 , D1 , RES 3 , PR3 , ACT 3 ,VR31 ,VR41 ,VR61 , PRU 4 ) STR14 STR34 .
At the fifth layer heuristics PRU 5 formalize images of situations sit 5 SIT 5 VT 5 and signs of
states st 5 ST 5 VT 5 :
vl 5 (STR14 , STR34 , PRU 5 ) SIT 5 ST 5 .
At the sixth layer, images of homogeneous prbh 6 PRBh 6 VT 6 and heterogeneous
prb PRB VT problems are specified on the basis of image images of the previous layers and
u6 u6 6
heuristics PRU 6 :
vl 6 ( P1 , D1 , RES 3 , PR3 , ACT 3 ,VR1 , ST 5 , PRU 6 ) PRBh 6 PRBu 6 .
At the seventh layer, heuristics PRU 7 form images of autonomous methods
met a 7 MET a 7 VT 7 for solving problems simulating the expert's reasoning:
vl 7 ( P1 , D1 , RES 3 , PR3 , ACT 3 ,VR1 , PRU 7 ) MET a 7 .
At the eighth layer, integrated methods met u 8 MET u 8 VT 8 for solving problems simulating the
reasoning of the team of experts are specified, based on images of the previous layers and heuristics
PRU 8 :
vl 8 ( P1 , D1 , RES 3 , PR3 , ACT 3 ,VR1 , ST 5 , SIT 5 , PRBh 6 , PRBu 6 , MET a 7 , PRU 8 ) MET u 8 .
Such multi-layer model of conceptual-visual language is a tool for a complex description of the
domain of various degrees of generalization, which forms it from a set of simpler models. As an
example, let us consider the use of this model for describing the alphabet of the first two layers of the
metalanguage for the FHIS of the operational and dispatching regional power systems management.
4. Basic layers of conceptual-visual metalanguage
Analysis of papers on visual control, cognitive graphics, methods of information visualization [3, 6,
10] made it possible to identify the main figures underlying the conceptual-visual metalanguage
(Figure 2a), and the set of pictograms [12], examples of which are shown in Figure 2b, for
constructing schematized images of resources, properties and actions in the management of the power
system.
(a) (b)
Figure 2. Basic shapes vocabulary of the language of the FHIS for visual control: (a) basic shapes of
visual metalanguage; (b) examples of pictograms for constructing images of resources, properties and
actions.
A point is the basis of measurements; it generates a line, a movement. A straight line is the
component of all geometric figures. The circle is a universal symbol of integrity, continuity and initial
perfection. The square symbolizes thing or resource category [12], the triangle symbolizes property
category and the arrow symbolizes action category. These forms in combination with the plane, color,
texture, the set of pictograms that represent the visual "names" of concepts, as well as syntactic rules
for recording visual role relationships VR are enough for constructing of a schematic image of any
complexity.
308
At the first layer vl1 of the metalanguage the derivative primitives, dimensions and relations are
constructed with heuristic rules PRU 1 from the kernel's graphic primitives. As can be seen from
Figure 2a limited, relatively small set of graphic primitives is required to represent the basic elements
of the visual meta-language of power system management: straight line segment, circle, pictogram and
filling (Figure 3). These primitives in turn can be described by the set of points that make up them.
The relation of the definition vr11 is represented graphically by a rectangle divided into two parts: the
top contains the primitive to be defined, and the lower one contains definition. Formally this relation is
written with the sign "=".
Figure 3. Graphical primitives of the first layer language: (a) point; (b) straight line segment;
(c) circle; (d) filling; (e) pictogram.
The point in this paper is understood as a square with the size of one pixel with coordinates on the
plane and color. For images drawn by hand fuzzy points and the methods of fuzzy geometry should be
used. The primitive "point" p11 has visual dimensions: the coordinates on the plane d11 [0, d1max1
],
d21 [0, d2max
1 1
] (Figure 4a), where, d1max 1
, d 2max are the maximum values of the coordinates; the color
d31 [0,100] on the scale "Grayscale" (Figure 4b). The point is displayed according to the rule:
p11 (d11 , d21 , d31 ) pru11 (d11 , d21 , d31 ). (2)
Figure 4. Visual measurements of the graphic primitive
"point": (a) the coordinates of the point on the plane; (b)
the color of the point.
When describing graphic primitives in this paper, the coordinates d11 and d 21 are given in the local
coordinate system of the primitive. When constructing a schematic image, the local coordinates of the
primitive are recalculated into the image coordinate system, and primitive rotation and scaling can be
performed. These operations are not considered in the present paper because of the absence of
semantic load.
According to the rule (2) of generating the visual primitive "point" p11 , the primitive "line segment"
can defined:
p12 ( p1b
1
, p1e1 , d31 ) p11 (d11 , d 21 , d31 ) | d 21 (d11 d1b1 )(d 2e
1
d 2b
1
)(d1e1 d1b1 ) 1 d 2b
1
, d1b
1
d11 ( p1b
1
),
1
d 2b d 21 ( p1b
1
), d1e1 d11 ( p1e1 ), d 2e
1
d 21 ( p1e
1
), d11 [d1b
1
, d1e1 ], d 21 [d 2b
1 1
, d 2e ], d11 , d 21 ў ,
where x is the rounding operation of the number x to the nearest integer.
The primitive “circle” is defined by the expression:
p31 ( p1c
1
, p1e1 , d31 ) p11 ( d11 , d 21 , d31 ) |(d11 d1c1 ) 2 ( d 21 d 2c
1 2
) ( d1e1 d1c1 ) 2 ( d 2e
1
d 2c
1 2
)
d1c1 d11 ( p1c1 ), d 2c
1
d 21 ( p1c
1
), d1e1 d11 ( p1e
1 1
), d 2e d 21 ( p1e
1
).
The fill is described by the expression:
309
p14 ({ p11}, d31 ) p11 (d11 , d 21 , d31 ) | p1h
1
p1l1 (d1l1 „ d11„ d1h
1
) p1h
1
p1l1 (d 2l1 „ d 21„ d 2h
1
),
1
p1h , p1l1 { p11}, d1h1 d11 ( p1h
1 1
), d 2h d 21 ( p1h
1
), d1l1 d11 ( p1l1 ), d 2l1 d 21 ( p1l1 ), d11 , d 21 ў .
Graphical representation of the pictogram is a set of points corresponding to its raster image; it’s
generated by the rule pru 2 :
p51 (d41 , d31 , p1dl
1 1
, p1tr ) { p11 (d11 , d21 , d31 ) | p11 (d11 , d21 , d31 ) pru12 (d41 , d31, p1dl
1 1
, p1tr )},
where d 41 is the bitmap image of the pictogram , d31 is its color, p1dl 1
, p1tr1
are the lower left and the
upper right points.
At the second layer of the metalanguage images of resources, actions, properties and relationships
are formed from the visual primitives of the first layer (Figure 4).
The graphical representation of the concept of "resource" res 2 RES 2 (Figure 4a) can be formally
represented by the expressions:
res 2 ( p1dl
1 1
, p1tr , d31 ) r12res pr (res 2 , p1dl
1
) o r12res pr (res 2 , p1tr
1
) o r12res pr (res 2 , d 31 )
p12 ( p1dl
1 1
, p1dr , d31 ) p12 ( p1dr
1 1
, p1tr , d 31 ) p12 ( p1tr
1 1
, p1tl , d 31 ) p12 ( p1tl
1 1
, p1dl , d 31 ),
1
p1dr p11 (d11 ( p1tr
1
), d 21 ( p1dl
1
), d31 ), p1tl
1
p11 (d11 ( p1dl
1
), d 21 ( p1tr
1
), d31 ),
1
d1dl d11 ( p1dl
1 1
), d 2dl d 21 ( p1dl
1 1
), d1tr d11 ( p1tr
1 1
), d 2tr d 21 ( p1tr
1
),
1 1
where o is the operation of gluing concepts [8], p1dl , p1tr are the coordinates of the vertices of the
element (Figure 5a), r12res pr is the relation "to have a property" of the second layer of the language of
the class "resource-property" [9].
The concept "property" pr 2 PR2 (Figure 5b) is represented by the expressions:
pr 2 ( p1dl
1
, p1t1 , p1dr
1
, p51 , d31) r12pr pr ( pr 2 , p1dl
1
) o r12pr pr ( pr 2 , p1t1 ) o r12pr pr ( pr 2 , p51) o
or12pr pr ( pr 2 , d31 ) p12 ( p1dl
1
, p1t1 , d31 ) p12 ( p1t1 , p1dr
1
, d31 ) p12 ( p1dr
1 1
, p1dl , d 31 ) p51 ,
p12 ( p1dl
1
, p1t1 , d31 ) p51 1, p12 ( p1t1 , p1dr
1
, d31 ) p51 1, p12 ( p1dr
1 1
, p1dl , d31 ) p51 …1,
where p1dl1
, p1t1 , p1dr
1
are the coordinates of the vertices of the element, p51 is the inscribed icon, r12pr pr is
the relation "to have a property" of the second layer of the language of the class "property-property".
The concept "action" act 2 ACT 2 (Figure 5c) is represented by the expressions:
act 2 ( p1dl
1 1
, p1tr , d31) r12act pr (act 2, p1dl
1
) o r12act pr (act 2, p1tr
1
) o r12act pr (act 2, d 31 ) p12 ( p1dl
1 1
, p1dr , d 31)
p12 ( p1dr
1 1
, p1cr , d31 ) p12 ( p1cr
1 1
, p1tr , d 31 ) p12 ( p1tr
1 1
, p1tl , d 31 ) p12 ( p1tl
1 1
, p1dl , d 31 ),
1
p1dr p11 (d11 ( p1tr
1
), d 21 ( p1dl
1
), d31 ), p1tl
1
p11(d11 ( p1dl
1
), d 21 ( p1tr
1
), d31),
1
p1сr p11 (d11 ( p1dr
1
) 0,5(d 21 ( p1tr
1
) d 21 ( p1dl
1
)), d 21 ( p1dr
1
) 0,5(d 21 ( p1tr
1
) d 21 ( p1dl
1
)), d31 ),
1 1
where p1dl , p1tr are the coordinates of the vertices of the element (Figure 5c), r12act pr is the relation "to
have a property" of the second layer of the language of the class "action-property".
310
Figure 5. Main graphic elements of the second layer of the conceptual-visual metalanguage: a)
resource; b) property; c) action; d) role relation "resource – resource"; e) role relation "resource –
property"; f) role relation "resource – action"; g) role relation "property – resource"; h) role relation
"property – property"; i) role relation "property – action"; j) role relation "action – resource"; k) role
relation "action – property"; l) role relation "action – action".
The visual relation "resource-resource" vr 2res res (resa2 , resb2 ) (Figure 5d) is formalized by the
expression:
vr 2res res (resa2 , resb2 ) (resa2 , resb2 ) | p1b
1
p1ah
1
p1al
1 1
(d1al„ d1b1 „ d1ah
1
)
p1b
1
p1ah
1
p1al
1 1
(d 2al„ d 2b
1
„ d 2ah
1 1
), p1ah 1
, p1al resa2 , d1ah
1
d11 ( p1ah
1 1
), d 2ah d 21 ( p1ah
1
), (3)
1
d1al d11 ( p1al
1 1
), d 2al d 21 ( p1al
1 1
), p1b resb2 , d1b1 d11 ( p1b
1
), d 21b d 21 ( p1b
1
), d11 , d 21 ў .
Formalized descriptions of visual relations "resource – property", "resource – action", "property –
resource", "property – property", "property – action", "action – action" (Figures 5e-5i, 5l) are
analogous to the equation (3) for the visual "resource – resource" relation, therefore, they are not given
here. The visual relation "action – resource" (Figure 5j) is represented by the expression:
vr 2act res (acta2 , resb2 ) (acta2 , resb2 ) | p1adl
1
p1bdl
1
p1atr
1
p1btr
1 1
, p1adl p1dl
1
(acta2 ),
1
p1atr p1tr
1
(acta2 ), p1bdl
1
p1dl
1
(resb2 ), p1btr
1
p1tr
1
(resb2 ) ,
and а visual relation "action – property" (Figure 5k) is represented by the expression:
vr 2act pr (acta2 , prb2 ) (acta2 , prb2 ) |( p1acr
1
p1bt
1
) ( p1atr
1
p1bdl
1
) ( p1adr
1
p1bdr
1
),
1
p1aсr p11 (d11 ( p1adr
1
) 0,5(d 21 ( p1atr
1
) d 21 ( p1adl
1
)), d 21 ( p1adr
1
) 0,5(d 21 ( p1atr
1
) d 21 ( p1adl
1
)), d 31 ),
1
p1atr p1tr
1
(acta2 ), p1adr
1
p11 (d11 ( p1atr
1
), d 21 ( p1adl
1
), d 31 ), p1adl
1
p1dl
1
(acta2 ), p1bdl
1
p1dl
1
( prb2 ),
1
p1bt p1t1 ( prb2 ), p1bdr
1
p1dr
1
( prb2 ).
Thus, the visual primitives (Figure 5), which constitute the alphabet (core) of the second layer of
the metalanguage for FHIS of the regional power systems management, are formally defined. On their
basis, a set of visual primitives is defined, examples of which are shown in Figures 6a-6d. It’s the
dictionary VN 2 of the second layer of the metalanguage. From the primitives in Figures 6a-6d, in turn,
the schematic images of resources, properties and actions are constructed as shown in Figure 6e.
311
Figure 6. Examples of schematic images of the second layer: a) the schematic image of the resource
"air power line" as a four-role visual relation; b) the property "measure"; c) the action "transfer";
d) "means" for the action; e) schematic image of the action "transfer of the object “electricity”, via an
air power line, from the CHP, to a transformer substation, start time and end time, name,
characteristics, estimates".
These primitives and schematized images can be used as an alphabet in higher-level languages to
organize visual reasoning about resource hierarchies, actions, properties, states, situations, etc.
The presence of interconnected formal and visual descriptions of the same entities allows the FHIS,
depending on the degree of uncertainty in the decision-making environment, to enable the expert to
participate in solving the problem, visualizing the current decision-making situation, and thus to
switch between verbal-logical and visual-shaped mechanisms of reasoning.
5. Conclusion
A formal model of conceptual-visual metalanguage is proposed as a multilayered structure which is
the basis for the automated solution of problems using integrated verbal-logical and visual-shaped
reasoning. An example of the application of the proposed model for describing the alphabet of the
metalanguage of operational and dispatching control of regional electric networks is considered.
The use of the proposed models makes it possible to implement FHIS that are able to dynamically
synthesize an integrated model and method over heterogeneous model and visual fields and simulate
the cooperation, relativity and complementarity of collective intelligence for finding solutions on
verbal-symbolic and visual-shaped languages. FHIS of this class will be able to manage the simulation
process depending on the uncertainty of the problem situation: when the domain of phenomena is
formalized (partially formalized) to use expert knowledge models from a heterogeneous model field to
search for solutions, and when there is a significant uncertainty not removed by accurate analysis and
logical-mathematical reasoning, to activate the mechanisms of visual-spatial, imaginative thinking of
FHIS users.
6. References
[1] Golin E J and Reiss S P 1990 The Specification of Visual Language Syntax Journal of Visual
Languages and Computing 1 pp 141 – 57
[2] Osipov G S 2002 Ot situatsionnogo upravleniya k prikladnoy semiotike [From situational
management to applied semiotics] Novosti iskusstvennogo intellekta [Artificial intelligence
news] 6(54) pp 3 – 7
[3] Bowman W J 1968 Graphic communication (New York: John Wiley) p 210
[4] Lakin F 1987 Visual grammars for visual languages Proceedings of the Sixth National
Conference on Artificial Intelligence (AAAI-87) (Menlo Park, California: AAAI Press) pp
683 – 8
[5] Narayanan N H and Hubscher R 1998 Visual language theory: Towards a human-computer
interaction perspective Visual Language Theory (New York: Springer-Verlag) pp 81–128
[6] Sibbet D 2013 Visual leaders: new tools for visioning, management, and organization change.
(Hoboken, New Jersey: Wiley) p 229
[7] Tarasov V B 2015 Problema ponimaniya: nastoyashcheye i budushcheye iskusstvennogo
intellekta [Problem of understanding: the present and future of artificial intelligence]
Otkrytyye semanticheskiye tekhnologii proyektirovaniya intellektual'nykh sistem (OSTIS-
2015): Materialy V Mezhdunarodnoy nauchno-tekhnicheskoy konferentsii [Open semantic
312
technologies for the design of intelligent systems (OSTIS-2015): Proc. of the V Int. Scientific
and Tech. Conf.] (Minsk: BGUIR) pp 25 – 42
[8] Kolesnikov A V and Kirikov I A 2007 Metodologiya i tekhnologiya resheniya slozhnykh zadach
metodami funktsional'nykh gibridnykh intellektual'nykh sistem [Methodology and technology
for solving complex problems using the methods of functional hybrid intelligent systems]
(Moscow: IPI RAN) p 387
[9] Kolesnikov A V 2001 Gibridnye intellektual'nye sistemy. Teoriya i tekhnologiya razrabotki
[Hybrid intelligent systems: theory and technology of development] (Saint Petersburg:
SPbGTU Publs.) p 711
[10] Kolesnikov A V and Listopad S V 2016 Kontseptual'no-vizual'nyye osnovy virtual'nykh
geterogennykh kollektivov, podderzhivayushchikh prinyatiye resheniy [Conceptual-visual
foundations of virtual heterogeneous groups supporting decision-making] Gibridnye i
sinergeticheskie intellektual'nye sistemy”: materialy III Vserossiyskoy Pospelovskoy
konferentsii s mezhdunarodnym uchastiem [Proc. of the 3rd All-Russia Pospelov conf. with
int. participation “Hybrid and synergistic intelligent systems”] (Kaliningrad: IKBFU Publ.)
pp 8 – 56
[11] Zavalova N D, Lomov B F and Ponomarenko V A 1986 Obraz v sisteme psikhicheskoy
regulyatsii deyatel'nosti [Image in the system of mental regulation of activity] (Moscow:
Nauka) p 172
[12] Kolesnikov A V 2018 Funktsional'nyye gibridnyye intellektual'nyye sistemy vizual'nogo
upravleniya [Functional hybrid intelligent visual control systems] Gibridnye i
sinergeticheskie intellektual'nye sistemy”: materialy IV Vserossiyskoy Pospelovskoy
konferentsii s mezhdunarodnym uchastiem [Proc. of the 4th All-Russia Pospelov conf. with
int. participation “Hybrid and synergistic intelligent systems”] (Kaliningrad: IKBFU Publ.)
pp 18 – 81
Acknowledgements
The work was supported by the Russian Foundation for Basic Research (project 16-07-00271a).
313