The solution to the problem of reducing data flows about graphic objects was carried out mainly by developing coding and compression formats. Among those methods of statistical compression considering psychovisual and psychophysical properties of human multimedia information perception are particularly effective. However, in terms of implementation, this is an extensive rather than intensive way of development solving the problem of reducing data flows between individual nodes of the system. To make decisions, the operator needs those graphic objects that provide a solution to the given task, which requires organizing the search for the necessary graphic objects among the entire set in the data store. The user compares the given fragment with the obtained aserch result using a graphical representation of the data, which causes additional time costs. Special complications arise graphic objects data is being processed, since information about each image can be characterized by different criteria. In addition, the constant growth of the volume of information causes a decrease in its effectiveness. It is appropriate to define criteria for data processing and retrieval and to outline approaches for evaluating the quality of retrieval. Existing methods and means of semantic description and analysis of a graphic object require the operator's participation at the pre-processing stage, so their implementation requires additional research to improve work efficiency. The analysis of existing methods shows that a significant simplification of the solution of individual storage and search problems can be achieved by structuring the description of data in repositories using a specially created semantic description for this purpose. However, now these methods are practically not used to organize information search, and their implementation in search engines requires significant time and system resources.
The article considers approaches of information technology creation for searching graphic data
by the content of graphic objects based on their semantic analysis. Before considering the
method, it is necessary to dwell on some conceptual provisions regarding semantics. First of all,
we note that the term "semantics" is used in different works in different aspects that determine
its meaning. In the theory of information, which investigates general scientific methodological
problems of the development of science, information and semantics are considered as concepts
that are close in essence [
The purpose of the work is to develop and research the principles of semantic analysis implementation and the use of its capabilities in the organization of searching by the graphic object content.
Today, there is a fairly large number of algorithms and methods of graphic data analysis
that are used in various areas of digital information processing: from the fight against
spam to systems for passenger traffic tracking [
We will dive into the most common methods of image content analysis: • • frequency-spatial display of a graphic object; image histograms.
Image analysis can be interpreted as its recognition, i.e. the maximum compression of the volume of information to eliminate the redundancy of processed images. Several difficulties arise in solving this problem: algorithmic and technical implementation. The compression algorithm should be sufficiently reliable and at the same time, if possible, economical. The search for effective algorithms is now carried out by researching the mechanisms of the brain, as well as the development and research of various recognition programs.
The technical complexity is caused primarily by relatively low speed and insufficient memory capacity. In addition, they are poorly adapted to the input and processing of multidimensional arrays of information, which are images.
The task of automatic analysis is reduced to finding some function that maps a set of images
into a set whose elements are image classes. The process of determining such function should
be carried out in three stages [
functions that determine the features, as a result of which the image is coded. Classification. As a result of these stages, a set of data appears, which can be considered the features of the initial imagef (x ,y ) .
This set should be considered as a point in n-dimensional space. If the areas occupied by a class in this space are indicated or the probability density for each class is specified on it, then, using measures of geometric proximity and maximum likelihood, this image can be assigned to a certain class, that is, "classified."
The first two stages — image pre-processing and feature selection — are quite difficult. This is due to the redundancy of the image due to the presence of an extra background; uncertainty of position, orientation, scale; therefore, the correct choice of various methods of image filtering, normalization, obtaining invariants, etc., is of great importance.
The procedure of forming signs or parameters of the primary description, which is a transition from a two-dimensional function (image) to a system of numbers (signs), can be interpreted as a task of some functionality.
The third stage of solving the problem of analysis — classification, associated with the performance of a complex of simple arithmetic or logical operations, can be successfully solved with the use of computer technology.
Therefore, in the images analysis there are mainly difficulties associated with pre-processing and selection of features.
During analysis tasks solving, two groups of tasks can be distinguished. The first one is when the information is contained in the integral properties of the entire image (textures — cloud cover, interferograms and shadow pictures, etc.). Such tasks are solved by analysing the statistical properties of a objects set that are combined simultaneously in the field of vision, without considering the individual local properties of each of them separately. To compress the information in such images, the histograms of the informative parameters’ distribution most related to image decomposition need to be used. The second group is the recognition of individual objects (or fragments) according to the given geometric parameters and their selection against the background of objects collection, their counting, which is another method of images abbreviated description.
The basis of any semantic analysis are the following factors: • • • • availability of an interpretation system; rules of the interpretation system usage; homomorphism between the rules for using the interpretation system and the rules for building and presenting the objects to be interpreted; semantic consistency of the description system and presentation of the graphic objects interpretation system.
First, it is needed to be noted that in this case only graphic objects will be discussed and only their semantic analysis will be investigated. The constituent parts of the interpretation system are the following components: semantic dictionary of the graphic objects system Sc ; semantic environment G ( Sc ) ; a system of environment rules, which is formally recorded in the form Σ ={ξ1 ,ξ2 ,...,ξn } .
A semantic dictionary Sc — is an ordered set of attributes, primitives, and elements used to describe graphical objects. Each of the attributes, primitives and elements is defined by a certain identifier, which will be used later for formal description as the value of a certain variable. It is considered that each individual graphic object as an object from which primary attributes or graphic primitives can be isolated.
Semantic dictionary Sc is a more complex structure than a traditional language dictionary,
as it is focused on solving problems related to the semantic analysis of the graphic object
content. Given that the attributes and graphic primitives that we will use to describe graphic
objects are a limited set, it is possible to structure a corresponding dictionary. If necessary, the
dictionary will only grow due to attributes or new primitives. For example, among the
dictionary elements, you can introduce a certain hierarchical dependence of the importance of
the elements in the description of the subject area. This principle is widely used in various
analysers and is known as the principle of determining key, or control, elements [
For the full operation of the system, it is necessary to investigate the possibility of creating a system for evaluating the search engine quality based on the use of semantic analysers. In this case, the task of creating and researching search result evaluation models for a certain system arises. To create them, you can use the following approaches that make it possible to solve the problem of determining the search evaluation in graphic objects at the semantic level: dictionary view of the analysis of the reduction degree in the effectiveness of the system's functioning; system load measures; the strategic significance of the object and the search result. • • • • • •
Let us consider in more detail the approaches to the implementation of semantic search of graphic objects by content.
The first approach consists of the semantic loading implementation of the its carriers, which for a graphic object are attributes, primitives, elements, their sets, fragments and the object in general. The listed components are means that are directly placed in the graphic object and actually form this object. Mediated carriers of semantics are the means by which objects are created or analysed. These include the following components: • • • semantic dictionary structure G (S c ) ; a system of rules for building graphic objects ψ = {ψ 1 ,...,ψn } ; a system of parameters characterizing the object P = {Pc ,...,Pk } .
Lets consider a possible formal description of the semantic structure of the dictionary. dictionary means the value of its semantic
Assigning a key value attribute x i from the S c significance for the entire environment of the subject area. It is obvious that the degree of such significance can be different for different attributes and primitives. This means that the S c dictionary can be partitioned into subsets that combine sets of attributes or primitives with equal or close values of semantic significance. Such subsets are independent of the specific interpretation of a particular set of elements I (x i ) . Therefore, it is possible to formally represent the simplest structure of the S c dictionary in the form of a D (S c ) tree. Semantic criticality k i of an attribute x i of ax i (k i ) set can have a certain level in the given context of using other attributes and primitives with lower levels of criticality, for which k is a j <k i critical element in S c if x i (k i ) . Such element will be called a key. We will assume that critical dependence on the context can be developed only at one level of the hierarchy of the degree of significance of the elements of the context. Further measures of significance, although they exist in other attributes or primitives, are not contextually determined for key or critical elements. This is because the measure of semantic element criticality is that it has the maximum value of its own semantic expressiveness or semantic value. In this case, the structure of the dictionary can be formally represented in the form of some hierarchical tree, which will be analytically written as follows:
G (S c ) =ik,1 S ik (x ,...,x ik ,m ) →S jz (x jz ,1 ,...,x jz ,m ) &S iz (z 1 ) & ... &S iz (z m ) , (1) where S ik
— set of key elementsx ik ,j ; S jz
— a set of contextually determined elements x jz ,i ; S iz (z i ) — sets with different levels of semantic significance of the elements listed in the dictionary S c .
Consider the relationship between the semantic significance x i elements x i (k i ) and the interpretation of the corresponding element I (x i ) . To ensure the of an element with key effectiveness of this approach, we will assume that the interpretation I (x i ) depends on the amount of semantic significance for key elements x i (k i ) and for other elements x i (z i ) equally in relation to their semantic significance. For this, we will assume that the value determined by the interpretation I (x i ) depends on the number of attributes or primitives used to describe this element. For example, a dictionary element Sc can be the following structure: (2) n :=<x i >→<x i 1 ,...,x i*k ,...,x in , where n — numeric or any other identifier of the S c dictionary elementx i ; x i*k — an element I (x ik ) that belongs to the subject area described by the S c dictionary and has its own interpretation.
It can be seen from the given expression that the Sc dictionary should not contain all the elements that exist in the description of the graphic object. The S c dictionary contains only the terminological base of the elements of the description of the object used in the format chosen to represent such a graphic object. The number of elements implementing the I (x i ) interpretation is not only the number of elements of one line of I (x i ) interpretation but is also summed up with the number of elements of the I (x ik ) interpretation extension for the element x ik from the S c dictionary. It can be written like this: π x i (k i ) =π I (x i ) =x i 1 +x i 2 + ... +x i ,k −1 + ... +x in + (x k 1 + ... +x km ) , (3) x ik interpretation within the S c dictionary. where π — the function of counting the number of the interpretive description elements x i and — elements of the interpretive description of the element x i*k , which has its own I (x ik ) Thanks to this definition of the interpretive I (x i ) description it is possible to determine the degree of semantic significance of individual elements of the S c dictionary. It is obvious that for S ik (x ik 1 ,...,x ikm ) does not necessarily hold for π (x ik 1 ) = π (x ik 2 ) = ... = π (x ikm ) equality. The value or the semantic significance of anx i (k i ) element in a S ik set can be specified by a certain interval of numerical values or π x i (k i ) = m ,n . Similarly, values of semantic significance are calculated for other elements that are not included in the set of S ik key elements, which will be equal to π x i (z i ) . Ranges of S iz values are also specified for such πi (x iz ) sets, which determine whether each x iz of them belongs to one or another S iz set. Such a structure of the semantic dictionary will be written in the form of a relation:
S ik πi (x i (k i )) → {S j π j (x j (z j )) ⊕ ... ⊕S m πm (x m (z m ))}, where ⊕ — the symbol of the logical function is the sum modulo two. Let’s consider the following statement.
Statement. For a set of key elements S ik (x i ) , such a set of semantically significant elements S jz (x j ) will be contextually conditioned, for which the relation holds:
π j (x j (z j )) = max{π 1 (x 1 (z 1 )) ,,πn (x n (z n ))} .
Assume that the statement is not correct or that condition (2) will not hold. In this case, it (5) can be specified that S ik (x i (k i )) →S jz (x j (z j )) and there is a relation:
π (x j (z j )) ≠ max{π 1 (x 1 (z 1 )) ,,πm (x m (z m ))} , It means that through S i (x i (z i )) there is x j* (z j* ) , for which:
π j* (x j* (z j* )) = max{π 1 (x 1 (z 1 )) ,,πm (x m (z m ))} . are determined α ,β in a S which there is a relation:
As each x j (z j ) with the value π j x j (z j ) automatically defines the range for eachx j (z j ) , that included in the corresponding setS j x j (z j ) , because for allS i (x i ) the ranges of their values c dictionary, then it can be claimed thatthere isS j (x j (z j )) in S c for Then the equality holds: π j 1 (x j 1 (z j 1 )) && π jm (x jm (z jm )) = = max {{ππin11((xxin11(z(zi n11))),) ,,π, πimnm(x(ixmn m(z i(mzn)m) =)S) =iS(xni (x(zni ()z)}n, ))},.
π j* (x j* (z j* )) =1(x π j j 1 (z j 1 )) ∨∨ π jm (x jm (z jm )) .
This equality indicates that the I ( xi ) interpretation in theS c dictionary is formed in such a way that all x i elements for whichπ (x i ) will fall into one α ,β range defined by I (x i ) are combined into oneS j (x j (z j )) set of semantically significant elements. Therefore, the assumption that there is no condition for statement (5) is controversial.
We will prove its sufficiency. (4) (6) (7) (8) (9) Let {S ik x i (k i ) →S j x j (z j )} & {S ik x i (k i ) →S r x r (z r )} , then S ik x i (k i ) allows an ambiguous interpretation: But, according to the
I i* (x i (k i )) =I i 1 (x i (k i )) &I i 2 (x i (k i )) .
statement, the following relations must hold π j x j (z j ) ∈π j S j (x j ) and for components πr x r (z r ) ∈πr S r (x r ) : з (10) for π j x j (z j ) = max{π 1 (x 1 (z 1 )) ,, πn (x n (z n ))} πr x r (z r ) = max{π 1 (x 1 (z 1 )) ,, πn (x n (z n ))} , where the maximum is taken for all elements of S c , except for the S ik (x ik ) set, which means also for all S ( j x j (z j )) . Then the ranges forπ j ( x j ( z j )) and for π r ( xr ( zr )) intersect or: {π j ( x j ( z j )) ∈ α j ,β j }∩{π r ( xr ( zr )) ⊂ [α r ,β r ]} ≠ 0 .
(11) And this is impossible according to the S (
j x j (z j )) structure within the S c limits, which are put in accordance with the I i (x i (z i )) interpretation for allS j (x j (z j ))∈S c , which proves the statement about the condition of the existence of a contextually conditioned set for a set S ik (x i (k i )) .
The structure of the S
dictionary can change in the direction of increasing individual
c
components, and, accordingly, the number of components can change during the transition
from one structure to another. We will call the structure that operates on S dictionary
c
elements or G (S c ) , the lowest structure in the hierarchy, by which we will distinguish them.
We take the name of the lowest structure since it operates with dictionary elements, which are
the smallest elements of the accepted set, which is used to describe graphic objects from the
point of view of their value as the minimum possible element that defines semantics as such
[
The next structure in the hierarchy is the structure that operates on sets of elements. An element set is a collection of semantically defined attributes or primitives. Formally, a set of elements can be written in the form of such a ratio:
V = {v 1 (x i 1 ) ,,v m (x im )} , where v j (x ij ) = {x 1j ,,x kj } .
It is obvious that each element of the expression receives and conveys a certain semantic
significance based on the expression that consists of these elements. It is clear that an expression
is not formed from an arbitrary set of elements, and the rules fo r forming a set of elements are
based on the rules defined by the content and on the rules of semantic admissibility. The rules
of semantic admissibility are the result of the interpretation accepted in the subject area. In the
case of subject areas of interpretation that are artificial and created by man, such prerequisites
are quite few, and one or another semantic significance in many cases is based on generally
accepted and agreed agreements. A well-known example of such arrangements are standards
or formats. Of course, any agreement about the semantic significance of a certain element or a
set of elements is justified by the nature of the object, which is defined by the corresponding
definitions, but the basis of such justification is the possibility of connecting or connecting the
new semantic significance with the semantic meanings of already defined objects. objects or
elements [
The subject area, in relation to which graphic objects are composed, is quite narrow and
quite precisely defined. This is due to the generally accepted practice of defining graphic
primitives, which originates from geometry. In addition, there is a significant part of graphic
objects so highly specialized that S dictionaries can describe such sets of elements only if they
c
are structured at the level of sets of elements. If we are not talking about objects — the works
of artists, then it can be assumed that in dictionaries structured at the level of sets of elements
or V (S c ) , each of them has its own specific and not very large in number of elements
interpretation I (v i ) :=<x i 1 ∗∗x > , where the sign "∗" means the function of consistency
between the elements that make up the description. Obviously, such an interpretation is not
sufficient for a non-specialist in the relevant field, but we will not consider these aspects.
Therefore, during the formation of graphic objects, it should be possible, regardless of the
specifics of the object, to use separate sets of elements contained in the S dictionary [
If a S c
dictionary with V (S c ) structure is created for the analysis of objects, then in this case, as well as for a dictionary with G (S c ) structure, it is possible to form hierarchical dependencies between sets of elements, depending on their semantic meanings. In many cases, it is possible to define control sets by analogy with control elements, which are quite convenient in solving the problems of analysing the content of a graphic object. c
The problem of extracting a graphic primitive from a graphic object can be formally
presented as a classification problem. If we use the results of the wave algorithm described
above for the input of the neural network, then we will get a classification problem when a
certain object P is a graphic primitive based on a set of features p — the list of key points of
i
the algorithm contour returns the value of the class c i — the type of graphic primitive to which
the contour refers, which is considered [
If you create a neural network with N inputs and M outputs and train it to return C c i ∈C , vector at the output, when P is input, we will solve the problem of determining graphic primitives.
To solve classification problems, it is advisable to use the Kohonen network. To determine the class to which an object belongs, it is necessary to select one of the neurons of the Kohonen layer with the maximum output. This is done by an interpreter, which is either a software product that selects the neuron with the highest output, or an additional layer of neurons. So, a Kohonen network will consist of 3-4 layers of neurons, the first of which is the input layer, the next is the Kohonen layer, the additional layer and the output. In order to obtain the sum of outputs approaching unity, it is advisable to apply the Softmax function, which can be interpreted as the probability of the object belonging to a certain class.
To build an image model and its semantic description, it is necessary to use the alphabet of conventional designations to describe graphic primitives. To do this, it is necessary to highlight their main types that can be used in the image, while the advanatge of their use is that the image model built on their basis is flexible, does not depend on scaling and positioning. In addition, such a model does not depend on the colour of the primitive execution (unless special conditions are specified).
Let's consider the proposed model of presentation of a conditional alphabet or graphic primitives.
First, let's build a dictionary of graphic primitives [
In addition, graphic primitives that must be used to create a semantic description of a graphic image must meet the following requirements: • • it is sufficient to fully submit the content of the graphic object; easy to formalize.
Therefore, the contours of the image selected from the graphic object should correspond as much as possible to a certain type of graphic primitives of the semantic dictionary. In order to maximally reduce the time of the process of identifying the contour of the image and its comparison with the elements of the semantic dictionary, it is necessary to choose such graphic primitives as the elements of the semantic dictionary, which can be formalized most simply.
To determine the set of elements of the semantic dictionary, we will classify image contours that may occur in a graphic object [25, 26, 27].
By continuity, the contour of the image can be closed and open. We consider such a contour of the image as closed, in which the wave, launched along the contour, returns to the starting point. An open loop is one in which the wave going around the loop fades out at two key points and does not return to the initialization point. According to this feature, graphic primitives are divided into lines and shapes. A line is an open contour, a shape is a closed one.
According to the presence of key points, there are contours that can have intermediate key points (except for the two points where the wave faded in the open contours) and contours without them.
After the segments connecting the key points, the contours can be with straight segments, with curved segments, and with mixed segments.
According to the given classification, the semantic dictionary should contain the following graphic primitives: • • • •
Graphical primitives describing open contours. We will call them graphic primitives of the second type.
Line — an open contour with two key points connected by a line segment.
Curve — an open contour that does not correspond to a straight line in any way. Graphical primitives describing closed contours. We will call them graphic primitives of the first type. Since most of the geometric shapes that will be used to denote the first type of graphic primitives generally consist of vertices connected by straight line segments, and perfect straight lines are rarely found in a graphic object, it is necessary to introduce additional types of graphic primitives that will identify contours with segments of curves. To designate such contours, we introduce the word "almost" (fuzzy), which means that formally the figure meets all the requirements for defining the contour and the key points are connected in it by segments of curves or by a mixed type. Circle — a closed contour without key points, all points of which are equidistant from the center of gravity of the contour. “Fuzzy" circle — a closed contour without key points, not all points of which are equidistant from the center of gravity of the contour.
Triangle — a closed contour with three key points connected by three straight line segments. "Fuzzy" triangle — a closed contour with three key points connected by three curve segments or mixed segments.
Quadrangle — a closed contour with four key points connected by four straight line segments. "Fuzzy" quadrilateral — a closed contour with four key points connected by four curve segments or mixed segments.
Polygon — a closed contour with five or more key points connected by line segments. "Fuzzy" polygon — a closed contour with five key points or more, which are connected to each other by curve segments or mixed segments.
Primitives of the first type are the most significant elements that unambiguously characterize the content of a graphic object. Other graphics primitives are smaller and may simply be elements of primitives of the first type in the case that one of the graphics primitives is derived from another by cutting graphics.
With the help of tools of the XML language, we will present the alphabet in the following way.
Having selected graphic primitives as elements, each of them, depending on the shape, can be assigned a value and get their semantic description: <Element GroupID> <ID>1</ID> <Name></Name> <Value></Value> </Element>
If necessary, the dictionary can be expanded by entering element attributes, such as fill colour, line colour, etc. XML tools make it possible to implement any changes by changing the format of the dictionary, introducing new elements into it, and reformatting existing descriptions.
For example, after expanding to include colour, fill, and line colour attributes, the dictionary description would have the following form: <Element GroupID> <ID>1</ID> <Name></Name> <Value></Value> <Attributes> <Attribute Name=”color”>
<Value></Value> </Attribute> <Attribute Name=”filling”>
<Value></Value> </Attribute> </Attributes> </Element>
The dictionary will consist of a set of elements with their meanings, and, accordingly, the semantic description will consist of dictionary elements (Fig. 1).
Graphical objects in their essence are most fully characterized by words of natural language, therefore it is advisable to describe them based on methods of formal grammars and semantic features.
To determine the semantic significance of graphic objects and their fragments, it is necessary to build a semantic dictionary from the subject area, which would include the main features of graphic objects, their attributes and hierarchy. In the subject area of the attributes of graphic objects, their completeness, semantic significance and consistency should be determined, which will reduce the search time and improve its quality.
The rules for forming a set of dictionary elements are based on the rules defined by the content and on the rules of semantic admissibility. The rules of semantic admissibility are the result of the interpretation accepted in the subject area. In the case of subject areas of interpretation, which are artificial and created by man, such prerequisites are quite few, and one or another semantic significance in many cases is based on generally accepted and agreed agreements. Proposed inference rules for the semantic analyser of graphic objects by the methods of formal grammars based on the consistency and completeness of the semantic dictionary.
The authors are appreciative of colleagues for their support and appropriate suggestions, which allowed to improve the materials of the article. Scientific and Technical Conference on Computer Sciences and Information Technologies, 2, art. no. 8929750, pp. 32-36, doi: 10.1109/STC-CSIT.2019.8929750. [25] Y. Wang, Z. Luo, D. Chen and Y. Li, "Semantic Segmentation of Fire and Smoke Images Based on Dual Attention Mechanism," 2022 4th International Conference on Frontiers Technology of Information and Computer (ICFTIC), Qingdao, China, 2022, pp. 185-190, doi: 10.1109/ICFTIC57696.2022.10075210. [26] A. Ortis, G. M. Farinella, G. Torrisi and S. Battiato, "Visual Sentiment Analysis Based on on Objective Text Description of Images," 2018 International Conference on Content-Based Multimedia Indexing (CBMI), La Rochelle, France, 2018, pp. 1-6, doi: 10.1109/CBMI.2018.8516481. [27] W. Wang, Y. Ding and C. Tian, "A Novel Semantic Attribute-Based Feature for Image Caption Generation," 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada, 2018, pp. 3081-3085, doi: 10.1109/ICASSP.2018.8461507.