Aggregate Parametric Representation of Image Structural Description in Statistical Classification Methods

Abstract

  Finding effective classification solutions based on the study of the processed data nature is one of the important tasks in modern computer vision. Statistical distributions are a perfect tool for presenting and analyzing visual data in image recognition systems. They are especially effective when creating new feature spaces, particularly, by aggregating descriptor sets in some appropriate way, including bits. For this purpose, it is natural to apply the number of criteria designed to compare the distribution parameters of the analyzed samples. The article develops a speed-efficient method of image classification by introducing aggregate statistical features for the composition of the description components. The metric classifier is based on the use of statistical criteria to assess the significance of the classification decision. The developed classification method based on the aggregation of the feature image set is implemented; the workability of the proposed classifier is confirmed. On the examples of the application of variants of the method for the system of the real images features, its effectiveness was experimentally evaluated.

Keywords

• Computer vision
• key point
• descriptor
• data aggregation
• metric classifier
• statistical distribution
• processing speed

1. Introduction 

The use of statistical data science tools in computer vision systems to build classifiers for visual object images aims to provide the necessary performance indicators based on the study of properties, content, the structure of the etalons and implementation of obtained knowledge in the classification process [ 1-5 ]. The finite set of descriptors of the image key points (KP) is considered here as an element of the image space in the environment of vector data with real or binary components in the implementation of structural recognition methods [ 2 ]. Recently, BRISK and ORB descriptors with binary components have become popular due to low computational costs [ 3-5, 11 ].

Statistical data distributions are a perfect tool for presenting and analyzing the data of visual object using image recognition systems. The number of statistical methods can be considered as fundamental apparatus for making a classification decision if the description of the recognized object is given by a set of vectors. The study of data distributions in the set of KP descriptors confirmed its effectiveness in sense of providing the indicators of the classification quality and processing speed [ 2 ].

It may be considered necessary to study in depth the statistical properties of the descriptor set in terms of the main issue of distinguishing multidimensional data to solve the classification problem. This task is especially important in the construction of new effective space features, particularly, by aggregating a descriptor set by their vector components [ 3, 9 ].

For this purpose, it is natural to have developed the use of the apparatus of statistical criteria aimed at comparing the distribution parameters of the studied samples. The classifier based on the aggregate features organizes a new data space as a set of descriptors to evaluate the similarity of the feature vectors of the recognized object and the etalon, and the classification is done by optimizing the degree of such similarity.

The probabilistic model of generating vector data of visual object description is a practical approach to formalizing the process of classifier construction, the essence of which is to build and study statistical distributions of object components based on the aggregation and optimization procedures on the set of classes [ 1, 5 ]. In spite of the widespread use and applied effectiveness of KP descriptors for the classification of visual objects [ 2-4 ], the issue of the statistical nature of these methods and the choice of effective ways to analyze their effectiveness for real data sets remains unexplored [ 1, 2 ].

The main task of the paper is to provide a statistical apparatus of data analysis to build and confirm the effectiveness of the image classifier on the aggregate data representation based on a set of key point descriptors.

2. Problem Statement 

Let’s consider a multidimensional space B n of binary n - dimension vectors, where descriptions of the object and etalons will be constructed. Description Z is defined on the basis of the KP descriptor set of the visual object in the form of a finite set of s binary vectors: n .,  In a more detailed form, let’s consider and analyze the description

Z  z s n

v v1 i1 ,  as a matrix of binary values of s  n size.

We will traditionally consider classification as a transformation

,  Z  z 

v vs1, zv  B where each class is represented by etalon descriptions E j , j  1,..., m , which are available for analysis [ 2 ].

The visual objects classification as assigning their description to one of the etalon classes based on the aggregate representation of the description data using the tools and criteria of mathematical statistics will be studied. Generally, the problem of classification is formally reduced to determining the degree of relevance of two vector sets with binary components.

We will build in a certain way a secondary integrated systems of features

P  pk k1

n on the basis of descriptions Z and E j mj1 and implement them in the classification solution. We use  a metric approach to determine the degree of similarity of feature values P for the object and etalons.

The introduction of aggregate features contributes to a significant acceleration of the classifier decision process, the gain in comparison with the traditional method of voting descriptors reaches hundreds of times [ 4, 5 ]. Also the separation properties of the newly created system of features using traditional statistical criteria will be investigated. The research is a development of the authors' works [ 2, 3, 5 ] in the sense of implementing a generalized parametric bitwise representation of a descriptor set and implementation of new variants of classifiers using statistical analysis for transformed data. Data of structural descriptions of etalons are taken from the article [ 2 ].

3. Literature review 

The formal definition of the classification problem with the description of the image as a set of KP descriptors is formulated in papers [ 2, 3 ], which also study the advantages of implementing a structural     (1)  description model in the methods of statistical classification [ 1, 4-8 ]. It is noted that the primary problem is the excessive computational costs for processing large spatial data sets.

Articles [ 3-5, 8, 13, 20 ] investigate statistical models for the synthesis of feature space modifications to reduce the amount of computation, in particular, the application of data aggregation methods by forming distributions and defining statistical data centers. Works [ 1, 7, 12 ] are devoted directly to the analysis of learning models for the fixed base of descriptions used in computer vision and the definition of the function of belonging to a fixed system of classes.

Articles [ 8, 11, 15, 16, 19 ] discuss the principle of construction feature detectors for the binary descriptors of KP. Studies [ 1, 2, 7 ] contain results on the applied implementation of statistical approaches to the visual images classification using an ensemble processing. In [ 1, 5-7, 13-15 ] methods of evaluating the effectiveness of intelligent systems using statistical and metric measures of similarity are described. The advantages of statistical solutions such as high processing speed, sufficient resistance to distortion and ensuring the required level of classification efficiency are discussed.

Work [10] is used as sources of traditional and modern methods of statistical evaluation, the book [12] contains a description of applied features of software modeling, and sources [ 2-5 ] include the results of authors' research in implementing statistical approaches to develop structural methods image classification. In particular, [ 2 ] proposed technologies of component analysis and spatial processing for the classification of visual objects using statistical characteristics of the structural description of the image.

4. Proposed Approaches 

We consider a transformation Z  P , Z  B n , from a fixed set Z of binary vectors – KP descriptors for a given object into a numerical vector P  pk k1 , which components are calculated by some rule. n This approach will give a possibility to identify and distinguish visual objects on the basis of smaller data, as set of vectors is transformed into a single vector [ 3 ].

We will carry out classification on the basis of estimating the differences in the values of vectors P for different descriptions. The representations of them are considered in two different ways which take into account the structural features of the studied data and, as a result, ensure the efficiency of the recognition process.

The first of the proposed ways for constructing a vector P is to find the average sum of binary values (number of units) consecutively for each bit with the number i separately, based on the full set of object description Z . For a fixed description we obtain vectors of the form:

P (1)  pi(1) in1, pi(1)  1 s zvi ,0  pi(1)  1 (2)  s v1 . 

The vector (2) can be represented as an aggregate parameter formed on a set of descriptors by bitwise analysis of data by adding the values of the corresponding bits (columns of the matrix (1)) and dividing by the dimension s of columns.

We believe it is possible to consider the distribution of values of the i-th bit of the object description close to binomial, which is determined by the Bernoulli formula. According to it the probability of occurrence of v units at the i-th bits in the description of s vectors can be found in the following way: Ps(i) (v)  Csv piv (1  pi ) sv ,    (3)  where pi is the distribution parameter. From the traditional point of view it is equal to the probability of occurrence of a bit equal 1 for the i-th component of the set Z . Also, from the applied point of view this probability is equal to the i-th component of the vector P determined by the formula (2).

Note that the process of comparing two objects can be based on bitwise comparison of values calculated for each of these objects by formula (3), which can be considered as aggregated by bits. But such an idea for a classification rule is not justified enough for application due to cumbersome calculations.

Based on general considerations, we will consider the tuple of values basis of the description Z , like its aggregated parametric representation.

In this paper, it is proposed to classify the studied objects on the basis of the distribution parameter pi . P(1)  p1(1) ,..., p n(1)  obtained on the

P (1 j) , j  1,...m Let’s consider

as a vector aggregated by columns of the matrix for the binary description of the etalon E j with the number j  1,...m , according to (2) and P (1O) as an aggregate vector for the description of the studied object O .

To compare the aggregate descriptions of objects of type (2) and, accordingly, to solve the classification problem, we introduce the classifier

K (1) : k (1)  arg min D(1 j) j1;m

,  D (1 j)  1 n pi(1 j)  pi(1O) , j  1,...m s i1

,  P (1)  p (1) n i i1, pi(1)  1 s

 zvi ,0  pi(1)  1 s v1 , 

Then P (2 j) , j  1,...m is a vector aggregated by the matrix’s rows of the etalon description E j with the number j  1,...m by expression (6); and P (2O) is an aggregated vector according to the description of the studied object O .

When aggregating vectors in the form (6) we obtain the independent samples and directly coordinate comparison of them is impossible. Therefore, for the correct application of the classifier according to scheme (4) - (5), we propose to perform pre-ranking (for example, in ascending order) aggregated according to (6) vectors of the two descriptions being compared. In this case, the classifier takes the form: where

D (1 j) , j  1,...m can be considered as a normalized measure of the similarity of the

pi(1 j)  pi(1O) , i  1,...n, j  1,...m corresponding vectors, and the expression as the Manhattan distance between them.

Classifier (4) implements the principle of analysis "object – etalon" based on the aggregate vector representation P. We emphasize that expression (4) can be considered as a decisive rule, which is formulated in terms of metrics, in particular, Manhattan.

To confirm the significance of the decision, as well as to control the obtained result of the classifier (4) with the involvement of aggregate vectors, we use methods of mathematical statistics, namely, a paired two-sample t-test for averages [10], which provides pairwise comparison of the studied objects as vectors P for a statistically significant difference in their average values. When using this test, two samples of the same volume are considered, in which the elements have a fixed location (as coordinates).

In the process of testing the null hypothesis regarding the equality of the averages in these samples, Student's statistics is used [10]; a level of significance α is established, equal to the probability of making an error of the І type, i.e. rejecting the null hypothesis if it was correct; based on the initial data, the pvalue is calculated as the maximum possible probability of error of the І type. Then, if the p-value is less than the established α, then the null hypothesis is rejected, and an alternative hypothesis is accepted regarding the significant difference of the means (at the level of significance α). Otherwise, there is no reason to reject the null hypothesis of no statistically significant differences between the means [10].

Note that for the sake of universality of the study, it may be appropriate to perform analysis of variance of aggregate vectors constructed from etalons Ej, j=1,…m, in order to ensure that the reduction of data dimensionality did not affect the difference in the etalon set. Also, since the structure of the vectors aggregated by formula (2) requires the consideration of paired samples, in this case it is possible to use only nonparametric analysis of variance, for example, in the form of the Friedman test [10].

The second way for constructing a vector P is to represent the components of the vector as the average sum of unit bits separately for each binary descriptor of the description. In this case we obtain aggregate description vectors in the form:

K (2) : k (2)  arg min D(2 j)

j1;m D(2 j)  1 s pi(2 j)  pi(2O) , j  1,...m n i1 ,  ,  (4)  (5)  (6)  (7)  (8) 

Due to the independent nature of the data testing for significant differences by statistical methods also in this case requires the use of appropriate approaches. The essence of two-sample t-test for averages for two independent samples which is appropriate in this situation is to compare the averages of two sets of disordered elements, which are calculated separately for each binary descriptor of the full description. The test procedure is the same as for the case of dependent samples, and differs only in the formula, according to which the statistics are calculated on the basis of the given data [10].

Also note that in order to confirm the fact of statistically significant differences of the aggregate vectors (6) constructed on the etalons Ej, j=1,…m, it can be appropriate to provide analysis of variance of them. As the structure of aggregate vectors requires the consideration of independent samples, it is possible to use the methods of both parametric and nonparametric variance analysis (for example, in the form of the Kruskal-Wallis test [10]).

5. Experiments 

Let’s consider an example with experimental descriptions of three fixed etalons E1, E2, E3, and E4, obtained from E1 by rotation. Examples of images based on the results of software modeling with the formed coordinates of the BRISK KP descriptors are shown in Figure 1 [ 2, 11, 16 ]. For the descriptions of these images in the form of a set of descriptors, our calculations are performed. In the example, n = 512 is the dimension of the descriptor, s = 500 is the number of descriptors in the description, m is the number of the etalons (m = 3).

We present the result of implementation of the proposed classification approach based on the values of the parameters P calculated on the database of three etalons E1, E2, E3 (Fig. 1) and the image E4, transformed by rotation of E1. Note that the representation using KP descriptors provides invariance to the transformations of displacement, rotation and scale of the analyzed object [ 3 ].

Fragments of the calculation results for aggregate vectors P (1 j) , j  1,2,3,4 of the form (2) for objects E1, E2, E3, E4 are given in the table 1.

Table 1 

Fragments of vectors

P (1 j)  p (1 j) 512

i i1 , j  1,2,3,4 applied to samples represented by aggregate vectors results are shown in the table 2.

Table 2  

The results of the application of a paired two‐sample t‐test  

According to the formulas (4, 5) using the data of E4 we have:

As we can see, the application of the classifier (4) gives the correct recognition of the object E4 as a transformed etalon E1 because it has the best similarity with the first etalon.

To confirm the fact of statistically significant closeness of E4 to E1 as well as statistically significant difference of E4 from other etalons E2, E3 we use a paired two-sample t-test for averages P (1 j)  p (1 j) 512 i i1 , j  1,2,3,4 , formed by (2). The

Samples  p – value  Significance 

Е1, Е4  0.252  no  Е2, Е4  0.002  yes 

Е3, Е4  0.0000000006  yes 

As we can see the first p - value is much higher the significance level α = 0.05 (equal to the probability of error of the first type). That indicates the absence of statistically significant differences between P (11) and P (14) (i.e. between the first etalon E1 and object E4 transformed from E1). Other obtained p - values are less than 0.05, confirming a statistically significant difference in pairs between P (14) , P (12) and P (14) , P (13) (i.e. between E4, E2 and E4, E3).

Note, that for a large sample size (in the example we have n = 512), checking the data for compliance with the normal distribution law when using a paired t-test is not mandatory [10].

Note also that the visual comparison of bar charts which is a graphical representation of aggregated vectors by formula (2) is a clear confirmation of the results obtained on the difference of objects (Fig. 2) (similarity between P (14) , P (11) and significant difference between pairs P (14) , P (12) and P (14) , P (13) .

Fragments of the calculation results for aggregate vectors P (2 j) , j  1,2,3,4 of the form (6) for objects E1, E2, E3, E4 are given in the table 3.   Table 3  

Fragments of vectors 

P (2 j)  p (2 j) 500

v v1 , j  1,2,3,4    Component number  1  2  3  4  5  6  …  494  495  496  497  498  499  500  components of the vectors P (2 j) , j  1,2,3,4 and using the data of E4: applied to samples represented by aggregate vectors P (2 j) , j  1,2,3,4 formed by (6). The results are shown in the table 4.

Table 4 

The results of the two‐sample t‐test  

Samples  p ‐ value  Significance 

Е1, Е4  0.843  no  Е2, Е4  0.024  yes 

Е3, Е4  0.0000000005  yes 

We also see that the first p - value is much higher the significance level α = 0.05. That indicates the absence of statistically significant differences between E1 and E4. Other p - values are less than 0.05, confirming a statistically significant difference in pairs between E4, E2 and E4, E3.

Similar to the previous one, the visual comparison of bar diagrams, which are a graphical interpretation of the vectors aggregated by formula (6), confirms the results obtained (Fig. 3).

P (21)   P (22)   P (24)   P (23)  

The main result of this research is the development of models for the image classification based on the apparatus of statistical analysis of component sets of the object description and metric means of classification deciding. The synthesis of an aggregate feature system based on KP descriptor set makes possible to build a classifier that works successfully for the real images database.

The proposed approaches of data analysis models are based on the degree of similarity between the object and etalons are workable and quite effective. Computational simulation performed on the example with 3 etalons confirmed efficiency of the proposed method using statistical criteria of significant data differences.

7. Conclusion and Future Work 

Statistical data analysis remains a powerful research factor for intelligent decision making, machine learning, and data science. The conducted research makes it possible to evaluate the applied efficiency of the application of the feature aggregate system for the effective implementation of the visual object classification by a set of key point descriptors. The research has shown that the available information in the form of a bit representation of the object description is quite sufficient for statistical differentiation of data for different visual objects.

The novelty of the investigation is the further development of the image classification method using an integrated statistical feature system for structural description, confirmation of its effectiveness and the significance of this system for classification within the given image database. The proposed classifier construction method allows further generalization in terms of fragment size aggregation that implies reduction of processing time.

8. Acknowledgements 

9. References 

The work was performed within the framework of the state budget research of Kharkiv National University of Radio Electronics "Deep hybrid systems of computational intelligence for data flow analysis and their rapid learning" (№ ДР0119U001403).

References

[1] P. Flach , Machine learning . The Art and Science of Algorithms that Make Sense of Data , New York, NY , Cambridge UniversityPress, 2012 .

[2] S.V. Gadetska , V.O. Gorokhovatskyi , N. I. Stiahlyk , N.V. Vlasenko , Statistical data analysis tools in image classification methods based on the description as a set of binary descriptors of key points . Radio Electronics , Computer Science, Control, № 4 ( 2021 58- 68 . doi: 10 .15588/ 1607 - 3274-2021-4-6

[3] V.O. Gorokhovatskyi , S. V. Gadetska , Determination of Relevance of Visual Object Images by Application of Statistical Analysis of Regarding Fragment Representation of their Descriptions . Telecommunications and Radio Engineering , 78 ( 3 ) ( 2019 211- 220 .

[4] Y.I. Daradkeh , V. Gorokhovatskyi , I. Tvoroshenko, S. Gadetska , M. Al-Dhaifallah , Methods of Classification of Images on the Basis of the Values of Statistical Distributions for the Composition of Structural Description Components . IEEE Access , 9 ( 2021 92964- 92973 , doi: 10.1109/ACCESS. 2021 .3093457

[5] S. Gadetska , V. Gorokhovatskyi , N. Stiahlyk , Image structural classification technologies based on statistical analysis of descriptions in the form of bit descriptor set . In CEUR Workshop Proceedings: Computer Modeling and Intelligent Systems (CMIS-2020) , 2608 , 2020 , pp. 1027 - 1039 . URL: http://ceur-ws. org/ Vol- 2608 /

[6] A. Oliinyk , S. Subbotin , V. Lovkin , The System of Criteria for Feature Informativeness Estimation in Pattern Recognition . Radio Electronics, Computer Science, Control. № 4 ( 2017 85- 96 . doi: 10 .15588/ 1607 -3274-2017-4-10

[7] J.F. Peters , Foundations of computer vision: Computational Geometry. Visual Image Structures and Object Shape Detection , Cham, Switzerland: Springer International Publisher ( 2017 ) 417 p.

[8] A. Hietanen and et al., A comparison of feature detectors and descriptors for object class matching , Neurocomputing , 184 ( 2016 ) 3 - 12 .

[9] O. Gorokhovatskyi , V. Gorokhovatskyi , O. Peredrii , Analysis of Application of Cluster Descriptions in Space of Characteristic Image Features . Data , 3 ( 4 ), 52 . ( 2018 ). doi: 10 .3390/data3040052. URL: https://www.mdpi.com/2306-5729/3/4/52