-The problem of boundary detection by the Canny (John F. Canny) algorithm is investigated as a complementary tool for the analysis, segmentation and classification of the hyperspectral (HSI) and multisensor images objects. The possibilities of various measures of the distance between the k-dimensional vectors of signatures in the detection of classes and states of HSI objects are investigated: the angular distance (in the form of the cosine of the angle); Pearson correlation coefficient; Euclidean norms. First of all, the possibilities were analyzed in a situation where the object of interest is determined by the features that appear in part of the HSI channels. Based on the feature vector, an object boundary is detected. Then the object inside the boundary is examined in another part of the channels (or in all channels) by the histogram of the corresponding metric or by the values in the individual channels. An adaptation of the John F. Canny algorithm has been implemented to detect the boundaries of the region of interest as a tool for the study and classification of HSI objects, which creates new opportunities for analysis. The angular distance is determined as the leading scalar metric for detecting boundaries. Values of standard deviations, an average of the signature, Euclidean norms of signatures are used as features of the second level classification. The references of the contoured objects can be used as references of the state of the object for comparative studies, and for further unmixing in units of the library reference objects.
The problem of detecting the boundaries of image objects is a classic problem of image processing. Many different methods and algorithms have been developed for twodimensional single-channel images that are well represented in open sources. The shape of an object 's boundaries, its area, and other characteristics associated with the shape are important features of the image object classification. One of the first, and still recognized as the best boundary detector, is the Canny boundary detector [1].
In the study of hyperspectration images (HSI), the role of detecting the boundaries and shape of an object would seem to become substantially more modest due to the ability to detect the class of the object by the spectral characteristic of a single pixel belonging to the object alone. And finding the boundaries of an object after it is classified appears to be a trivial task.
In fact, by working with hyperspectration imaging, we are able to detect the frequency features of an object and all its possible states by many sensors operating in different wavelength ranges. We may allocate the boundaries of the object in any channel or group of channels, where it is expressed quite contrasting. Then we can transfer these boundaries already as an object mask to other channels, where we can analyze the object as a whole, and also its subclasses or states. Using simultaneously several channels or even several sensors to detect the boundary of an HSI object or to learn its properties, each time we must use the mapping of a pixel signature to a scalar metric capable of detecting the boundary and the required features of its subclass or its state. Under these conditions, the problems of boundary detection, segmentation, and classification of HSI objects become complementary tools for exploring hyperspectral and multisensory images. The disclosure of the potential embedded in multi-sensor image research and intersensory correlation of data on object patterns and their states requires research and development of tools, including an adaptation of well-established single-channel algorithms. One area of management, waiting for solutions and tools in that area is precision farming.
In the publication [2] of 2014, the problem of extracting boundaries of HSI objects based on different measures of similarity of spectral characteristics (signatures) of image pixels and refining their estimation and gradient estimation under known noise characteristics is considered. Examples are given in which correlation estimates give significantly better results on contour estimation than brightness in channels. Suggested by polynomial smoothing of signatures in the interests of image compression and a method for correcting atmospheric distortions on this basis, involving the estimation of the statistical profile of atmospheric distortions of a particular HSI.
Research is underway on the possibility of detecting boundaries on the basis of multidimensional statistical methods of image recognition. Thus, in the work [3] of 2006, it is proposed to use an estimate of the joint probability density function of two neighboring signatures. The basic idea behind the approach is that pixel combinations characteristic of object boundaries are rare and can be seen as emissions. Object boundaries are detected by regions with low joint probability density. The approach is interesting in connection with the possibility of additional indication of the boundary to ensure its closure, which very often turns out to be relevant despite many advantages of the Canny method. As the authors note, the proposed approach is computationally expensive.
A common approach is that in the first stage, preliminary clustering of N-dimensional pixels is carried out using various spectral similarity measures, followed by the segmentation or classification of objects of interest, which greatly simplifies the task of forming boundaries. For example, [4] examined Spectral Angle Mapper (SAM). The spectral angle classification method is used to analyze the correspondence of the spectrum of unknown material and the a priori specified reference value of the spectrum characterizing the class. In terms of HSI, a spectrum refers to a vector of features (signature) that characterizes an HSI pixel. The advantage of the SAM method is noted that it is sensitive only to the direction of the signature vector, and ignores their length, which provides classification stability with different illumination of the objects under study.
The use of a combined feature space (spatial-spectral) that takes into account both spectral and spatial correlation between pixels is another direction in solving the problem of clustering and detecting HSI objects.
The publication [5] of 2017 proposed an approach to the
classification of HSI objects based on gravitational models
(GEDHSI), which provide for the assessment of spatial
proximity with subsequent detection of edges. The main
ideas of the method: (
The 2017 publication [6] further developed the approach [3]. The probability density of similarity of adjacent pixels within an object is also expected to be substantially higher than at the boundary. For a local estimate of spatial-spectral proximity within a radius of 5 pixels, a similarity matrix is formed for each of the three evaluation metrics: Spectrum Angle Mapper (SAM); Spectral Gradient Angle (SGA); Spectral-Spatial Variance (SSV). The resulting matrix is formed as the product of the three data. The matrix is then used to construct a characteristic equation whose solution defines the boundary between HSI objects.
In [7], a classification algorithm based on a self-learning recognition method is proposed, which determines values of alignment conversion parameters for each signature of a script compared to a reference. Similarity with reference is established by value of standard deviation of transformed signature from reference. For contouring of formed dense groups of detected objects (for example, oil spots, trees, etc.), a geometric algorithm for construction of a nonconvex shell has been developed.
An important task when using HSI is to eliminate redundancy while maintaining maximum information value. Therefore, much attention is paid to managed and unmanaged methods of reducing the size of data. In [8], the task of segmentation is solved by a three-stage procedure: reduction of the dimension of the hyperspectration image; One of the classical segmentation algorithms; Area consolidation procedure based on priority queues. Known segmentation quality indicators (global consistency error and Rand index) have been used to optimize algorithm parameters and analyze different segmentation approaches. In [9], it is assumed that HSIs supplemented with polarization information and transformed into polarized hyperspectration images (PHSIs) will have even greater potential in object detection and clustering tasks due to increased informativity.
Thus, it can be said that research supporting the task under consideration is actively being carried out. Currently, the solution quality assessment stage is a bottleneck due to the lack or insufficient quality of real benchmark test materials. A pleasant exception is the extremely elaborate and reference-rich USGS Spectroscopy Lab (www.usgs.gov/labs/spec-lab) tool [10]. The presence of a library of accurate standards oriented to unique spectral characteristics, including those implemented on a small number of channels up to single ones, encouraged the creation of a unique tool there to remove continuum, allowing the step-by-step allocation of the necessary additive components. In these circumstances, the creation of deep research tools for the material itself of a particular and, at the same time, unique HSI becomes even more relevant.
Canny's multi-stage algorithm is known as the most successful one for detecting the boundaries of halftone image objects. The algorithm works reliably, has open implementations. Applying the Canny algorithm to an individual HSI channel is thus not a problem. In the case of HSI, in order to use a Canny detector, it is necessary to at least collapse the N-dimensional signature vector of the image pixel to a scalar value so as to make sufficient use of the information of all or a predetermined portion of the HSI channels. First of all, it is information about classes (subclasses) and their condition. The problem with adapting the algorithm for HSI is to make it also work well on an arbitrary set of HSI channels, to investigate the parameters of its operation depending on the metric of an arbitrary part of the N-dimensional pixel used.
When selecting metrics, let us assume that the class of objects represented by N-dimensional HSI pixel vectors includes those unidirectional to the class reference vector in N-dimensional space or its k-dimensional subspace essential for classification, with accuracy to a certain corresponding to the metric used. That is, vectors bound by a linear relationship must be assigned to one class.
A number of values can be used as a detector of the linear
relationship between the two x and y pixel vectors of the
hyperspectration image on a given sample of k channels:
a) covariance (
The most interesting class detectors are the normalized
values, i.e. (
The influence of the boundaries of an object detected by the classical Canny algorithm over only a part of the spectrum on the classification and the boundaries of hyperspectral image objects over the entire spectrum or its other part is investigated. The possibility and metrics of constructing a hierarchical classification within the boundaries of the detected contour of an object are investigated. The classification hierarchy confirms the priority of angular metrics at the stage of detecting the contour of an object and the formation of classes, and the role of the Euclidean norm at the stage of distinguishing subclasses. As signs of the classification of lower levels, normalizing values of standard deviations or mean values for a signature can also be used. We also study the detection of unique states (sequences of states), which may require an estimate of the Chebyshev metric in the characteristic part of the spectrum. For the analysis and quantitative assessment of the unique states of class objects, the mean and variance values are used for HSI channels.
The study was performed on the examples of the different object types present at HSI Moffett Field. This HSI is chosen because of the greatest, from open HSIs, diversity and heterogeneity of the image objects, and also the diversity of water bodies. It is the territory of the USA's first wetland reserve, which is located within urban areas due to circumstances. This HSI is most interesting for distinguishing small variations in the state of objects as a hierarchical system of classes. The result of the initial experiment determining the best metric for detecting the contours of the objects claiming to be classes is shown in Fig. 1.
From the comparison of cases (a), (b), (c), it can be seen that the best metric is the cosine of the angle of deviation of the signature of the current pixel from the reference. Distinct contours were obtained for most objects. The experiment was conducted for signatures of different dimensions. Increasing the number of channels increases the detail of the image. Fig. 2 shows the same Moffett Field fragment #1 (only truncated from below) from which the boundaries will be detected. Further, five boundaries are used as contours of areas defining the corresponding class or subclass on the whole spectrum of channels (Fig.3). The color defined for each class fills the area inside the boundary and all other HSI pixels that are similar to the class reference, which has defined by the boundaries (see the result also on Fig.3).
Fig. 4 shows the image of fragment No. 1 for channel 17 HSI obtained as a result of the removal of the continuum. One can observe the correspondence between the classification shown in Fig. 3 and the coloring of the channel image in Fig.4. The channel number is randomly selected from channels with significant differences with channel 43.
Fig. 5 shows the results of the HSI analysis over only the first 43 channels. Fig. 5a shows the result of class selection by window, using the threshold value of 0.945 for the cosine metric of the spectral angle between the current signature vector and the class reference-vector. As the class referencevector is used the signature average over the window. Fig. 5b shows the subclasses of this class classified by the second level criterion, Euclidean norm value, at value intervals approximately equal to the standard deviation for the class.
Fig. 6 shows the signature of the water class-reference, which is red-colored in Fig. 5a, and the references of its three most potent subclasses classified by Euclidean norm value; the relationship between the reference signatures is quite close to linear.
Fig. 7 shows the water class reference-signature and the references of the classes corresponding to the 2-5 areas (zones) of Fig. 3, and show together the presence of components orthogonal to the water class reference signature.
Fig. 8 shows the reference curves for the differential
analysis of the 2-5 zone references, which obtained by
subtracting the scaled water reference from the reference
signatures of the 2-5 classes (see Fig. 7). After the
transformation, the water reference signature turns
transformed onto the X-axis of Fig.8. The differential
reference curves are shown in Fig. 8 may be visually broken
down into two types of curves: (
The task of detecting boundaries by the Canny (John F. Canny) method has been investigated as a complementary tool of analysis, segmentation and classification of hyperspectration and multisensory image objects. Based on the tools for working with HSI multi-dimensional pixel signature, the possibilities of different distance measures between N-dimensional signature vectors in the detection of classes and states of HSI objects have been investigated, such as angular distance (in the form of angle cosine); coefficient of correlation of Pearson; Euclid's norms. Their possibilities were analyzed both for the situation where the class templates are set and for the case where the class is to be determined based on the natural classification by the histogram the values of the corresponding metric inside the object border detected by the Canny algorithm over the one channel or channel sequence data.
The cosine of the deviation of the pixel signature from the reference is recommended as the leading scalar metric for the detector of the boundaries of the HSI region over several channels. The normalizing values of standard deviations, or average values for the signature, are used as signs of the classification of lower levels.
Based on the results of the study, an adaptation of the John F. Canny algorithm is implemented to highlight the boundaries of the classes of HSI objects and their states. It is shown that the detection by the Canny method of the boundaries of the HSI region over an arbitrary set of channels specified by the mask of interest channels, and the transfer of the action of these boundaries to other channels creates new opportunities for the analysis and classification of hyperspectral and multisensor images.
This work was supported by the Russian Science Foundation grant No. 16-11-00068-P.