=Paper= {{Paper |id=Vol-3006/18_regular_paper |storemode=property |title=Joint processing of images in two spectral channels for small objects detecting |pdfUrl=https://ceur-ws.org/Vol-3006/18_regular_paper.pdf |volume=Vol-3006 |authors=Valeriy P. Kosykh,Gennadiy I. Gromilin,Nikolay S. Yakovenko }} ==Joint processing of images in two spectral channels for small objects detecting== https://ceur-ws.org/Vol-3006/18_regular_paper.pdf

Joint processing of images in two spectral channels for
small objects detecting
Valeriy P. Kosykh1 , Gennadiy I. Gromilin1 and Nikolay S. Yakovenko1
1
Institute of Automation and Electrometry SB RAS, Novosibirsk, Russia

Abstract
The article is devoted to the problem of detecting low contrast small-sized objects in two-color images
with a powerful spatially non-stationary background. An increase of the detecting reliability is achieved
through a combination of three factors: attenuation of the background based on the construction of
its locally stationary model; improving the estimation of model parameters by excluding statistically
significant outliers from the initial data; joint processing of two-color images with a weakened background
component. A method of constructing a linear boundary for detecting a useful signal in a two-dimensional
space is proposed. The performance characteristics of a two-channel detector of small-sized objects are
presented.

Keywords
Low contrast small-sized objects detection, spatially non-stationary background suppression, linear
separating boundary.

1. Introduction
One of the urgent tasks of the Earth space monitoring is the early detection of potentially
dangerous objects in near-earth space by analyzing images generated by onboard spacecraft
equipment. A particular version of this problem is the detection of dim "point" objects, the sizes
of which are determined mainly by the point spread function (PSF) of the equipment, against a
random spatially non-stationary background created by the Earth’s surface and cloud cover.
Preliminary suppression of the background, based on knowledge of its statistical characteristics,
facilitates to effective selection of objects. In conditions of spatial non-stationarity, this is
ensured by constructing a locally stationary background model from a fragment of an image
near the supposed presence of an object [1, 2]. If the signal from the object has a wide spectral
range, it is advisable to jointly process images of the same area of the observed space obtained
in different spectral channels with different statistical characteristics of the background. The
suppression of the main structural features of the background in each of the channels by its local
models leads to a decrease in the interchannel correlation of the background component, while
maintaining the correlation between the images of objects. Therefore, with joint processing,
one can expect a decrease in the probability of a false alarm, or, for a given probability of a false
alarm, an increase in the probability of detection.
The most effective way of joint processing would be to build a multichannel local stationary

SDM-2021: All-Russian conference, August 24–27, 2021, Novosibirsk, Russia
" kosych@iae.nsk.su (V. P. Kosykh)
© 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org)

142
Valeriy P. Kosykh et al. CEUR Workshop Proceedings 142–151

background model, however, even for two channels (i.e., when the size of the model is doubled),
the computational costs for estimating its parameters increase manifold. Therefore, the work
considers a compromise variant of joint processing of two-channel images, when at the first stage
in each spectral channel the background component is suppressed using its locally stationary
model, and at the second stage, as a result of the analysis of the experimentally estimated
two-dimensional distributions of the residual background brightness, a useful signal is detected.
In what follows, we shell assume that the analyzed images are discrete and in each channel
can be represented in the form
𝐷(𝑖, 𝑗) = 𝐵(𝑖, 𝑗) + 𝐴 · 𝑂(𝑖 − 𝑖0 , 𝑗 − 𝑗0 ) + 𝜉(𝑖, 𝑗). (1)
Here 𝑖, 𝑗 are the current coordinates, 𝐵 is the background component, 𝑂 is the normalized
image of the object with the center located at the point (𝑖0 , 𝑗0 ) and with the amplitude 𝐴, 𝜉 is
the random uncorrelated registration noise. Then, with the known background model 𝐵 ^ (𝑖, 𝑗),
it is advisable to search for objects in the modified image
^ (𝑖, 𝑗) = 𝐴 · 𝑂(𝑖 − 𝑖0 , 𝑗 − 𝑗0 ) + 𝜃(𝑖, 𝑗), (2)
˜ (𝑖, 𝑗) = 𝐵(𝑖, 𝑗) + 𝐴 · 𝑂(𝑖 − 𝑖0 , 𝑗 − 𝑗0 ) + 𝜉(𝑖, 𝑗) − 𝐵
𝐷
where 𝜃(𝑖, 𝑗) = 𝐵(𝑖, 𝑗) − 𝐵
^ (𝑖, 𝑗) + 𝜉(𝑖, 𝑗) is disturbance, consisting of the residual part of
the background, in which, with a well-chosen model, the spatial correlation is significantly
weakened, and the noise component of the original image.

2. Locally stationary background model
If we can consider the background in some domain Ω as a realization of a stationary random
process, then its model at the point (𝑖′ , 𝑗 ′ ) ∈ Ω can be represented as a linear combination of
the image samples in its vicinity
∑︁
𝐵^ (𝑖′ , 𝑗 ′ ) = 𝐷(𝑖 − 𝑖′ , 𝑗 − 𝑗 ′ )ℎ(𝑖, 𝑗), (3)
𝑖,𝑗∈𝑊

where 𝑊 ∈ Ω is the vicinity of point (𝑖′ , 𝑗 ′ ), ℎ(𝑖, 𝑗) is a set of weight coefficients, which,
according to the optimal linear prediction (OLP) [3], can be estimated from the condition
⎧ ⎫
⎨ ⎬
∑︁ [︀ 2
^ = arg min 𝐽(h) = 𝐷(𝑖′ , 𝑗 ′ ) − h𝑇 d𝑖′ ,𝑗 ′ (4)
]︀
h .
⎩ ′ ′
⎭
𝑖 ,𝑗 ∈Ω

Here d𝑖′ ,𝑗 ′ is a vector consisting of lexicographically ordered samples of the vicinity 𝑊 , ℎ is a
vector of similarly ordered weight coefficients. This approach allows the model to be adjusted
to the current statistical characteristics of the background.

3. Reducing the influence of objects on the background model
estimate
The minimum of the functional 𝐽(h) is attained at
^ = (P𝑇 P)−1 P𝑇 c,
h (5)

143
Valeriy P. Kosykh et al. CEUR Workshop Proceedings 142–151

where the rows of the matrix P are the vectors d𝑇𝑖′ ,𝑗 ′ , and the components of the vector c are
the values 𝐷(𝑖′ , 𝑗 ′ ), taken in the same order as the vectors d𝑖′ ,𝑗 ′ . This decision is optimal in
the case, when all the samples of the image participating in (3)–(4) belong to the background.
If the samples of the object are among them, the estimate of the background model will be
distorted. In work [4], devoted to the recursive suppression of the background, this drawback
is partially eliminated due to the weakening of the contribution of the image samples, the
difference of which with the previously constructed model is large. We propose to introduce
such a weakening directly into the procedure for estimating the coefficients of the model, which,
of course, complicates the estimation, but makes it possible to exclude samples for which the
model has not yet been built. Let h ^ be the set of coefficients obtained in (5), then the components
^
of the vector c = Ph are the “smoothed” model values of the components of the vector c. We
introduce a diagonal matrix K, the elements of which are calculated as
{︂
1, |c ^𝑖 − c𝑖 | < 𝑡,
K𝑖𝑖 =
0, |c ^𝑖 − c𝑖 | ≥ 𝑡.

The corrected expression for estimating the coefficients of the current background model will
take the form
^ 𝑐 = (P𝑇 KP)−1 P𝑇 Kc,
h (6)
The threshold 𝑡 is determined by the current value of the residual between c
^ and c:
√︃
^ − c)𝑇 (c
(c ^ − c)
𝑡=𝜅 ,
𝑁𝑐 − 𝑁ℎ

where 𝑁𝑐 is the number of components in the vector c, and 𝑁ℎ is the number of weight
coefficients in (3). Rigorous justification of the coefficient 𝜅 requires certain assumptions about
the type of residual distribution. In this work, this coefficient was selected experimentally
according to the quality of background suppression near objects. By means of numerical
simulation, it was found that the value 𝜅 = 2.5 provides a sufficiently effective suppression of
the background and relatively small distortions of the amplitude of objects.

4. Detection of objects in two-color images. Linear detection
boundary for a given false alarm probability
As mentioned in the Introduction, it is assumed here that background suppression is performed
in each spectral channel separately, and at the detection stage, the samples of the modified
images with the same spatial coordinates are considered as vectors in the two-dimensional
˜ (1) × 𝐷
space 𝐷 ˜ (2) (superscripts indicate belonging to the corresponding spectral channel):
(︁ (1) (2) )︁𝑇
D𝑖𝑗 = 𝐷˜ ,𝐷˜ .
𝑖𝑗 𝑖𝑗

To construct in this space the boundary between the background and objects, which satisfies
the given criterion of optimality of object detection, it is necessary to know the two-dimensional

144
Valeriy P. Kosykh et al. CEUR Workshop Proceedings 142–151

probability density functions of the background and objects brightness. Unfortunately, in real
conditions, instead of probability density functions, we can only obtain a two-dimensional
histogram of the joint distribution of the background and object brightness. This histogram,
on the assumption that the appearance of objects is a very rare event, can be considered as
an estimate of the discretized probability density function of the background. As for the two-
dimensional probability density function characterizing objects, it can be constructed only based
on a priori data on their spectral and brightness characteristics. In this work, we assumed the
presence of minimal information about objects, namely, only knowledge of the ratio of their
average brightness in different spectral channels. This is not enough to determine the specific
shape of the boundary, so here we propose to construct a linear boundary. This boundary should
provide, for a given false alarm probability, the maximum detection probability, being guided by
the experimentally obtained two-dimensional histogram of the background and the direction
in two-dimensional space, given by the ratio of the average brightness of objects 𝐴 ¯ (2) /𝐴
¯ (1) in
different channels.

4.1. Evaluation of the boundary for a given false alarm probability
According to expression (2), the modified images are obtained by subtracting the background
component adjustable model from the original images; therefore, it should be expected that the
average value of the brightness in them would be close to zero. Accordingly, we will construct a
two-dimensional histogram 𝐻(𝐷 ˜ (1) , 𝐷
˜ (1) ) of brightness in the space 𝐷
˜ (1) × 𝐷
˜ (2) in the range
[−𝐷max , 𝐷max ] with step 𝛿. To determine the linear boundary dividing the space 𝐷 ˜ (1) × 𝐷
˜ (2)
in a given direction in accordance with a given false alarm probability 𝑃𝑓 𝑎 , we calculate the
𝑅(𝑠𝑘 , 𝜙) — Radon transform [5] of normalized histogram. (Here 𝑠𝑘 = 𝑠0 + 𝑘𝛿 is the distance
from the origin in the space 𝐷˜ (1) × 𝐷
˜ (2) to the straight line along which the histogram is
(1)
integrated, 𝜙 is the angle between the direction 𝐷 ˜ and the normal to this line). Then the
distance 𝑠𝑡 from the origin to the straight line, corresponding to the given false alarm probability
𝑃𝑓 𝑎 , is determined by the relations
𝑡
∑︁ 𝑡+1
∑︁
𝑅(𝑠𝑘 , 𝜙) < 1 − 𝑃𝑓 𝑎 , 𝑅(𝑠𝑘 , 𝜙) ≥ 1 − 𝑃𝑓 𝑎 . (7)
𝑘=0 𝑘=0

These relations show at what distance from the origin of coordinates the straight line given
by the angle 𝜙 should lie, providing the given probability of false alarm, and exhaustively
determine the set of straight lines that can be considered as possible variants of the boundary.

4.2. Choosing the border direction based on the known ratio of the average
brightness of objects
Apparently, with such scant information on the distribution of the objects brightness, the only
justified choice of the detection boundary direction is the choice of such an angle 𝜙 at which
˜ (1) × 𝐷
the straight line satisfying (7) intersects in the space 𝐷 ˜ (2) direction to objects at the

145
Valeriy P. Kosykh et al. CEUR Workshop Proceedings 142–151

point closest to the origin. The normal equation of a straight line satisfying (7) has the form

˜ (1) cos 𝜙 + 𝐷
𝐷 ˜ (2) sin 𝜙 − 𝑠𝑎 = 0,

and the direction to objects, taking into account the above, is set by the straight line

(2) ˜ (1) 𝐴
𝐷 ¯ (2)
˜
𝐷 = .
𝐴¯ (1)

It is easy to show that the distance from the point of intersection of these straight lines to the
origin is equal to ⎯ [︂ ]︂
⎸ 2 (︁ ¯ (1) )︁2 (︁ ¯ (2) )︁2
⎸
⎸ 𝑠𝑎 𝐴 + 𝐴
(8)
⎸
𝐿(𝜙) = ⎷ (︁
⎸ )︁2 .
¯ (1) cos 𝜙 + 𝐴
𝐴 ¯ (2) cos 𝜙

The minimum of this expression with respect to 𝜙 determines the sought boundary for
detecting objects. The second way to choose the direction of the border is to choose, taking
into account (7), a straight line orthogonal to the direction to the objects. For this straight line,
the angle 𝜙 must satisfy the condition

¯ (1) sin 𝜙 + 𝐴
𝐴 ¯ (2) cos 𝜙 = 0.

Since the angle 𝜙 takes only discrete values, its estimate, provided that relations (7) are
satisfied, is obtained by minimizing the expression
⃒ ⃒
⃒ ¯ (1) ¯ (2) cos 𝜙⃒⃒ .
𝐽(𝜙) = ⃒𝐴 sin 𝜙 + 𝐴 (9)

We will compare these methods of constructing the boundary in the experimental part of the
work.

5. Computational experiment
For the computational experiment, we used multispectral images obtained by the Landsat-7
satellite [6]. The purpose of the experiment was to test the above technique for identifying
small-sized objects in two-color images with a complex spatially non-stationary background.
Pairs of images obtained synchronously in different spectral channels made up two-color images
and served as a spatially non-stationary background, on which images of small-sized objects
were additively applied.

5.1. Formation of objects images
Since one of the objectives of this study is to assess the probability of detecting objects with
an accuracy of 10−3 , the experiment should contain a representative (∼ 105 ) sample of their
images. Therefore, firstly, more than 600 objects were simultaneously applied to the background

146
Valeriy P. Kosykh et al. CEUR Workshop Proceedings 142–151

Figure 1: Dependence of the object shape on the position relative to the PSE lattice.

image, and secondly, the application procedure was repeated many times (more than 100 times)
with a change in the position of the objects. Let us note one, in our opinion, rather important
feature of the formation of images of objects, which consists in the following. The dimensions
of the images of “point” objects projected into the focal plane of the receiving equipment are
determined by its PSF. As a rule, to maximize the signal-to-noise ratio, the dimensions of the
photosensitive elements (PSE) of the receiver and the characteristic PSF size are matched in
such a way that the integral radiation power from the object incident on the area of one PSE
is approximately 60–80% of the total power falling from the object onto photodetector [7, 8].
Therefore, the shape of the image formed by the photosensitive elements of the receiver depends
significantly on the position of the object relative to the elements recording it. In the experiment,
when generating images, this fact was taken into account by specifying various displacements
of objects, which are not multiples of the step of placing the PSE [9]. Figure 1 shows examples
of objects images located in various ways relative to the PSE array of the receiver.

5.2. Background suppression using a locally stationary model
In the experiment, the domain of the model for predicting the background value at the image
point (𝑖′ , 𝑗 ′ ) was its square neighborhood 𝑊 of 7 × 7 pixels with a 3 × 3 pixel square cut out in
the center. As the region of stationarity of the background Ω, according to which the parameters
of the model were estimated, the neighborhood of the current point with a size of 13 × 13 pixels
was considered. The sizes and shapes of the domains Ω and 𝑊 were selected experimentally.
Background suppression was performed according to expressions (2)–(6).
Figure 2 illustrates the effectiveness of background suppression without correction and with
correction of the objects occurrence in Ω. Figure 2, a shows a fragment of a satellite image
obtained in the spectral range of 0.45 ÷ 0.515 𝜇m with objects applied to it, the average
brightness of objects is ∼ 2.2 RMSD of the background brightness calculated over the entire
image. All data are presented in a single brightness scale. The background standard deviation in
the original image is 100, the average object brightness is 212, and the object brightness standard
deviation is 12.7. In Figure 2, b, this fragment is shown after suppressing the background based
on the model (5). The background RMSD is 10.8, while the average brightness of objects is 158,

147
Valeriy P. Kosykh et al. CEUR Workshop Proceedings 142–151

a b c

Figure 2: Background suppression using a locally stationary model.

and their RMSD is 23.8. In Figure 2, c — the same fragment after suppressing the background
by applying the model (6) (the RMSD of the residual background is 10.6, the average brightness
of objects is 183, RMSD is 21.9).
Experiments with images of different spectral ranges confirm the assumption that the use of
a locally stationary background model makes it possible to weaken the background component
many times (∼10 times) when the quality of object images deteriorates by about half.

5.3. Estimation of the boundaries of a given false alarm probability using
two-dimensional histograms
The images of “point” objects to be detected, as shown in Figure 1, are small outliers with
a brightness exceeding the brightness of neighboring pixels, therefore, after suppressing the
background, it is advisable to search for them among the local maxima of the modified images.
The first step in processing a pair of images belonging to different spectral channels is to split
the images into regions of the background and objects. Since the objects were applied artificially,

a b c

Figure 3: Family of boundaries with 𝑃𝑓 𝑎 = 10−4 for two-color images.

148
Valeriy P. Kosykh et al. CEUR Workshop Proceedings 142–151

such a partition is easy to implement. Further, for each of the regions, two-dimensional
histograms of the maxima are constructed. Pairs of maxima are selected in each of the images
and have the same (or close) coordinates. Based on the histogram belonging to the background
region, according to expression (7) and the given false alarm probability 𝑃𝑓 𝑎 , the possible linear
boundaries of the space division 𝐷 ˜ (1) × 𝐷˜ (2) are calculated. Figures 3, a and b show fragments
of two synchronous images of the same area of the terrain, obtained in the spectral ranges of
0.45÷0.515 and 1.55÷1.75 𝜇m. Figure 3, c with blue color shows a two-dimensional histogram
of the distribution of maxima in the background area of their modified images, green — a family
of straight lines dividing the space 𝐷˜ (1) × 𝐷
˜ (2) in accordance with a fixed false alarm probability
𝑃𝑓 𝑎 = 10−4 at different angles 𝜙. Two straight lines are selected from this family, one of which,
𝐿𝑇 , provides a minimum to functional (8), the second, 𝐿𝑁 , to functional (9).

5.4. Evaluation of the object detection probability
The second stage of the experiment is to estimate the detecting probability of objects that are
applied on the original background images. Similarly to the histogram of the background maxima
for the area of objects, a histogram of the maxima of objects is built and, taking into account
the already found boundaries, the probabilities of detection 𝑃𝑑𝑒𝑡𝑇 and 𝑃𝑑𝑒𝑡𝑁 are calculated. In
Figure 4 for the above frames are shown in blue — the histogram of the background maxima
distribution, in red — the histogram of the objects maxima distribution, the green straight line
indicates the direction determined by the a priori known ratio of the average brightness of
objects 𝐴¯ (2) /𝐴
¯ (1) in different channels. Straight lines mark the boundaries corresponding to
the false alarm probability 𝑃𝑓 𝑎 = 10−4 . Boundaries drawn independently from the first and
second frames are shown in white, the 𝐿𝑇 boundary is indicated in cyan, and the 𝐿𝑁 boundary
is in magenta.
The summary result of the experiment with this pair of images is shown in Figure 5, a.
Here, the dependence of the objects detecting probability on the angle 𝜙, which determines
the orientation of the boundary, is shown for three values of the false alarm probability 𝑃𝑓 𝑎 :
10−2 , 10−3 and 10−4 . The blue points on the graphs correspond to the boundaries determined

Figure 4: Background and object maxima distributions with boundaries of a different type.

149
Valeriy P. Kosykh et al. CEUR Workshop Proceedings 142–151

a b

Figure 5: Dependence of the detection probability on the orientation of the boundary.

according to the minimum of the functional (8), purple — to the minimum of the functional (9).
A significantly worse detection quality with the same false alarm probability values is obtained
for a pair of images with close spectral ranges of 0.45–0.515 and 0.525–0.605 𝜇m (Figure 5, b).
Here, at 𝑃𝑓 𝑎 = 10−4 , the graph of the detection probability is raised by 0.8 to be in the range
of 0.8 ÷ 1.0. Nevertheless, as in the previous case, for all three values of 𝑃𝑓 𝑎 the boundaries
corresponding to functional (8) provide a higher detection probability than the boundaries
corresponding to functional (9).

6. Conclusion
The paper proposes an approach to solving the problem of detecting small-sized objects in a pair
of synchronous images of a scene with a powerful spatially non-stationary background, obtained
in different spectral ranges. It is shown that the separate application of locally stationary models
to describe the background to each of the images makes it possible to weaken their background
component many times (up to 10 times). The exclusion of statistically significant outliers from
the data used to estimate the parameters of the model makes it possible to significantly reduce
(by an average of 20%) the distortion of the objects amplitude. A method for joint processing
of a pair of images with a suppressed background is proposed, based on the analysis of two-
dimensional histograms of the background brightness, which ensures the detection of objects
with a given false alarm probability. Assuming that only the ratio of the average values of
the brightness of objects in different spectral ranges is known, it is proposed to search for the
boundary between the background and objects in a two-dimensional color space as a straight
line intersecting the line of direction to objects at a minimum distance from the origin in this
space. It has been shown by means of modeling that among all linear dividing boundaries that
provide a given false alarm probability, this boundary allows one to obtain the maximum of the
object detection probability.

150
Valeriy P. Kosykh et al. CEUR Workshop Proceedings 142–151

Acknowledgments
The work has performed with the financial support of the Ministry of Education and Science of
Russia (project 121022000116-0).

References
[1] David S.K. Chan, David A. Langan, and Daniel A. Stayer. Spatial processing techniques for
the detection of small targets in IR clutter // Proc. SPIE 1305, Signal and Data Processing
of Small Targets 1990, (1 October 1990); doi: 10.1117/12.2321786
[2] Ffrench P.A., Zeidler J.R., Ku W.H. Enhanced detectability of small objects in correlated
clutter using an improved 2-D adaptive lattice algorithm // IEEE Transactions on Image
Processing. 1997. Vol. 6. No. 3. P. 383–397.
[3] Vaidyanathan P.P. The theory of linear prediction. Morgan & Claypool Publishers, 2008.
184 p.
[4] Jin L., Hong-Bing J. Improved background prediction algorithm for IR small targets de-
tection // Proceedings of 2nd International Congress on Image and Signal Processing
(CISP 2009). Tianjin, China, 17–19 October, 2009. P. 3363–3367.
[5] Bracewell R.N. Two-dimensional imaging. Englewood Cliffs, NJ: Prentice Hall, 1995.
P. 505–537.
[6] Global land cover facility. Available at: http://glcf.umiacf.umd.edu (accessed 23.09.2017).
[7] Winick K.A. Cramer – Rao lower bounds on the performance charged-coupled-device
optical position estimator // JOSA A. 1986. Vol. 3. No. 11. P. 1809–1815.
[8] Ivanov V.A., Kirichuk V.S., Kosykh V.P. et al. Specific features of detecting point objects in
images formed by a detector array // Optoelectronics, Instrumentation and Data Processing.
2016. Vol. 52. P. 113–120.
[9] Gromilin G.I., Ivanov V.A., Kosykh V.P., Popov S.A. Specific features of scanning with
FPA in time delay integration mode // Computational Technologies. 2016. Vol. 21. No. 1.
P. 60–89.

151