=Paper=
{{Paper
|id=Vol-2391/paper37
|storemode=property
|title=A technique for detecting concealed objects in terahertz images based on information measure
|pdfUrl=https://ceur-ws.org/Vol-2391/paper37.pdf
|volume=Vol-2391
|authors=Dmitry Murashov,Alexei Morozov,Fedor Murashov
}}
==A technique for detecting concealed objects in terahertz images based on information measure ==
https://ceur-ws.org/Vol-2391/paper37.pdf
A technique for detecting concealed objects in terahertz
images based on information measure
D M Murashov1, A A Morozov2 and F D Murashov3
1
Federal Research Center “Computer Science and Control” of RAS, Vavilov str., 44-2,
Moscow, Russia, 119333
2
Kotel'nikov Institute of Radio Engineering and Electronics of RAS, Mokhovaya, 11-7,
Moscow, Russia, 125009
3
Moscow Aviation Institute (National Research University), Volokolamskoe Shosse, 4,
Moscow, Russia, 125993
e-mail: d_murashov@mail.ru, morozov@cplire.ru
Abstract. In this paper, a new technique for detecting concealed objects in the images acquired
by a passive THz imaging system is proposed. The technique is based on a method for mutual
information maximization successfully used for image matching. For reducing computational
expenses, we propose to analyze the mutual information at local maxima of the cross-
correlation function computed in the Fourier domain. The proposed technique does not require
parameter tuning. A computing experiment approved the efficiency of the proposed technique
and the possibility of its implementation in security systems.
1. Introduction
One of the areas of public safety protection is the development of concealed object detection systems
in images obtained in the terahertz band (300 GHz - 3 THz). As a rule, these images are characterized
by a low signal-to-noise ratio, low resolution, low contrast, and fuzziness of objects (see Figure 1 (a)).
One of the approaches to solving the problem for detecting concealed objects in the terahertz
images consists of image segmentation and recognition of selected objects. Many of the proposed
segmentation methods are based on the assumption that there are three areas in the terahertz image
(background, a human body, and hidden objects) characterized by their radiometric temperature range
corresponding to the ranges of gray tone levels. Gaussian mixture models are usually used to represent
the images. The authors of works [1-3] used these models as the basis for developing multi-level
segmentation algorithms. Noise suppression algorithms are applied at the preprocessing stage to
improve the segmentation quality. For example, the anisotropic diffusion algorithm and the nonlocal
means (NL-means) algorithm are used in [1]. In paper [3], the authors developed a multi-level EM
algorithm to localize objects. At the first level, the algorithm segments two objects: the background
and the human body. At the second level, the EM algorithm highlights the hidden object over the
body. In [4], the author proposed an algorithm based on the maximum likelihood method for
recognizing hidden objects. In order to reduce the effects of noise and low contrast, it may be useful to
combine images taken in different spectral ranges. To increase the performance of the concealed
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019)
Image Processing and Earth Remote Sensing
D M Murashov, A A Morozov and F D Murashov
objects detection, the authors of work [5] combined images obtained in the terahertz and visible
spectral bands. In [6], the objects hidden under clothes are localized using the images obtained in four
channels of a system operating in 3- and 8-millimeter wavelengths with vertical and horizontal
polarization. A multilevel segmentation algorithm combines the k-means algorithm, the EM algorithm
for estimating the parameters of a Gaussian mixture, and the Bayesian classifier. An alternative
approach to object detecting in terahertz images is to use matching the template image of an object and
the input image. In [7], a method based on the localization of the maximum of the correlation
coefficient was applied to detect objects. In this paper, for detecting objects in terahertz images, we
propose to apply an image matching method based on the mutual information maximization [8].
2. Problem statement
Suppose there is a set of images of objects that should be detected under the clothes of people, using a
passive terahertz camera. Objects that are visible in the image should be compared with the templates,
and it is necessary to make a decision about their identity or difference. Suppose there are N template
images of objects U n , n 1,2,..., N . Let the image V be the input image. It is necessary to identify the
object U n located in the image V .
To solve such a problem, the segmentation of the input image is usually preliminarily performed,
feature descriptions of the images of objects U n are formed, and then in the feature space, the
separating surfaces between the classes of the objects under consideration should be found.
The properties of terahertz images listed above complicate the segmentation of objects. Therefore,
it is preferable to use methods that do not require segmentation. In this paper, we propose to solve the
object detection problem as an image matching problem [9]. Let the similarity measure of the template
and the objects in the image V be the criterion J (U n ,V ) . Then the problem can be formulated as
follows. It is necessary to find a template image V which provides the maximum value of the
similarity measure J (U n ,V ) :
U m arg max{ J (U n ,V )} , n 1,2,..., N . (1)
Un
Several methods that do not require segmentation of images are known, in particular, methods
based on the maximization of the cross-correlation function values, maximization of the correlation
coefficient [7,9], and maximization of the mutual information [8,10]. These methods do not require a
priori knowledge about the relationship between the input image and the template as well as
information on the statistical characteristics of the images. However, the correlation matching of
terahertz images fails in some cases (see Figure 1). The mutual information is calculated directly from
the joint distribution of the gray levels of these images and involves the data contained in the
compared images more completely than the correlation parameters. The mutual information is a
quantitative characteristic of the statistical dependence of images. The greater similarity of the
template and the input image produces a greater value of the mutual information. In this work, the
mutual information is used as a similarity measure to solve the problem (1).
To apply the information-theoretic approach, a stochastic model of the dependency between the
template image and the input image is necessary. Let the grayscale values in the compared images at
the point x with coordinates ( x, y) be described as discrete random variables U (x) and V (x) with
values u and v quantized into a finite number of levels K and L respectively. If images U (x) and
V (x) represent the same scene, there is a relationship between the variables U (x) and V (x) . We
will use a model similar to that proposed in [8,11]:
V (Tr (x)) F (U (x)) (x), (2)
where Tr is a coordinate transformation (for registered images V (Tr (x)) U (x) ); F is a grayscale
value conversion function that simulates the relationship between two images of an object; (x) is a
random variable that simulates noise. Model (2) can be considered as a model of a discrete stochastic
information system with the input U and the output V .
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 270
Image Processing and Earth Remote Sensing
D M Murashov, A A Morozov and F D Murashov
a) b) c)
Figure 1. The result of the correlation image matching: (a) an image of a human with a hidden object
obtained from the THERZ-7A complex; (b) the image of the template object; (c) the result of an
erroneous correlation matching. The contour of the template is shown in red.
The mutual information between the system input and output is defined by the expression:
K L
p(uk , vl )
I (U ;V ) p(uk , vl )log , (3)
k 1 l 1 p(uk ) p(vl )
where the functions p(u ) , p(v) , and p(u, v) define one-dimensional and two-dimensional discrete
probability distributions of the gray tone levels of the images U and V . If mutual information (3) is
used as a similarity measure, the problem (1) can be formulated as follows. It is necessary to find an
image of the template U n maximizing the criterion (3):
U m arg max{ I (U n ;V )} . (4)
Un
In the next section, an algorithm for solving the problem is proposed.
3. Comparison of the template images with the images obtained in the terahertz spectral band
The direct computing of the mutual information values on the set of pixels corresponding to the
intersection of the images U n and V at different positions of U n relative to V requires large
computational expenses. Therefore, in practice, methods for accelerated computing of the mutual
information magnitude are necessary. For example, in [8], a differential equation was obtained for
searching for the extremum of function (3) in the image matching problem.
In this paper, we propose to compute the mutual information values I (U n ;V ) at the points of the
input image V corresponding to the local maxima of the cross-correlation function [9], where the
content of the input image V matches the template image U n . The number of the local maxima is
significantly less than the number of pixels in the input image V and the cross-correlation function of
the images can be rapidly computed in the frequency domain. These circumstances make it possible to
significantly speed up the computing of mutual information and, accordingly, solving the problem (4).
Since the cross-correlation function is sensitive to changes in the amplitudes of the grayscale levels in
the compared images, we will use contour preparations of the U n and V images for its computing
because the boundaries of the objects have a fairly stable high grayscale value in the contour
preparations. The cross-correlation function is determined as follows:
cUn,V ( x, y)
CUn (s, t )CV ( x s, y t ) ,
s t
(5)
where CUn and CV are the contour preparations of the images U n and V , x and y are the spatial
coordinates of the points of the image CV , s and t are the coordinates of the points in the
intersection area of the images CUn and CV . Given the high noisiness of terahertz images, masks of
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 271
Image Processing and Earth Remote Sensing
D M Murashov, A A Morozov and F D Murashov
stochastic gradients [12] for signal-to-noise ratio SNR = 1 will be used to obtain contour preparations.
The computing of the function (5) will be carried out in the frequency domain.
Thus, the search algorithm on the terahertz images of an object corresponding to condition (4)
includes the following operations: (a) smoothing the image V and obtaining contour preparations
CU 1 and CV of the images U n and V respectively; (b) calculating the cross-correlation function of
the contour preparations CU 1 and CV by the formula (7); (c) finding the points of local maxima of
the cU 1,V ( x, y) function; (d) calculating the mutual information I (U1;V ) at the cU 1,V ( x, y) local
maxima points; (e) finding the maximum of mutual information I max (U1;V ) ; (f) repeating steps (a) –
(e) for the images U1 , ..., U N ; finding the maximum of the mutual information I max (U m ;V ) among the
values of I max (U1;V ) , I max (U 2 ;V ) , ..., I max (U N ;V ) and the corresponding template image U m which
will be the solution of the problem (4). It should be noted that an operation of impulse noise filtering
[13] can be applied at step (a) for input image enhancement.
The proposed algorithm has no parameters and does not require tuning. Figure 2 illustrates the
main stages of localizing the objects. The input image captured by the THERZ-7A complex is shown
in Figure 2 (a), and the image of the template object is given in Figure 2 (b). Contour preparations of
these images are presented in Figures 2 (c) and 2 (d). Figure 2 (e) visualizes the cross-correlation
function of the contour preparations. The figure shows the local maxima of the surface formed by the
function cUn,V ( x, y) . The result of localizing the object in the input image is shown in Figure 2 (f).
a) b) c) d) e) f)
Figure 2. The main stages of localizing the objects: (a) image captured by the THERZ-7A complex;
(b) the image of the template object; (c) the contour preparation of the input image; (d) the contour
preparation of the template image; (e) visualized cross-correlation function of contour preparations
cUn,V ( x, y) ; (f) the result of localizing the object in the input image, corresponding to the maximum
of the mutual information I (U n ;V ) .
4. Experiment
To estimate the applicability of the above approach to solve the problem of localizing and identifying
concealed objects, a computational experiment was carried out. The experiment was organized as
follows. We obtained template images of objects, which should be detected in images captured at the
output of the terahertz imaging system. Template images are shown in Figure 3.
a) b) c)
Figure 3. Images of template objects: (a) Kalashnikov submachine gun (AK); (b) TT handgun;
(c) Walther handgun.
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 272
Image Processing and Earth Remote Sensing
D M Murashov, A A Morozov and F D Murashov
We consistently placed objects under the clothes of a person and recorded image sequences using
the THERZ-7A system. Using the proposed method, we compared the template images shown in
Figure 3 (a) with frames of the obtained video sequences and fixed the values of the similarity
measure (3). The shooting range was about 1.5 meters and varied slightly, so the template images were
used with the constant value of the scale factor. The cross-correlation function and the mutual
information are not invariant with respect to rotation. Therefore, it is also necessary to analyze the
rotated and flipped variants of the template images. The results of the object localizing are shown in
Figures 4 (a-e), and the computed values of mutual information are presented in Table 1. It follows
from Figure 4 and Table 1 that concealed objects are successfully localized by the proposed method
based on a combination of correlation matching and the mutual information maximum matching.
From the data in Table 1, we can draw the following conclusion. When comparing images with
concealed objects obtained at the output of the terahertz imaging system and the images of template
objects, the maximum of the mutual information is achieved if the template corresponds to the
concealed object. Consider an example. Suppose a terahertz image is obtained in which a hidden
object is visible (for example, a Walther handgun). This image is matched with the AK submachine
gun, TT handgun, and Walther handgun templates. The computed values of the mutual information
between the matched images are given in the third row of Table 1. The maximum value of the mutual
information is obtained when comparing the input image and the Walther template. The system then
makes a decision that the concealed object is a Walther handgun. From this, it follows that the
proposed method can be used to detect and recognize prohibited concealed objects in security systems.
To eliminate false alarms, the calculated maximum value of the mutual information is compared with
a predetermined threshold value. If the found maximum of the criterion (3) is less than the threshold,
then the decision is made that the prohibited object is not found. The threshold value is determined
empirically.
a) b) c) d) e)
Figure 4. The result of the matching images of template objects and the images of hidden objects: (a)
TT handgun hidden under the clothes and its template; (b) hidden Walther handgun and its template;
(c) hidden AK submachine gun and TT template; (d) hidden Walther handgun and TT template;
(e) hidden AK submachine gun and its template.
Table 1. The values of the mutual information computed when matching the template images
and the images of concealed objects.
Template objects
Concealed object
AK TT Walther
AK 0.307398 0.295407 0.276332
TT 0.191471 0.404531 0.345492
Walther 0.162791 0.267373 0.342771
5. Conclusions
The paper proposes a new algorithm for localization and identification of objects in images taken
using the passive terahertz imaging system. The proposed algorithm combines matching methods
based on maximizing the value of the cross-correlation function and the mutual information
magnitude. To reduce computational expenses, the criterion of the mutual information maximum is
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 273
Image Processing and Earth Remote Sensing
D M Murashov, A A Morozov and F D Murashov
analyzed at points of local maxima of the cross-correlation function computed in the frequency
domain. The proposed algorithm does not require parameter tuning. Future research will be aimed at
improving the quality of detection and recognition of concealed objects using preprocessing of
terahertz images and applying detection methods that are invariant to geometric transformations.
6. References
[1] Shen X, Dietlein C R, Grossman E, Popovic Z and Meyer F G 2008 IEEE transactions on
Image Processing 17 2465-2475
[2] Lee D S, Yeom S, Son J Y and Kim S H 2010 Optics Express 18 10659-10667
[3] Martinez O, Ferraz L, Binefa X, Gomez I and Dorronsoro C 2010 Concealed object detection
and segmentation over millimetric waves images IEEE Computer Society Conference on
Computer Vision and Pattern Recognition-Workshops 31-37
[4] Antsiperov V E 2016 Automatic target recognition algorithm for low-count terahertz images
Computer Optics 40(5) 746-751 DOI: 10.18287/2412-6179-2016-40-5-746-751
[5] Kowalski M, Palka N, Piszczek M and Szustakowski M 2013 Acta Physica Polonica A 124
[6] Yeom S, Lee D, Lee H, Son J and Gushin V P 2013 Progress in Electromagnetics Research
39 169-180
[7] Trofimov V A, Trofimov V V, Shestakov I L and Blednov R G 2016 Concealed object
detection using the passive thz image without its viewing Passive and Active Millimeter-Wave
Imaging XIX (International Society for Optics and Photonics) 9830 98300E
[8] Viola P and Wells III W M 1997 International journal of computer vision 24 137-154
[9] Gonzalez R C and Woods R E 2008 Digital Image Processing (New Jersey: Pearson Education,
Inc)
[10] Maes F, Collignon A, Vandermeulen D, Marchal G and Suetens P 1997 IEEE transactions on
Medical Imaging 16 187-198
[11] Escolano F, Suau P and Bonev B 2009 Information theory in computer vision and pattern
recognition (Springer Science & Business Media)
[12] Jain A K 1989 Fundamentals of digital image processing (Englewood Cliffs, NJ: Prentice Hall)
[13] Chervyakov N I, Lyakhov P A and Orazaev A R 2018 Two methods of adaptive median
filtering of impulse noise in images Computer Optics 42(4) 667-678 DOI: 10.18287/2412-6179-
2018-42-4-667-678
Acknowledgments
Authors are grateful to the Astrohn Technology Ltd and OOO ASoft who provided us with the
THERZ-7A terahertz scanning device. This research was supported in part by the Russian Foundation
for Basic Research (grants No 18-07-01295 and No 16-29-09626).
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 274