In this paper, a new technique for detecting concealed objects in the images acquired by a passive THz imaging system is proposed. The technique is based on a method for mutual information maximization successfully used for image matching. For reducing computational expenses, we propose to analyze the mutual information at local maxima of the crosscorrelation function computed in the Fourier domain. The proposed technique does not require parameter tuning. A computing experiment approved the efficiency of the proposed technique and the possibility of its implementation in security systems.
One of the areas of public safety protection is the development of concealed object detection systems in images obtained in the terahertz band (300 GHz - 3 THz). As a rule, these images are characterized by a low signal-to-noise ratio, low resolution, low contrast, and fuzziness of objects (see Figure 1 (a)).
One of the approaches to solving the problem for detecting concealed objects in the terahertz
images consists of image segmentation and recognition of selected objects. Many of the proposed
segmentation methods are based on the assumption that there are three areas in the terahertz image
(background, a human body, and hidden objects) characterized by their radiometric temperature range
corresponding to the ranges of gray tone levels. Gaussian mixture models are usually used to represent
the images. The authors of works [
Suppose there is a set of images of objects that should be detected under the clothes of people, using a passive terahertz camera. Objects that are visible in the image should be compared with the templates, and it is necessary to make a decision about their identity or difference. Suppose there are N template images of objects U n , n 1, 2,..., N . Let the image V be the input image. It is necessary to identify the object U n located in the image V .
To solve such a problem, the segmentation of the input image is usually preliminarily performed, feature descriptions of the images of objects U n are formed, and then in the feature space, the separating surfaces between the classes of the objects under consideration should be found.
The properties of terahertz images listed above complicate the segmentation of objects. Therefore,
it is preferable to use methods that do not require segmentation. In this paper, we propose to solve the
object detection problem as an image matching problem [
Un
Several methods that do not require segmentation of images are known, in particular, methods
based on the maximization of the cross-correlation function values, maximization of the correlation
coefficient [
To apply the information-theoretic approach, a stochastic model of the dependency between the
template image and the input image is necessary. Let the grayscale values in the compared images at
the point x with coordinates (x, y) be described as discrete random variables U (x) and V (x) with
values u and v quantized into a finite number of levels K and L respectively. If images U (x) and
V (x) represent the same scene, there is a relationship between the variables U (x) and V (x) . We
will use a model similar to that proposed in [
V (Tr(x)) F(U (x)) (x), (2) where Tr is a coordinate transformation (for registered images V (Tr(x)) U (x) ); F is a grayscale value conversion function that simulates the relationship between two images of an object; (x) is a random variable that simulates noise. Model (2) can be considered as a model of a discrete stochastic information system with the input U and the output V .
a) b) c) Figure 1. The result of the correlation image matching: (a) an image of a human with a hidden object obtained from the THERZ-7A complex; (b) the image of the template object; (c) the result of an erroneous correlation matching. The contour of the template is shown in red.
The mutual information between the system input and output is defined by the expression: I (U ;V ) K L p(uk , vl ) log p(uk , vl ) ,
k1 l1 p(uk ) p(vl ) where the functions p(u) , p(v) , and p(u, v) define one-dimensional and two-dimensional discrete probability distributions of the gray tone levels of the images U and V . If mutual information (3) is used as a similarity measure, the problem (1) can be formulated as follows. It is necessary to find an image of the template U n maximizing the criterion (3):
Um arg max{I (Un;V )} .
Un In the next section, an algorithm for solving the problem is proposed. (3) (4)
The direct computing of the mutual information values on the set of pixels corresponding to the
intersection of the images U n and V at different positions of Un relative to V requires large
computational expenses. Therefore, in practice, methods for accelerated computing of the mutual
information magnitude are necessary. For example, in [
In this paper, we propose to compute the mutual information values I (Un;V ) at the points of the
input image V corresponding to the local maxima of the cross-correlation function [
s t where CUn and CV are the contour preparations of the images Un and V , x and y are the spatial coordinates of the points of the image C , s and t are the coordinates of the points in the
V intersection area of the images CUn and C . Given the high noisiness of terahertz images, masks of
V
stochastic gradients [
Thus, the search algorithm on the terahertz images of an object corresponding to condition (4)
includes the following operations: (a) smoothing the image V and obtaining contour preparations
CU1 and CV of the images Un and V respectively; (b) calculating the cross-correlation function of
the contour preparations CU1 and CV by the formula (7); (c) finding the points of local maxima of
the cU1,V (x, y) function; (d) calculating the mutual information I (U1;V ) at the cU1,V (x, y) local
maxima points; (e) finding the maximum of mutual information Imax (U1;V ) ; (f) repeating steps (a) –
(e) for the images U1 , ..., U N ; finding the maximum of the mutual information Imax (Um;V ) among the
values of Imax (U1;V ) , Imax (U2;V ) , ..., Imax (UN ;V ) and the corresponding template image Um which
will be the solution of the problem (4). It should be noted that an operation of impulse noise filtering
[
The proposed algorithm has no parameters and does not require tuning. Figure 2 illustrates the main stages of localizing the objects. The input image captured by the THERZ-7A complex is shown in Figure 2 (a), and the image of the template object is given in Figure 2 (b). Contour preparations of these images are presented in Figures 2 (c) and 2 (d). Figure 2 (e) visualizes the cross-correlation function of the contour preparations. The figure shows the local maxima of the surface formed by the function cUn,V (x, y) . The result of localizing the object in the input image is shown in Figure 2 (f).
a) b) c) d) e) f) Figure 2. The main stages of localizing the objects: (a) image captured by the THERZ-7A complex; (b) the image of the template object; (c) the contour preparation of the input image; (d) the contour preparation of the template image; (e) visualized cross-correlation function of contour preparations cUn,V (x, y) ; (f) the result of localizing the object in the input image, corresponding to the maximum of the mutual information I (Un;V ) .
To estimate the applicability of the above approach to solve the problem of localizing and identifying concealed objects, a computational experiment was carried out. The experiment was organized as follows. We obtained template images of objects, which should be detected in images captured at the output of the terahertz imaging system. Template images are shown in Figure 3.
We consistently placed objects under the clothes of a person and recorded image sequences using the THERZ-7A system. Using the proposed method, we compared the template images shown in Figure 3 (a) with frames of the obtained video sequences and fixed the values of the similarity measure (3). The shooting range was about 1.5 meters and varied slightly, so the template images were used with the constant value of the scale factor. The cross-correlation function and the mutual information are not invariant with respect to rotation. Therefore, it is also necessary to analyze the rotated and flipped variants of the template images. The results of the object localizing are shown in Figures 4 (a-e), and the computed values of mutual information are presented in Table 1. It follows from Figure 4 and Table 1 that concealed objects are successfully localized by the proposed method based on a combination of correlation matching and the mutual information maximum matching.
From the data in Table 1, we can draw the following conclusion. When comparing images with concealed objects obtained at the output of the terahertz imaging system and the images of template objects, the maximum of the mutual information is achieved if the template corresponds to the concealed object. Consider an example. Suppose a terahertz image is obtained in which a hidden object is visible (for example, a Walther handgun). This image is matched with the AK submachine gun, TT handgun, and Walther handgun templates. The computed values of the mutual information between the matched images are given in the third row of Table 1. The maximum value of the mutual information is obtained when comparing the input image and the Walther template. The system then makes a decision that the concealed object is a Walther handgun. From this, it follows that the proposed method can be used to detect and recognize prohibited concealed objects in security systems. To eliminate false alarms, the calculated maximum value of the mutual information is compared with a predetermined threshold value. If the found maximum of the criterion (3) is less than the threshold, then the decision is made that the prohibited object is not found. The threshold value is determined empirically.
a) b) c) d) e) Figure 4. The result of the matching images of template objects and the images of hidden objects: (a) TT handgun hidden under the clothes and its template; (b) hidden Walther handgun and its template; (c) hidden AK submachine gun and TT template; (d) hidden Walther handgun and TT template; (e) hidden AK submachine gun and its template.
The paper proposes a new algorithm for localization and identification of objects in images taken using the passive terahertz imaging system. The proposed algorithm combines matching methods based on maximizing the value of the cross-correlation function and the mutual information magnitude. To reduce computational expenses, the criterion of the mutual information maximum is analyzed at points of local maxima of the cross-correlation function computed in the frequency domain. The proposed algorithm does not require parameter tuning. Future research will be aimed at improving the quality of detection and recognition of concealed objects using preprocessing of terahertz images and applying detection methods that are invariant to geometric transformations.
Authors are grateful to the Astrohn Technology Ltd and OOO ASoft who provided us with the THERZ-7A terahertz scanning device. This research was supported in part by the Russian Foundation for Basic Research (grants No 18-07-01295 and No 16-29-09626).