Image Processing, Geoinformatics and Information Security IMPULSE RESPONSE IDENTIFICATION BY ENERGY SPECTRUM METHOD USING GEOINFORMATION DATA IN CASE OF REMOTE SENSING IMAGES A.Y. Denisova, V.V. Sergeyev Samara National Research University, Samara, Russia Abstract. In this paper a modification of spectral energy method of a linear ob- servation model identification using data from geoinformation systems is de- scribed. Spectral energy method uses relation between energy spectra of origi- nal and output images. Original undistorted image is assumed to be unknown. The modification described in this paper supposes that boundaries of all image regions are known. It applies these boundaries to construct an image with ener- gy spectrum similar to original undistorted one. The algorithm is considered in terms of remote sensing images, for which the boundaries of image regions can be presented as vector map data of the same territory and can be recieved from geoinformation systems. Keywords: linear observation model, identification, geoinformation systems Citation: Denisova AY, Sergeyev VV. Impulse response identification by en- ergy spectrum method using geoinformation data in case of remote sensing im- ages. CEUR Workshop Proceedings,2016; 1638: 290-295. DOI: 10.18287/1613-0073-2016-1638-290-295 Introduction Linear observation model is often used to describe the process of image capturing in remote sensing systems. In many cases it is necessary to obtain an impulse response of distortion system using only observed image as input. This problem is called blind channel identification. There are several approaches to blind system identification, for example, solving convolution equation [1], using assumption about some statistical properties of the input image [2], estimating parametric models of impulse response [3]. The main disadvantages of these approaches are that they can not perform well with large images because of their computational complexity or instability in two dimensional case. It is seemed to be a good decision for remote sensing data to use earlier developed in [4-6] energy spectrum method of impulse response identification. The method allows to estimate smoothing impulse response in two dimensional case by means of calcula- Information Technology and Nanotechnology (ITNT-2016) 290 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… tion of observed image autocorrelation function and energy spectrum. For large data volumes, which are typical for remote sensing data, required characteristics are statis- tically stable and there are fast algorithms based on Fourier transform, which allow to compute them effectively. 1 Energy spectrum method In accordance with the linear observation model the relation between original image and distorted image is described as follows: K yn1, n2    hk1, k2 xn1  k1, n2  k2   vn1, n2 , n1, n2  0, N  1, (1) k1 ,k2  K where y n1 , n2  is observed image, hk1, k2  is unknown impulse response, x n1 , n2  is unknown original image without distortions, vn1, n2  is statistically independent from signal white noise, N is image size. The model includes an assumption that impulse response is normalized: k ,k   K hk1 , k 2   1 . K 1 2 Energy spectrum method is based on the relation between energy spectra of original and observed images [7]:    Y ei1 , ei 2  H ei1 , ei 2   e , e  D 2 X i1 i 2 V (2)   where Y ei1 , ei2 is energy spectrum of observed image,  X ei1 , ei2 is energy   spectrum of undistorted image, H e , e  i1 i 2  is frequency response, DV is dispersion of white noise, and 1 , 2 are cyclic frequencies. In real remote sensing systems negative frequency response components corresponds to frequency values higher than maximum frequency presented in a periodic spec- trum of sequence, because they are designed to take into account blur produced by detector and they have sampling step agreed with the blur size. Therefore, the follow- ing expression can be applied to estimate frequency response (2):  H ei1 , ei2    Y ei1 , ei2  DV  X e ,e i1 i 2  . (3) Impulse response can be obtained through inverse Fourier transform of frequency response. The method of impulse response identification using expression (3) is called as spectral energy method. The key issues of the method are how to estimate observed image energy spectrum, noise dispersion and energy spectrum of unknown undistort- ed image. To compute energy spectrum of observed image classical methods of digi- tal spectral analysis can be used [15]. There also many methods for noise dispersion evaluation. In this study averaging of observed energy spectrum values correspond- Information Technology and Nanotechnology (ITNT-2016) 291 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… ing to high frequencies was used, brief description of it can be found in [10]. The most interesting subproblem is estimation of unknown undistorted image energy spectrum. 2 Energy spectrum method using GIS data The modification of energy spectrum method is a nonparametric method and it re- quires following assumptions to be legal: 1. The brightness of undistorted image for any image section is piecewise-constant function. This assumption seems natural for remote sensing images, since the ob- jects on the Earth has clear boundaries. 2. The impulse response is smoothing and affects brightness only on the edges of re- gions with constant brightness levels. It is known that the autocorrelation function properties are defined by the intensity of brightness leaps in each direction on the image. Estimation of undistorted autocorrela- tion function can be achieved from sharpened image with restored border information. Then such estimation can be used to compute undistorted energy spectrum. The fact, that remote sensing images represent a part of Earth surface, makes possible to use electronic map data about boundaries of the objects on the image. Geoinfor- mation systems are the most widespread and well organized electronic map source. The common way to store objects’ boundaries is a vector map, which includes a set of spatial coordinates for each object. Vector map can be transformed into raster with given accuracy. Then raster is used as boundary mask to construct piecewise-constant image with sharp edges from the observed one. Received image is used to get energy spectrum similar to energy spectrum of unknown original image. Further, it is assumed that noise dispersion is has been already estimated. Therefore, proposed modification can be written as follows: 1. Generate raster mask Dm1 , m2 , 0  m1 , m2  MN  1 of regions’ borders by vec- tor map. The raster mask should be in M times larger than observed image to rep- resent borders more precisely. For each region Di in mask the pixels are colored with the index i  1,..., I of region in vector map, where I is the total number of image objects on image. It is obvious that raster mask should have the same refer- ence system as the observed image. 2. Increase the size of observed image y n1 , n2 ,0  n1 , n2  N  1 though the bilinear interpolation with step 1 M : y m1 , m2 ,0  m1 , m2  MN  1 . 3. Construct image with sharp edges xˆ m1 , m2 ,0  m1 , m2  MN  1 by averaging of observed image pixels within the boundaries from the vector map: xˆ m1, m2   yi , m1, m2  Di , i  1, I , yi   yin m1, m2 , 1 Di m1 ,m2Di where Di is amount of pixels, corresponding to i -th mask region. Information Technology and Nanotechnology (ITNT-2016) 292 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV…  4. Estimate energy spectrum of the original undistorted image  X ei1 , ei2 as en-  ergy spectrum   i1 ˆ X e ,e i2  of the image xˆ m1 , m2  received on step 3. Compute energy spectrum of observed image. It should be calculated using ob- served image before interpolation. However, formula (3) supposes that both spec- tra have the same sampling step. To get energy spectra of observed image with smaller sampling step zeroes should be added in frequency domain. It corresponds to interpolation with sinc basis functions in spatial domain [7]. If the influence of other spectrum periods is low, an energy spectrum of sinc-interpolated signal with sampling step T takes the form:  in 1 ,  2   2  in 1 ,  2  , where 1 T  in 1 ,  2  is interpolated energy spectrum and 1 ,  2 are frequencies. So the   ˆ Y ei1 , ei2 used in expression (3) is a result of multiplying energy spectrum    observed image energy spectrum Y ei1 , ei2 on M 2 and adding zeroes to it 2 up to size MN . The factor M corresponds to sampling step 1 M . 5. Calculate frequency response using formula (3). 6. Estimate impulse response though the inverse Fourier transform of received fre- quency response. Interpolation of observed image on step 4 is required to provide boundaries more precisely. Borders accuracy is highly correlated with the energy spectrum estimation quality. To compute energy spectra on the stages 4 and 5 any standard method of digital spectral analysis can be applied [8]. 3 Experimental Research The experimental research was made for set of mosaic images. Firstly, to model un- distorted input image the images with fine sampling step were generated. Correlation coefficient of neighbor pixels for undistorted images equaled to 0.99. Raster masks of boundaries were obtained simultaneously with the original mosaic images and had the size of 4096×4096 pixels. After the original images set had been prepared the set of distorted images was constructed according to the linear observation model described above. Distortions were made with the following impulse response corresponding to the impulse response of MODIS sensor built in Terra and Aqua spacecrafts [9]: hM m1 , m2   h1 m1 , m2 * *h2 m1 , m2 * *h3 m1 , m2 , (4) where ** is a convolution,    h1 k1, k2   A exp  k12  k22 22 , h2 k1 , k 2   rectk1 wrectk 2 w, h3 k1 , k 2   rectk1 s ,  K  k1 , k 2  K . In our notation the parameters of the impulse response were   w  s  8 . Generated impulse response was used as etalon in comparison to the restored impulse responses. The sampling step of the impulse response was the same as for the fine images. Information Technology and Nanotechnology (ITNT-2016) 293 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… Finally, observed image was obtained as a result of convolution of etalon impulse response and original image, sampling with bigger sampling step than sampling step of original image and putting of independent “white” noise into final image. Observed images size was 512×512 pixels. Reconstruction was made with bilinear interpolation on the stage 2 with the step 0,125 and with parameter M  8 . The example of observed image is presented in fig.1. The results of impulse response estimation in case of signal to noise ratio equal to 120 are shown in fig.2. Fig. 1. The fragment of observed image -3 -3 x 10 x 10 7 7 6 6 5 5 4 4 hM(0,k2) hM(k1,0) 3 3 2 2 1 1 0 0 -1 -1 -40 -20 0 20 40 -40 -20 0 20 40 a) k2 b) k1 Fig. 2. The central section in a) transverse and b) longitudinal directions of ideal (gray) and restored (black) impulse responses The error of impulse response restoration was estimated by normalized root mean square error:  hk1 , k2   hˆk1 , k2  , K 1 2  2K  1h0,0 (5) k1 , k 2   K where hk1 , k 2  is ideal impulse response, hˆk1, k2  is estimated impulse response. It (5) is considered as relative error. Table 1 contains average values  of the error and its standard deviation   in cases with different values of signal to noise ratio d in observed images. Table 1. Average value and standard deviation d=250 d=120 d=15       0.0039 0.0001 0.0045 0.0001 0.0075 0.0001 Information Technology and Nanotechnology (ITNT-2016) 294 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… From table 1 it can be seen that in the error is less than 1%. So proposed method al- lows to reconstruct impulse response with high accuracy even in presence of noise with SNR value more than 15. Conclusion Experiments in our study show that proposed method can be applied to impulse re- sponse restoration in remote sensing images. The main advantage of the proposed method is that it allows to reconstruct impulse an response with sampling step smaller than the sampling step of observed image. Acknowledgements The research was financially supported by RSF, grant №16-37-00043_mol_a «Devel- opment of methods of using data from geoinformation systems in remote sensing data processing», grant №16-29-09494 ofi_m «Methods of computer processing of multi- spectral remote sensing data for vegetation areas detection in special forensics». References 1. Fursov VA. Image restoration using filters with finite impulse response by means of direct identification of inverse tract. Computer Optics, 1996: 16: 103-108. [in Russian] 2. Goriachkin OV, Erina EI. Blind channel identification by manifolds of given correlation generated by random polynoms. Achievements of Modern Radioelectronics, 2008; 8: 70- 77. [in Russian] 3. Bavrina AY, Myasnikov VV, Sergeyev AV. Parametrical identification of optical- electronic tract of optical remote sensing system. Computer Optics, 2011; 35(4): 500-507. [in Russian] 4. Sergeyev VV, Denisova AY. Spectral-Energy Identification Method of the Linear Obser- vation Model for Remote Sensing of the Earth. Pattern Recognition and Image Analysis, 2011; 21(2): 321-323. 5. Sergeyev VV, Denisova AY. Iterative method of piecewise-constant image restoration in case of regions border knowledge. Computer Optics, 2013; 37(2): 239-243. [in Russian] 6. Sergeyev VV, Denisova AY. Spectral energy identification method of the linear observa- tion model in the absence of a covariance function model. Pattern Recognition and Image Analysis, 2014; 24(4): 561-565. 7. Soifer VA. Computer Image Processing, Part II: Methods and algorithms. VDM Verlag, 2009. 8. Marple JS, Lawrence. Digital spectral analysis with applications. Englewood Cliffs, NJ, Prentice-Hall, Inc., 1987; 512 p. 9. Schowengerdt RA. Remote sensing: models and methods for image processing. Academic press, 2006. Information Technology and Nanotechnology (ITNT-2016) 295