Image Processing, Geoinformatics and Information Security USING GIS DATA TO IDENTIFY LINEAR OBSERVATION MODEL ON REMOTE SENSING IMAGES IN CASE OF SPATIAL MISMATCH OF INPUT IMAGE AND VECTOR MAP A.Y. Denisova, V.V. Sergeyev Samara National Research University, Samara, Russia Image Processing Systems Institute - Branch of the Federal Scientific Research Centre "Crystallography and Photonics" of Russian Academy of Sciences, Samara, Russia Abstract. This paper presents an experimental research of identification meth- od proposed by the authors in their previous works. The method was developed to identify a linear observation model in remote sensing images. It uses the rela- tion between energy spectra of input and output images, and it is called energy spectrum method. The latest modification of the method applies vector map da- ta from geoinforation system to construct an image with energy spectrum simi- lar to energy spectrum of unknown undistorted image. Vector map and input image had been supposed to be exactly spatially coherent. In this paper the al- gorithm’s quality is investigated in case of inexact spatial matching of input im- age and vector map. Accuracy of input image georeference and accuracy of vector map borders are considered as major spatial mismatch factors. Experi- mental evaluation of the these factors influence on quality of impulse response restoration is provided. Keywords: identification of linear observation model, impulse response, en- ergy spectrum method, geoinformation systems, vector map, georeferencing. Citation: Denisova AY, Sergeyev VV. Using GIS Data to Identify Linear Ob- servation Model on Remote Sensing Images In Case of Spatial Mismatch of In- put Image and Vector Map. CEUR Workshop Proceedings, 2016; 1638: 296- 303. DOI: 10.18287/1613-0073-2016-1638-296-303 1 Introduction The common way to describe the process of image acquisition in remote sensing sys- tems is a linear observation model [1]. For different applications, for example, for image correction, it is important to estimate the parameters of the model using only observed image. State of the art methods of system’s impulse response identification [2-4] work mainly with one dimensional signals or have high computational complex- ity, that significantly impedes their use in case of remote sensing images. Information Technology and Nanotechnology (ITNT-2016) 296 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… In papers [5-6] we described an approach of system impulse response estimation based on the relation between energy spectra of observed image and original un- distorted image. It is called energy spectrum method. It supposes that original image is unknown. The method allows to identify two dimensional symmetrical nonnegative impulse response. Occurrence of geoinformation systems (GIS) and electronic map services makes pos- sible to use accumulated vector data for remote sensing image processing. In article [7] we proposed a modification of energy spectrum method using GIS data. This modification was considered in case of exact geometrical matching of observed image and vector map. In practice this condition may not be satisfied because of georefer- encing errors and borders changes cased by lower updating rates for vector data. The aim of present research is to evaluate experimentally an influence of spatial mismatch factors on the quality of impulse response estimation. The paper is orga- nized as follows. In first section we adduce the computer realization of energy spec- trum method using GIS data. The second section describes experimental research of the method in case of inexact georeferencing of observed image and mismatch of object borders on observed image and vector map. 2 Energy spectrum method modification using GIS data Let us suppose that observed discrete image y d n1 , n2  has sampling step T and size N  N , where n1 , n2  0, N  1 are integer pixel coordinates, and unknown original image xm1 , m2  , m1 , m2  0, M  1 is also discrete with sampling step T1  T . To simplify further descriptions it is assumed that M and N are even numbers and the ratio of sampling steps is denoted as   T T1 . If the noise dispersion D̂V is already evaluated, the energy spectrum method using GIS data for estimation of unknown impulse response hk1 , k 2  , k1 , k 2   K , K 2 K 1  M can be written as follows: 1. Build raster mask Dm1, m2 , m1, m2  0, M  1 of object borders from the vector map with sampling step T1 . Each object Di corresponds to pixels with value i, i  1, I , where I is amount of objects on the image. 2. Interpolate observed image y d n1 , n 2  with step 1  and size M  M . Received image is denoted as y in m1 , m2 , m1 , m2  0, M  1 . 3. Make piecewise-constant image with sharp borders xˆm1 , m2 , m1 , m2  0,..., M 1 by means of averaging pixels for each object xˆ m1 , m2   yi , m1 , m2  Di , i  1, I ,  yin m1, m2 , 1 yi  Di m1 , m2 Di (1) Information Technology and Nanotechnology (ITNT-2016) 297 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… where Di is the number of pixels for object with index i . ˆ X l1 , l2  , 4. Calculate an estimation of unknown original image energy spectrum  l1 , l 2   M2 ,..., M2  1 using xˆm1 , m2 , m1 , m2  0,..., M 1 by means of discrete Fourier transform with length M . Pixels with indexes l1 , l 2   M2 ,..., M2  1 cor- 2l1 2l2 respond to frequencies 1  , 2  . M M ˆ Yd   p1 , p2 , p1 , p2   N2 ,..., N2  1 of observed image 5. Calculate energy spectrum  y d n1 , n2  , n1 , n2  0,..., N 1 using discrete Fourier transform with length N . Pixels with indexes p1 , p2   N2 ,..., N2  1 corresponds to frequencies 2p1 2p2 1  , 2  . N N ˆ Yd   p1 , p2  : ˆ Y l1 , l2  by adding zeros to spectrum  6. Calculate energy spectrum  ˆ Yd  l1 , l2   DV ,  2   при  N2  l1  N2 и  N2  l2  N2 ;  0,   Y l1 , l2   при  M2  l1   N2 и  M2  l2  M2 , ˆ (2)  N l  M и  M l  M ,  2 1 2 2 2 2   N l  N и  M l   N ,  2 1 2 2 2 2   N l  N и N l  M .  2 1 2 2 2 2 7. Achieve an estimation of frequency response Hˆ l1 , l2  , l1 , l 2   M M ,..., 1 : 2 2 ˆ Y l1 , l2   H l1 , l2   (3) ˆ X l1 , l2   8. Find impulse response hˆk1 , k 2  using discrete Fourier transform with length M . Optional pixels with k1 , k 2  K can be set to zero. An estimations of energy spectrum on stages 4 and 5 could be obtained using standard methods of digital spectral analysis. To know more about these methods see [7]. Information Technology and Nanotechnology (ITNT-2016) 298 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… 3 Experimental research Preparation of input data for the experiments includes following stages: 1. Generate original undistorted image and borders mask corresponding to it. Both images have sampling step T1 and size M  M pixels. Correlation coefficient be- tween neighbor pixels of original undistorted image is denoted as  . 2. Incorporate distortions into image by means of convolution of original image with given impulse response having sampling step T1 . This impulse response was used as etalon. 3. Digitize distorted image with size M  M in  times. Resulting image had size N  N pixels and sampling step T . It was interpreted as observed image with ac- curate geometrical parameters. 4. The last stage is including spatial mismatch to observed image and objects border mask. In all experiments sampling steps and impulse responses were defined according to chosen sensor model. In this study we present results for  MODIS sensor (Terra\Aqua) [1] with parameters T1  31,25 m, T  250 m and impulse response: hM k1 , k 2   h1 k1 , k 2  * *h2 k1 , k 2  * *h3 k1 , k 2  ,  ETM+ sensor (Landsat-7) [1] with parameters T1  3,75 m, T  30 m and impulse response: hL k1 , k 2   h1 k1 , k 2  * *h2 k1 , k 2  ,   2 2 2   where * * is convolution, h1 k1 , k 2   A exp  0.5 k1  k 2  models smoothing of image due to defocusing; h2 k1, k2   rect k1 w rect k2 w is impulse response of detector; h3 k1, k2   rect k1 s  corresponds to motion-blur, k1 , k 2   K , K . In the first case parameters of impulse response were   123,5 m [10], w  s  250 m [1] and K  20 , in the second case they were   30 m, w  30 m [9], K  30 . Quality of impulse response restoration was evaluated as root mean square error (RMSE) normalized by maximum (central) value of ideal impulse response:  hk1 , k 2   hˆk1 , k 2  , K 1 2  (4) 2K  1h0,0 k1 ,k2  K where hk1 , k 2  is ideal impulse response, hˆk1 , k 2  is estimated impulse response. Expression (4) can be interpreted as relative error of impulse response restoration. The experimental research was made with absence of noise. Information Technology and Nanotechnology (ITNT-2016) 299 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… 3.1 Quality of impulse response restoration in case of inexact georeference of input image To investigate the influence of georeference accuracy on the quality of impulse re- sponse estimation mosaic images were used as input. Input images were generated in accordance with the process described above with following parameters M  4096 ,   0,99 , N  512 . The example of observed picture is shown in fig. 1. Fig. 1. The example of observed image On the fourth stage observed images were shifted horizontally (diagonally) relatively to initial position by B  1,..., 5 pixels of observed image. Experimental results are shown in fig. 2. 0.25 0.2 0.15  0.1 0.05 0 B 0 1 2 3 4 5 a) horizontally diagonally 0.3 0.25 0.2  0.15 0.1 0.05 0 B 0 1 2 3 4 5 b) horizontally diagonally Fig. 2. Impulse response restoration RMSE depending on shift parameter B for different sensor models: a) MODIS, b) ETM+ Information Technology and Nanotechnology (ITNT-2016) 300 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… It is seen from the charts that average value of impulse response estimation error does not exceed 10% in case of MODIS sensor model and shift parameter value about 2-3 pixels of observed image. Such georeference accuracy meets the requirements of many image processing methods and can be achieved by standard georeferecing methods. As for the impulse response modeling ETM+ sensor, shifting becomes a problem and does not allow to reconstruct it with appropriate accuracy. The reason is wider im- pulse response with higher smoothing characteristics in terms of observed image pixels. 3.2 Quality of impulse response restoration in case of objects borders mismatch on vector map and observed image In order to estimate the influence of inexact vector map on the quality of impulse response restoration, simulated images were used. The images of agricultural land- scape from Landsat 7 were transformed into piecewise-constant form by averaging pixel values within the borders of agricultural fields vector map for the same sowing season. Piecewise-constant images represented original undistorted image data and had following parameters M  2048 ,   0,95 , N  256 . Ideal border mask was made as a result of initial vector map rasterization. Distortions in vector map were made as a result of partial replacement of objects’ borders by borders from previous sowing season of the same objects. We used the following ratio: S  S~ S As quantitative characteristic of borders mis- ~ match, where S is an area of disagreement of ideal and distorted border masks, and ~ S is total area of objects on ideal mask. Both values S and S are measured in pix- els of original undistorted image. The example of observed image and binary image of mismatched vector regions are presented in fig. 3. The value of S , corresponding to the example on fig. 3, equals to 0,025. a) b) Fig. 3. Examples of: a) observed image and b) binary image of mismatched vector regions (black color - changes in vector mask) Information Technology and Nanotechnology (ITNT-2016) 301 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… In fig. 4 the graph of impulse response RMSE depending on S value is presented. It can be seen that border changes up to 2% of total objects area do not affect on quality of restoration. 0.14 0.12 0.1 0.08  0.06 0.04 0.02 0 0 0.005 0.01 0.015 0.02 0.025 S IR MODIS IR ETM+ Fig. 4. Impulse response restoration RMSE depending on mask border changes area However, RMSE is significantly higher than in previous experiment. The reason is that mosaic images had simpler borders geometry than the real vector map objects. The influence of borders geometry on quality of impulse response restoration is the question of future research. Conclusion The experimental research presented in paper shows that energy spectrum method using GIS data can be applied in case of inexact spatial matching of observed image and vector map. It was shown that allowable georeferencing error is about 2 or 3 pixels of observed image as a result of simulation for MODIS impulse response. Ex- periments also shown that allowable changes in borders of vector map are about 2% from total area of objects. Further research deals with estimation border geometry influences on the quality of impulse response restoration and with developing of method’s modification that will be stable to various spatial mismatches in vector map and image data. Acknowledgements The research was financially supported by RSF, grant №16-37-00043_mol_a «Devel- opment of methods of using data from geoinformation systems in remote sensing data processing», grant №16-29-09494 ofi_m «Methods of computer processing of multi- spectral remote sensing data for vegetation areas detection in special forensics». Information Technology and Nanotechnology (ITNT-2016) 302 Image Processing, Geoinformatics and Information Security Denisova AY, Sergeyev VV… References 1. Schowengerdt RA. Remote sensing: models and methods for image processing. Academic press, 2006. 2. Fursov VA. Image restoration using filters with finite impulse response by means of direct identification of inverse tract. Computer optics, 1996; 16: 103-108. [in Russian] 3. Goriachkin OV, Erina EI. Blind channel identification by manifolds of given correlation generated by random polynoms. Achievements of Modern Radioelectronics, 2008; 8: 70- 77. [in Russian] 4. Bavrina AY, Myasnikov VV, Sergeyev AV. Parametrical identification of optical- electronic tract of optical remote sensing system. Computer optics, 2011; 35(4): 500-507. [in Russian] 5. Sergeyev VV, Denisova AY. Spectral-Energy Identification Method of the Linear Obser- vation Model for Remote Sensing of the Earth. Pattern Recognition and Image Analysis, 2011; 21(2): 321-323. 6. Sergeyev VV, Denisova AY. Spectral energy identification method of the linear observa- tion model in the absence of a covariance function model. Pattern Recognition and Image Analysis, 2014; 24(4): 561-565. 7. Sergeyev VV, Denisova AY. Impulse response identification for remote sensing images using GIS data. Computer Optics, 2015; 39(4): 557-563 [in Russian]. DOI: 10.18287/0134-2452-2015-39-4-557-563. 8. Sergeyev VV, Denisova AY. Iterative method of piecewise-constant image restoration in case of regions border knowledge. Computer Optics, 2013; 37(2): 239-243. 9. Storey JC. Landsat 7 on-orbit modulation transfer function estimation. International Sym- posium on Remote Sensing, 2001: 50-61. 10. Huang C, Townshend JRG, Liang S., Kalluri RS. Impact of sensor's point spread function on land cover characterization: assessment and deconvolution. Remote Sensing of Envi- ronment, 2002; 80(2): 203-212. Information Technology and Nanotechnology (ITNT-2016) 303