Computer Technology of High Resolution Satellite Image Processing Based on Packet Wavelet Transform Vita Kashtan1 [0000-0002-0395-5895], Volodymyr Hnatushenko2 [0000-0003-3140-3788] 1 Oles Honchar Dnipro National University, Dnipro, 49010, Ukraine 2 Dnipro University of Technology, Dnipro, 49005, Ukraine vitalionka@gmail.com, vvgnatush@gmail.com Abstract. This paper discusses spatial quality improvement of multispectral satellite images with minimizing color distortion. Technology is based on bicu- bic resampling, HSV-, packet wavelet-transform involves the loading of prima- ry different spatial resolution images of the same scene; transform after the spectral correction of primary images in color space HSV, optimal packet wavelet based decomposition of the synthesized panchromatic image until the specified decomposition level according to the chosen information value func- tion linear forms. The new technology of high resolution satellite image pro- cessing has been tested on the satellite images. Comparison of quantitative indi- cators as well as the visual results shows the advantage of using proposed tech- nology. Keywords: remote sensing, panchromatic and multispectral images, resolution, HSV-transform, packet wavelet transform, Shannon entropy. 1 Introduction In recent years the systems and methods of optical remote sensing have become the basic tools of objects state and events control on the Earth surface. For monitoring natural phenomena consequences and Earth surface state it is needed to use satellites with high-spatial resolution: Pleiades-1A, Pleiades-1B, TripleSat Constellation (DMC-3), DubaiSat-2, Jilin-1, WorldView-1,2,3, RapidEye, Cartosat -3 etc. Such satellites allow to obtain hundreds of images digitally of a target local area. The anal- ysis of such multichannel data is a very difficult task and comes down to specified objects emphasizing, obtaining their characteristics, and relative position. The typical data set of remote sensing apparatus mounted on satellites includes: multispectral (multichannel) image and panchromatic image (PAN). A panchromatic image usually has a higher spatial resolution than multispectral one, which substantially complicates objects recognition and imposes restrictions on the used processing methods. For information content of primary data improvement, the existing methods of images processing have a set of disadvantages, the main of which is color distortions [1-4]. The aim of the work is improvement of primary multichannel image spatial resolution minimizing color distortion. Images taken from WorldView-2 satellite are used as input data. To determine the effectiveness of the proposed information technology Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attrib- ution 4.0 International (CC BY 4.0) CMiGIN-2019: International Workshop on Conflict Management in Global Information Networks. quantitative quality assessment of synthesized multispectral images will be obtained, in particular: Shannon entropy, signal entropy etc. 2 State of art Nowadays there are different methods of obtaining synthesized multispectral im- ages with spatial resolution increase by merging them with panchromatic images: Brovey-transform, PC-sharpening, independent component analysis (ICA), Gram- Schmidt, IHS-transform. But these methods do not take into account constructing characteristics of modern scanning devices, appropriate structures and high resolution data formats [1, 3-7]. Chu Heng and Zhu Weile [8] proposed method based on the transition to color-difference metrics of computer graphics, where the question about decorellation of primary data is solved. However, these methods allow us to take into account only spectral components of primary grayscale image. One of the most per- spective and effective mathematical apparatus for aerospace images analysis is packet wavelet transform. Its appliance allows to get photogrammetric scanner images which are obtained by traditional methods, and the methods that use discrete wavelet trans- form [9-11]. 3 Methods and materials This work proposes a fusion method based on packet wavelet bases building with decorrelation of primary aspectual data. The proposed algorithm scheme is shown in the fig. 1. Fig. 1. Technology scheme The main stages of primary multichannel image processing are: 1. Download multichannel image in RGB color space and image resampling. Fig. 2 shows fragments of scene panchromatic channel (PAN) and RGB composi- tion (Bands 5-3-2) from satellite WorldView-2. Image resampling is a process in which new pixel values are interpolated from existing pixel values, whenever the raster’s structure is modified during, for example, projection, datum transformation, or cell resizing. There are many resampling methods available through a number of platforms, including image-processing software. Bilinear interpolation, nearest neigh- bor, and cubic convolution are most commonly used resampling methods in remote sensing. We used bicubic resampling. a) b) Fig. 2. Satellite images of WorldView-2: a) panchromatic; b) multispectral 2. Geometric and spectral correction: decompose the appropriate RGB and PAN image luminance channel to the sixth decomposition level (L) by the packet wavelet transform of the bior 6.8 class according to the logarithmic information value func- tion. Calculate Shannon entropy in its extended definition; choose the maximum val- ue between the two ones and get the optimal wavelet tree based on the RGB image; inverse packet wavelet decomposition [9, 12, 13]. 3. Decorrelation in HSV: f RGB (r )  f HSV (r ), . (1) f PAN ( p )  f HSV ( p ) 4. Optimal packet wavelet base decomposition of the PAN until the specified de- composition L built at the previous stage [9, 12]: L f P (r) = Tc PL (r) +  [Td Pl,1(r),Td Pl,2 (r),Td Xl,3 (r)] . (2) l=1 5. New components formation according to the specified rule of coefficients merg- ing [9]: App XL (r) = Tc XL (r), L . (3) Det X (r) =  [Td Pl,1(r),Td Pl,2(r),Td Pl,3(r)] l =1 6. Reverse wavelet packet decomposition and transition to HSV color metrics: f XYZ (r) = App XL (r) + Det X (r), . (4) f XYZ (r)  f HS V (r) 7. In the reverse transform from the HSV color space in the RGB-space, choose H and S components of multichannel component images and the resulting V after wave- let-transformation of panchromatic image and getting the Fusion result. For displaying the research results of the information characteristics (IC) of differ- ent wavelet bases and merger methods, there are the next items used: conical coordi- nates system, the base radius of which is equal to the maximum function value all the way through the arguments set. The lateral surface is divided into sectors and subsec- tors depending on the problem. Within the sector or subsector framework the results are represented as colorful markers in which the color matches the wavelet decompo- sition level. The marker position matches the final value of radius-vector, the begin- ning of which is in the inner radius of the circle – conditional zero. The diagrams show the values of the conditional zero and the maximum value among the absolute values (I – II quarters), and the values which determine the reserve of dynamic infor- mation criteria range – D (reflected in the III – IV quarters) and are defined in accord- ance with the following expression:  C  Min  D = 20log 10   , db , (5)  Max  where Min, Max – appropriate minimum and maximum absolute values ІC, С – cur- rent ІC value. For providing the comparison analysis of the mathematical models, the minimal geometric size of the primary data is necessary. It is established that primary data geometric size is the most influencing one for definition of the models built on the wavelet packet transform base, i.e. for building optimal wavelet trees with a specified information value function (IVF) and wavelet filter. Impact of the specified factors consists in obtaining (or not obtaining) the optimal packet wavelet structure – Epw. The cases when optimal wavelet tree is not obtained are: transition of packet tree structure to normal wavelet; getting a full packet wavelet tree. For establishing the fact that the optimal packet wavelet tree was obtained, the next criteria is used: n  n0 E pw = 1  , (6) N  n0 where n is the total number of nodes of the obtained wavelet packet tree, N is the total number of nodes of the full wavelet tree, n0 is the total number of nodes of the normal wavelet structure. The specified criterion acquires its maximum value (1) when getting a full packet wavelet tree, and minimal (0) – in case of normal wavelet structure. The fact of ob- taining the optimal wavelet tree (Epw) is determined by indicator (6), which does not have to take specified limit values. As further each class of the wavelet filters is sub- mitted by its two members, within the task of determining the minimum geometric primary data size, the wavelet filter will be considered with the highest order within the class. The results of solving this problem are shown in fig. 3, where the results are within the sector regarding to the representative of each wavelet decomposition class, name- ly the Daubechies filters of the 12 order (db12), Symlet filters of the 12 order (sym12), Koyflet filters of the 5 order (coif 5), biorthogonal filters of the 9/11 order (bior 6.8); the result is within each subsector. The following RGB are used: Shannon, norm, log entropies, entropy [13] for primary images with geometric size within range of from 350х350 to 800х800 with 50х50 step. Inside the brackets, next to the sizes of the primary data, there are maximum decomposition level which is typical for the specific geometric size and type of the wavelet filter; exactly with such decomposi- tion level the problem solving is provided. As most results are specified in the range of [0…0.9] D, the fig.3 shows its scale increase. Fig. 3. Fact of getting the optimal wavelet packet tree dependence on the primary image size, the wavelet order, and class According to the results, geometric sizes of primary images are equal to 650х650 pixels, for such size it is possible to get Epw structure, according to the specified wavelet filters and IVF. It is a typical behavior for the provided comparative analysis with the specified criteria (6) with IVF. The IVF is defined as the logarithm, because regardless of the size of the original image and wavelet filters types, it takes its max- imum value which matches the case of getting a full wavelet packet tree. When analyzing the obtained IC "signal entropy" results: there are a greater differ- ence between the second and the third decomposition levels than between quality scores of the previous criteria, and more precise definition of the global minimum observed. While analyzing the results got by ІC "conditional signal entropy" in relation to the primary RGB D is not very different from the conditional Shannon entropy criteria (fig.4). Fig. 4. Quality assessment results of the images obtained by the fusion method with packet wavelet decomposition (ІC – conditional signal entropy) When analyzing the results by ІC “Shannon entropy”, the maximum quality score is calculated with wavelet filter bior 2.4; the wavelet decomposition involvement is ineffective at the first decomposition level. Moreover, the least appropriate quality score is got when involving the wavelet decomposition based on wavelet filter db4. While analysing the results got by ІХ "conditional Shannon entropy" in relation to the primary MSI: the maximum of the score obtained by wavelet filter bior 2.2; rapid growth of the informative value until the second decomposition level by wavelet filter bior db4, and until the third decomposition level by wavelet filter bior 2.2 prove the statement regarding to the informative value of the Shannon entropy about inefficien- cy of the first wavelet decomposition appliance inefficiency. While analysing the results got by ІХ "standard deviation": the quality maximum score calculated with wavelet filter db4; rapid growth of the informative value until the third decomposition with further less rapid growth; considerable difference be- tween informativeness score and fusion methods obtained by the first, second, and third decomposition levels. While analysing the results got by ІХ "integrated informativeness by Shannon" and "signal integrated informativeness" in relation to the primary MSI: ІХ Shannon En- tropy dynamics are decreasing depending on the level of wavelet decomposition – the maximum function decline is observed on the second decomposition level with fur- ther minor current ІХ increase; dynamics of the ІХ Signal Ventropy is also decreasing depending on the wavelet decomposition level. For packet wavelet transforms, the best indicators by ІC and computational com- plexity are obtained for the case of missing the stages of wavelet trees structures op- timization by the chosen IVF, what reduces the condition until minimal geometric primary aspectual data sizes, i.e. the geometric sizes are limited only by capacity of the sets, which define the low pass and high pass filters, and by the necessary wavelet decomposition level. 4 Results Fig. 5 shows the synthesized image after working new technology based on bicubic resampling, HSV-, packet wavelet- transforms. Quality analysis of the fig.5, which was obtained by the fusion method when using the geometric indicator, has shown absence of any affine distortions. It proves linearity of the proposed mathematical models. Visual "quality" image can be evaluated according to the criteria of maxi- mum information content characteristics. Entropy is used to measure the amount of information [13]. Fig. 6 shows a graphical comparison of the absolute values and dynamical ІC rang- es in relation to the images, which were got by the fusion procedure based on packet wavelet transforms. The results show the effectiveness of the proposed technology. The correlation coefficient (CORR) is an important indicator reflecting the differ- ence between the fused image and the original image [13]. Table 1 shows the values CORR for the Gram-Schmidt, wavelet, packet wavelet, HSV, PCA, and new technol- ogy image fusion methods. Table 2 shows value of SSIM of the pansharpenined results. The structural similar- ity image quality paradigm is based on the assumption that the human visual system is highly adapted for extracting structural information from the scene. Fig. 5. The resulting image on new technology Fig. 6. Graphical representation of results of fusion based on packet wavelet transforms and proposed technology Table 1. Correlation value CORR Methods R G B PCA 0.75 0.79 0.87 Gram-Schmidt 0.86 0.85 0.84 HSV 0.48 0.57 0.54 Wavelet 0.72 0.82 0.71 Packet wavelet 0.78 0.85 0.79 New technology 0.96 0.95 0.97 The value measure of the structural similarity is calculated as follows [14]:    2 XY   2   SSIM   XY    2  X Y 2  , (7)  X Y    X   Y    X   Y       2 2 1 M N 1 M N X  ij MN i 1 j 1 x , Y   yij , MN i 1 j 1 (8) x  X  , M N 1  M  1  N  1   2  X2  ij (9) i 1 j 1  y Y  , M N 1  M  1  N  1   2  Y2  ij (10) i 1 j 1  x  X  y  Y  , M N 1  M  1  N  1   2  XY  ij ij (11) i 1 j 1 where SSIM – structural similarity (quality) index; X  xij  , Y   yij  – Images are compared; M, N - the size of the image;  xy – covariance between x and y , and  x2 and  y2 - standard deviation [14]. Table 2. SSIM of the pansharpening results. Methods R G B PCA 0.55 0.47 0.47 Gram-Schmidt 0.42 0.43 0.44 HSV 0.58 0.67 0.65 Wavelet 0.45 0.56 0.49 Packet wavelet 0.66 0.67 0.69 New technology 0.71 0.79 0.77 Can see that these methods may enhance the detail of the image but result in much loss of spectral information. These results point out one of the main advantages of our technology: original spectral information is maintained, while image detail is en- hanced. So, much less extremes in dynamics of quality indicators (table 2) shows more sta- bility of the proposed technology and monotonously increasing Shannon, signal en- tropies, conditional Shannon, and conditional signal entropies dependence on the level of the wavelet decomposition. The analysis of the obtained results by the «Peak Signal-to-Noise Ratio», concerning the primary MSI (fig. 7), helps determine the fact that visual quality is lower when using the existing methods but not the suggested technology. The technology influences the quality of objects recognition and increas- es the quality of primary satellite images by 10–12%. Fig. 7. Results of the obtained images by the roposed technology and existing methods quality assessment 5 Conclusions In this paper we propose computer technology of high resolution satellite image processing based on packet wavelet transform. Most of the fusion techniques that have been proposed are based on the spectral consistency. In this paper, a computer technology based on HSV- and packet wavelet transform has been adopted for high resolution satellite multichannel image fusion without spectral distortions in local areas. Compared with the already existing fusion methods the proposed technology helps avoid substantial color distortions and improve accuracy of the further objects recognition in pictures. It is obtained, particularly, by the previous correction of the primary images and data processing in localized spectral bases which is optimized by information characteristics. The qualitative experimental results, based on different testing data sets, show that the proposed technology to reduce the time of correspond- ing processing of data without loss of accuracy. Our future research will focus on perfection of the proposed technology of im- proving multi-channel data informativeness, taking into account different types of wavelet packet characteristics and selecting the optimal decomposition. 6 References 1. Pohl, C., Van Genderen, J.: Multisensor image fusion in remote sensing: concepts, meth- ods and applications. International journal of remote sensing, nol. 19, No. 5, pp. 823-854 (1998). 2. Hnatushenko, V., Hnatushenko, Vik., Kavats, O., Shevchenko, V.: Pansharpening technol- ogy of high resolution multispectral and panchromatic satellite images. Scientific Bulletin of National Mining University, Issue 4, 91-98 (2015). 3. Manu, C.S., Jiji, C.V.: A novel remote sensing image fusion algorithm using ICA bases. Advances in Pattern Recognition (ICAPR), Eighth International Conference. P. 244-249 (2015). 4. Shedlovska, Y., Hnatushenko, V., Kahtan, V.: Satellite imagery features for the image similarity estimation. International Young Scientists Forum on Applied Physics 2017, p. 359-362. Lviv (2017). 5. Nikolakopoulos, K.G.: Comparison of Nine Fusion Techniques for Very High Resolution Optical Data. Photogrammetric Engineering & Remote Sensing. Vol. 74. No 5. P. 647-659 (2008). 6. Hnatushenko, V., Kavats, O., Kibukevych, I.: Efficiency determination of scanner data fu- sion methods of space multispectral images. International Young Scientists Forum on Ap- plied Physics «YSF-2015», Dnipropetrovsk (2015). 7. Chen, F., Qin, F., Peng, G., Chen S.: Fusion of remote sensing images using improved ICA mergers based on wavelet decomposition, , pp. 2938-2943, Procedia Eng. (2012). 8. Heng, C., Weile, Z.: Fusion of IKONOS Satellite Imagery Using IHS Transform and Local Variation, IEEE Transactions on Geoscience and remote sensing, vol. 5, no. 4, pp. 653 – 657 (2008). 9. Hnatushenko, V., Vasyliev, V.: Remote sensing image fusion using ICA and optimized wavelet transform. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLI-B7, pp. 653-659 (2016). 10. Cheng, J., Liu, H., Liu, T., Wang, F., Li, H.: Remote sensing image fusion via wavelet transform and sparse representation. Elsevier. Elsevier-ISPRS Journal of Photogrammetry and Remote Sensing. Vol. 104. P. 158–73 (2015). 11. Kahtan, V., Hnatushenko, V., Shedlovska, Y.: Processing technology of multispectral re- mote sensing images. International Young Scientists Forum on Applied Physics. 2017, p. 355-358. Lviv (2017). 12. Mallat, S.: Theory for Multi-resolution signal decomposition: the wavelet representation. IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 11, no. 7, pp. 674-693 (1987). 13. Nikolakopoulos, Konstantinos, Oikonomidis, Dimitrios: Quality assessment of ten fusion techniques applied on Worldview-2, European Journal of Remote Sensing, 48:1, 141-167 (2015). 14. Wang, Z., Bovik ,A. C., Sheik, H.R., Simoncelli, E.P. Image quality assessment: From er- ror visibility to structural similarity. IEEE Trans. Image Processing 13 (4), pp. 600–612 (2004).