=Paper=
{{Paper
|id=Vol-2588/paper31
|storemode=property
|title=Computer Technology of High Resolution Satellite Image Processing Based on Packet Wavelet Transform
|pdfUrl=https://ceur-ws.org/Vol-2588/paper31.pdf
|volume=Vol-2588
|authors=Vita Kashtan,Volodymyr Hnatushenko
|dblpUrl=https://dblp.org/rec/conf/cmigin/KashtanH19
}}
==Computer Technology of High Resolution Satellite Image Processing Based on Packet Wavelet Transform==
https://ceur-ws.org/Vol-2588/paper31.pdf
Computer Technology of High Resolution Satellite Image
Processing Based on Packet Wavelet Transform
Vita Kashtan1 [0000-0002-0395-5895], Volodymyr Hnatushenko2 [0000-0003-3140-3788]
1
Oles Honchar Dnipro National University, Dnipro, 49010, Ukraine
2
Dnipro University of Technology, Dnipro, 49005, Ukraine
vitalionka@gmail.com, vvgnatush@gmail.com
Abstract. This paper discusses spatial quality improvement of multispectral
satellite images with minimizing color distortion. Technology is based on bicu-
bic resampling, HSV-, packet wavelet-transform involves the loading of prima-
ry different spatial resolution images of the same scene; transform after the
spectral correction of primary images in color space HSV, optimal packet
wavelet based decomposition of the synthesized panchromatic image until the
specified decomposition level according to the chosen information value func-
tion linear forms. The new technology of high resolution satellite image pro-
cessing has been tested on the satellite images. Comparison of quantitative indi-
cators as well as the visual results shows the advantage of using proposed tech-
nology.
Keywords: remote sensing, panchromatic and multispectral images, resolution,
HSV-transform, packet wavelet transform, Shannon entropy.
1 Introduction
In recent years the systems and methods of optical remote sensing have become the
basic tools of objects state and events control on the Earth surface. For monitoring
natural phenomena consequences and Earth surface state it is needed to use satellites
with high-spatial resolution: Pleiades-1A, Pleiades-1B, TripleSat Constellation
(DMC-3), DubaiSat-2, Jilin-1, WorldView-1,2,3, RapidEye, Cartosat -3 etc. Such
satellites allow to obtain hundreds of images digitally of a target local area. The anal-
ysis of such multichannel data is a very difficult task and comes down to specified
objects emphasizing, obtaining their characteristics, and relative position. The typical
data set of remote sensing apparatus mounted on satellites includes: multispectral
(multichannel) image and panchromatic image (PAN). A panchromatic image usually
has a higher spatial resolution than multispectral one, which substantially complicates
objects recognition and imposes restrictions on the used processing methods. For
information content of primary data improvement, the existing methods of images
processing have a set of disadvantages, the main of which is color distortions [1-4].
The aim of the work is improvement of primary multichannel image spatial resolution
minimizing color distortion. Images taken from WorldView-2 satellite are used as
input data. To determine the effectiveness of the proposed information technology
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons License Attrib-
ution 4.0 International (CC BY 4.0) CMiGIN-2019: International Workshop on Conflict Management in
Global Information Networks.
quantitative quality assessment of synthesized multispectral images will be obtained,
in particular: Shannon entropy, signal entropy etc.
2 State of art
Nowadays there are different methods of obtaining synthesized multispectral im-
ages with spatial resolution increase by merging them with panchromatic images:
Brovey-transform, PC-sharpening, independent component analysis (ICA), Gram-
Schmidt, IHS-transform. But these methods do not take into account constructing
characteristics of modern scanning devices, appropriate structures and high resolution
data formats [1, 3-7]. Chu Heng and Zhu Weile [8] proposed method based on the
transition to color-difference metrics of computer graphics, where the question about
decorellation of primary data is solved. However, these methods allow us to take into
account only spectral components of primary grayscale image. One of the most per-
spective and effective mathematical apparatus for aerospace images analysis is packet
wavelet transform. Its appliance allows to get photogrammetric scanner images which
are obtained by traditional methods, and the methods that use discrete wavelet trans-
form [9-11].
3 Methods and materials
This work proposes a fusion method based on packet wavelet bases building with
decorrelation of primary aspectual data. The proposed algorithm scheme is shown in
the fig. 1.
Fig. 1. Technology scheme
The main stages of primary multichannel image processing are:
1. Download multichannel image in RGB color space and image resampling.
Fig. 2 shows fragments of scene panchromatic channel (PAN) and RGB composi-
tion (Bands 5-3-2) from satellite WorldView-2. Image resampling is a process in
which new pixel values are interpolated from existing pixel values, whenever the
raster’s structure is modified during, for example, projection, datum transformation,
or cell resizing. There are many resampling methods available through a number of
platforms, including image-processing software. Bilinear interpolation, nearest neigh-
bor, and cubic convolution are most commonly used resampling methods in remote
sensing. We used bicubic resampling.
a) b)
Fig. 2. Satellite images of WorldView-2: a) panchromatic; b) multispectral
2. Geometric and spectral correction: decompose the appropriate RGB and PAN
image luminance channel to the sixth decomposition level (L) by the packet wavelet
transform of the bior 6.8 class according to the logarithmic information value func-
tion. Calculate Shannon entropy in its extended definition; choose the maximum val-
ue between the two ones and get the optimal wavelet tree based on the RGB image;
inverse packet wavelet decomposition [9, 12, 13].
3. Decorrelation in HSV:
f RGB (r ) f HSV (r ),
. (1)
f PAN ( p ) f HSV ( p )
4. Optimal packet wavelet base decomposition of the PAN until the specified de-
composition L built at the previous stage [9, 12]:
L
f P (r) = Tc PL (r) + [Td Pl,1(r),Td Pl,2 (r),Td Xl,3 (r)] . (2)
l=1
5. New components formation according to the specified rule of coefficients merg-
ing [9]:
App XL (r) = Tc XL (r),
L . (3)
Det X (r) = [Td Pl,1(r),Td Pl,2(r),Td Pl,3(r)]
l =1
6. Reverse wavelet packet decomposition and transition to HSV color metrics:
f XYZ (r) = App XL (r) + Det X (r),
. (4)
f XYZ (r) f HS V (r)
7. In the reverse transform from the HSV color space in the RGB-space, choose H
and S components of multichannel component images and the resulting V after wave-
let-transformation of panchromatic image and getting the Fusion result.
For displaying the research results of the information characteristics (IC) of differ-
ent wavelet bases and merger methods, there are the next items used: conical coordi-
nates system, the base radius of which is equal to the maximum function value all the
way through the arguments set. The lateral surface is divided into sectors and subsec-
tors depending on the problem. Within the sector or subsector framework the results
are represented as colorful markers in which the color matches the wavelet decompo-
sition level. The marker position matches the final value of radius-vector, the begin-
ning of which is in the inner radius of the circle – conditional zero. The diagrams
show the values of the conditional zero and the maximum value among the absolute
values (I – II quarters), and the values which determine the reserve of dynamic infor-
mation criteria range – D (reflected in the III – IV quarters) and are defined in accord-
ance with the following expression:
C Min
D = 20log 10 , db , (5)
Max
where Min, Max – appropriate minimum and maximum absolute values ІC, С – cur-
rent ІC value.
For providing the comparison analysis of the mathematical models, the minimal
geometric size of the primary data is necessary. It is established that primary data
geometric size is the most influencing one for definition of the models built on the
wavelet packet transform base, i.e. for building optimal wavelet trees with a specified
information value function (IVF) and wavelet filter. Impact of the specified factors
consists in obtaining (or not obtaining) the optimal packet wavelet structure – Epw.
The cases when optimal wavelet tree is not obtained are: transition of packet tree
structure to normal wavelet; getting a full packet wavelet tree.
For establishing the fact that the optimal packet wavelet tree was obtained, the next
criteria is used:
n n0
E pw = 1 , (6)
N n0
where n is the total number of nodes of the obtained wavelet packet tree, N is the total
number of nodes of the full wavelet tree, n0 is the total number of nodes of the normal
wavelet structure.
The specified criterion acquires its maximum value (1) when getting a full packet
wavelet tree, and minimal (0) – in case of normal wavelet structure. The fact of ob-
taining the optimal wavelet tree (Epw) is determined by indicator (6), which does not
have to take specified limit values. As further each class of the wavelet filters is sub-
mitted by its two members, within the task of determining the minimum geometric
primary data size, the wavelet filter will be considered with the highest order within
the class.
The results of solving this problem are shown in fig. 3, where the results are within
the sector regarding to the representative of each wavelet decomposition class, name-
ly the Daubechies filters of the 12 order (db12), Symlet filters of the 12 order
(sym12), Koyflet filters of the 5 order (coif 5), biorthogonal filters of the 9/11 order
(bior 6.8); the result is within each subsector. The following RGB are used: Shannon,
norm, log entropies, entropy [13] for primary images with geometric size within range
of from 350х350 to 800х800 with 50х50 step. Inside the brackets, next to the sizes of
the primary data, there are maximum decomposition level which is typical for the
specific geometric size and type of the wavelet filter; exactly with such decomposi-
tion level the problem solving is provided. As most results are specified in the range
of [0…0.9] D, the fig.3 shows its scale increase.
Fig. 3. Fact of getting the optimal wavelet packet tree dependence on the primary image
size, the wavelet order, and class
According to the results, geometric sizes of primary images are equal to 650х650
pixels, for such size it is possible to get Epw structure, according to the specified
wavelet filters and IVF. It is a typical behavior for the provided comparative analysis
with the specified criteria (6) with IVF. The IVF is defined as the logarithm, because
regardless of the size of the original image and wavelet filters types, it takes its max-
imum value which matches the case of getting a full wavelet packet tree.
When analyzing the obtained IC "signal entropy" results: there are a greater differ-
ence between the second and the third decomposition levels than between quality
scores of the previous criteria, and more precise definition of the global minimum
observed.
While analyzing the results got by ІC "conditional signal entropy" in relation to the
primary RGB D is not very different from the conditional Shannon entropy criteria
(fig.4).
Fig. 4. Quality assessment results of the images obtained by the fusion method with packet
wavelet decomposition (ІC – conditional signal entropy)
When analyzing the results by ІC “Shannon entropy”, the maximum quality score
is calculated with wavelet filter bior 2.4; the wavelet decomposition involvement is
ineffective at the first decomposition level. Moreover, the least appropriate quality
score is got when involving the wavelet decomposition based on wavelet filter db4.
While analysing the results got by ІХ "conditional Shannon entropy" in relation to
the primary MSI: the maximum of the score obtained by wavelet filter bior 2.2; rapid
growth of the informative value until the second decomposition level by wavelet filter
bior db4, and until the third decomposition level by wavelet filter bior 2.2 prove the
statement regarding to the informative value of the Shannon entropy about inefficien-
cy of the first wavelet decomposition appliance inefficiency.
While analysing the results got by ІХ "standard deviation": the quality maximum
score calculated with wavelet filter db4; rapid growth of the informative value until
the third decomposition with further less rapid growth; considerable difference be-
tween informativeness score and fusion methods obtained by the first, second, and
third decomposition levels.
While analysing the results got by ІХ "integrated informativeness by Shannon" and
"signal integrated informativeness" in relation to the primary MSI: ІХ Shannon En-
tropy dynamics are decreasing depending on the level of wavelet decomposition – the
maximum function decline is observed on the second decomposition level with fur-
ther minor current ІХ increase; dynamics of the ІХ Signal Ventropy is also decreasing
depending on the wavelet decomposition level.
For packet wavelet transforms, the best indicators by ІC and computational com-
plexity are obtained for the case of missing the stages of wavelet trees structures op-
timization by the chosen IVF, what reduces the condition until minimal geometric
primary aspectual data sizes, i.e. the geometric sizes are limited only by capacity of
the sets, which define the low pass and high pass filters, and by the necessary wavelet
decomposition level.
4 Results
Fig. 5 shows the synthesized image after working new technology based on bicubic
resampling, HSV-, packet wavelet- transforms. Quality analysis of the fig.5, which
was obtained by the fusion method when using the geometric indicator, has shown
absence of any affine distortions. It proves linearity of the proposed mathematical
models. Visual "quality" image can be evaluated according to the criteria of maxi-
mum information content characteristics. Entropy is used to measure the amount of
information [13].
Fig. 6 shows a graphical comparison of the absolute values and dynamical ІC rang-
es in relation to the images, which were got by the fusion procedure based on packet
wavelet transforms. The results show the effectiveness of the proposed technology.
The correlation coefficient (CORR) is an important indicator reflecting the differ-
ence between the fused image and the original image [13]. Table 1 shows the values
CORR for the Gram-Schmidt, wavelet, packet wavelet, HSV, PCA, and new technol-
ogy image fusion methods.
Table 2 shows value of SSIM of the pansharpenined results. The structural similar-
ity image quality paradigm is based on the assumption that the human visual system is
highly adapted for extracting structural information from the scene.
Fig. 5. The resulting image on new technology
Fig. 6. Graphical representation of results of fusion based on packet wavelet transforms and
proposed technology
Table 1. Correlation value CORR
Methods R G B
PCA 0.75 0.79 0.87
Gram-Schmidt 0.86 0.85 0.84
HSV 0.48 0.57 0.54
Wavelet 0.72 0.82 0.71
Packet wavelet 0.78 0.85 0.79
New technology 0.96 0.95 0.97
The value measure of the structural similarity is calculated as follows [14]:
2 XY
2
SSIM XY
2
X Y
2
, (7)
X Y X Y X Y
2 2
1 M N 1 M N
X ij
MN i 1 j 1
x , Y yij ,
MN i 1 j 1
(8)
x X ,
M N
1
M 1 N 1
2
X2 ij (9)
i 1 j 1
y Y ,
M N
1
M 1 N 1
2
Y2 ij (10)
i 1 j 1
x X y Y ,
M N
1
M 1 N 1
2
XY ij ij (11)
i 1 j 1
where SSIM – structural similarity (quality) index; X xij , Y yij – Images
are compared; M, N - the size of the image; xy – covariance between x and y , and
x2 and y2 - standard deviation [14].
Table 2. SSIM of the pansharpening results.
Methods R G B
PCA 0.55 0.47 0.47
Gram-Schmidt 0.42 0.43 0.44
HSV 0.58 0.67 0.65
Wavelet 0.45 0.56 0.49
Packet wavelet 0.66 0.67 0.69
New technology 0.71 0.79 0.77
Can see that these methods may enhance the detail of the image but result in much
loss of spectral information. These results point out one of the main advantages of our
technology: original spectral information is maintained, while image detail is en-
hanced.
So, much less extremes in dynamics of quality indicators (table 2) shows more sta-
bility of the proposed technology and monotonously increasing Shannon, signal en-
tropies, conditional Shannon, and conditional signal entropies dependence on the
level of the wavelet decomposition. The analysis of the obtained results by the «Peak
Signal-to-Noise Ratio», concerning the primary MSI (fig. 7), helps determine the fact
that visual quality is lower when using the existing methods but not the suggested
technology. The technology influences the quality of objects recognition and increas-
es the quality of primary satellite images by 10–12%.
Fig. 7. Results of the obtained images by the roposed technology and existing methods quality
assessment
5 Conclusions
In this paper we propose computer technology of high resolution satellite image
processing based on packet wavelet transform. Most of the fusion techniques that
have been proposed are based on the spectral consistency. In this paper, a computer
technology based on HSV- and packet wavelet transform has been adopted for high
resolution satellite multichannel image fusion without spectral distortions in local
areas. Compared with the already existing fusion methods the proposed technology
helps avoid substantial color distortions and improve accuracy of the further objects
recognition in pictures. It is obtained, particularly, by the previous correction of the
primary images and data processing in localized spectral bases which is optimized by
information characteristics. The qualitative experimental results, based on different
testing data sets, show that the proposed technology to reduce the time of correspond-
ing processing of data without loss of accuracy.
Our future research will focus on perfection of the proposed technology of im-
proving multi-channel data informativeness, taking into account different types of
wavelet packet characteristics and selecting the optimal decomposition.
6 References
1. Pohl, C., Van Genderen, J.: Multisensor image fusion in remote sensing: concepts, meth-
ods and applications. International journal of remote sensing, nol. 19, No. 5, pp. 823-854
(1998).
2. Hnatushenko, V., Hnatushenko, Vik., Kavats, O., Shevchenko, V.: Pansharpening technol-
ogy of high resolution multispectral and panchromatic satellite images. Scientific Bulletin
of National Mining University, Issue 4, 91-98 (2015).
3. Manu, C.S., Jiji, C.V.: A novel remote sensing image fusion algorithm using ICA bases.
Advances in Pattern Recognition (ICAPR), Eighth International Conference. P. 244-249
(2015).
4. Shedlovska, Y., Hnatushenko, V., Kahtan, V.: Satellite imagery features for the image
similarity estimation. International Young Scientists Forum on Applied Physics 2017, p.
359-362. Lviv (2017).
5. Nikolakopoulos, K.G.: Comparison of Nine Fusion Techniques for Very High Resolution
Optical Data. Photogrammetric Engineering & Remote Sensing. Vol. 74. No 5. P. 647-659
(2008).
6. Hnatushenko, V., Kavats, O., Kibukevych, I.: Efficiency determination of scanner data fu-
sion methods of space multispectral images. International Young Scientists Forum on Ap-
plied Physics «YSF-2015», Dnipropetrovsk (2015).
7. Chen, F., Qin, F., Peng, G., Chen S.: Fusion of remote sensing images using improved ICA
mergers based on wavelet decomposition, , pp. 2938-2943, Procedia Eng. (2012).
8. Heng, C., Weile, Z.: Fusion of IKONOS Satellite Imagery Using IHS Transform and Local
Variation, IEEE Transactions on Geoscience and remote sensing, vol. 5, no. 4, pp. 653 –
657 (2008).
9. Hnatushenko, V., Vasyliev, V.: Remote sensing image fusion using ICA and optimized
wavelet transform. Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLI-B7,
pp. 653-659 (2016).
10. Cheng, J., Liu, H., Liu, T., Wang, F., Li, H.: Remote sensing image fusion via wavelet
transform and sparse representation. Elsevier. Elsevier-ISPRS Journal of Photogrammetry
and Remote Sensing. Vol. 104. P. 158–73 (2015).
11. Kahtan, V., Hnatushenko, V., Shedlovska, Y.: Processing technology of multispectral re-
mote sensing images. International Young Scientists Forum on Applied Physics. 2017,
p. 355-358. Lviv (2017).
12. Mallat, S.: Theory for Multi-resolution signal decomposition: the wavelet representation.
IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 11, no. 7, pp. 674-693
(1987).
13. Nikolakopoulos, Konstantinos, Oikonomidis, Dimitrios: Quality assessment of ten fusion
techniques applied on Worldview-2, European Journal of Remote Sensing, 48:1, 141-167
(2015).
14. Wang, Z., Bovik ,A. C., Sheik, H.R., Simoncelli, E.P. Image quality assessment: From er-
ror visibility to structural similarity. IEEE Trans. Image Processing 13 (4), pp. 600–612
(2004).