=Paper=
{{Paper
|id=Vol-3271/Paper15_CVCS2022
|storemode=property
|title=A New Optimization Model for the Restoration of the Deteriorated Hyperspectral Images
|pdfUrl=https://ceur-ws.org/Vol-3271/Paper15_CVCS2022.pdf
|volume=Vol-3271
|authors=Shanthini K. S.,Sudhish N. George,Sony George
|dblpUrl=https://dblp.org/rec/conf/cvcs/ShanthiniGG22
}}
==A New Optimization Model for the Restoration of the Deteriorated Hyperspectral Images==
https://ceur-ws.org/Vol-3271/Paper15_CVCS2022.pdf
A New Optimization Model for the Restoration of the
Deteriorated Hyperspectral Images
Shanthini K.S.1,* , Sudhish N. George2 and Sony George3
1
Department of Electronics and Communication Engineering,
National Institute of Technology Calicut, Kerala, India
2
Department of Computer Science, Norwegian University of Science and Technology, Gjovik,Norway
Abstract
Hyperspectral imaging technology has great role in performing computer vision tasks efficiently. How-
ever the acquired hyperspectral images (HSIs) are contaminated by different types of noises and other
unwanted signals. This paper proposes a new tensor svd based low rank decomposition together with
spatial spectral total variation (SSTV) regularization for removing the noise artefacts in HSIs. The
proposed optimization model uses tensor decomposition to express the correlation among the different
frequency bands. The sparse noise is detected using a ๐1 norm, and in addition, a Frobenius norm is
added to remove heavy Gaussian noise from the images. A SSTV norm is added to preserve the piecewise
smoothness structure in the spatial and spectral domains. An efficient solution for the optimization
problem is developed based on the alternating direction method of multipliers (ADMM). From the exper-
iments conducted on noisy HSIs, it can be observed that our method achieves better results compared to
the already existing ones.
Keywords
Hyperspectral image (HSI), low-rank tensor decomposition, denoising, spatial spectral total variation
(SSTV)
1. Introduction
Hyperspectral imaging which combines spectroscopy and digital imaging has become one of the
powerful tools for solving many computer vision tasks such as quality inspection and sorting
efficiently. Hyperspectral imaging has been widely used in remote sensing applications by
virtue of modern sensor technologies which can cover large surfaces of earth. In recent years,
ground-based hyperspectral imaging has emerged as a powerful tool for quality and safety
evaluation of food items, forensic science, medical surgery and diagnosis, military applications,
and restoration of artworks[1]. HSI is more effective in detecting the external or internal quality
and chemical compositions of horticultural products [2] and therefore there is a tremendous
increase in the research on hyperspectral imaging used for quality and safety checking of fruits
and vegetables. A hyperspectral imaging system obtains a two dimensional (2D) matrix for
each wavelength ranging from VIS (visible) to NIR (Near Infrared). The resulting structure is
a three dimensional (3D) image dataset, which is called a hypercube. The data acquisition of
The 11th Colour and Visual Computing Symposium, September 08โ09, 2022, Gjovik, Norway
*
Corresponding author.
$ shanthini.ks@gmail.com (S. K.S.); sudhish@nitc.ac.in (S. N. George); sony.george@ntnu.no (S. George)
ยฉ 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR
Workshop
Proceedings
http://ceur-ws.org
ISSN 1613-0073
CEUR Workshop Proceedings (CEUR-WS.org)
hyperspectral images can be done in 4 modes- line scanning, area scanning, point scanning,
and single shot [2]. In a pushbroom based hyperspectral system, the sensor captures spectral
information at each line, and by moving the camera or the object, the system captures the entire
spatial region[2]. The major components of a pushbroom hyperspectral imaging system are:
illumination, camera, objective lens, transportation plate, and a data processing unit, as shown
in Fig.1 [3]. The spectral range produced by the system depends on illumination conditions,
camera, and the typical range of a VNIR system is 400 to 1000 nm. Even if HSI is a significant
Hyperspectral
camera
VNIR
PC
Illuminant Illuminant
Fruit Spectralon tile
Moving platform Translator stage
Motor
Figure 1: Hyperspectral imaging system used for the present study
non-invasive quality assessment technique, real HSIs always suffer from various degradations.
The anomalous observations can be due to several reasons, such as the dysfunction and noise
in the sensor or from different stages in the workflow, shape and geometry of the scene, and
the radiation technique [4]. In this paper, we try to develop a new model for removing specific
noise artefacts affecting the HS images.
The pushbroom sensor used for image acquisition produces stripes, and random noise [5].
The random noise can be viewed as an adaptive white Gaussian(AWGN) random process with
variance ๐ 2 and mean zero. Dead detectors will give rise to single line drop-out for pushbroom
sensors [6]. The neighbouring elements in the CCD(Charge Coupled Device) array may have
sensitivity variations, which causes vertical stripe noise [7]. HSIs also suffer from impulse
noise [8]. These noises can affect the subsequent internal and external quality evaluation and
defect detection processes. Improving the HSI quality merely through a hardware scheme is
unsustainable and impractical. Therefore, suitable image pre-processing techniques are to be
applied to obtain a high-quality HSI before subsequent applications.
HS imaging was actually developed for remote sensing applications, and so far, a huge number
of proposals have been made in the HSI denoising. These techniques can be broadly classified
into five categories [9]. In the first category, each band in the HSI is treated as an image, and
then commonly used two dimensional (2D) denoising methods are applied in each band to
remove noise [10],[11]. But the results are not promising since the band-wise processing does
not consider the strong correlations between bands. Another method is to treat the HSI as a
multidimensional data cube, and the volumetric data denoising methods can accomplish the
denoising [12],[13]. However, the results are unsatisfactory because these methods do not
consider the correlation among the bands. Therefore, it is necessary to consider the spatial and
spectral information simultaneously to enhance the denoising quality. The third type of HSI
denoising method combines spatial and spectral information, and several algorithms have been
proposed based on this concept.e.g. [14], [15]. The results obtained from this fusion method
are promising compared with single view(either spatial or spectral) methods. A few transform
domain-based techniques were also proposed in this category [16], [17].
The fourth type of HSI denoising method falls under the category of low rank based techniques.
In [18], a randomized singular value decomposition is introduced and a noise adjusted iterative
framework is proposed for low rank matrix approximation (NAILRMA). A SSTV based low
Tucker rank model (LRTDTV) was proposed to preserve the spectral signatures of HSI in [19].
The method proposed by Chen et al.[20] is based on weighted group sparsity regularization,
and this concept is incorporated for low rank tucker decomposition (LRTDGS), in which results
were shown to be improved in comparison with the previous TV based methods. Zheng et al.
[21] proposed the fibered rank and suggested the convex (3DTNN) and non-convex models
(3DLogTNN) to separate clean HSI from noisy images.
Recently, many deep learning based algorithms have been proposed, and their performance
is encouraging[22], [23]. But their results are highly dependent on the quality and volume of
training data which limits their application.
Restoration of HS images for remote sensing applications has been well studied in the litera-
ture. But the researchers have less focused on restoring the images for close range applications.
Moreover, the algorithms used for remote sensing applications may not be suitable for HSI
restoration of data captured at proximity range. This is evident from Fig.2 wherein one of
the recently proposed remote sensing HSI denoising techniques fails to restore the original
fruit image. Hence we propose and test an algorithm specifically for restoring deteriorated
hyperspectral images for close range applications.
(a) (b)
Figure 2: Performance of denoising method 3DLogTNN(Band 129) [21] : (a)Noisy image, (b)Denoised image
The low rank based HSI restoration techniques can be broadly classified as matrix based and
tensor based approaches. Even though several methods have been proposed based on low rank
matrix modelling for removing mixed noise, these methods failed to utilise the correlation in
spatial and spectral modes effectively, leading to suboptimal denoising results under severe noisy
conditions. So many recent research works included the direct tensor modelling techniques, and
it has been found that these techniques are superior to matrix based techniques in computing
higher order data. Such studies motivated us to propose a new optimisation model for restoring
noise-contaminated hyperspectral images using direct tensor modelling techniques exploiting
the spatial and spectral information.
Our Contributions:
. As per our knowledge, this is the first effort to realise a tensor based low rank(LR) and
SSTV model for the removal of noise artefacts from the HSI data captured at proximity
range.
. Further, ๐1 norm and Frobenius norms are used to address the sparse noise and heavy
Gaussian noise in the images.
. The formulated optimisation problem is decomposed into several subproblems and solved
using the Alternating Direction Method of Multipliers (ADMM).
. The algorithm is implemented successfully on the fruit HSI data with simulated noise.
The rest of the paper is organised as follows. Section 2 explains the tensor based low rank and
SSTV model and the optimisation solution in detail. The experimental results are presented in
section 3. Finally, the conclusions are derived in section 4.
2. Proposed Model
A noisy HSI cube โ may be represented as,
โ=๐ฅ +โฐ (1)
where ๐ฅ represents the clean HS image and โฐ represents noise artefacts. The aim of HSI
restoration task is to obtain the noise-free HS image ๐ฅ from the noise-contaminated image โ.
The noise term includes Gaussian noise ๐ฒ and the sparse noise ๐ฎ. Dead pixels, stripes and
impulse noise fall into the category of sparse noise. Based on this, the degraded HSI can be
modelled as,
โ=๐ฅ +๐ฎ +๐ฒ (2)
Using the TRPCA (Tensor Robust Principal Component Analysis) model [24], the image can
be expressed as a combination of a rank function which describes the low rank property and a
โ0 norm to represent the sparse noise.
argmin rank(๐ฅ ) + ๐โ๐ฎโ0
๐ฅ
๐ .๐ก. โ=๐ฅ +๐ฎ (3)
The optimization problem Eq.(3) is a nonconvex optimization problem. So we use the convex
surrogates, tensor nuclear norm โ.โโ and โ1 norm โ.โ1 to approximate the rank(๐ฅ ) and โ๐ฎโ0 ,
respectively. The reformulated convex optimization problem is,
argmin โ๐ฅ โโ + ๐โ๐ฎโ1 + ๐ฝโ๐ฒโ2F ,
๐ฅ
๐ .๐ก. โ=๐ฅ +๐ฎ +๐ฒ (4)
where โ,๐ฅ ,๐ฎ, and ๐ฒ are 3๐๐ order tensors. The low rank approximation can be obtained using
tensor singular value decomposition algorithm[25]. The objective of using the TV regularization
is to take advantage of the piecewise smoothness property in both the spectral and spatial
domains. The widely used 2D TV regularizer consider only the spatial smoothness structure of
HSI. So we also use a SSTV regularizer to explore the spectral smoothness.
โ๏ธ
โ๐ฅ โSSTV := w1 |pi,j,k โ pi,j,k-1 | + w2 |pi,j,k โ pi,j-1,k |
i,j,k (5)
+ w3 |pi,j,k โ pi-1,j,k |
where ๐๐,๐,๐ , is the (i, j, k)๐กโ entry of ๐ฅ . The term ๐ค๐ (l = 1, 2, 3) is the weight along the l๐กโ
mode of ๐ฅ . These weights decide the strength of regularization. When the SSTV regularization
is also added to the rank-constrained RPCA, the model will become,
argmin โ๐ฅ โโ + ๐โ๐ฎโ1 + ๐ โ๐ฅ โSSTV ,
๐ฅ
๐ .๐ก. โ=๐ฅ +๐ฎ +๐ฒ (6)
By adding some auxiliary variables Eq.(6) can be rewritten as,
argmin โ๐ฅ โโ + ๐โ๐ฎโ1 + ๐ฝโ๐ฒโ2F + ๐ โโฑโ1 ,
๐ฅ
๐ .๐ก. โ = ๐ฅ + ๐ฎ + ๐ฒ, ๐ฅ = ๐ต, ๐ท๐ค (๐ต) = โฑ (7)
where ๐ท๐ค (ยท) is the third order weighted difference operator. It can be given by ๐ท๐ค (ยท) = [ ๐ค๐ ร
๐ท๐ (ยท); ๐ค๐ ร ๐ท๐ (ยท); ๐ค๐ ร ๐ท๐ (ยท)] where ๐ท๐ , ๐ท๐ , and ๐ท๐ represents the first order difference
operators in the different orientations of a 3D HSI. The optimization problem in Eq.(7) can be
rewritten using the augmented Lagrangian multiplier (ALM) method:
argmin โ๐ฅ โโ + ๐โ๐ฎโ1 + ๐ฝโ๐ฒโ2F + ๐ โโฑโ1
๐ฅ
๐
+ โจ๐บ1 , โ โ ๐ฅ โ ๐ฎ โ ๐ฒโฉ + โโ โ ๐ฅ โ ๐ฎ โ ๐ฒโ2F
2 (8)
๐
+ โจ๐บ2 , ๐ฅ โ ๐ตโฉ + โ๐ฅ โ ๐ตโ2F
2
๐
+ โจ๐บ3 , ๐ท๐ค (๐ต) โ โฑ โฉ + โ๐ท๐ค (๐ต) โ โฑ โ2F
2
๐ฅ update
๐ฅ ๐+1 =argmin โ๐ฅ โโ + โจ๐บk1 , โ โ ๐ฅ โ ๐ฎ k โ ๐ฒ k โฉ
๐ฅ
๐
+โโ โ ๐ฅ โ ๐ฎ k โ ๐ฒ k โ2F + โจ๐บ2 , ๐ฅ โ ๐ต k โฉ
2
๐ (9)
+ โ๐ฅ โ ๐ต k โ2F
2
๐
=argmin โ๐ฅ โโ + 2( )โ๐ฅ โ โณโ2F
๐ฅ 2
where
(๐บk1 โ ๐บk2 )
(๏ธ )๏ธ
1 k k k
โณ= โ+๐ต โ๐ฎ โ๐ฒ +
2 ๐
๐ฎ update
๐ฎ ๐+1 =argmin ๐โ๐ฎโ1 + โจ๐บk1 , โ โ ๐ฅ k+1 โ ๐ฎ โ ๐ฒ k โฉ
๐ฎ
๐
+ โโ โ ๐ฅ k+1 โ ๐ฎ โ ๐ฒ k โ2F
2
๐ ๐บ1 2 (10)
=argmin ๐โ๐ฎโ1 + โ๐ฎ โ (โ โ ๐ฅ k+1 โ ๐ฎ โ ๐ฒ k + )โ
๐ฎ 2 ๐ F
๐บk1
(๏ธ )๏ธ
k+1 k
=๐ ๐๐ ๐ก ๐ โ โ ๐ฅ โ๐ฒ +
๐ ๐
๐ฒ update
๐ฒ ๐+1 =argmin ๐ฝโ๐ฒโ2F + โจ๐บ๐1 , โ โ ๐ฅ k+1 โ ๐ฎ k+1 โ ๐ฒโฉ
๐ฒ
๐
+ โโ โ ๐ฅ k+1 โ ๐ฎ k+1 โ ๐ฒโ2F
2
๐(โ โ ๐ฅ k+1 โ ๐ฎ k+1 ) + ๐บk1 โฆ (11)
โฆ โฆ2
๐ โฆ
=argmin (๐ฝ + )โฆ๐ฒ โ
โฆ โฆ
๐ฒ 2 ๐ + 2๐ฝ โฆ
F
๐(โ โ ๐ฅ k+1 โ ๐ฎ k+1 ) + ๐บk1
=
๐ + 2๐ฝ
๐ต update
๐ k+1
๐ต ๐+1 =argmin โจ๐บk2 , ๐ฅ k+1 โ ๐ตโฉ + โ๐ฅ โ ๐ตโ2F
2
๐ต
(12)
๐
+ โจ๐บk3 , ๐ท๐ค (๐ต) โ โฑ k โฉ + โ๐ท๐ค (๐ต) โ ๐น k โ2F
2
The solution to the above equation can be obtained from the following linear equation.
๐
(๐I + ๐๐ท๐ค ๐ท๐ค )๐ต = ๐๐ฅ ๐+1 + ๐๐ท๐ค
๐
(โฑ ๐ ) + ๐บ๐2 โ ๐ท๐ค
๐
(๐บ๐3 )
where ๐ท๐ค ๐ indicates the adjoint operator of ๐ท . We adopt the Fourier Transform to solve
๐ค
Eq.(12).
Hz = ๐๐ฅ k+1 + ๐๐ท๐ค
๐
(โฑ ๐ ) + ๐บ๐2 โ ๐ท๐ค
๐
(๐บ๐3 )
2
Tz = ๐ค๐ |fftn(๐ท๐ )|2 + ๐ค๐2 |fftn(๐ท๐ )|2 + ๐ค๐2 |fftn(๐ท๐ )|2
(๏ธ )๏ธ (13)
k+1 fftn(๐ป๐ง )
๐ต = ifftn
๐I + ๐๐๐ง
โฑ update
โฑ ๐+1 =argmin ๐ โโฑโ1 + โจ๐บk3 , ๐ท๐ค (๐ต k+1 ) โ โฑ โฉ
โฑ
๐
+ โ๐ท๐ค (๐ต k+1 ) โ โฑ โ2F
2
๐ ๐บk (14)
=argmin ๐ โโฑโ1 + โโฑ โ (๐ท๐ค (๐ต k+1 ) + 3 )โ2F
โฑ 2 ๐
๐บ๐3
=๐ ๐๐ ๐ก ๐๐ (๐ท๐ค (๐ต ๐+1 ) + )
๐
The updates of the multipliers are given by
๐บk+1 k
1 = ๐บ1 + ๐(โ โ ๐ฅ
k+1
โ ๐ฎ k+1 โ ๐ฒ k+1 )
๐บk+1 k
2 = ๐บ2 + ๐(๐ฅ
k+1
โ ๐ต k+1 ) (15)
๐บk+1 k
3 = ๐บ3 + ๐(๐ท(๐ต
k+1
) โ โฑ k+1 )
Algorithm 1: HS image denoising
1 Input: Observed noisy HSI โ, convergence criteria ๐, the parameters of regularisation ๐, ๐, ๐ฝ and the
weights [ ๐ค๐ , ๐ค๐ , ๐ค๐ ]
2 Output: Denoised HSI ๐ฅ
3 Initialize: ๐ฅ = ๐ฎ = ๐ฒ = ๐ต = 0,๐บ1 , ๐บ2 , ๐บ3 = 0,
4 ๐๐๐๐ฅ = 10 , ๐ = 1.5 and ๐ = 0
6
5 while not converged do
6 update ๐ฅ , ๐ฎ, ๐ฒ, ๐ต, โฑ via Eqns. (9),(10),(11),(12),(13) and (14)
7 update ๐บ1 , ๐บ2 , ๐บ3 via Eq. (15)
8 update parameter ๐ :=min(๐๐, ๐๐๐๐ฅ )
9 check for the convergence criteria
โ๐ฅk โ๐ฅk+1 โ2
10
โโโ2
๐น
โค๐
๐น
11 end
3. EXPERIMENTAL RESULTS AND DISCUSSION
The proposed method is compared with five other best-performing denoising techniques to
show the effectiveness in HSI restoration. These methods include LLRGTV [26], 3DTNN[21],
3DLogTNN [21], NAILRMA [18], and LRTDTV [19]. The experiments for all the compared
methods are conducted using the same parameters given by the authors in the respective papers
to ensure optimum performance. An Intel Xeon CPU@ 3.50 GHz with 64-GB RAM is used to
conduct the experiments in MATLAB R2022a
3.1. Dataset and Experiments:
Hyperspectral images with different spectral ranges can be used to detect the quality attributes
and defects of horticultural products[27]. The dataset for this study is obtained using a camera
HySpex-VNIR-1800 which has a spectral sensitivity from 400 nm to 1000 nm and spectral
sampling of 3.18 nm [3]. This VNIR push broom scanner records 186 spectral bands and 1800
pixels across the field of view. It uses a polariser to avoid specular reflection. In the present case
study, we used the fruits cherry and strawberry. Since the size of the dataset is computationally
intensive, it is cropped to 500ร500 pixels for cherry and 600ร700 pixels for strawberry. Real
HSIs are affected by a variety of noises, such as Gaussian noise and sparse noise in different
amounts. In order to simulate real noise conditions, we consider different combinations of these
noises in varying proportions. Thus we can define the following noise cases as follows.
Noise case 1: Gaussian noise + Impulse noise: Gaussian noise and salt and pepper noise with
equal distribution are added to each and every band. The variance of the Gaussian noise is set
as 0.2 with zero mean, and the percentage of the impulse noise added is 0.2.
Noise case 2: Gaussian noise + Impulse noise + Dead lines: Here, the variance of Gaussian
noise and percentage of impulse noise were fixed as 0.15. Besides this, we add dead lines whose
width varies randomly between 1 and 3. The dead lines are added to bands ranging from 101 to
140, and the number of lines randomly varies between 3 and 10.
Noise case 3: Gaussian noise + Impulse noise + Dead lines + Stripes: Here, in addition to the
noises as in case 2, we add some stripes also to bands ranging from 91 to 130. The number of
stripes randomly varies from 30 to 40. The variance of the Gaussian noise was 0.05 with zero
mean, and the percentage of the impulse noise was 0.05.
The visual quality and quantitative metrics for the above noise cases are tested for all methods
used for comparison.
Visual quality comparison: The denoising results for case 1 and case 3, of band 120 are shown
in Figs. 3,4,5 and 6. From Figs. 3(b),4(b),5(b) and 6(b) it can be observed that the original HSIs
are severely affected by a combination of various noises. In all the noise cases, it is found
that the proposed method gives the best performance. This demonstrates that the low rank
minimisation, together with the TV regularisation, perform well for HSI restoration.
Quantitative comparison: For quantitative evaluation, we use the metrics such as the mean peak
signal-to-noise ratio (MPSNR) and the mean structural similarity index (MSSIM). The higher
the values for the PSNR and SSIM, the better the restoration results.
๐
1 โ๏ธ
MPSNR = ๐๐ ๐๐๐
๐
๐=0
๐
1 โ๏ธ
MSSIM = ๐ ๐ ๐๐๐
๐
๐=0
where ๐๐ ๐๐๐ and ๐ ๐ ๐๐๐ are the PSNR and SSIM values for the ๐ th band, respectively. Table 1
and Table 2 show the restoration results of cherry and strawberry respectively for all methods
considered for comparison. The psnr and ssim values obtained for the different noise cases
clearly shows that the proposed method gives better restoration results than the other methods.
Table 1
Quantitative evaluation of different methods for cherry
Noise Eval.
LLRGTV 3DTNN 3DLogTNN NAILRMA LRTDTV Proposed
case index
Case1 PSNR 20.85 14.81 11.45 17.55 16.75 20.97
SSIM 0.90 0.25 0.11 0.547 0.58 0.86
Case2 PSNR 20.97 16.91 16.53 20.21 18.17 21.04
SSIM 0.89 0.40 0.41 0.60 0.63 0.93
Case3 PSNR 21.01 16.56 17.04 20.56 20.60 21.10
SSIM 0.94 0.39 0.42 0.79 0.81 0.96
Table 2
Quantitative evaluation of different methods for strawberry
Noise Eval.
LLRGTV 3DTNN 3DLogTNN NAILRMA LRTDTV Proposed
case index
Case1 PSNR 32.29 19.69 13.22 18.18 18.33 32.66
SSIM 0.79 0.21 0.12 0.48 0.51 0.75
Case2 PSNR 33.82 27.15 27.33 20.61 20.82 34.18
SSIM 0.79 0.59 0.68 0.53 0.56 0.80
Case3 PSNR 38.19 27.10 27.26 29.62 30.91 38.93
SSIM 0. 91 0.80 0.81 0.70 0.70 0.94
Table 3
Running time comparison for cherry(in seconds)
LLRGTV 3DTNN 3DLogTNN NAILRMA LRTDTV Proposed
622.01 551.2 659.9 163.48 266.25 300.59
Table 4
Running time comparison for strawberry(in seconds)
LLRGTV 3DTNN 3DLogTNN NAILRMA LRTDTV Proposed
1000.10 869.5 1071.99 252.86 390.72 527.45
3.2. Discussion:
Parameter setting: There are several parameters in the algorithm that need to be carefully
adjusted in order to get optimum restoration results. The sparsity regularization parameter ๐
can be selected as ๐ = 100x โ๐ถ๐๐ , where a is the height and b is the width of a single HSI band,
and C is a parameter for tuning. In all the simulated data experiments, the value of C is fixed as
10. ๐ฝ is the Frobenius norm regularization parameter, which is selected as the inverse of the
variance of Gaussian noise. The SSTV regularization parameter ๐ can be fixed as a constant
equal to 1. The weights of the SSTV are chosen as 1 along the spatial mode and a value in the
range of 0 to 1 in the spectral mode. The typical value used in the simulation experiment is
[1,1,0.8].
Computational speed: The running time of different models are compared on the simulated
(a) (b) (c) (d)
(e) (f) (g) (h)
Figure 3: Restoration results for all the compared methods for noise case 1 for cherry: (a) Original image, (b) Noisy image,
(c)LLRGTV (d) 3DTNN (e) 3DLogTNN, (f) NAILRMA,(g) LRTDTV,(h) Proposed
(a) (b) (c) (d)
(e) (f) (g) (h)
Figure 4: Restoration results for all the compared methods for noise case 1 for strawberry : (a) Original image, (b) Noisy image,
(c)LLRGTV (d) 3DTNN (e) 3DLogTNN, (f) NAILRMA,(g) LRTDTV,(h) Proposed
data . Table 3 and Table 4 show the details for different methods. It can be observed that the
NAILRMA method has the lowest computation time among all the compared methods. But, the
quantitative and visual comparison results of this method are not good. The converging time
for LLRGTV, 3DTNN and 3DLogTNN methods are much higher than the other methods since
they require running an expensive iterative algorithm. The computation time of the proposed
method is relatively higher than some other methods such as LRTDTV and NAILRMA but
significantly lower than LLRGTV, 3DTNN and 3DLogTNN.
(a) (b) (c) (d)
(e) (f) (g) (h)
Figure 5: Restoration results for all the compared methods for noise case 3 for cherry: (a) Original image, (b) Noisy image,
(c)LLRGTV (d) 3DTNN (e) 3DLogTNN, (f) NAILRMA,(g) LRTDTV,(h) Proposed
(a) (b) (c) (d)
(e) (f) (g) (h)
Figure 6: Restoration results for all the compared methods for noise case 3 for strawberry : (a) Original image, (b) Noisy image,
(c)LLRGTV (d) 3DTNN (e) 3DLogTNN, (f) NAILRMA,(g) LRTDTV,(h) Proposed
4. Conclusion
A low rank tensor based model is proposed for removing noise artefacts in the line scan-based
hyperspectral images. Additionally an SSTV regularization is used, which preserves the spatial
smoothness and spectral correlation. We have used ๐1 norm to detect the sparse noise effectively,
and in order to tackle the heavy Gaussian noise conditions which may occur in real situations,
we have also considered a Frobenius norm. A new algorithm based on the ALM method is
designed to solve the resulting nonconvex optimization model. The algorithm is implemented
and tested successfully on the fruit HSI data with simulated noise. The experiments show that
the proposed method performs better than some popular methods used for comparison in terms
of quantitative evaluation and visual comparison.
References
[1] M. J. Khan, H. S. Khan, A. Yousaf, K. Khurshid, A. Abbas, Modern trends in hyperspectral
image analysis: A review, IEEE Access 6 (2018) 14118โ14129.
[2] Y. Lu, Y. Huang, R. Lu, Innovative hyperspectral imaging-based techniques for quality
evaluation of fruits and vegetables: A review, in: Appl. Sci., volume 7, 2017.
[3] B. Devassy, S. George, Estimation of strawberry firmness using hyperspectral imaging: a
comparison of regression models, Journal of Spectral Imaging 10 (2021) a3.
[4] M. Vidal, J. M. Amigo, Pre-processing of hyperspectral images. essential steps before
image analysis, Chemometrics and Intelligent Laboratory Systems 117 (2012) 138โ148.
[5] W. Kang, S. Yu, D. Seo, J. Jeong, J. Paik, Push-broom-type very high-resolution satellite
sensor data correction using combined wavelet-fourier and multiscale non-local means
filtering, Sensors 15 (2015) 22826โ22853.
[6] M. Bouali, S. Ladjal, Spectral Inpainting for the Restoration of Missing Data from Multi-
spectral Satellite Sensors: Case study on Aqua MODIS Band 6, 2011.
[7] L. Gรณmez-Chova, L. Alonso, L. Guanter, G. Camps-Valls, J. Calpe, J. Moreno, Correction of
systematic spatial noise in push-broom hyperspectral sensors: application to chris/proba
images, Appl. Opt. 47 (2008) F46โF60.
[8] S. Wang, Z. Zhu, R. Zhao, B. Zhang., Hyperspectral image denoising via nonconvex
logarithmic penalty, Mathematical Problems in Engineering (2021).
[9] F. Yang, X. Chen, L. Chai, Hyperspectral image destriping and denoising using stripe
and spectral low-rank matrix recovery and global spatial-spectral total variation, Remote
Sensing 13 (2021).
[10] A. Buades, B. Coll, J. Morel, A non-local algorithm for image denoising, in: 2005 IEEE
Computer Society Conference on Computer Vision and Pattern Recognition (CVPRโ05),
volume 2, 2005, pp. 60โ65 vol. 2.
[11] K. Dabov, A. Foi, V. Katkovnik, K. Egiazarian, Image denoising by sparse 3-d transform-
domain collaborative filtering, IEEE Tran. on Image Process. 16 (2007) 2080โ2095.
[12] Y. Qian, Y. Shen, M. Ye, Q. Wang, 3-d nonlocal means filter with noise estimation for
hyperspectral imagery denoising, in: 2012 IEEE International Geosci. and Rem. Sens.
Symposium, 2012, pp. 1345โ1348.
[13] M. Maggioni, V. Katkovnik, K. Egiazarian, A. Foi, Nonlocal transform-domain filter for
volumetric data denoising and reconstruction, IEEE Trans. on Image Process. 22 (2013)
119โ133.
[14] Q. Yuan, L. Zhang, H. Shen, Hyperspectral image denoising employing a spectralโspatial
adaptive total variation model, IEEE Trans. on Geoscience and Remote Sensing 50 (2012)
3660โ3677.
[15] X. Huang, B. Du, D. Tao, L. Zhang, Spatial-spectral weighted nuclear norm minimization
for hyperspectral image denoising, Neurocomputing 399 (2020).
[16] G. Chen, S.-E. Qian, Denoising of hyperspectral imagery using principal component
analysis and wavelet shrinkage, IEEE Transactions on Geoscience and Remote Sensing 49
(2011) 973โ980.
[17] B. Rasti, J. R. Sveinsson, M. O. Ulfarsson, J. A. Benediktsson, Hyperspectral image denoising
using 3d wavelets, in: 2012 IEEE International Geoscience and Remote Sensing Symposium,
2012, pp. 1349โ1352.
[18] W. He, H. Zhang, L. Zhang, H. Shen, Hyperspectral image denoising via noise-adjusted
iterative low-rank matrix approximation, IEEE Journal of Selected Topics in Applied Earth
Observations and Remote Sensing 8 (2015) 3050โ3061.
[19] Y. Wang, J. Peng, Q. Zhao, Y. Leung, X.-L. Zhao, , D. Meng., Hyperspectral image restoration
via total variation regularized low-rank tensor decomposition, IEEE Journal of Selected
Topics in Applied Earth Observation and Remote Sensing. 11 (2018) 1227โ1243.
[20] Y. Chen, W. He, N. Yokoya, , T. Huang., Hyperspectral image restoration using weighted
group sparsity-regularized low-rank tensor decomposition, IEEE Trans. on Cybernetics.
50 (2020) 3556โ3570.
[21] Y. Zheng, T. Huang, X.-L. Zhao, T. Jiang, T. Ma, , T. Ji, Mixed noise removal in hyperspectral
image via low-fibered rank regularization, IEEE Trans. Geosci. Remote Sens. 58 (2020)
734โ749.
[22] Y. Gou, B. Li, Z. Liu, S. Yang, X. Peng, Clearer: Multi-scale neural architecture search
for image restoration, in: H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin (Eds.),
Advances in Neural Information Processing Systems, volume 33, Curran Associates, Inc.,
2020, pp. 17129โ17140.
[23] Q. Yuan, Q. Zhang, J. Li, H. Shen, L. Zhang, Hyperspectral image denoising employing a
spatialโspectral deep residual convolutional neural network, IEEE Trans. on Geosci. and
Rem. Sens. 57 (2019) 1205โ1218.
[24] C. Lu, J. Feng, Y. Chen, W. Liu, Z. Lin, S. Yan, Tensor robust principal component analysis
with a new tensor nuclear norm, IEEE Trans. on Pattern Analysis and Machine Intelligence
42 (2020) 925โ938.
[25] M. E. Kilmer, K. Braman, N. Hao, R. C. Hoover., Third-order tensors as operators on
matrices: A theoretical and computational framework with applications in imaging, SIAM
J. Matrix Anal. Appl. 34 (2013) 148โ172.
[26] W. He, H. Zhang, H. Shen, L. Zhang, Hyperspectral image denoising using local low-rank
matrix recovery and global spatialโspectral total variation, IEEE Journal of Selected Topics
in Applied Earth Observations and Remote Sensing 11 (2018) 713โ729.
[27] A. Gowen, C. OโDonnell, P. Cullen, G. Downey, J. Frias., Hyperspectral imaging โ an
emerging process analytical tool for food quality and safety control, Trends in Food
Science & Technology 18 (2007) 590โ598.