This paper introduces an alternative image model for video data that is represented as a series of images afected by shot noise. This type of noise not only disrupts the current frame in the video sequence but also goes over to subsequent frames, gradually diminishing over time until it vanishes. The shot-noise process is characterized by a sequence of jumps that decay as time passes. Under specific conditions, this process converges to a Gaussian distribution. To tackle this issue, the Kalman filter is proposed as a solution for removing noise and restoring the compromised image sequence. Numerical experiments demonstrate the efectiveness of the proposed approach in denoising videos corrupted by shot noise. The results of the proposed method were compared to the results provided by spatial Wiener filter, median filter, bilateral filter and a multilayer perceptron model. PSNR was calculated for the above methods.
In certain applications digital devices can introduce noise to the original image or sequence of images. While there are various image denoising techniques that are efective at restoring noisy images, shot noise still remains mostly unexplored. Currently, no satisfactory algorithm exists that can efectively denoise images sequence of images afected by shot noise.
We propose a novel method for restoring images afected by shot noise. Unlike noise that vanishes
immediately, shot noise diminishes gradually over time. This type of noise arises from defects in the
hardware of a device or issues within the camera sensor [
When the intensity of the underlying Poisson process is high, shot noise can be efectively
approximated by Gaussian noise. Numerous algorithms have been proposed in the literature to address various
types of noise. These include linear filters, such as median and mean filters, as well as non-linear filters,
which are discussed in sources [
The goal is to create a method for restoring images afected by shot noise by utilizing a Gaussian approximation of the shot-noise process in combination with Kalman filtering. • (, ) is the true image intensity level at (, ); • is the pointwise noise; • is the observed image intensity level at (, ).
The sequence of 2-D images can be modeled in the following way:
= (, ) + ,
equation [
Therefore, the sequences (, ), = 1, 2, . . . , are the 2-D profiles, and monitoring the image sequence is equivalent to monitoring the 2-D profile sequence. 2.1. Shot noise process and its approximation Shot noise is a type of random noise marked by abrupt intensity fluctuations that dissipate over time. This phenomenon frequently appears in real-time image processing, where individual pixels may experience sudden intensity spikes caused by defects in image recording devices. The impact of shot noise diminishes with time, resulting in reduced interference on the same pixel in subsequent frames of the image sequence.
Shot noise is defined in the following way:
= 0− + ∑︁
− (− ) = 0− + − ∑︁
,
where
1;
intensity ;
beta distribution [
• 0 is the initial value of ; • {}=1,2,... is the sequence of iid random variables with distribution function () and () = • {}=1,2,... is the sequence representing the event times of a Poisson process with constant • is the rate of exponential decay.
The distribution of the random variables {} can be arbitrary, for example normal distribution or
The expectation of the shot noise , assuming that 0 is known, is as follows from [
( ) = + 0 − − →
→ ∞, 1 )︁ and, moreover, if the initial value 0 equals 1/ , then we have a stationary case and the mean value ( ) will be equal to 1/
and will not depend on time .
The variance of the shot noise process is as in [
2 + √︂ 2 .
2 = (, ) + , model: where of the 2-D images: where (4) (5) (6) (7) (8) distribution to 0. Then () converges in law to , where
The main result of the paper [[
This implies that is normally distributed with mean () = 0− () = 1 − − 2 → 1 →
∞. If 0 = 1/ , then 0 = 0 and, therefore, () = 0.
Following the linear transformation (5) the shot-noise process has the following form: → 0 → ∞ and variance Define ˆ as Gaussian approximation of as follows:
When the intensity of jumps is relatively high, shot noise can be efectively approximated as Gaussian noise. This allows for the application of Kalman filtering techniques in real-time image restoration processes. A series of 2-D images afected by shot noise can be represented through the following • = 1, ..., ; = 1, ..., ., = 1, ..., .; • (, ) is the (, )ℎ pixel of the ℎ image; • (, ) is the true image intensity level at (, ) of the ℎ image; • is the pointwise shot-noise of the ℎ image; • is the observed image intensity level at (, ) of the ℎ image.
Applying the Gaussian approximation of the shot noise we obtain the following model of the sequence = (, ) + ˆ = (, ) + • is normally distributed with mean 0 and variance 1 as → ∞; • 1 and 2 are first and second initial moment of the random variable ; • is the intensity of the underlying Poisson process; • is the rate of exponential decay.
There occurs a correction term 1 in equation (8), which is constant additive part to the image intensity. Therefore where = (, ) + ˆ = (, ) + 2.2. Compound Poisson process and its approximation Compound Poisson noise process is defined in the following way: This definition is a special case of the shot-noise process with = 0.
We recall the result from previous subsection: = ∑︁ .
=1 ( ) = − 1 √
→ , → ∞,
This means that compound Poisson process where = ∑︀ =1 , is Poisson process with intensity and is Brownian motion. = √︀ 2/ + 1 ˆ = √ 2 + 1 can be approximated by = (, ) + ˆ = (, ) + √ 2, (, ) = (, ) + 1 .
It can be noticed that both shot-noise and compound Poisson noise process both have Gaussian approximations, but these noise processes have diferent impact on the images. Shot noise decays and disappears as time passes. But the Compound Poisson noise process does not disappear, its efect stays at that part of the image where it occurred.
Both types of noise can be filtered out using Kalman filter if the intensity of the underlying Poisson
process is high and thus the Gaussian approximation holds.
2.3. Kalman filtering for image restoration
Kalman filtering is used to estimate the variables of the control system subject to stochastic disturbances
caused by noisy measurements of the input variables [
We assume that the image sequence is captured by several cameras and the noise interferes during the image acquisition process. Then the image sequence is processed for further analysis. The aim is to construct the filter that would eficiently denoise the image sequence and would produce acceptable results with such kind of noise. The representation of the image for the Kalman filter is pixelwise.
The noise free pixel value is , which is assumed to be the first order autoregression model. The process looks as follows:
+1 = + , where • is a constant and depends on signal parameters; • is Gaussian noise with zero mean and 2 variance.
Such a model is often used to represent pixel values in video signals.
The measured signal is given by the following equation:
= + , where the independent of additive zero mean Gaussian white noise with variance 2. We need to construct a linear unbiased estimate ˆ of having the observations 1, 2, ..., . The estimation error is denoted by ˜ = − ˆ . The variance of the error is denoted by ˜ and defined as follows:
˜ = [(˜ )2].
The discrete Kalman filter is assessed through the iterative application of the following equations: (13) (14) (15) (16) (17) (18) (19) • filter equations; • variance of the estimation error and the coeficient ;
* = 2ˆ − 1, ˆ = * + { − * };
* = 2˜− 1 + 2 , = *{* + 2}− 1,
˜ = * − *.
Note that *+1 = [(2˜ + )2]. This algorithm is applied to each pixel and at each time instant.
The estimated pixel value ˆ the output for each iteration . It provides the filtered image for the further analysis.
We consider the following parameters for the shot-noise process: • intensity of the Poisson process equals 1.04, • jump size y is normally distributed N(0, 0.007).
The shot-noise and compound Poisson process look as shown in figures 1 and 2.
The example consists of a sequence of 2D RGB images, where certain pixels in the video are afected
by a shot noise process, leading to corruption as noted in [
Figure 3 shows one image taken from the original video sequence of images. The original image sequence is not corrupted by any noise. Figure 4(a) shows the same image as in figure 3 but corrupted by the additive shot-noise. One can see that not the whole image is corrupted but only a part of it (the upper part). This means that not every pixel is noisy but only the pixels on the upper side. In real life this may occur due to the problems with the camera sensor. Next, the shot-noise corrupted image sequence is denoised using the Kalman filtering techniques. In figure 4(b) the corresponding denoised image is shown. By comparing figures 3 and 4(b) (calculating the MSE), one can evaluate the quality of restoration. Although, the noise is not removed completely, the quality of the image becomes much better. The similar results concerning compound Poisson noise are show in figures 4(c) and 4(d).
The results of the performance of other denoising techniques, such as spatial Wiener filter, median iflter, bilateral filter, multilayer perceptron model, comparing to the proposed approach are provided in table 1.
The higher PSNR value value means the better image filtration quality, therefore, the use of proposed ifltering approach (Kalman filtering) gave better results amoung other filters. Filters
PSNR(dB) value The proposed filtering approach
Spatial Wiener filter
Median filter
Bilateral filter Multilayer perceptron model
In this paper we have solved the problem of filtering the pixels of the video corrupted by shot-noise and compound Poisson noise. Shot noise brings the efect of instant jump of the pixel value that slowly decays as time passes. Compound Poisson noise gives the efect of the instant jump of the pixel intensity that does not decay as time passes. Therefore it is very important to filter this kind of noise. Standard filtering techniques would not work well for such type of noise. Therefore, we considered approximating shot-noise and compound Poisson noise with Gaussian process and applying Kalman ifltering to remove shot noise from the sequence of images. Kalman filtering uses the filtered image from the previous step to denoise (filter) the image on the current step. The graphical illustration was shown on several figures. In average, the shot noise corrupted image sequence is 90% restored using Kalman filter comparing to the original image sequence. For compound Poisson process the filtering quality is 80% on average.
The central result of this paper allows to model the efects of the random shot-noise and compound Poisson jumps that corrupts the video data, to approximate the both kinds of noise by Gaussian noise and to apply the Kalman filtering techniques for the video restoration.
Declaration on Generative AI: The authors have not employed any generative AI tools.