In the work, the following two variants of the direct and inverse problems about image smearing along a rectilinear trajectory are compared: 1) Uniform smearing, the same at all points of the image (smear Δ = const). This variant is described by a set of one-dimensional Fredholm integral equations (IEs) of the first kind of convolution type with direction of x axis along the smear trajectory and y is perpendicular to a smear, as well as by one two-dimensional IE of convolution type, moreover the axis x is directed horizontally and y vertically down. IEs are solved by Tikhonov regularization (TR) method (since the problem for solving them is ill-posed) and Fourier transform (FT). 2) Non-uniform image smearing of several moving objects (smear Δ = Δ (x)). This variant is described by IE of general type and solved by TR method and quadrature method (in case of set of one-dimensional IEs) or cubature method (in case of one two-dimensional IE). It is shown that in case of non-uniform smear, use of a set of one-dimensional IEs is preferable to a two-dimensional IE. The results of numerical experiments are obtained.
Consider one of the actual problems of distorted image processing – the
elimination of image smearing via mathematical processing ([
To date, in a number of publications, the variant of rectilinear uniform image
smearing is considered in detail [
The purpose of this work is a comparative consideration of two variants for
straight-line image smearing – uniform and non-uniform one. Example: a
smeared image of runners on a track, running at the same, as well as at different
speeds obtained by a fixed IRD. Note that in [
First, we recall the well-known case of uniform rectilinear smear [
The mathematical description of uniform rectilinear image smearing Consider the direct and inverse problems. 2.1
The direct problem
The direct problem of uniform and rectilinear smear is described by an integral
[
The inverse problem The inverse (more important and complex) problem can be solved in two approaches.
In the first approach, a set of one-dimensional Fredholm integral equations
(IEs) of the first kind of convolution type (for each value of y) is solved to
eliminate the smearing [
ì1 D , - D £ x £ 0, h(x) = í
î 0, otherwise.
IE (2) is obtained from relatio (1). Here, h is mathematically the kernel of IE,
and physically and technically it is the point spread function (PSF) [
The problem of solving IE (2) is ill-posed [
¥ wa y (x) = 1 2p òWa y (w) e-iwx dw ,
-¥ Wa y (w) =
H (-w) Gy (w)
H (w) 2 + a w2 p
is the regularized spectrum, or FT of the solution; H(w) = F(h(x)) and
Gy (w) = F (g y (x)) are the Fourier spectra of functions h(x) and g y (x) , where
F is a FT symbol; a > 0 is the regularization parameter; p ³ 0 is the
regularization order (usually p = 1 or 2). To select the parameter a, a number of
methods have been developed: the discrepancy principle, the method of teaching
example-images, etc. [9, p. 236], [
In the second approach, a two-dimensional Fredholm IE of the first kind of
convolution type (cf. (2)) is used to eliminate the smearing (and the defocusing)
[
In this approach, the calculation of the direct problem is based on the
m-functions fspecial.m and imfilter.m [
Wa (w1, w2 ) =
H *(w1, w2 ) G(w1, w2 )
H (w1, w2 ) 2 + a (w12 + w2 ) p
2
where H (w1, w2 ) = F(h(x, y)), G(w1, w2 ) = F(g(x, y)) . MatLab has the
m-functi-on deconvreg.m [
The mathematical description of non-uniform rectilinear image smearing Based on relations (1)–(8), we consider the non-uniform rectilinear image smearing along the smear trajectory. Suppose that from a smeared picture, we determined in some way the dependence D = D(x) of the smear Δ on coordinate x, directed along the smear. 3.1
The direct problem In this case, the PSF h will not be difference, or spatially invariant and the direct problem will be written as (cf. (1)): (9) (10) (11) To solve IE (10), the FT cannot be applied, but the quadrature method is well suited and leads IE (10) to a system of linear algebraic equations (SLAE) [9, p. 126]:
Awy = g y , The inverse problem in case of the first approach is written as a set of onedimensional Fredholm integral equations of the first kind of general type (for each value of y) [9, p. 125]:
b Awy º ò h(x,x) wy (x) dx = g y (x), c < x < d ,
a where A is an integral operator; [a, b] and [c, d] are limits for ξ and x. PSF h will be written as:
ì1 D(x) , x £ x £ x + D(x), h(x, x) = í
î 0, оtherwise. where A is the matrix associated with h (the same to all y-rows), w y is the desired vector, g y is the right-hand side of the SLAE. A stable solution of the SLAE (12) is given by the Tikhonov regularization method [9, p. 126] where a > 0 is the regularization parameter, I is the unit matrix, AT is the transposed matrix, and w ya is the regularized solution in y-row equal to (aI + AT A) wya = AT g y , wya = (aI + AT A)-1 AT g y .
For computer implementation of formulas (10)–(14), the m-function desmearq_n.m was developed.
Note that the quadrature method with Tikhonov's regularization (14) can also be used to solve a IE of convolution type (2) with PSF (3). The m-function desmearingq.m has been developed for this.
The inverse problem in case of the second approach can be written in the form of a two-dimensional Fredholm integral equation of the first kind of general type (cf. (6)):
b d Aw º ò ò h(x, x, y, h) w(x, h) dx dh = g(x, y), a c a £ x £ b, c £ y £ d .
(15) Equation (15) can be solved by a quadrature method (more precisely, cubature) (cf. [13, p. 167]). According to this method, each of the integrals in (15) is replaced by a finite sum on discrete node grids with respect to x, ξ, y, η and we obtain a SLAE with a four-dimensional matrix A and a two-dimensional righthand side g. To solve such a SLAE, one needs to transform the four-dimensional matrix A into a two-dimensional one, two-dimensional right-hand side g to transform into a one-dimensional one, and the resulting one-dimensional solution w to transform into a two-dimensional one. Although the (successful) attempt to solve a two-dimensional IE by the cubature method took place [13, p. 167–169], nevertheless, this is a cumbersome method and its use for restoration of non-uniform smeared image is problematic.
It is also possible to apply for solving IE (15) an iteration method, for
example, the Friedman iterative regularization method [13, p. 272], [
As a result, it should be recognized that the most effective method is (10)– (14), based on line-by-line image processing, according to which one-dimensional IE (10) and SLAE (12) need to be solved with a two-dimensional matrix. 4
Illustrative example The following numerical example was solved. The original image of seven runners on the track is shown in Fig. 2. Fig. 3b shows the result of solving the inverse problem – the line-by-line restoration of the image by the quadrature method with Tikhonov's regularization according to (14) using the developed m-function desmearq.m. Regularization parameter a = 10-6 (chosen by selection). We see that despite the considerable smearing (Fig. 3a), the image is well restored, and without the Gibbs effect due to diffusing the image edges (in Fig. 3a).
The inverse problem of non-uniform image smearing The next step is non-uniform image smearing. We suppose that the runners run at different speeds v, which means that they have different smears D = v × T during the exposure time T. On the basis of Figure 2, we determine the boundaries between the runners and the values of the smears Δ runners (see Table). As a result, a smear D(x) or D(i) is represented as a piecewise constant function. Each runner has its own smear value within his range. Fig. 4a shows an image smeared piecewise non-uniformly, namely, the smearing increases from the left runner, for which smear is 5 px, to the right runner, for which smear is 23 px, i.e. smearing is substantially non-uniform. This smearing is performed according to (9) using the m-function smear_n.m (with diffusing the edges).
Fig. 4b shows the result of image restoration (the inverse problem) by the quadrature method with Tikhonov's regularization according to (10)–(14) using the m-function desmearq_n.m. Regularization parameter a = 5 ×10-3 (chosen by selection). Fig. 4b shows that the images of the runners is restored, but with an uneven background. 4.4
The alignment of image background The background alignment in Fig. 4b has been done: the intensity values in Fig. 4b more than 100 are replaced by 170 (this is the background value). Fig. 4c shows the final result of image restoration after background alignment. We see that the images of the runners are restored quite satisfactorily (cf. Fig. 3b).
Conclusion and future plans
The described technique can be used in practice for restoring group images of
several objects (people, airplanes, cars) moving at different speeds and therefore
received different smears Δ on the image during the exposure by a fixed IRD.
In subsequent publications, the question about a method for determining
nonuniform smear Δ (x), as well as general case of smear Δ = Δ (x, y) will be
considered. Example: a smeared image of a stream of cars on a wide highway
moving at different speeds. A comparison will also be made with the technique
described in [
This work was supported by the grant MFKTU ITMO (Project No. 619296).