Low-illumination Image Enhancement Method Based on Adaptive MSRCR Algorithm

Abstract

The low-illumination image enhancement method based on the adaptive MSRCR algorithm is proposed to address the problems of the Retinex algorithm in processing low-illumination images, such as the need to manually adjust parameters and blurred details. In the HSV color space of the original image, the luminance V component is decomposed by mean filtering to create a detail layer, and the detail layer information is enhanced by using enhancement weights. The improved Salp Swarm Algorithm (LLSSA) is proposed for adaptive parameter adjustment of Multi-Scale Retinex with Colour Recovery (MSRCR) and detail layer weights, which uses Logistic Chaos to initialise the salps population and introduces Lévy flights into the updated positions of leaders and followers to enhance the global search capability. Finally, the adaptive MSRCR enhancement map and the detail layer enhancement map are images fused to produce a final enhanced image with clear details. The experimental results show that compared with several typical algorithms, the algorithm in this paper can effectively maintain the image details, improve the image brightness and have better visual effects. Low-illumination image enhancement, MSRCR algorithm, Salp Swarm Algorithm, parameter Images are an indispensable part of today's information age. However, because of weather, lighting, equipment, and other factors, low-illumination images make up the majority of the images. Lowillumination images generally contain more obvious noise, low brightness, and inconspicuous details, which are detrimental to subsequent computer vision and other applications while affecting the visual experience. Although low-illumination image enhancement has been extensively studied, it is difficult to balance the relationship between contrast enhancement and naturalness. Aspects such as brightness, detail, and color perception of the enhanced image are considered as the subjective perception of the individual is related to the measure of naturalness.

1. Introduction

[6] proposed an approach to lightweight deep networks by converting the task to an image-specific curve estimation problem, and setting the non-reference loss function. Work [7] present an effective unsupervised generative adversarial network that builds unpaired mappings between low-light and

2023 Copyright for this paper by its authors. normal-light image spaces. ( 3 ) Retinex theory, first proposed by Land and McCann [8] in the 1960s. Based on this theory, Jobson and his collaborators have advanced some classical algorithms: Single Scale Retinex (SSR) [9], Multi-Scale Retinex (MSR) [10], and Multi-Scale Retinex with Color Restoration (MSRCR) [11], which introduces a color factor C for color correction to make the picture more natural. However, traditional algorithms are not conducive to automating image enhancement. Work [12] proposed a Retinex-based image enhancement algorithm that uses Particle Swarm Optimization and multi-objective functions to control parameters, which can effectively enhance the brightness, contrast and color of an image. Work [13] and [14] enhance images by converting color spaces.

To further improve the brightness and image details of the enhanced image, this paper recommends a low-light image enhancement method based on the adaptive MSRCR algorithm. The adaptation function in this study adds the color metric and information entropy, which not only ensures the augmentation of brightness but also better preserves the naturalness and detail aspects, in contrast to the prior function that solely concentrates on brightness. The five parameters of the MSRCR algorithm and the enhancement weight parameters of the detail layer enhancement part are optimized by using the improved Salp Swarm Algorithm. The proposed algorithm is tested by using images from different shooting environments.

2. Retinex theory and related algorithms

The Retinex theory is also called the retinal cortex theory. Retinex theory is the decomposition of a given image into illumination and reflection components with the following expressions.

I  x, y   L  x, y   R  x, y  ( 1 ) where is the image observed by the human eye; is the information of the incident component of the object. refers to the reflected part of the object. and are the locations of the pixel points. In the calculation process, since the logarithmic form is closest to the properties of the process with which one perceives luminance, it is usually transferred to the logarithmic domain for the solution.

SSR constructs a Gaussian surround function to obtain the estimated light component. The final image color is easily distorted. The following equation is shown.

lg R  x, y   lg I  x, y   lg I  x, y  G  x, y  where ∗ is the convolution and ( , ) is the Gaussian surround function.

MSR, which has the benefits of constant color, is presented to weight and total the SSR of various scales in order to address the problems with SSR. The following equation is shown.

n lg R  x, y   i1Wi lg I  x, y   lg I  x, y   Gi  x, y  ( 3 ) where is the number of Gaussian surround functions, which is generally 3; is the weight of the ith filter function, and the sum of all scales is 1.

However, there are still problems with distortion of the image due to operation in the RGB space. To address this problem, the researcher once again proposed the new algorithm MSRCR, which multiplies the color recovery factor on the MSR output result to achieve color improvement and correction. The following equation is shown.

RMSRCRj  G C j RMSRj  b where , is the gain and offset; refers to the th color recovery factor, there are generally three color recovery factors representing R, G, B three color channels, respectively. ( 2 ) ( 4 )

3. Improved Salp Swarm Algorithm

Proposed by Mirjalili [15], the Salp Swarm Algorithm (SSA) is a population intelligence optimization algorithm which is computationally small and simple to understand. The researchers found that the salps often forms a salp chain. This is an optimization behavior to change their trajectory faster and better to obtain food, and this behavior has led to the formation of SSA, which has a high utilization rate and convergence speed.

Since there is no guidance of prior knowledge, the standard SSA adopts random initialization. Chaotic sequence helps the algorithm improve the ability of global search [16, 17]. Therefore, in this paper, Logistic chaotic mapping was used to initialize the population of salps. The system definition of Logistic chaos is expressed as follows.

x i  1  x i  1  x i  where is the branching parameter and the value of μ in this paper is 4, which can achieve better results. Compared with the random population individual positions, the use of Logistic chaotic map makes the algorithm more capable of finding the optimal value.

When processing complex image enhancement, the results obtained by standard SSA are not satisfactory . The randomness of step size and direction of Lévy flight [18] allows the algorithm to jump out of the local optimal solution. The length of Lévy flight step is expressed as follows: =

| | 1 ) 1 ~ (0, 2), ~ (0, 2) {

2 = { Γ(1+ )sin( ∙Γ[(1+ )]2 22−1 } , = 1 where is the Lévy flight step, 2, 2 are the variances, and follows normal distribution. Γ is the gamma function. Parameter β is 1.5.

A salps chain consists of leaders and followers. The leader move towards the food, and the update of its position is only related to the food position. In this paper, introducing the Lévy flight step into the update of the position of the leader and follower of salp, the leader can improve the ability of global optimization. The improved position of the salp leader is updated as follows.

x1j   F  c1  ubj  lbj   s  lbj  , c3  0.5  j Fj  c1  ubj  lbj   s  lbj  , c3  0.5 where 1 is the position of the first leader in the th dimension, is the position of the food in the th dimension, is the upper bound of the search space, is the lower bound of the search space, 3 is role between global exploration and local exploitation before both. 1 is expressed as follows. random numbers in [0,1], the direction of leader movement is controlled by 3. 1 plays a balancing where is the number of current iterations and max_ represents the maximum number of times the population can iterate.

Followers can follow the leader to move more targeted, improving the speed of searching for the global optimal solution. The location of the follower of salp is updated as follows. c1  2e  max_iter 

  4l  2 s 2 x ij' 

 xij  xij1  where ’

is the position of the updated follower salp, is the position of the follower salp before update, −1 shows the position of − 1th follower salp in th dimension.

4. LLSSA-MSRCR image enhancement

The parameters in MSRCR usually use fixed values for image enhancement, which is not conducive to achieving the best subjective perception of the human eye and the best quality of the image itself. In this paper, LLSSA is used to improve the multiscale Retinex algorithm with color recovery, and LLSSA-MSRCR image enhancement algorithm is proposed. The algorithm is mainly divided into three (5) (6) (7) (8) (9) parts: the first part is to separate the detail level image of the input low-illumination image, and then some image details are enhanced according to the weight value set after the algorithm optimization; In the second part, according to the carefully set fitness function, LLSSA algorithm optimizes the six parameters of three Gaussian kernels, gain, offset and detail layer weight in MSRCR, and achieves the best image quality enhancement for the differences of each image; The third part is the fusion of enhancement image and detail image according to the weight to generate the final enhancement image. 4.1.

Setting of fitness function

It is very important to use an effective objective standard to evaluate the image quality. In this paper, brightness, color measurement and information entropy are selected as the objective function, as the direction of salp optimization, to measure the image quality.

First, Transfer the image to HSV color space, select the lightness component V in HSV color space for evaluation, and perform subsequent image enhancement.

The calculation of image brightness is shown as follows.

BR  mean Vz   mean V  (10) where is the V component of the enhanced image and is the V component of original image.

Second, to enhance the color of the image, the image color index proposed by work [19] is adopted to measure the overall color sense in the natural scene, reflecting the vividness of the image. For the enhanced low-illuminance image, the higher vividness indicates that the image is more consistent with human perception.

C   rgyb  0.3  rgyb (11) where ∙ is the standard deviation and ∙ is the mean.

Thirdly, a performance indicator information entropy is selected to evaluate the image quality. Image information entropy represents the amount of information contained in the image. The calculation formula of information entropy is shown as follows.

255 H   p( j) lg P( j) (12)

j0 where ( ) is the probability of pixel j in the image.

In this paper, Formula (13) is used as the fitness function of the whole to search for optimization.

F  BR  H  0.3  C (13) 4.2.

Enhancement of image detail layer

In the HSV color space, the brightness component V usually contains a large number of details of the picture. The enhancement of the detail layer of the image is to separate the detail layer in the V channel and carry out the restoration and enhancement of the detail layer.

First, convert images to HSV space, and the mean filter is used to blur the V channel of the original image, and the basic image B is obtained. The mean filter is a commonly used low-pass filter, which takes the set region as the template, calculates the average value of the set region and sets the value as the center. The difference image obtained by subtracting the base image from the original image is the detail layer image . The formula for obtaining the detail layer is shown as follows.

D  V  B (14)

Second, set the enhancement weight of the detail layer, and enhance the separated detail layer times, which can effectively retain the edge information of the image and restrain the image overexposure. Combined with base and space enhanced by LLSSA-MSRCR, V channel image after detail enhancement is formed, as shown in Formula (15).

V  tD+B+Vmsrcr (15) where

is the value component V channel after image enhancement by LLSSA-MSRCR.

Image fusion

In the process of image enhancement, image fusion is also a crucial part. Only with appropriate fusion weights can we generate images with higher quality and more consistent with human visual perception. The image fusion formula is shown as follows.

P=a  X  b  M (16) where is the final enhancement image of low illumination image enhancement, is the detail layer enhancement weight, is the MSRCR enhancement weight, is the detail layer enhancement image, and is the MSRCR enhancement image. After extensive experimentation, set to 0.2 and to 0.8.

5. Results and Analysis

out in this paper. F2  x  maxi  xi ,1  i  n F3  x   100  xi1  xi2 2   xi 12    F4  x   ixi4  rand 0,1 F5  x    xi2 10 cos 2 xi  10  xi2   cos  n i1  xi   i 

 1 F6  x  F7  x  

  u  xi ,10,100, 4 F8  x   ai  x1 bi2  bi x2   bi2  bi x3  x4  2 2  

Dim 10 10 10 10 10 10 10 4

Range [-100,100] [-100,100]

[-30,30] [-1.28,1.28] [-5.12,5.12] [-600,600] [-50,50] [-5,5] 0 0 0 0 0 0 0 3E04 n1 i1 10 sin  y1     yi 12 1 10 sin 2  yi1    yn 1 

For the comparison experiment of standard test functions, there are 8 standard test functions selected in this paper, including single-peak standard test function (F1-F4), multi-peak standard test function (F5-F7) and fixed-dimension multi-peak standard test function F8, as shown in Table 1. In this section, LLSSA is compared with other optimization algorithms, such as Salp Swarm Algorithm SSA, Ant Lion Optimizer ALO, Moth Flame Optimization MFO, and Whale Optimization Algorithm WOA. In all optimization algorithms, the initial population value is set to 30, and the maximum number of iterations

According to the experimental results of comparison between LLSSA and four optimization algorithms on eight test functions, it can be seen from the table that in terms of average value, the optimization performance of LLSSA is better than that of SSA, WOA, ALO and MFO when the optimization accuracy of LLSSA is solved on unimodal function, multimodal function test function and fixed-dimension multimodal function test function. The convergence accuracy of LLSSA in functions F5 and F6 is the highest, which is close to the theoretical value 0. In terms of standard deviation, the standard deviation of LLSSA is smaller than that of SSA, WOA, ALO and MFO, which indicates that the stability of LLSSA and the ability to jump out of the local optimal are stronger than other algorithms. It can be seen from the above analysis that the stability, global optimization ability and ability to jump out of local optimization of LLSSA are stronger than SSA, WOA, ALO and MFO. 5.1.

Subjective evaluation

In this paper, a total of twenty low-illumination condition images from different low-light image data sets are used, including DICM [20], LIME [21] low-light datasets and images downloaded from some company websites. The parameters of SSR, MSR and MSRCR were set as follows: ( 1 ) the Gaussian kernel selection of SSR was 80. ( 2 ) The three Gaussian kernels of MSR are 15, 80 and 250 respectively, and the weight of each Gaussian kernels is one-third. ( 3 ) MSRCR The Gaussian kernels are the same as those of MSR, the gain is 30, and the offset value is -15.

Figure 1 (a) shows the street image at night. Compared with the original image. Pictures in (b) and (c) are relatively white. The overall color is slightly dark, and the distant architectural details are not obvious. Pictures in (d) makes the overall tone of the picture more real and the color more vivid. However, both (d) and (e) have the blurred details, and the light is overexposed, which reduces the visual quality. The algorithm in this paper let the contour details of the human figure, mountains, bridges and street lamps in the image are significantly enhanced, and the overall brightness is improved. Compared with (d), the overall color of the image is more vivid and clearer.

(a)Original image (b)SSR (C)MSR (d)MSRCR (e)MSRCP-PSO (f)Proposed

Figure 2 (a) shows the low-light plant image. In (b) and (c), the images are gray, which destroys the visual effect. (d) solves the ash phenomenon well on the whole. However, there are still problems such as unsatisfactory detail processing and unclear details on plant edges. (e) has blurred details, and the effect is not ideal. The overall brightness of the image enhanced by the algorithm in this paper is improved very well with plants and color palette as the center. The brightness enhancement is reasonable. The outline details of plants, especially small succulent ones, are more obvious. （a）Original image (b)SSR (c)MSR (d)MSRCR (e)MSRCP-PSO (f)Proposed

Figure 3 (a) shows the image of the bridge under the setting sun. Due to the influence of poor light at night, houses, green plants and high-voltage lines can hardly be distinguished. (b) and (c) enhance the brightness to a certain extent. Houses and Bridges can be clearly seen, but there is a lack of details. The overall tone is somewhat gray, and the color and vividness are poor. The overall tone of (d) is better than the previous pictures. The details of the house in (e) are blurred and the effect is not ideal. The outlines of clouds and building in (f) are more detailed, the overall tone is brighter, and the clarity of the image is improved compared with the previous image.

From the above analysis, it can be concluded that compared with other algorithms, the algorithm in this paper has good results in brightness, color saturation, detail and clarity, and the overall visual perception of human eyes is better. 5.2.

Objective evaluation

In addition to the subjective visual perception, the results of image enhancement also need to be evaluated objectively with some measurement data. In this paper, three objective evaluation indexes, namely mean value, variance and information entropy, are used to evaluate the experimental results of each algorithm. The mean value can reflect the overall brightness of the image. The larger the mean value is, the higher the brightness of the image is. The variance reflects the richness of the image gray level. The larger the variance is, the higher the contrast and the more obvious the detail. Image information entropy reflects the storage of image information.

Table 3 shows the enhanced effect of street side image. As can be seen from Table 3, compared with the traditional MSRCR algorithm, the low illumination image enhancement proposed in this paper improves by 6.25%, 35.16% and 3.64% respectively in the three evaluation indexes of mean value, variance and information entropy. It shows that the image enhanced by the proposed algorithm is better than that enhanced by other algorithms in brightness, contrast, detail and information contained in the image.

Evaluation original SSR MSR MSRCR MSRCP- This paper index PSO

Mean 48.2371 166.9349 166.9549 150.2321 149.5434 159.6194 Variance 885.6733 1420.2753 1417.9889 1131.0718 1147.7752 1528.7883

IE 6.7404 7.0182 7.0174 7.0531 7.0600 7.3100 Table 4 shows the enhanced effect of plant image. According to the data in Table 4, it can be seen that the mean value, variance and information entropy have increased by 7.25%, 50.43% and 3.77% respectively, and these three evaluation indicators have significantly improved.

As can be seen from the objective evaluation indexes in Table 3-Table 5, the five image enhancement algorithms have improved low-illumination images to a certain extent, and the algorithm in this paper has the most obvious enhancement effect on low-illumination images.

6. Summary

Aiming at the problems of low brightness, fuzzy detail and low contrast of low illumination image, this paper proposes a low illumination image enhancement method based on adaptive MSRCR algorithm. The proposed algorithm effectively combines the MSRCR enhanced image with the detail enhanced image and can enhance the image quality by improving the advantages of brightness and detail. Brightness, color measure and information entropy were used as fitness functions to judge the quality of the enhanced image. Parameters such as Gauss kernel in MSRCR algorithm and the weight of detail layer were optimized by using Salp Swarm Algorithm to obtain the best parameters of image enhancement in different scenes. The enhanced weight is used to enhance the detail layer and then fuse with the MSRCR enhanced map, which not only preserves the detail information but also balances the brightness information. The experimental results show that the proposed algorithm can improve image brightness and contrast, and achieve high performance in image fidelity, full color and detail clarity. It is superior to the traditional algorithm in vision and objective evaluation, and is conducive to the subsequent application of computer vision and other aspects.

References

[1] R. Dyke , K. Hormann , Histogram equalization using a selective filter, The Visual Computer ( 2022 ) 1 - 15 . doi: 10 .1007/s00371-022-02723-8

[2] A. K. Bhandari , S. Maurya , Cuckoo search algorithm-based brightness preserving histogram scheme for low-contrast image enhancement , Soft Computing 24 ( 2020 ) 1619 - 1645 . doi: 10 .1007/s00500-019-03992-7

[3] C. Wei , W. Wang , W. Yang , J. Liu, Deep retinex decomposition for low-light enhancement , arXiv preprint arXiv: 1808 . 04560 ( 2018 ).

[4] Y. Zhang , J. Zhang , X. Guo . Kindling the darkness: a practical low-light image enhancer , in: Proceedings of the 27th ACM international conference on multimedia , 2019 , pp. 1632 - 1640 . doi: 10 .1145/3343031.3350926