-Blurred Image Matching (BIM) is based on image pre-processing and Blob detection. BIM has been designed to function with images presenting a strong level of noise of different kinds. The technique shows an excellent robustness, speed and unique features when compared to existing methods. This article investigates the process BIM is based on, proposes a new way to improve the range of noise the technique can process with a good range of success by adding image normalization. Moreover, the article investigates the technique's performances when confronted to different parameters, thus suggesting an ideal brightness for the blob detection to perform at the best of its capacities.
and matches them from an image to another [4].
As shown in previous researches, one of the main issues in stitching images using BIM is their brightness level [4]. Thresholding algorithms are by essence heavily impacted by brightness levels. Therefore, it was difficult to find key points images different expositions.
Different expositions can result from different angles of view, time of the day, metrological phenomena.
In this context, it was necessary, for many samples to first normalize the images characteristics. This normalization would allow finding comparable shapes as used by BIM.
The dataset used in this experiment are sets of aerial pictures taken by drone, those images include a wide range of colorimetry and brightness in their original state. 917 Images were used, divided in 4 categories. Those categories are sets of images representing the same area or an area of proximity to
It allowed us to determine whether a unified, ideal,
features range existed. These dataset contains only aerial
views, however the experimentation on different kind of
images has already be reviewed and deemed insignificant [
one of pixel’s most significant characteristics; however, there is no standard formula for its Copyright © 2020 for this paper by its authors. =0 ( + + ) 3 Where n is the amount of pixels in the image and r, g and b, the value for each pixel in red, green and blue.
Fig. 1. Illustration of the objective of brightness normalization: Matching two images with different exposition.
Fig. 2. Samples of images from the datasets used in the framework of this experiment.
BIM present optimal performance when the difference of brightness between images stays under 5%, especially when using shape contouring for comparison instead of quadrilaterals [7]. Therefore, the formula serving to assess the necessity of normalization presents itself as such | 1 − 1 + 2 2 | < 0.05
Fig. 3 represents the amount of points found depending on images brightness; it shows that the different sets of images with a significant range of characteristics, present comparable areas of matching. Although colorimetry plays a role into shape
matching, it seems not to be a relevant feature
regarding brightness [
Meanwhile, Fig. 4 highlights that the totality of the points were found in between a brightness of 38 and 161 (23% of the spectrum, hereafter referred as partial brightness spectrum). Which highlight the importance of the brightness for BIM processing. Moreover, the majority of the points, (0.25σ to 0.25σ) are situated in a range between 89 and 113, or just under 7% of the full brightness spectrum and about 32% of the partial brightness spectrum.
The ideal interval is to be found where the average brightness of the final image lays in the interval defined in B Define ideal brightness and confirmed in 0.
Therefore, the ideal interval is in a brightness range between 78 and 110, where shapes are the more distinctive and where brightness of both images is equal. Which leaves two areas, on the centre of both diagonals for which all images respect the following formula: 110 >
> 78
III. HISTOGRAM NORMALIZATION
Average brightness changes between pictures, whether
they come from different exposition time or were taken at
different time, with different environmental conditions.
Concretely, our objective was to equalize histograms in order
to find the same objects of interest from an image to another
[
As shown on Fig. 6, where the X and Y axis are different values of images’ average brightness, from -255 to 255. The results of successful matching are presented on a pair of diagonal Fig. 6.1.a, the two original images having different brightness, there are two combination for each coordinate, for example (-21;33) and (-33;21) as shown on Fig. 5.
A smaller brightness difference between pairs of images is showed on Fig. 6.1 and Fig. 6.2 (histogram 1 for the pair with the highest delta, histogram 2 for the pair with the lowest delta), the closer the two diagonals, the lower the delta, which results in a less significant brightness correction. The best results appearing when no brightness difference exists, then the two diagonals are merged into one.
On Fig. 6, the area directly below (Fig. 6.1.b) the diagonal shows dispersed points; those are either noise or isolated points that are very distinctive shapes on an image. Brightness change has a lesser impact on such shapes; they are usually caused by a sudden change of colour in the landmark (such as a red roof in a green forest). The area directly above the diagonal (Fig. 6.1.c) is empty as it represents the part of the array where images are brightness correction of both images diverge in opposite directions; any point is this area is extremely likely to be noise.
Fig. 6.2.a also confirms the results obtained in II.B Define ideal brightness, it is then mentioned that 60% of the points could be found in 32% of the partial brightness spectrum. The shape 2.a contains 33% of the given spectrum and contains 62.8% of the points matched. This information also establishes a direct link bet ween the amount of points and their quality as defined in [4].
Experiments showed that equalizing the source by channel (RGB) was not necessary and has indeed a negative impact as it was frequent that from an image to another the colour distribution would change (new building, area of terrain) and treat colours independently would in that context induce mistakes into the image.
The process of image matching with source image having a brightness delta higher than 1.5% goes as displayed on Fig. 7.
On this figure, the two source images (1.a and 1.b) are similar. 2.a and 2.b shows both images respective brightness histogram. Follows 3.a and 3.b, which displays the images after their treatment of brightness correction. There they give the impression to the human eyes to be perfectly similar. However, it is possible to see slight differences on their resulting histograms: 4.a and 4.b. Those differences are the result of an information alteration due to the process of brightness modification. Finally, 5.a and 5.2 show the resulting shape detection after BIM processing.
The process of histogram normalization adjusts an image intensity and enhance its contrast [8]. Two features that are important for BIM to reach good performances. In most cases, this process is not necessary, but in the context of this research, the process of histogram normalization showed itself especially useful.
By default, the histogram normalization function equalizes the images to their mean brightness according to the following formula [9]: , =
(( − 1) ∑ =, 0 )
This results in images having the same brightness, and consequently, as shown on Fig. 8, the two combinations presented previously (Fig. 5) are joined. Which is a significant advantage for us to find optimal points, as they are now joined on one area as shown on Fig. 8.
However, as seen previously, histogram normalization uses the image’s mean to adjust brightness, resulting in a constant brightness of 128 [10]. Which as seen previously in B Define ideal brightness, is out of the recommended brightness range. It is therefore still necessary to define an ideal area
Normalization has two effects in the framework of our
experiments, as highlighted previously it does correct
brightness of images, equalizing their histogram. But also as
it is one of the technique’s well known purpose is to improve
an image’s contrast, which is extremely helpful to BIM’s
processing [
1 2
In most ways similar to the process presented in chapter III, the following process involves one more step, represented in Fig. 9 as 3.a and 3.b. This step corresponds to the image’s histogram normalization by colour channel, resulting in the brightness presented in 4.a and 4.b. The brightness of 3.a and 3.b in 128, it is then corrected as in 5.a and 5.b. resulting in an average brightness ranged between -0.5σ and 0.5σ of the matched points distribution.
1.a) 2.a) 3.a) 4.a) 5.a) 6.a) 7.a) 1.b) 2.b) 3.b) 4.b) 5.b) 6.b) 7.b)
The following figure shows the distribution of point found without and with histogram normalization. The second having a much higher variance and shows comparable amount on its smaller sigma.
As shown on Figure 10 histogram normalization by channel also flattens matching differences relatively to colours. As such colorimetry doesn’t have to be considered in finding the ideal parameters.
Although results are comparable with and without histogram normalization on the highest sigma, the brightness equalization and the independence taken from image’s colorimetry maintains the utility of this technique. However, our experiments showed steadier results when, using histogram normalization, images in the range of brightness between -0.25σ and 0.25σ were selected, which corresponds precisely to the same range showed in chapter 2.ranged between -0.5σ and 0.5σ of the matched points distribution. 60 d n uo50 f s itn40 o p f30 o t un20 o a10 m e g ra 0 e v A 0 20 40 60 80 100 120 140 160 180 200 220
It occurs that the best approach to merge a set of images is to use image normalization, not only increases the amount of points found but most importantly equalizes all image’s brightness. However, this technique sets the image brightness to its mean, which is outside the ideal range for blob detection defined earlier in this article. Therefore, in order to keep to a minimum, the modifications inflicted to the image, it is best to bring images to the 0.3σ value of amount of points found relatively to brightness, 113.
The experiment presented in this paper allowed to successfully determine sets of pre-processing parameters, while proposing a process modification allowing BIM to perform with a higher robustness than previously, introducing the treatment of new noise sources in BIM’s abilities range.
The research was supported by the Ministry of Science and Higher Education of the Russian Federation (Grant # 07772020-0017) and partially funded by RFBR, project number # 19-29-01135.