Image segmentation is an important stage in image processing. The approach for the satellite images segmentation based on objects edge detectionis proposed. The approach usesthe random Markov fields as a mathematical model of an image. It is proposed to use the methods of contour and texture segmentation on different color components of a satellite image. The contour segmentation detects objects with different colors. It is applied to component with color information. The transition probability in two-dimensional Markov chains is used as a texture feature. The texture segmentation is applied to component with brightness information. The simulation results of the proposed approach in different color models, such as RGB, HSV, Lab, are presented. The accuracy of detecting contours was estimated using the set of test images on the base of five criteria. The use of a combination of color and texture characteristics of regions, made it possible to improve the accuracy of objects edge detection.
Remote sensing data are widely used in different applications, including agriculture, forestry and
water management, monitoring of the environment and emergencies, urban planning, cartography, etc.
The thematic processing is one of the ways to process the satellite images in such systems. It includes
detection, decoding and objects recognition stages. Using the thematic decoding of the satellite
images, it is possible to allocate different classes of objects, such as forests, fields, rivers, urban zones,
etc. [
In general, image decoding consists of such stages: image acquisition; image enhancement (filtering, contrast enhancement, increase of resolution, etc.); object detection (edge analysis, segmentation into homogeneous regions); objects classification (sorting the allocated objects to finite number of classes).
Each stage of such process uses data obtained on the previous stage. Therefore, the quality of each stage affects the accuracy of the recognition results. Usually at each next stage the more complex algorithms are used. They require more time for processing and have the less degree of their automation. Therefore, it is preferable to use algorithms, which have a small number of parameters for setting, require small computational resources and minimum operator participation. At the same time they must provide high quality processing. It is especially actually for the algorithms applying at the first stages.
This work is about the object detection stage, the main method of which is segmentation. Different
features can be used for image segmentation. Among them are object brightness, color, texture, shape,
etc. In general, all segmentation methods can be divided into two classes: methods of contour analysis,
and texture methods. Contour methods are based on the objects edge detection for some feature [
In this work, the approach for image segmentation of satellite images is proposed. It is based on objects edges detection using the color and texture information. It increases the accuracy of objects edge detection. The approach uses the mathematical model based on Markov random fields for image description.
In the previous work [
According to the used model, g-bits digital halftone images (DHI) are represented by the set of g bit binary images (BBI). Each BBI is the superposition of two one-dimensional Markov random chains with two equiprobablestatesM1 and M2 and matrices of transition probabilities in the horizontal and vertical directions: 1Π = ‖ 11 11 11 1222‖ , 2Π = ‖ 22 1211
The entropy approach was applied for calculating the probabilities of the binary elements states.
Thus, the amount of information in the element 3 relative to the states of the neighboring
elements 1, 2is calculated by equation (2) [
( 3| 1, 2)= − ( 3| 1) ( 3| 2)
, ( 3| 2, 1) where ( 3| 1), ( 3| 2)are one-dimensional densities of transition probability of the neighboring elements, ( 3| 2, 1)is the density of transition probability in two-dimensional Markov chain.
The transition probability density in the binary two-dimensional Markov chain can be expressed by equation (3),where (∙)is the delta function.
( 3| 2, 1)=
( 3 = | 1 = , 2 = )× ( 1 − )× ( 2 − )
Taking into account the equation (3), the transition probability matrix for various combinations of the neighboring elements states has the form (4).
2
∑
, , =1
Π = ‖
1
4
1′
4′
‖ = ‖‖ 23 23′′‖‖ ; , = ̅1̅,̅2̅; ≠
sliding window method.
as[
This method [
, ̃ ( , ) =
1 ∙
∑
∑ ̂ =1 =1 ( , ), (3) (4) (5) (6) of (7) where ̂( ) is the estimate of the average sequence length of the identical BBI elements; 1 is the initial probability ( 1 = 0,5).
From the second line, the estimate of transition probability for vertical 2 ̂ and estimate ̂ transition probability in two-dimensional Markov chain are calculated by the matrix (4).
All the obtained estimates are averaged within the window to produce a mean estimate of transition probability ̃ : where m, n are height and width of the sliding window.
This mean value is used as a texture feature for the central element of the window.
A window of fixed size is moved from left to the right and top to bottom on lth BBI to get texture feature for each image element.
Then image element is marked by comparing the calculated texture feature with the threshold.
As a result, each image element has the label corresponding to a certain texture. The threshold can be selected on the basis of the analysis of texture feature histogram. If there are several textures on the image, it is needed to select several thresholds.
In the case of color image processing, each color component can be represented as the DHI. All color components are processed separately. The threshold is selected for each component. The segmentation results obtained on different color components are combined into a single color image. On this image different colors correspond to the regions of different textures.
To detect objects edges the amount of information between the element 3 and the various combinations of the neighboring elements is calculated. It is determined with the matrix (4) and the equation (2) for each element of l th BBI.
The amount of information in the element 3 will be minimal, if the neighboring elements 1, 2
have the same states with the 3 [
On the edge of other brightness region one or two neighboring elements have different states with 3.In this case the amount of information in the element 3 is increased.If the amount of information in the element 3is greater than to h the pixel belongs to the contour. The element 3belongs to homogeneous region in the other case.
The threshold h is calculated for each BBI taking into account the minimal amount of information and the amount of information, when one of the neighboring elements has a different state. ℎ = 0,5 ∗ ( ( 3 = | 1 = , 2 = )+ ( 3 = | 1 = , 2 = )). (8) It is supposed that the transition probability matrices are a priori known.
In the case of color image the contours are detected on each color component. Then the contour maps of each component are combined to asingle contour image.
Most often, satellite images are multispectral (multicomponents). They are displayed as the color
images, which have three channels. The three channels may be three multispectral bands of the same
scene. Various types of color images can be prepared based on the different band combinations.
Truecolor images use visible red, green and blue bands. False-color images use the combination of near
infrared, red and green bands. Pseudocolor images contain medium and near infrared and green
bands.[
Color is an important characteristic of objects that often simplifies there segmentation and recognition. There are several ways to specify colors. The RGB color model is the simplest and the most nature. In this case a color image consists of three components (red, green and blue), described by their corresponding intensities. This model has large color coverage, but it is poorly suited for processing tasks, because the color and the brightness information are encoded in the same three channels. In Lab and HSV color models color and brightness information are separate into different components. So they are much more convenient for processing.
In Lab color model, the a and b components encode color. The first component a determines the color position between green and magenta, the second component b – its position between blue and yellow. The third component Lis independent of color information and encodes brightness only.
The HSV color model uses only one channel to describe color. The image contains of three components. They are the hue H, the saturation S and the value (or brightness) V components. The hue H component is the color position; the saturation S component is the amount of gray in the color. The value V is the brightness or intensity of the color.
The main idea of the proposed approach is that different segmentation methods are appliedto different components of a color image. The texture segmentation method, which was described in subsection 2.2, is used on the component with brightness information. The contour segmentation method, described in the previous subsection, is applied to the component with color information. As a result of texture segmentation the regions of different textures are marked by different labels. To get the contour map the second stage after texture segmentation was added. On the second stage the contour segmentation is applied to the marked image output by the texture segmentation. Then the contours detected on different components are combined into single contour image. Thus, the proposed approach takes into account the color and texture information for image segmentation. This improves the accuracy of the objects edges detection.
To estimate the performance of the proposed approach, we have simulated it on images in RGB, Lab and HSV color models. The software used was Matlab. The experiments were designed to analyse the role of texture and color characteristics played on image segmentation. In the first experiment we used RGB color model and the contour segmentation method discussed earlier in subsection 2.3. In this case the image was divided into three components (R, G and B components), and each component was processed by the contour segmentation method. In the second experiment we applied the texture segmentation method to each component of RGB image. To detect the edges, the contour segmentation method was applied to the segmented regions output by the texture segmentation. Then, we simulated the combination of texture and contour segmentations on the images in Lab color model. The component L(lightness) was proposed by the texture segmentation method with the next contour detection. The color components a and b were processed by the contour segmentation method. In the final experiment the images in HSV color model were processed. The texture segmentation was applied to V component (brightness), and the contour segmentation was applied to H component (hue). The S component (saturation) was not used.
The qualitative performance was evaluated by comparing the segmenting results with a benchmark. For lack of benchmarks for the real satellite images, we tested the proposed approach on the images from the Berkeley Segmentation
Dataset [
The quantitative comparison for performance is based on five measure metrics. They are
FOM(Figure of Merit) [
The FOM (figure of merit) is an empirical distance between the image with contours from the segmentation resultsg and the corresponding ground truthf. It shows how similar the ground truth and the segmentation result are. The FOM is defined as: where card(f) is the number of contour elements in the image f, card(g) is the number of the contour elements in the image g, di is the distance between ith pixel in f and the nearest pixel to it in g.
The RMS is the root mean squared error. It shows how different the groundtruth and segmented image are. The RMS is defined as
The FOM, P, R and F-measure are the higher the better segmentation results. The RMS is the lower the higher segmentation accuracy. models (the bold letters means best value). The values are averaged over all processed test images. different algorithms and color models. We use the highest BBI for segmentation, because there are the more significant region details on it. = ( ) ; = ( )
, = 2 ∙ ∙ + . where w and h are width and height of the image, fi and gi are the intensities of ith pixel in the ground truth and segmented contour image.
The P (precision) is the relation between the correctly detected contour elements and all elements detected as contours on the image g. The R (recall) is the relation between the correctly detected contour element and all elements detected as contours on the ground truth image f. They are calculated by the equations: where TP is the number of true positives decisions of the algorithm, i.e. the number of image elements, which are contours both on the segmented image and the ground truth.
F-measure is a widely used metric to evaluate segmentation results that combines precision and recall. It is the weighted harmonic mean of precision and recall. The F-measure is calculated by the equation (12).
( ) ∑ =1 1⁄2
(9) (10) (11) (12)
F 0.249 0.274 0.287 0.322 c) d) Figure 3. Segmentation results. e)
Figure 3b shows the results of contour segmentation using the RGB color model. The edges are detected by the method based on two-dimensional Markov chains on each color component (R, G, B).Then all contours are combined into one resulting image (figure 3b). Figure 3c shows the result of the second experiment. Here the texture segmentation method is applied to each color component of RGB color image. It is assumed that the initial image contains only two different textures. On the segmented image the regions of the first texture are marked as “1”, and the regions of the another texture - as “0”. As a result of texture segmentation binary image were obtained. The contour segmentation method is applied to the texture segmentation results. Thus, the edges of texture regions are detected. The contour images of three components are combined into one resulting image (figure 3c).
Figure 3d illustrated image segmentation result in Lab color model. Here only the final resulting contour image is shown. It is a combination of contour segmentation results of a and b components and contours of the texture regions detected on the L component. Figure 3e shows the resulting contour image obtained on satellite image in HSV color model. It is a combination of contours detected on the H component and contours of the texture regions detected on the V component.
From the simulation results, a conclusion can be draw that the contour segmentation method allows detecting edges of regions with different colors on the image. But it gives unsatisfactory results for texture regions. Such texture regions are often observed on satellite images. They do not have pronounced edges in terms of brightness or color. This leads to the significant over-segmentation. This case is shown on figure 3b. The texture segmentation allows detecting the edges of texture regions more clearly. But in the same time, some edges between objects of different colors can be lost what is illustrated on figure 3c.The sea region and the flat part of coast are differ significantly in color. But in the same time, they have close values of transition probability of the elements in two-dimensional Markov chain. Consequently, the algorithm is detected them as one region.
The segmentation based on Lab and HSV color models provides similar results. Herewith, the edges of objects of different colors and the edges of different texture regions are detected more precisely. The use of HSV color model has another advantage that it is enough to process only two components. This allows to reduce the computational time significantly.
The results of simulation on the test images confirm these conclusions. In table 1, segmentation based on HSV color model gets one of the best values in the most quality criteria. We can also find that the contour segmentation based on RGB color model has the best values in FOM and P. The reason for this is that such segmentation belongs to over-segmentation which gives more details. Therefore, there are a lot of coincidences between the ground truth and segmented contours. But in addition in this case there are also a lot of false detected contour pixels. As a result such segmentation has the worst values in RMS and R. So the results of contour segmentation based on RGB color model are unsatisfactory.
Thus, the proposed approach consists of the joint use of contour and texture segmentation on the different image components. It takes into account color and texture characteristics of objects for segmentation. Due to this the segmentation results are more accurate. It is recommended to use HSV color model, because it shown the best results.