This paper describes a method that tries to improve the accuracy of a machine vision algorithm for frayed edge detection in coldrolled electrical grain oriented steel plate with usage of the Singular Value Decomposition. The algorithm being improved is based on preprocessing the image with the Multi Scale Retinex algorithm, application of the Sobel filter and additional evaluation logic.
To enhance the quality of an input image and highlight the frayed edges in the input image many preprocessing methods can be used. In the process of design of our detection algorithm we have tried number of preprocessing method, some of them are described in this section. 2.1
The histogram equalization aims to increase global contrast of a processed
image to adjust local intensities. This method is useful when an image is composed
mainly of close values as it spreads the most frequent values of an image and
allows areas with close values to gain much higher contrast. More detailed
description of this method can be found in [
The Self Quotient Image is an extension of The Quotient Image technique first
introduced by Riklin-Raviv and Shashua in [
These methods are both class recognizing methods and they are widely used
for an object classifications (for example in face recognition [
Original method (Quotient Image) uses series of bootstrap images to identify ideal illumination free representation of recognized class of objects. Quotient Image of two objects belonging to the same class is then defined as ratio of their albedo functions, thus being illumination free and normalizing lightning conditions and luminance variations.
Self-Quotient Image is extension of previous method that doesn’t need training set of images, instead it derives Quotient Image directly from analyzed image. This means, that it can be used purely as image preprocessing method, since no direct knowledge of object’s class is required. 2.3
Anisotropic diffusion (also refered to as Perona-Malik diffusion) is technique
first proposed by P. Perona and J. Malik in [
In real life huge difference in color quality of observed scene and detail of recorded image can often be perceived. The most apparent difference is loss of color accuracy and image detail, especially in darker areas covered by shadows. As a result recorded images often seem dimmed compared to observed scene. This is caused mainly by inability of camera to distinguish between ambient illumination of the scene and reflectance. Illumination is by its nature independent of the scene, so all the characteristics of observed objects are described only by reflected light component. Recorded image is product of these two components and once it has been evaluated, there is no way we can separate these two values and obtain reflectance, which is critical for correct visual representation of the scene, but human eye still seems to be able to do so.
In 1986, Edwin Land [
MSRCR can be considered most advanced of these techniques and is thoroughly
described in [
N Ri(x, y) = X (wslogIi(x, y) − log [F (x, y) ∗ Ii(x, y)]) s=1 (1) Where i is the index of color band of the image, Ri(x, y) is resulting value of pixel (x, y) of i-th color band, Ii(x, y) is value of i-th color band of original image, F (x, y) denotes Gaussian function and ∗ represents convolution. (a) (b)
Basically the MSRCR performs set of Gaussian filter operations on input image and computes difference between the filtered and unfiltered image. Each of the steps performed is dependent on so called scale. Images filtered with smaller scales contain strong details and dynamic compression, but fail to provide faithful color representation. Large scales behave the opposite. The MSRCR merges all of these images and combines strengths of each scale to provide best image detail and color quality possible.
Great advantage of the MSRCR is that once desired input parameters are found technique performs constantly well with any image provided.
Example of the MSRCR applied to color image can be seen in Fig. 1. Notice how all the details (mainly bricks on the tower) became clearly visible. 4
Singular value decomposition (SVD ) is well known because of its application
in information retrieval – Latent semantic indexing (LSI ) [
Formally, we decompose the matrix of images A by singular value decomposition (SVD ), calculating singular values and singular vectors of A.
We have matrix A, which is an n × m rank-r matrix and values σ1, . . . , σr are calculated from eigenvalues of matrix AAT as σi = √λi. Based on them, we can calculate column-orthonormal matrices U = (u1, . . . , ur) and V = (v1, . . . , vr), where U T U = In a V T V = Im, and a diagonal matrix Σ = diag(σ1, . . . , σr), where σi > 0, σi ≥ σi+1.
The decomposition is called singular decomposition of matrix A and the numbers σ1, . . . , σr are singular values of the matrix A. Columns of U (or V ) are called left (or right ) singular vectors of matrix A.
Now we have a decomposition of the original matrix of images A. We get r nonzero singular numbers, where r is the rank of the original matrix A. Because the singular values usually fall quickly, we can take only k greatest singular values with the corresponding singular vector coordinates and create a k-reduced singular decomposition of A.
Let us have k (0 < k < r) and singular value decomposition of A A = U ΣV T ≈ Ak = (UkU0) Σk 0 0 Σ0
VkT V0T We call Ak = UkΣkVkT a k-reduced singular value decomposition (rank-k SVD) (U0, Σ0, and V0 represent matrices filled with zeros).
Instead of the Ak matrix, a matrix of image vectors in reduced space Dk = ΣkVkT is used in SVD as the representation of image collection. The image vectors (columns in Dk) are now represented as points in k-dimensional space (the feature-space). For an illustration of rank-k SVD see Figure 2.
Rank-k SVD is the best rank-k approximation of the original matrix A. This means that any other decomposition will increase the approximation error, calculated as a sum of squares (Frobenius norm) of error matrix B = A − Ak. However, it does not implicate that we could not obtain better precision and recall values with a different approximation.
To execute a query Q in the reduced space, we create a reduced query vector qk = UΣkT−q1U(akTnqo)t.hIenrstaepapdrooafchA isagtaoinusste qa, mthaetrmixatDrik0x =DVkkTagianisntsetadqkof(oDr kq,k0a)nids qk0 = k evaluated.
Once computed, SVD reflects only the decomposition of original matrix of
images. If several hundreds of images have to be added to existing decomposition
(folding-in), the decomposition may become inaccurate. Because the
recalculation of SVD is expensive, so it is impossible to recalculate SVD every time
images are inserted. The SVD-Updating [
The Retinex as the most appropriate image normalization technique was chosen for preprocessing of images in inspection of quality in grain oriented electrical steel making process.
Frayed edges detection is part of surface quality monitoring system. Goal of the system is to monitor grain oriented electrical steel plate’s surface during manufacturing process and to detect set of defects degrading quality of final product. Steel plate is coiled up into the coils. Approximate length of one coil is 4000 meters.
Steel plate continuously runs through the de-carbonization line and it’s surface is monitored by set of cameras from both sides. System then analyses input images for defects in real-time.
One of the most problematic defects to detect is frayed edge. In the input image it appears only as a small deviation in brightness in horizontal direction (see Fig. 3). This type of defect is captured from one side of the plate only (as it is visible from both sides) using monochrome digital camera. Resolution of one image is 2400 × 600 pixels. Width of area captured by one camera is approximately 0.5 m, which means that each millimeter of captured area is represented almost by 5 pixels in final image. Images are automatically archived so they can be worked with to improve quality of defect detecting algorithms and now we currently have base of more then two million test images.
Frayed edges arise on a plate because of insufficient MgO powder coverage of the edges. In the annealing process the uncovered edges are stuck together, because of high annealing temperature, and the defect is formed on a plate when it is unwinded on the next processing line and stuck edges are torn off. (a) (b)
Common edge detecting algorithms used on non-preprocessed images do not provide any meaningful results because frayed edge appears only as very small deviation in input image and is suppressed by noise that is introduced into the image due to low exposition time requirements and environmental conditions that do not allow for better lighting of the scene.
To highlight our area of interest – the frayed edge – and to suppress the light non-constancy is the core of the problem.
Many preprocessing algorithms were tried before the Retinex was chosen for
this problem. Among others these light normalization algorithms where tried
out: Histogram Normalization [
In our experiments we will try to detect frayed edges on plate using simple Sobel
filter [
SVD’s result will be used as a mask on preprocessed image. This will allow us to perform detection only on areas highlighted by the SVD. First we will try to apply this mask to original input image, so we can see if the SVD itself is able to replace the Retinex (if the SVD’s mask is accurate enough then the Sobel filter might yield correct results). Additionally we will try to apply mask to image already preprocessed with the Retinex to see whether it can improve accuracy of the Sobel filter. Finally we will run the Sobel filter on image processed purely the Retinex for reference.
SVD image can be computed either from input image or from image already processed by the Retinex algorithm. Both variants will be tried in our experiments.
As test database life data captured with system described in Sect. 5 will be used. Database consists of 100 images containing Frayed Edge on left edge of the steel plate.
All relevant parameters of preprocessing algorithms used for experiments are summarized in Table 1. First column Name of the table contains identification name of the run. Second column SVD source specifies whether the SVD was computed from original image or from image with the Retinex applied. Third column Source image specifies whether the SVD’s mask will be used on original image or on image with the Retinex. Column Multiplier contains value that will be used to multiply all values in resulting image to enhance the brightness of the image. Column Lower bound contains lower bound constant, all values lower than this bound will be clipped to 0. Last column Upper bound contains upper bound constant, all values higher then this constant will be automatically adjusted to 255. First row of the table represents reference algorithm run.
First we will run our reference algorithm and store locations and depths of frayed edges present on the image. Then all the other settings will be run and their results will be compared to those of the reference run.
To conclude contributions of SVD to the detection the following metric will be used: – if the Sobel filter is able to detect at least 80 % of frayed edge’s length, then the detection is considered as successful, – if the algorithm detects false frayed edge of length at least 10 % of the image’s size, then it is considered as false positive, – otherwise detection is considered unsuccessful.
Ratio of successful detections and sum of false positives and unsuccessful detections will then be our metric that will allow us to compare individual settings. This metric is described in Eq. (2).
s m = s + f + u · 100 % (2) Where: – m - final number specifying accuracy in percents of given run, the higher the number the better, – s - number of successfully detected images, – f - number of false positive detections, – u - number of unsuccessful detections. 7
Table 2 summarizes results we have obtained. Structure of the table is similar to Table 1 only column Success rate is added. This column contains result of the experiment, how this number was obtained is described in Sect. 6 and in Eq. (2).
We can see that the SVD by itself was not able to highlight defected areas sufficiently. All runs that were detecting the defect from original image failed to detect single defected image from the testing set.
Runs that used the Retinex image as source image for detection were able to detect defects with some success. Those that used the Retinex as source both for the SVD and for detection performed much better. Those that used the Retinex only as source image and SVD mask was computed from original image performed unconvincingly. This is caused by fact that SVD highlighted only significantly different areas in the image and omitted the less distinctive ones thus shortening the length of detected defect.
Sample images of all settings can be found in Fig. 4. We can see original image 4(a), Retinex image 4(b), SVD (upper bound: 200, lower bound: 200) computed from original image 4(c), SVD(upper bound: 200, lower bound: 200) computed from the Retinex image 4(d), SVD (upper bound: 225, lower bound: 125) computed from original image 4(e) and SVD(upper bound: 225, lower bound: 125) computed from Retinex image 4(f).
From the results obtained we can see that the SVD itself led to no improvements in detection. As an algorithm to highlight defected areas the SVD itself failed and even with help of the Retinex algorithm it was not able to achieve 100 % success rate, so the addition of the SVD will not be an improvement over the current the Retinex solution and will not help to detect defects that the Retinex previously was not able to. 8
In this paper we have tried to use Singular Value Decomposition to improve accuracy of frayed edge detection. Proposition that the SVD itself might be able to successfully highlight defected areas has proven to be wrong as it was not able to correctly detect single frayed edge in test images.
When used in combination with the Retinex the detection algorithm was able to achieve 91 % success rate of reference the Retinex algorithm. This means, that with addition of this mask the algorithm performed worse then without it. Our hopes were that with usage of the SVD mask the detection algorithm will detect defects in all test images and we might be able to lower the threshold on edge detection algorithm thus finding more subtle frayed edges and improving the accuracy of the algorithm. With success rate of 91 % this idea is proven to be wrong.
To summarize results presented in this paper – method introduced in our
previous paper [
In our future work we would like to test more lighting normalization methods and try to improve detection accuracy so that the algorithm would be able to reliably detect even more subtle frayed edges.
This work was supported by the European Regional Development Fund in the IT4Innovations Centre of Excellence project (CZ.1.05/1.1.00/02.0070) and by the Development of human resources in research and development of latest soft computing methods and their application in practice project, reg. no. CZ.1.07/2.3.00/20.0072 funded by Operational Programme Education for Competitiveness, co-financed by ESF and state budget of the Czech Republic.
Wiley