This article describes, how color textures can be reliably detected and classified in the production process independent of external parameters such as brightness, object positions (translation), angulars (rotation), object distances (scaling) or curved surfaces (rotation + scaling). The methods described here are also suitable for reliably classifying at least 18 color textures even if they differ only slightly from each other optically. The online classification of color textures is a classic task in the wood, furniture and textile industry. For example, unwanted defects or partial soiling on moving webs can be reliably detected regardless of fluctuations in brightness and/or shadows during process operation. Algorithms has been developed for teach-in with RGB-HSI-transform, set fewer segments on the color textures of each class with e.g. 24x24 Pixel, use suitable transformations {HSI}, e.g. 2D-FFT for formation characteristic 2D spectral mountains in these segments, extraction of statistical features and setting up the individual classifiers. Algorithms has been developed for identification & classification in process operation with extraction of statistical characteristics and methods of robust classification. The implementation of the methods, the triggering of the color cameras, the processing of the color information including the output of the results to the process control is done with the data analysis program Xeidana®.
The applications of optical image processing cover almost all areas of daily life as well as in production, manufacturing and research. They can be assigned to five essential fields, see Fig.1.1.
Copyright © 2020 for this paper by its authors. Use permitted under Creative Commons
License Attribution 4.0 International (CC BY 4.0).
Important fields of application are object recognition and classification, monitoring and
control of visible areas, inspection of object surfaces, recording the (3D) positions
of objects or measuring their 3D geometries. In this paper the automatic recognition and
classification of objects using existing textures will be presented. Textures are
characteristic regular or disordered patterns, which can be found on the surfaces of
objects, see Fig.1.2.
During the actual classification process, all objects of one type are grouped together in
a uniform class. New classes are created for objects of a new type. An important task
in classification is to separate textures of different classes from each other (selectivity),
but at the same time to allow textures with small deviations within the class (immunity
to interference) [
Identification and Classification of Color Textures 3 In particular, the correct result of the classification should not be affected even if typical deviations occur in the production process, e.g.
The objects change their positions in the camera image, (T-) translation invariance The color textures of individual objects are rotated, (R-) rotation invariance The sizes of the textures change, (S-) scaling invariance The ambient brightness changes and thereby the integral brightness of the image changes Partial shadows appear on the textures There is partial soiling on the colour textures The textures lie on spherically curved surfaces and are therefore distorted in the camera image (RST invariance) 3
Mostly color cameras are used for optical image processing. The method of classification described here is therefore based on color textures, but is also well suited for b/w textures. The actual process of classification is divided into 2 steps; the prior learning of suitable features of the textures (1.) as well as their recognition and online classification (2.) in the process, see Fig.3.1.
Both for teaching the textures and for their classification in process operation, the textures are divided into individual segments (segmentation). This corresponds to the model of nature with a successive rejuvenation of the number of image channels from the retina to the following bipolar cells according to the visual process. 3.1
Segmentation (tiling)
During the teach-in process one or more representative segments (tiles) with e.g.
16x16 or 32x32 pixels edge length are taken from the textures of each relevant object
class, see Fig.3.2.
A previous HSI transformation of each RGB color pixel proves to be advantageous. It
forms the basis for obtaining invariant features and ensures by simple means an
additional invariance against changes in brightness, partial shading etc., Fig.3.3.-6.
The HSI model used for this purpose is based on the HSI color sphere model according
to [
* after transformation from cartesian coordinates to polar coordinates
The transformations {HSI} listed below offer, among other things, the possibility of bringing about an extraction of suitable and partially invariant characteristics after ap- propriate preparation of the segments:
Fast Fourier Transformation (FFT) Hough Transformation Gabor transformation
Local Binary Pattern (LBP) approach
With methods such as the Local Binary Pattern (LBP) approach [
In order to achieve additional better properties regarding immunity to interference (shadows, contamination) and at the same time high separation efficiency for similar textures belonging to other classes, methods with robust classification are used. Modern methods of classification use numerical compensation methods, e.g. the method of least squares, as well as methods of fuzzy classification based on probabilistic models, see Fig.3.7. For their description, for example, the minimum sympathy difference (shown in green), which is a measure of the minimum distance between two texture classes in
Identification and Classification of Color Textures 7 the n-dimensional feature space, is suitable. The sympathy difference then describes the affiliation to a texture class.
In [
The methods listed in Table 3.2. were implemented in the existing universal data
analysis program Xeidana® [
Procedures
Support Vector
Machine [
It comprises of an extensible development environment for solving data analysis tasks in the industrial sector, Fig. 3.8.
Xeidana® is also used for the classification and visualization of large amounts of data. With the help of the extensive repertoire of algorithms and procedures, colour textures, for example, can be classified. The software has a modular structure and can be extended with new functionalities using a variety of different libraries.
Fig.3.9. shows the user interface with which colour textures are both taught and classified. The left window shows 18 different colour textures which are to be taught. The right part of the user interface contains the functionality for selecting the RGB and HSI color channels, setting segments of variable size, e.g. with 16x16 or 32x32 pixels, and classifying the textures.
The segments are set with the mouse on the texture, see red tile on the upper texture. The extraction of the characteristics can be done with Histogram Analysis, Fast Fourier Transformation (FFT) or Gabor Transformation
In Fig.3.10. n-dimensional feature vectors of the individual classes are shown. By pair- wise combination of two characteristics in each case, in the plan view (left picture) suitable characteristics are compared and selected by assessing their separation perfor- mance (Features of the LBP procedure).
Fig.3.11. shows the module for Robust Classification used for setting the parameters of the classifier and for automatic parameter optimization using the Support Vector Machine as an example.
Identification and Classification of Color Textures 11 4
In order to create environmental conditions for the classification of color textures similar to those in process operation, a special test stand with a conveyor belt (width 1m, length 4m) was set up at Fraunhofer IWU. The asynchronous triggering of the individual cameras, the processing of the image information and the transfer of the classification results are based on the data analysis program Xeidana®, see Fig.4.1.
To demonstrate the performance of the Xeidana® program, a demonstrator was cre- ated, see Fig. 4.2. which consists of a camera and a turntable with 10 different types of wood. The grains and colors of the 10 types of wood are different, but in some cases they differ only slightly from each other. The aim was to achieve a correct clas- sification over their entire surface despite the spherically curved surfaces and the re- sulting variations in colour textures in distance and angular position. The result of the classification is shown in Fig.4.3. With the methods for classification described here, color textures can be reliably detected and classified in the production process independent of external parameters such as brightness, object positions (translation and rotation), object distances (scaling) or curved surfaces (rotation + scaling).
The methods described here are also suitable for reliably classifying at least 18 colour textures even if they optically differ only slightly from each other.
The online classification of color textures is a classic task in the wood, furniture and textile industry. For example, unwanted defects or partial soiling on moving webs can be reliably detected regardless of fluctuations in brightness and/or shadows during process operation.
The implementation of the methods, the triggering of the color cameras, the processing of the color information including the output of the results to the process control is done with the data analysis program Xeidana®. The following algorithm has been developed for teaching and classifying HSI color textures is inserted: a)
1. RGB-HSI transformation of the color textures 2. set fewer segments on the color textures of each class with e.g.
24x24Pixel 3. suitable transformations {HSI}, e.g. 2D-FFT for formation characteristic 2D spectral mountains in these segments
extraction of statistical features from the 2D spectral mountains setting up the individual classifiers
1. RGB-HSI transformation of all pixel values of the image 2. segmentation of previously defined areas of the image (ROI) 3. suitable transformations {HSI} of all segments 4. extraction of statistical characteristics 5. robust classification