This article presents the results of a study of the Elman neural network to identify defects (delamination) in composite materials and estimate their area. The registration zone is a square matrix consisting of 100 elements. For training, we used images with different defect spot area, moving along the matrix and additively mixed with Gaussian noise of various intensity. The network structure optimal by the criterion of testing error/training time is determined. Testing showed that when the defect spot area changes by more than 8 times and with a noise level of 27%, the test error reaches 6%. This error is significantly decreased when narrowing the range of the defect area changing and reducing the noise intensity. A block diagram of the corresponding intelligent system is proposed.
At present in a number of industries composite materials are widely used due to their
high mechanical and thermodynamic properties at low density [
A composite material is heterogeneous in structure. The spatial arrangement of reinforcing elements, such as fiber boundless, is often not ideal. Furthermore, in the balk of the material and on its surface, especially at phase boundaries, there are accumulations of pores and various kinds of microdefects. Cracking of the composite matrix and fibers breaks are observed. The concentration of microdefects is especially high at the boundaries of delamination, i.e. in places where the material begins to disintegrate. All these structural imperfections distort the defect image fixed by the sensor element. Therefore, to identify the defect and estimate the true area of the delamination, it is necessary to process its noisy image.
The perspective of using neural networks for these purposes is based on the possibility of parallelizing the processing of information and their ability to learn, i.e. make generalizations. This makes it possible to identify images that have not been encountered in the learning process. In the problem of classification of defects images considered by us, the processing of information by a neural network in a number of key positions is close to the methods of non-parametric statistical training.
Recurrent neural network differs from the classical network of feed-forward by the
presence of feedback loops from hidden or output neurons. Their presence has a direct
impact on the ability of such networks to learn [
The article aims to create an intelligent system with a core in the form of an Elman neural network to identify and estimate the area of distorted defect images moving within the control zone. In the process of solving this problem, we want to study the ability of recurrent neural networks to identify dynamic noisy images of various sizes defects in the process of scanning by the measuring transducer the product surface. 3
Over the years, neural network image processing methods have been successfully used in the field of non-destructive testing, as well as in a number of other engineering fields.
In the article [
The article [
The authors of [
Elman network [
The standard back-propagation algorithm used in Elman neural network is called
Elman’s back-propagation algorithm. To increase the convergence, speed a new
learning rate scheme is proposed [
Multilayer perceptron network (MLP) and Elman neural network were compared
in four different time series prediction tasks [
One of the major problems facing researchers in the recurrent networks is the
selection of the hidden neuron’s numbers in neural network layers [
L is the number of hidden layers, Nin is the number of input neurons and N p is the number of input images.
Kallan R. [
Generalization performance varies over time as the network adapts during training.
The necessary numbers of hidden neurons approximated in the hidden layer using a
multilayer perceptron were found by Trenn [
Shibata and Ikeda [
and N0 is the number of output neurons. A tradeoff is formed that if the number of hidden neurons becomes too large, the output of neurons becomes unstable, and if the number of hidden neurons becomes too small the hidden neuron’s output becomes unstable again.
Hunter et al. [
The hidden layer nodes number was determined by using the empirical formula
[
In accordance with [
For preliminary processing of information received from the sensor element of the ultrasonic transducer, we used the Elman network, which is an example of a feedback network. The feedback has a profound impact on the learning capacity and the challenge in the design of a neural network is the fixation of hidden neurons with minimal error. The accuracy of training is determined by the parameters: neural network architecture, number of hidden neurons in hidden layers, activation function, inputs number, and weights updating procedure.
In our case, the control zone of the ultrasonic transducer is displayed in the form of a square matrix of 100 100 elements. The Elman network, which performs image processing, has 100 inputs and 100 outputs. In the process of learning the network, we used 8967 images. The results of calculations of the number of neurons in the hidden layer are shown in Table 1. where yk and x j represent the output of the context state neuron and input neuron, respectively; ik and ij represent their corresponding weights; f [] is the sigmoid transfer function (1).
Network training was conducted in sequential mode. This mode requires less internal memory for each synaptic link and is preferable for real-time processes. The initialization procedure was carried out without preliminary using the priory information. Herewith, the initial network parameters were set using a generator of uniformly distributed random numbers.
The number of examples for training the network should not be too large, as this
can lead to overtraining of the network. In this case, the learning process can end only
memorizing learning data. Overtraining network loses the ability to generalize. In
accordance with Widrow’s rule of thumb, the size of the training set necessary for a
good generalization should be of the order of N O W , where W is the total
0
number of free parameters, i.e. synaptic weights and thresholds, 0 is the permissible
classification error. Let nin be the number of input nodes of the network, and nh be
the number of neurons in the hidden layer. If the product nin nh corresponds to the
total number of free parameters W network [
Hidden neurons of the Elman network, learned by the method of back propagation, play the role of feature detectors. Therefore, it is advisable to use this network as a replicator or identity map. At the same time, the input and output layers of our network have the same size nin nout 100 neurons. The network is fully connected.
A sigmoidal logistic function is used as the transfer function of neurons of the hidden layer. One of the important advantages for us of this function is the high enough speed of calculating the derivative. The network training time largely depends on this. In the process of network learning, we used the conjugate – gradient method. This was due to the need to increase the sufficiently low rate of convergence of the quickest descent method and avoid computational difficulties caused by operations with the Hessian matrix in Newton’s method. The computational complexity of the quasiNewton methods is estimated as OW 2 . In contrast, the computational complexity of the conjugate – gradient method is estimated as OW . Thus, in our case when
5
The image of the defect (delamination) has the form of a continuous black spot, which is formed by the n matrix cells with a color level L 5 adjacent to each other. This spot is surrounded by a layer of gray cells with L 4 , the next layer of cells corresponds to L 3 and so on to L 1 (white color). White color characterizes the areas of the solid defect-free material that are completely transparent for an ultrasonic signal.
The defect image (pattern) No.1 is a black square with the size 2 2 cells having L 5 . The spot area of the defect is S1 4 . This is the target image for the first class of images. Moving this image on a matrix of 10 10 cells is carried out line by line. Checkpoint of this pattern, like all subsequent ones, is its lower left cell. In the process of moving, the coordinates of this cell for the first row j 1 change from i 1 to i 9 , then the same is repeated for j 2 and so on to j 9 . Herewith the black spot is not distorted at the edges of the matrix. The number of images used for network training in this case is N1c 81 . Here, the index c means that the displayed images are clean, i.e. not containing noise.
In patterns No.2 and No.3, the left and right upper cells of the black square (No.1) are replaced by gray cells with L 4 . The number of training images does not change N 2c N3c 81 .
Image No.4 is a black square with the size 3 3 cells having L 5 . This is the target image for the second class of images. During the scanning process, the checkpoint of this square was moved along the coordinates i 1...8 , j 1...8 . The number of patterns used for network training in this case is N 4c 64 . Patterns No.5, 6 and 7 are formed similarly to No.2 and 3. For patterns No. 8 and 9 in the upper layer of the black square, two cells on the left and two on the right are replaced by grey ones.
Image No.10 is a black square with the size 4 4 cells having L 5 . This is the target image for the third class of images. The scanning process is similar to that described above. The number of training images is N10c 49 . Patterns No.11, 12 and 13 differ from the target pattern No.10 by the alternate replacement of each of the black corner cells with the exception of the checkpoint on the gray. Patterns No.14 and 15 are formed similarly to patterns No.8 and 9 only for the target image No.10.
Image No.16 is a black square of 5 5 cells with L 5 . This is the target image for the fifth class of images. The scan coordinates for the checkpoint are i 1...6 , j 1...6 . The number of training images is N16c 36 . Patterns No.17, 18 and 19 are formed similarly to patterns No.11, 12 and 13. For images No. 20, 21, 22, 23, 24 and 25 a pair of cells of the outer black layer, shifting along the periphery of the spot, is replaced by gray ones.
The distortion coefficient of the target image is determined by the ratio of the total area of the cells with the changed color to the area of the corresponding black spot (defect image).
The defect images described above were noisy. The absolute noise level P was set by us as follows: where bic is the elements of a clear image matrix and bin is the elements of the noisy image matrix.
Noise is additively mixed with a clean image. We used P 10, 30, 50, 70, 90 levels (2). Considering that the full image contains cells with a gray color gradation from L 1 (white color) to L 5 (black color), the question arises about the distribution of the distortion magnitude L 1, 2, 3, 4 by the number of distorted image elements. We assume that this distribution is Gaussian with a zero mean value and a standard deviation which is equal to 1. Then the probability density is
M P bin bic ,
i1 f L 1 2 e (2L)2 .
(2) (3) (4) , %
P f L 1 f L 2 f L 3 f L 4 .
The corresponding distribution of L by the number of distorted image elements q is shown in Table 2 (3), (4). The relative value of the noise intensity we calculated according to Table 2.
Lmax k , 1 100
Li 2 , Lmax 5 100 1 (5) The value of the normalizing factor k was determined based on the following. If in all 100 cells of the matrix the black color with L 5 (presence of a defect) is changed to white with L 1 (defect-free material), then Li 4 and, in accordance with (5),
Lmax 100% error corresponding to 1 , we get k 5 / 4 1.25 . In the article, we used the following restrictions: if the total value of the image intensities and the noise in this cell of the matrix exceeds the maximum level Lmax 5 , then L 5 , if the summary . Taking such a change in L in the entire matrix as a value was below Lmin 1 , then L 1 was kept. 6
The outputs of the Elman neural network are connected to the inputs of a logical matrix connected in turn to a combinational circuit, which has four outputs by the number of classes of identifiable images. The functional scheme of the intelligent system for defect identification is shown in Figure 1. To estimate the area of defect, we propose the following methodology: the matrix (10 10 elements) represents the control zone of an ultrasonic transducer operating in shadow mode and "sounding", for example, the product wall. The full image of this zone is captured by the sensor unit of the transducer.
During physical scanning of the product surface by the transducer, a noisy image of a defect in the form of a fuzzy dark spot may appear in the lateral part of the matrix. By a local displacement of the transducer, this spot must be moved to the central part of the matrix. The neural network allows you to significantly clear the image of the defect from the noise during its movement to the center of the matrix and make this image more distinct.
In the center of the matrix, there are positioned defect images capture zones, as shown in Figure 2. These zones correspond to images No.1 ( 2 2 elements), No.4 ( 3 3 elements), No.10 ( 4 4 elements), and No.16 ( 5 5 elements), which are targeted for the respected classes. The logic matrix has 100 inputs, corresponding to the number of outputs of the neural network that perform the function of a replicator. These inputs are numbered in accordance with the pair jk , where j denotes the row number and k denotes the column number of the matrix, as shown in Figure 2. The Boolean function FI , implemented at the same output of the logic matrix, describes images of class I, to which images No. 1, 2, and 3 belong.
FI y44 y45 y54 y55 .
Herewith FI 1 , if elements y44 and y45 have a level of L 5 (black color), which corresponds to a logic-1 level, and at least one of the elements y54 or y55 has the same level (6).
The Boolean function FII describes images of class II, to which images No. 4, 5, 6, 7, 8, and 9 belong.
FII y44 y45 y54 y55 y56 y65 y64 y66 y46 .
Herewith FII 1 , if all elements involved in conjunction have a level of L 5 , and at least one of the corner elements of the target image (No.4), i.e. one of the disjunctive members in brackets had the same level L 5 (7).
The Boolean function FIII describes images of class III, to which images No.10, 11, 12, 13, 14, and 15 belong.
FIII y44 y45 y46 y54 y55 y56 y57 y64 y65 y66 y67 y75 y76 y47 y74 y77 . (8) Disjunctive terms of expression (8) correspond to the corner elements of the target image No. 10. (6) (7)
The Boolean function FIV describes images of class IV, to which images No. 16 – No. 25 belong.
FIV y44 y45 y46 y54 y55 y56 y57 y64 y65 y66 y67 y68 y75 y76 y77 y74 y84 y85 y87 y88 y78 y47 y48 y58 . (9) When forming the FI FIV functions, we made assumptions about the optional presence of all corner elements of the target images in the defect image capture zones. This is consistent with control practice.
The analysis of expressions (6) – (9) shows that the patterns described by them are sequentially nested one into another, namely FI is included in FII , FII – in FIII , FIII – in FIV . Thus, when fixing an image of a senior class, for example No. 16, logical unit will be present not only at the output FIV of the logical matrix, but also at outputs FIII , FII , and FI . The combinational circuit shown in Figure 3 helps to eliminate this effect. The circuit uses inverters and four–input conjunctors and provides separate fixation of each class of image. Herewith, the appearance of a logical unit at one of the outputs DI DIV of the combinational circuit indicates the capture of the defect image of the corresponding class. The proposed system of the defect area estimation allows monitoring both in the manual scanning mode with indication of the defect magnitude by sound or light signals and automating the testing process. In the latter case, when scanning the product along parallel paths with a given step, it is easy to automatically register the area.
To simulate the operation of a neural network, we developed the program functioning in the Matlab environment. Network training was carried out sequentially in three stages. 1. First, a clean image No. 1 located in the lower-left corner of the matrix is fed to the network input. This image then moves sequentially along the rows of the matrix within i 1 ... 9 , j 1 ... 9 . Here i and j are the horizontal and vertical coordinates of the matrix, respectively. The specified scan coordinates describe the movement of the lower-left cell of the image. 2. At the second stage, a clean image No.1 ( 0 ) was again fed to the network input and it was moved along the matrix as described in paragraph 1. Then, image No. 1, additively mixed with noise corresponding to 10% , was fed to the network input, and this noisy image was moved along the matrix. We repeated this process sequentially for all remaining noise levels, up to 31% . It should be noted that this scanning process was repeated 100 times for each value for the noise intensity. 3. After completing the second stage of training, in order to verify the absence of retraining effect, we repeated paragraph 1.
The network training process described above we fulfilled with each pattern from the training set. Table 3 shows the data on the image classes used for training.
Figure 4 shows a graph of the dependence of the training time, relative to one pattern on the area of the black spot S, reflecting the area of the defect. But at the same time, characterized training on clean images, without noise. Training on noisy images is carried out after the training on clean images. And here the training time relative to one pattern is significantly less. So, for images of the first type, 1 0.045 . For images of the second, third and fourth types, we have: 2 0.027 , 3 0.032 , and 4 0.021 . Thus, we can state that the network training time at the second stage, assigned to one noisy image, is ten times less than the corresponding time spent at the first stage when training the network at the clean image.
The phenomenon of retraining the network was not observed.
Network testing was carried out by moving images from the set described above (No. 1 – No. 25) with noise levels from Table 2 along with the matrix. Testing results showed the following.
If testing was carried out after a full cycle of network training on patterns of one class using then a noisy image with 31% from the same class, then the test error did not exceed 3%. This result is valid for all four classes.
When combining images of classes I and II, i.e. with the expansion of the defect spot area from S 3 to S 9 , the testing error on noise patterns ( 31% ) increased to 8%. When combining all the images of I-IV classes in a single training set, the maximum testing error increased to 12%. And in the first and second cases, the parameter β increased significantly, its value became comparable with α. The reduction of the noise level during testing significantly decreases this error. So, with a reduction in from 31 to 27%, the test error decreases by 40-50%, from 31 to 22% by 60-70% and so on.
The above results were obtained by using the Elman neural network as a replicator with nin nout 100 , nh 200 and the number of layers l 1 . An increase in the number of neurons in the hidden layer to 400 on average by 25% reduces the testing error, however, the network training time increased by 30%. A network with two hidden layers, while reducing the test error by 20%, trained 4 times longer than a network with one hidden layer. Training a network with three hidden layers took 14 times more time than the single-layer network we used.
We investigated the Elman neural network as an identity map (replicator) for the task of identifying and estimating the area of a delamination in composite materials in the process of ultrasonic testing. Twenty-five images were used as training patterns. They were divided into four classes, according to the size of their defect area. Deterministic distortions caused by the nature of the testing object were introduced into the images of each class. Each image was moved step by step on a square matrix consisting of 100 cells, which represents the registration area of the ultrasound transducer. Images were additively mixed with white Gaussian noise, the intensity of which varied from 10 to 31%. At each scan point, the noise was superimposed 100 times.
The process of training the network began with presenting it with the clean (without any noise) patterns. In the process of training the network, it was found that the training time with clean patterns, assigned to one pattern fixed in a given position inside the matrix, increases with increasing the pattern area. Network training using noisy patterns was carried out after training with clean patterns. In this case, the training time, defined in the same way, turned out to be ten times shorter.
Network testing has provided the following. If testing was carried out after a full cycle of network training on patterns of one class using a test pattern with a noise level of 31%, then the test error did not exceed 3% for any class. When combining images of classes I and II, i.e. with the expansion of the defect spot area from S 3 to S 9 , the testing error increased to 8%. When combining the images of all four classes in a single training set, i.e. with the expansion of the defect spot area from S 3 to S 25 , the maximum testing error increased to 12%. The reduction of the noise level during testing significantly decreases this error. So, with a reduction in from 31 to 27%, the test error decreases by 40-50%, from 31 to 22% by 60-70% and so on.
These results were obtained by modeling the Elman’s neural network with the number of input and output neurons nin nout 100 and with the number of neurons in one hidden layer nh 200 . This network structure turned out to be optimal by the criterion of error testing/training time.
To estimate the delamination area, four nested defect spot capture zones were implemented in the central part of the matrix. For this purpose, a cascade-connected logical matrix and the combinational circuit having four outputs by the number of classes of identifiable images are connected to the output of the neural network.
Estimation of the defect area serves as the basic for determining the residual strength of a given unit of a product and making a decision on the possibility of its further operation.