=Paper=
{{Paper
|id=Vol-2258/paper21
|storemode=property
|title=The Use of Neural Networks for Testing and Failure Analysis of Electronic Devices
|pdfUrl=https://ceur-ws.org/Vol-2258/paper21.pdf
|volume=Vol-2258
|authors=Roman Girin,Sergey Orlov
}}
==The Use of Neural Networks for Testing and Failure Analysis of Electronic Devices==
https://ceur-ws.org/Vol-2258/paper21.pdf
The use of neural networks for testing and failure analysis of
electronic devices
R V Girin1 and S P Orlov1
1
Institute of Automatic and Information Technologies, Samara State Technical University,
Samara, 443100, Russia
Abstract. The paper deals the problem of remote technical diagnostics of complex objects.
This allows detecting failures not only during testing of equipment, but also during operation.
One effective way of non-contact measurement of technical state of the test object is to obtain a
thermogram their surface. The article describes the structure of the information-measuring
system, which includes a measuring channel with a thermal imager. Method thermography
analysis using the comparison of measured data with the calculated temperature fields is
proposed. To solve the incorrectness of the inverse heat conduction problem, it was suggested
to use a two-branch artificial neural network. The structure of such a convolutional neural
network is described. The influence of neural network parameters on the quality of detection of
defective thermograms was studied. It is shown that such a technique allows increasing the
detection accuracy of failures and defects of electronic devices during remote monitoring.
1. Introduction
Technical diagnostics of complex objects develops in the direction of operational control of technical
states using remote measuring instruments. This allows detecting failures not only during testing of
equipment, but also during operation. In many cases, controlled technical objects have a set of
measurable parameters that can identify their technical state [1]. However, it is not always possible to
build measurement channels for continuous monitoring of these parameters. Often, access to
measurement is impossible due to closed design, restrictions on the weight and volume of the object to
be monitored. Furthermore, the measuring procedure may contributions distortion in the object
operation process. In this connection, the methods of contactless remote monitoring with the help of
infrared thermography are promising [2, 3].
In this case, a channel for remote measurement of the temperature field of the object should be
organized, including a thermal imager, a thermogram processing unit and a temperature field analysis
unit, in which a decision is made on the operability of the object, and the failure facts are
differentiated.
The problem of technical diagnostics using thermograms is as follows:
The set of obtained thermograms of the object and the set of inoperative states do not have a
one-to-one correspondence. Different faults and inoperative states can correspond to the same
temperature distribution on the object surface. Essentially, we are incorrect inverse heat
conduction problem, which is necessary to find the location and intensity of the internal heat
sources in the test object.
A large number of thermograms that are to be analyzed leads to complex algorithms for
selecting the desired thermal images of the object.
160
In the report, to solve this problem, it proposed to use a hybrid intellectual model based on a
convolutional neural network and a fully connected neural network.
2. The information-measuring system for remote technical diagnostics
In [4, 5], an information-measuring system IMS for remote monitoring of electronic devices in ground
tests have been described. Figure 1 shows a block diagram of a system that consists of measuring
channels:
Determining environmental parameters via Termohigrometr Poly MI 6401;
Measuring of the temperature field of the surface based on the thermal imaging NEC R500;
Measuring of the device electrical parameters using a digital oscilloscope GDS-2104.
Control of test modes of devices is carried out using a computer and SPS-3610 and FPGA
XC3S500E units.
The main idea is based on a set of mathematical models of the thermal state of the device
corresponding to the failures in it. Classes, corresponding to different defects in the electronic device,
differentiate these models. In this case, the decision on the operability of the electronic device was
made by comparing the measured thermogram with the set of calculated thermograms obtained with
the help of mathematical models. However, as noted above, it was not always possible to classify the
inoperative state due to the incorrectness of the inverse heat conduction problem.
Figure 1. Structure of information-measuring system for electronic devices testing.
To solve this problem, the development of this approach with the use of intelligent tools for
measuring information analysis is proposed.
161
3. Hybrid intelligent model
It is known that the artificial neural network ANN effectively recognize complex images [6, 7]. There
are a large number of ANN applications for analyzing information under uncertainty [8, 9]. In this
report, we propose an approach to the regularization of incorrect inverse problem of recognition of
diagnostic images, which is based on the combined use of two neural networks.
For each type of technical object, mathematical models of the thermal state of its surface are
developed depending on the behaviour of internal heat sources:
Ti M Fi ( x, y),
when i I ; I an index set of technical states corresponding to various defects and failures.
The developed mathematical models of the thermal state and history of previous tests of objects are
used to form the fact base and the rule base that are components of the knowledge base. They are used
in the expert system with a forward chaining on the production rules.
Thermal states of technical objects are characterized by thermograms obtained with the thermal
imager T j ( x, y), j 1, J , J number of thermograms in the knowledge base of IMS. Standing heat
sources Qi ( x, y ), i 1, I , I number of sources inside the object is used to determine the object's
performance. We define the set of states Dn ( x, y ) , n 0 ,N , in which a technical object can be
located (one of failures or a working state). In this case, we assume that to all serviceable states there
will correspond one state D0 . Figure 2 shows a graph model of the technical object states.
Figure 2. Graph model of the technical object states.
In general, we have incorrect inverse problem, since the same surface thermograms may
correspond to multiple heat sources states. Consequently, the one-to-one mapping of thermograms and
inoperative states of the object is violated. We use an additional vector of measured process
parameters, where Vm , m 1, M , M is the number of parameters monitored by measuring means.
This may be, for example, input and output voltages of the electrical signals, supply voltages,
frequency, and phase of the signals, and others. We carry out a regularization of the problem using
more information on the connection of electrical parameters with thermal conditions. Thus, the inverse
problem becomes correct by narrowing the infinite set of solutions to finite compact sets
corresponding to the chosen defects.
We require the following condition: for any pair of classes T l and T k , l , k 1, 2,..., N , l k ,
there is at least one pair of elements Tjl Ti k , j Jl , i J k or Vml Vrk , m M l , r M k .
162
Consequently, the separation of thermograms with the help of the vector V into subsets
T ( x, y,V ) Dn puts them in a one-to-one correspondence with the inoperative states of the
n
controlled object. Thus, the set of classes is formed: T n (T jn ,Vmn ), j J n ; m M n , n 0, N ,
where Jn and Mn - index sets of thermograms and vectors of the object parameters included in the
n-th class (Figure 3).
Figure 3. Separation signs of inoperability classes.
The implementation of the proposed approach is to build a convolutional neural network ANN 1,
which is trained on a set of calculated thermograms, obtained using mathematical models. Another
network ANN 2 is a fully connected neural network that processes the vector of additional parameters.
In figure 4 the structure of the hybrid intellectual model is shown.
Figure 4. Structure of the hybrid intellectual model.
4. A measuring channel with a convolutional neural network
The thermal imager and the convolutional neural network ANN 1 form the main measuring channel.
The organization of the convolutional network is based on the Y. LeCun approach [6] also combines
some feature which were introduced in [10, 11] and is presented in figure 5. The network uses batch
normalization, which was first introduced in [12].
163
Figure 5. Neural network with two branches used in the experiments.
Proposed artificial neural network consists of two branches. The main branch is well known
convolutional neural network. It consists of several convolutional layers united in feed-forward neural
network. Thermograms are used as input for this branch.
Thermograms as any other image can be presented as an 3D array in which every value represents
colour of the corresponding pixel. The width and height of the array is the same as width and height of
the image (in pixels) and the depth of the array is 3, one for each of three (red, green, blue) channels.
The size of thermograms used in our experiments were 30x30 pixels.
The first convolution layer’s receptive field size is 5x5, padding is 3 and stride 1 pixel. The number
of feature maps produced with the layer is equal to 6. The output of the first convolution layer is fed to
max pooling layer with receptive field 2x2 and stride 2. This layer’s output conveyed to the input of
another convolutional layer which receptive field size is 5x5 and the stride is equal to 1. At this layer
number of produced feature maps is 16. After the layer max pooling layer is used. The receptive field
of the layer is 2x2 and stride is equal to 2. This reduces size of the feature maps with factor of 2. The
output is fed to the third convolutional layer with receptive field of 6x6. Because size of the feature
maps used as input for the layer is also 6x6 this layer can be considered as fully-connected. The output
of this layer is vector which length is 120. Output of this convolutional layer is output of the whole
first branch.
As easy to see, parameters of the convolutional layers, such as receptive field, stride and padding,
were chosen so that with application to the given size of thermograms output of the last convolution
layer is a vector.
In order to take into account signals from the controlled unit’s build-in sensors an auxiliary branch
of ANN was introduced. This branch consists of fully connected neurons layer. Input for the branch is
a vector with normalized data from the sensors which length is equal to 6. Output of the layer is also a
vector of length 2. The vector is merged with output vector of the main branch and result vector gotten
after the merge is passed in as input for neurons layer that performs softmax or two-staged linear [13]
normalization. This layer is the output layer of the whole ANN and performs categorization of failure
in controlled unit.
ANN was used for categorization of four major and critical failures in controlled device. Therefore,
output layer contains five neurons (one per each failure category and one corresponds to the normal
condition of the unit).
Additional details of the network, such as number of trainable params per layer and some
additional information, is presented in table 1.
164
Table 1. Neural network’s summary.
Input Recept Output Depth
Feature
Type height/ ive Stride Padding height/ Neurons per Weights Params
maps
width field width layer
the main branch
conv 30 5 1 3 32 6144 1 6 25x6 156
max
pool 32 2 2 0 16 1 6
conv 16 5 1 0 12 2304 6 16 150x16 2416
max
pool 12 2 2 0 6 1 16
conv 6 6 6 0 1 120 16 120 576x120 69240
the auxiliary branch
fc 2 6x2 14
the common part of neural network
Y-junc-
tion
fc 3 122x5 615
Total Total
8570 71826
Some recent papers [14] consider in details application of some architectures that allow emulating
sparsity in connections in neural networks. In our experiments we didn’t use some of those techniques
and used sparsity explicitly in similar manner as it was used in [6], retrieving different sub-set of
feature maps for input of the third convolution layer. And as for neurons in convolutional layer each of
them connected to the input within its receptive field.
Considering the fact that the main branch of our neural network is widely used convolutional
network the weights for the layers can be initialized with weights of some pre-trained convolutional
network which parameters compatible with our network. Although we didn’t use this approach and
initialized all the parameters with random value from range (-2, 2) and trained the network from
scratch. But using pre-trained weights can reduce the training time in cases when it’s important.
In our experiments we trained our network using back-propagation technique [15] with learning
rate 0.001 using 100 epochs. Achieved precision of thee network is 99%. For training we used model
thermograms generated programmatically for each of classified failure and for case when controlled
unit is operating normally.
Although traditionally neural networks are trained on dataset that comprises subset of samples
which network will process during its exploitation we trained the network on model thermograms, not
on thermograms taken from some controlled unit. In our case we considered the problem from
metrological point of view. Similar to any other metrological instrument it first passes its calibration
on standard data in laboratory and then it is used in field. In similar manner we collected network's
training dataset from thermograms that were generated programmatically based on mathematical
model of surface temperature field of our control device. In practice even thermograms of two unit of
the same model can differ slightly. By mean of using model thermograms we reduce influence of such
variations.
Some sample thermograms used in network’s training are shown on figure 6.
165
Figure 6. Some of the thermograms used in the experiments (best viewed in electronic version).
Figure 7. Dependency of networks’ precision from number of parameters.
In addition to the experiments described above also some comparisons of precision of networks
that differ in number of their params were made. Using conceptually the same network we vary
number of used feature maps in convolutional layers and number of neurons in fully-connected layers.
So that we achieved several networks with similar architecture but with different number of params.
After we have trained each of them on the same dataset with thermogram we compared their precision.
The discovered dependency of the networks’ precision from number of their params is shown on
Figure 7. The dependency is pretty much what would be expected to get: precision increases with
number of params up to the certain point and after that point network tend to get overfit and precision
decreases. In spite of that the dependency well known in case of metrology application this
information can be used for sizing network by precision. It is essential that with number of parameters
computation cost of feeding forward signal through network increases. In some cases calculation
expensive network is not an option but refusing some amount of precision is allowed, so using
diagrams similar to the one on figure 7 network sizing to some particular application can be
performed.
5. Conclusion
Described approach can be easily generalized to application with wide range of units which technical
state is controlled via thermograms and some auxiliary data. Using artificial neural network for
interpretation thermograms and signals has many advantages. One of them is that even to unit of the
same model can in practice have slightly different thermograms that corresponds to their state.
Comparing (which is implicitly done during signal processing in network) thermograms of such unit
with just canonical model sample not gives as good results as comparing thermograms taken from this
particular instance of unit. And the latter is easily achievable with network fine-tuning on a given
166
controlled unit. This kind of fine-tuning is nothing more like the same training with back-propagation
but on thermograms from the given instance of controlled unit.
And the architecture of network can be easily corrected to take into account some additional signals
as long as training dataset for the network is available.
6. References
[1] Schklyar V N 2009 Reliability of control systems (Tomsk: Tomsk Polytechnical University
Press) p 126
[2] Afonin A V, Newport R K, Polyakov V S, Sergeev S S and Tadjibaev A I 2000 Infrared
thermography in power engineering Fundamentals of infrared thermography vol 1 (Saint-
Petersburg: PEIEK) p 240
[3] Uvaysov S U and Yurkov N K 2012 Bull. Samara univ. Aerospace engineering, technology and
engineering 7 pp 16-22
[4] Orlov S P and Akhpolova E A 2016 Technical diagnostics of electronic blocks on thermal fields
of elements Proc. Int. Conf. Perspective information technologies vol 1 ed S Prokhorov
(Samara: Samara Scientific Center of RAS) pp 139-142
[5] Orlov S P and Vasilchenko A N 2016 Intelligent measuring system for testing and failure
analysis of electronic devices Proc. 19th IEEE Int. Conf. on Soft Computing and Measurements
(Saint-Petersburg) vol 1 pp 401-403
[6] LeCun Y, Bottou L, Bengio Y and Haffner P 1998 Gradient-based learning applied to document
recognition Proc. IEEE vol 86 issue 11 (IEEE Press) pp 2278-2324
[7] Haykin S 2009 Neural networks and learning machines 3rd ed. (Pearson Prentice Hall) p 905
[8] Norvig P and Rassell S 2010 Artificial intelligence: A modern approach, 3rd ed. (Pearson
Prentice Hall) p 1109
[9] Nielsen M 2017 Neural Networks and Deep Learning Free online book URL:
http://neuralnetworksanddeeplearning.com
[10] Krizhevsky A, Sutskever I and Hinton G E 2012 ImageNet Classification with Deep
Convolutional Neural Networks International Conference on Neural Information Processing
Systems pp 1097-1105
[11] Nair V and Hinton G 2010 Rectified Linear Units Improve Restricted Boltzmann Machines
Proc. of ICML 27 pp 807-814
[12] Ioffe S and Szegedy C 2015 Batch normalization: Accelerating deep network training reducing
internal covariate shift Cornell University Library, URL: https://arxiv.org/abs/1502.03167v3
[13] Girin R V and Orlov S P 2017 Two-stage normalization of output signals of artificial neural
networks Bull. Samara State Technical University. Technical Sciences vol 56 no 4 pp 7-16
[14] Szegedy C, Liu W, Jia Y, Sermanet P, Angelov D, Erhan D, Vanhoucke V and Rabinovich A
2015 Going deeper with convolutions Proc. IEEE Conf. on Computer Vision and Pattern
Recognition (CVPR) pp 1-9
[15] Rumelhart D E, Hinton G E and Williams R J 1986 Learning representation by back-
propagating errors Letters to Nature vol 323 pp 533-36
167