=Paper=
{{Paper
|id=Vol-2416/paper5
|storemode=property
|title=The electrocardiogram signal morphology analysis based on convolutional neural network
|pdfUrl=https://ceur-ws.org/Vol-2416/paper5.pdf
|volume=Vol-2416
|authors=Mikhail Zavoyskih,Alexander Korobeynikov,Aleksey Menlitdinov,Vladislav Lyuminarskiy,Yurij Kuzelin
}}
==The electrocardiogram signal morphology analysis based on convolutional neural network ==
https://ceur-ws.org/Vol-2416/paper5.pdf
The electrocardiogram signal morphology analysis based on
convolutional neural network
М Zavoyskih1, A Korobeynikov1,2, A Menlitdinov2, V Lyuminarskiy1 and
Yu Kuzelin3
1
Kalashnikov Izhevsk State Technical University, Studencheskaya 7, Izhevsk, Russia, 426069
2
IzhTeleMed Ltd., Lenina, 110, Izhevsk, Russia, 426009
3
Republican Clinical Diagnostic Center, Lenina 87B, Izhevsk, Russia, 426009
e-mail: grizz8575@gmail.com, kav33@inbox.ru
Abstract. The analysis of electrocardiogram signal morphology based on
convolutional neural network is considered. Input data is obtained by splitting the
signal into cardiac cycles. The calculation the average cycle is performed to exclude
the artefacts. The Haar wavelet transform of the average cycle is performed. The
images of size 200x6 are input data for the recognition system: 200 – number of
counts constituting the cycle; 6 – number of Haar transform time scales. This work is a
reconsideration of the previous work of the authors. The training samples base of
marked cardiac cycle segments is the same (1500 cycles): the average cycle and the
segment’s start and end indexes. In the previous work, the original composite system
consisting of several modules was used as a recognition system. In current work it is
proposed to use the convolutional neural network with the special structure: 4
convolutional layers, 2 dense layers, and 200 outputs for every of 3 segment. The
recognition system based on neural network showed results slightly superior to the
previous system. The percent of acceptable localization of the segments is the
following: P – 82.2%, QRS – 88.7%, and T – 85.4%. The proposed system effectively
solves the problem using the standard modules of modern artificial neural networks.
1. Introduction
Among the many methods of research, observation and monitoring of human condition, the leading
place belongs to electrocardiography (ECG). The need to monitor the ECG during anesthesia,
resuscitation, intensive care and diagnosis is beyond all doubt. The ECG signal carries a large amount
of information, and a detailed automatic analysis of the patient's ECG signal allows to timely generate
alarms that prevent death, as well as to make prognostic conclusions.
When processing the ECG signal, taking the advantages of modern computing resources, it is
required to automatically analyze the shape of the ECG signal cycle (cardiocycle), i.e. automatically
recognize the characteristic segments and determine their time and amplitude parameters. [1]
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019)
Data Science
М Zavoyskih, A Korobeynikov, A Menlitdinov, V Lyuminarskiy and Yu Kuzelin
An example of the ECG signal cardiac cycle shape and it’s characteristic points is shown in Fig. 1.
It is common in cardiology to analyze the segments, i.e. the regions between the characteristic points:
the segment of P (points P0… P1), QRS segment of a point (Q...S), and the segment T (point T0… T1).
The morphology analysis input receives information obtained at the stage of splitting the ECG signal
into cycles: an array of cardiocycle signals accumulated over a some period of time. [2-3]
Figure 1. The average shape of the cardiac cycle.
Figure 2. The software used to obtain the training sample base of marked cardiac cycles segments.
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 35
Data Science
М Zavoyskih, A Korobeynikov, A Menlitdinov, V Lyuminarskiy and Yu Kuzelin
Sample base
of cardiocycles
Manual markup
segments MHAT wavelet tranform
of cardiocycles
Segments points
Results of MHAT
of manual
wavelet tranform
localization
Training Application
Calculation of statistical Calculation of statistical
parameters parameters of all
samples
Statistical
Statistical
parameters of
parameters of all
segments
samples
Clustering of cardiocycle
segments Calculating of fuzzy
variables of classes
Array of segments
classes The values of fuzzy
variable of classes
Calculation of fuzzy and samples
characteristics
parameters of classes Calculation of the
conformity degree of
classes and samples
Array of fuzzy
characteristics The conformity
parameters of degree of classes
classes and samples
Determination of the
maximum conformity
degree
Time and Class number
Calculation of time and
amplitude and segments
amplitude parameters
parameters points localization
Figure 3. The stages of the previous version of the cardiac cycle morphology analysis.
2. The previous version of the cardiac cycle morphology analysis
Based on the fact that the normal cardiocycle shape changes slowly over time and each current
cardiocycle can be distorted by artifacts and noises, it makes sense to recognize the characteristic
segments of average cardiocycle shape. The average cardiocycle of several (5-20) cardiocycles having
a high correlation coefficient with each other is calculated. Then the selected cardiocycles are
averaged pointwise. An example of the obtained signal is shown in Fig. 1 [2]. In the following stages,
the shape (characteristic points and segments) of the cardiocycle is recognized.
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 36
Data Science
М Zavoyskih, A Korobeynikov, A Menlitdinov, V Lyuminarskiy and Yu Kuzelin
The training of the recognition system is performed on the basis of the information provided after
the manual cardiocycle marking, i.e. the localization of the start and end indexes of the segments P,
QRS, and T is known. Such information can be formed using specially developed software. Marking
of cardiocycles segments should be made by a medical specialist in this area. The screenshot example
of the software developed by LLC Izhtelemed for performing of such marking is presented on Fig. 2.
[4]
In [2] it was proposed to use a special algorithm based on the pattern recognition of the ECG signal
(Fig. 3.). An original composite system consisting of several modules was developed.
Sample base
of cardiocycles
Manual markup
segments Haar wavelet tranform
of cardiocycles
Segments points
Results of Haar
of manual
wavelet tranform
localization
Training Application
Training Application
of neural network of neural network
Values of weight
coefficients
Neural network
Time and Calculation of time
Segments points
amplitude and amplitude
localization
parameters parameters
Figure 4. The stages of the cardiac cycle morphology analysis based on neural network.
3. The cardiac cycle morphology analysis based on neural network
In this paper, a reconsideration of the approach based on the pattern recognition of the cardiocycle
morphology is carried out, and it is proposed to use an artificial neural network (ANN) as a
recognition system of cardiocycle segments. Fig. 4 presents the stages of morphology analysis based
on convolutional ANN.
Currently, most studies on ECG analysis algorithms use wavelet transform [5-7]. In the course of the
study in [2] it was found that the wavelet transform application of the signal simplifies the ECG
cardiac cycle segmentation. In this paper, in contrast to the previous work [2], the Haar wavelet
transform of the average cardiocycle was chosen due to the simplicity of its implementation [8]:
𝑥 𝑘2𝑖 −𝑥 𝑘2𝑖+1 𝑥 𝑘2𝑖 +𝑥 𝑘2𝑖+1
𝑑𝑘+1𝑖 = 2
, 𝑥 𝑘+1𝑖 = 2
, (1)
where x is the signal for which the wavelet transform of the next time scale is calculated; d is the result
of the wavelet transform calculation; k is the time scale number; i is the signal point index.
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 37
Data Science
М Zavoyskih, A Korobeynikov, A Menlitdinov, V Lyuminarskiy and Yu Kuzelin
As a result of wavelet transform of each cardiocycle at different time scales, a matrix of half-
differences is obtained based on which the image is formed. This image then will be fed to the ANN
input layer. A total of 6 time scales of transformation is used, an example is shown on Fig. 7.
At the moment, most researchers in signal, speech and image recognition algorithms use a neural
network [9-13]. Some researchers combine wavelets and neural network [14-15].
To solve the problem of cardiocycle segments localization based on the wavelet transform results, the
apparatus of convolutional ANN (which have been successfully used for image processing in recent years)
was chosen.
The first convolutional ANN (which borrowed for computer science the ideas embodied by nature in
the visual cortex) was the ANN of Kunihiko Fukushima (Neocognitron, 1975-1980) [16]. The
convolutional network in already quite modern form appeared in the works of Yann LeCun (LeNet, 1989).
[9]
The input data for the ANN in this paper are the Haar wavelet transform results performed on the
average cardiocycle, presented in the form of images of size 200x6, where 200 – the number of counts
constituting the cardiocycle (1 second), 6 – the number of time scales of the Haar wavelet transform.
The base of the cardiac cycle segments recognition system are the modules of the convolutional
ANN. The special structure of the developed ANN is shown on Fig. 5.
The main modules of the proposed ANN structure are convolutional layers. To avoid the network
overfitting problem to the ANN it is added the layers of random transmission shutoffs of the
particular outputs (dropout) with a probability of outputs disconnection equal to 25%.
The last two layers of ANN are dense (fully connected) layers. A formed structure of the ANN
outputs based on the requirements of the solved problem is following: an array of 200 outputs for each
segment (P, QRS, T), i.e. total of 600 outputs.
Figure 5. The structure of proposed convolutional artificial neural network.
4. Experiments
To train the recognition system, it is necessary to form a correct training sample base. Each training
example should include the following: the image obtained as a result of the Haar wavelet transform of
the average cardiocycle, as well as the coordinates of the start and end indexes of the segments (P,
QRS, T) which was marked by the expert.
Before training the ANN, it was necessary to modify the training sample to bring the format of
target responses for each training example in accordance with the ANN outputs. On the basis of a set
of 6 coordinates of 3 segments from each cardiac cycle allocated by the cardiologist 3 array of outputs
target values for each segment separately are formed according to the following formula (2):
1, 𝑖 ∈ [𝑖 𝑚0 , 𝑖 𝑚1 ]
𝑡 𝑚𝑖 = � , (2)
0, 𝑖 ∉ [𝑖 𝑚0 , 𝑖 𝑚1 ]
m
where i = {1...200} is the ANN’s output index and the coordinate on the average cardiac cycle; t i is
the target output value; m is the segment number; im0 and im1 is the indexes of the start and end of the
segment marked by the expert.
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 38
Data Science
М Zavoyskih, A Korobeynikov, A Menlitdinov, V Lyuminarskiy and Yu Kuzelin
After obtaining a set of outputs for each segment by using ANN it is necessary to interpret them.
Each training example after transformation by formula (2) contain an image of the wavelet transform
result of the average cardiocycle and an array of 600 values, the localization of each segment in which
will be allocated units (Fig. 7).
The mean square error (MSE) is chosen as the optimization criterion in this work. The ANN’s
output MSE is the difference between the desired result (target) and the actual output. In the process of
ANN fitting the regression problem is solved, i.e. the values of all outputs obtained during the work of
ANN should match with all target outputs values in the training sample. To measure the recognition
quality we use the standard error function:
1 1
MSE = ∑ ∑ (𝑒 𝑚𝑖 )2 = ∑ ∑ (𝑡 𝑚𝑖 − 𝑦 𝑚𝑖 )2, (3)
𝑚⋅𝑖 𝑚 𝑖 𝑚⋅𝑖 𝑚 𝑖
m m
where e i is the error value of the segment’s m output i; y i is the output obtained by ANN (depends
on the ANN’s current weights values and the input example); tmi is the target output value.
The ANN learning process is reduced to correcting the ANN weights, so as to minimize the MSE.
The gradient descent method is generally used to minimize the MSE. In this work, an adaptive Adam
algorithm based on gradient descent with smoothed versions of the mean and standard gradients is
used to optimize MSE. [17]
Training samples base consisting of 1500 elements was used to fit the ANN. All samples were
divided into training and test subsamples with the ratio of 80% by 20%, i.e. training subsample
contains 1200 examples and the test subsample contains 300 examples.
The average period of the ANN’s training was 1 hour and 40 minutes on Lenovo G505s laptop
(CPU AMD A10-5750M, RAM 4 GB). The graph of the error function obtained during ANN training
is shown on Fig. 6, where the abscissa axis is the learning epoch number.
Training error displays the ANN’s fitting accuracy on a training subsample. However, it does not
give a full confidence that the trained model will also be working well on a new data that was not in
the training subsample. The model accuracy on the new data is the ANN generalizing ability. To
estimate the model generalization ability, the generalization error obtained on the test subsample was
used (Fig. 6).
a) the training subsample error
b) the generalization error
Figure 6. Changing an errors when training ANN.
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 39
Data Science
М Zavoyskih, A Korobeynikov, A Menlitdinov, V Lyuminarskiy and Yu Kuzelin
The ANN learning process is iterative. Each iteration is called an epoch. During one epoch, all the
examples from the training subsample are submitted to the ANN input. It is also possible to validate
the fitting error on the test subsample. As can be seen on the charts, the error function reaches a
minimum at epoch 19.
After the ANN training phase, it is possible to use the cardiac cycle segments recognition system
on unknown signals. During operation, as well as during training, the wavelet Haar transform of the
average cardiocycle is applied, and then the resulting image is fed to the ANN input. The ANN was
localized precisely enough the P and QRS segments on the EGG signal which neither was in the
training nor in the test subsample (Fig. 7). Segment T localization error could be the result of an
inaccurate marking by the expert.
a) the average cardiac cycle signal
b) the Haar wavelet transform result (ANN input)
c) the segments recognition result (ANN outputs)
Figure 7. The example of segments localization results.
To perform experiments with the ECG morphology analysis system based on the proposed ANN,
the Keras library was used – an open-source ANN library written in Python. It is capable of running
on top of the Deeplearning4j, TensorFlow, or Theano frameworks. In this work, Keras was used along
with TensorFlow – an open-source software library for machine learning. [18-19]
The Table 1 is showing the results of practical comparison of the proposed cardiocycle segments
recognition system (based on convolutional ANN) and the previous version of recognition system
proposed in [2] (pattern recognition algorithm) both carried out on the same signals sample (1500
samples). Column names mean the following:
a) elimit is the absolute difference between manual marking and automatic recognition of a segment
coordinate, the error expressed in counts;
b) point is the name of the cardiac cycle characteristic points: P0, P1, Q, S, T0, T1;
c) % is percentage of samples satisfying the value elimit.
According to the clinical requirements for the quality of the algorithms results for the cardiocycles
segments localization on real signals, a maximum error of no more than 40 MS is allowed. In our case,
when the quantization frequency is of 200 Hz, maximum allowable error is equal 8 counts.
According to these requirements, the recognition system based on ANN with a maximum allowable
error of 8 counts showed satisfactory results for the following segments (rows in table 1 marked are
grayed out): P – 82.18 % of samples, QRS – 88.74 % of samples, and T – 85.36 % of samples. The
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 40
Data Science
М Zavoyskih, A Korobeynikov, A Menlitdinov, V Lyuminarskiy and Yu Kuzelin
system based on the pattern recognition proposed in [2] showed a satisfactory results for the following
segments: P – 81.44 % of samples, QRS – 89.20 % of samples, and T – 80.50 % of samples.
Table 1. The results of the recognition systems comparison.
%
elimit point pattern convolutional
recognition [2] ANN
2 P0 39.77 34.15
2 P1 41.59 39.31
2 Q 58.76 46.25
2 S 60.64 57.18
2 T0 43.46 45.69
2 T1 43.62 41.27
4 P0 58.33 53.28
4 P1 60.89 56.34
4 Q 76.76 73.56
4 S 77.93 68.93
4 T0 64.58 62.80
4 T1 61.58 60.56
8 P0 81.44 84.93
8 P1 83.35 82.18
8 Q 90.38 90.82
8 S 89.20 88.74
8 T0 84.65 86.15
8 T1 80.50 85.36
16 P0 93.54 96.93
16 P1 93.95 92.99
16 Q 95.31 94.53
16 S 95.31 95.18
16 T0 94.56 93.62
16 T1 94.15 92.23
5. Conclusion
The system based on convolutional ANN proposed in this paper showed results slightly superior to the
previous system based on pattern recognition [2] and similar to the work [20].
The proposed system effectively solves the problem used the standard modules of modern
convolutional ANN, which simplifies the development of signal and image analysis systems.
The obtained results should be considered as satisfactory given that the training sample was formed
mainly on the basis of the ECG from the functional diagnosis cardiological department with large
percentage of it were pathological ECG [3].
In the future researches it is possible to modify the developed ANN structure by adding new layers
to ANN. It is also necessary to increase the variety and volume of training sample signals. Thus, it is
possible to improve the recognition system and use it in a clinical practice.
6. References
[1] Velic M, Padavic I. and Car S 2013 Computer aided ECG analysis - State of the art and
upcoming challenges IEEE EUROCON International conference on computer as a tool
DOI:10.1109/EUROCON.2013.6625218
[2] Korobeynikov A V 2004 Algorithms and software of computer monitoring systems for analysis of
electrocardiograms morphology and rhythm (Doctoral dissertation, ISTU, Izhevsk, Russia)
[3] Kalyadin N I, Lemenkov V A, Korobeinikov A V, Perevozchikov S M, Odiyankov E G,
Rodionov A N and Kotkov S M 2002 Development and Clinical Experience with a Computer
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 41
Data Science
М Zavoyskih, A Korobeynikov, A Menlitdinov, V Lyuminarskiy and Yu Kuzelin
Monitoring System Used at the Intensive Cardiological Care Unit. Biomedical Engineering 36(1)
44-49 DOI: 10.1023/a:1015417424098
[4] IzhTeleMed Ltd. (n.d.). URL: http://izhtelemed.ru/
[5] Li W 2019 Wavelets for Electrocardiogram: Overview and Taxonomy IEEE Access 7 25627-
25649 DOI: 10.1109/access.2018.2877793
[6] Lannoy G D, Frenay B, Verleysen M and Delbeke J 2009 Supervised ECG Delineation Using
the Wavelet Transform and Hidden Markov Models IFMBE Proceedings 4th European
Conference of the International Federation for Medical and Biological Engineering 22-25 DOI:
10.1007/978-3-540-89208-3_7
[7] Zaniol C, Varriale M C and Manica E 2018 Apnea Recognition with Wavelet Neural Networks
TEMA (São Carlos) 19(2) 277 DOI: 10.5540/tema.2018.019.02.277
[8] Mallat S G, Peyré G 2009 A wavelet tour of signal processing: The sparse way (Amsterdam:
Academic Press)
[9] LeCun A and Bengio Y 1995 Convolutional Networks for Images, Speech, and Time-Series, in
Arbib (The Handbook of Brain Theory and Neural Networks, MIT Press)
[10] Özbay Y, Ceylan R and Karlik B 2006 A fuzzy clustering neural network architecture for
classification of ECG arrhythmias Computers in Biology and Medicine 36(4) 376-388 DOI:
10.1016/j.compbiomed.2005.01.006
[11] Ronneberger O, Fischer P and Brox T 2015 U-Net: Convolutional Networks for Biomedical
Image Segmentation Lecture Notes in Computer Science Medical Image Computing and
Computer-Assisted Intervention 234-241 DOI: 10.1007/978-3-319-24574-4_28
[12] Hinton G, Deng L, Yu D, Dahl G, Mohamed A, Jaitly N and Kingsbury B 2012 Deep Neural
Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research
Groups IEEE Signal Processing Magazine 29(6) 82-97 DOI: 10.1109/msp.2012.2205597
[13] Deng L, Hinton G and Kingsbury B 2013 New types of deep neural network learning for speech
recognition and related applications: An overview IEEE International Conference on Acoustics,
Speech and Signal Processing DOI: 10.1109/icassp.2013.6639344
[14] Efitorov A and Dolenko S 2018 A New Type of a Wavelet Neural Network Optical Memory
and Neural Networks 27(3) 152-160 DOI: 10.3103/s1060992x18030050
[15] Shoaib M, Shamseldin A Y, Melville B W and Khan M M 2016 Hybrid Wavelet Neural
Network Approach Artificial Neural Network Modelling. Studies in Computational Intelligence
628
[16] Wasserman P D 1989 Neural computing: Theory and practice (New York, NY: Van Nostrand
Reinhold)
[17] Goodfellow I, Bengio Y and Courville A 2017 Deep Learning (MIT Press) URL:
http://www.deeplearningbook.org
[18] TensorFlow (n.d.) URL: https://www.tensorflow.org/
[19] Pattanayak S 2017 Introduction to Deep-Learning Concepts and TensorFlow Pro Deep
Learning with TensorFlow 89-152 DOI: 10.1007/978-1-4842-3096-1_2
[20] Sampath A and Sumithira T 2016 ECG Morphological Marking using Discrete Wavelet
Transform Intelligent Decision Technologies 10(4) 373-383 DOI: 10.3233/idt-160264
V International Conference on "Information Technology and Nanotechnology" (ITNT-2019) 42