=Paper=
{{Paper
|id=Vol-3047/paper16
|storemode=property
|title=Application of convolution neural networks of the DeepLabv3 architecture for preprocessing of magnetic resonance imaging for the differential diagnosis of Parkinson’s disease and essential tremors
|pdfUrl=https://ceur-ws.org/Vol-3047/paper16.pdf
|volume=Vol-3047
|authors=Vladislav Abramov,Anna Alekhina,Mikhail Dorrer,Maria Tunik,Xenia Tutsenko,Alina Khoroshavina,Michael Sadovsky
}}
==Application of convolution neural networks of the DeepLabv3 architecture for preprocessing of magnetic resonance imaging for the differential diagnosis of Parkinson’s disease and essential tremors==
https://ceur-ws.org/Vol-3047/paper16.pdf
Application of Convolution Neural Networks of the DeepLabv3
Architecture for Preprocessing of Magnetic Resonance Imaging
for the Differential Diagnosis of Parkinson’s Disease and
Essential Tremors
Vladislav Abramov1 Anna Alekhina2, Mikhail Dorrer2, Maria Tunik3, Xenia Tutsenko3, Alina
Khoroshavina3 and Michael Sadovsky1,3,4
1
Siberian research & clinical center of FMBA of Russia, Krasnoyarsk, Russia
2
Reshetnev Siberian State University of Science and Technology, Krasnoyarsk, Russia
3
V.F. Voino-Yasenetsky Krasnoyarsk State Medical University, Krasnoyarsk, Russia
4
Institute of Computational Modeling of the Siberian Branch of the Russian Academy of Sciences,
Krasnoyarsk, Russia
Abstract
Preprocessing of data for implementing an artificial intel- ligence system solving the problem
of differential diagnosis of Parkin- son’s disease and essential tremor based on the data of
nuclear magnetic resonance imaging is discussed. A database of 500 images of magnetic
resonance imaging and positron emission tomography of patients with Parkinson’s disease,
essential tremor, and unclear diagnoses is used. For preprocessing a neural network of the
DeepLab v3 + architecture was used. The accuracy of preprocessing “raw” images reached
87 %. Furthermore, the proposed solution adequately highlights the area corresponding to the
brain tissue on the magnetic resonance tomography. Therefore, the solution provides a
feasible mechanism to prepare data for a neural network implemented for the differential
diagnosis of Parkinson’s disease and essential tremor.
Keywords 1
Parkinson’s disease, essential tremor, diagnostics, magnetic resonance imaging, computer
vision, convolution neural net-works
1. Introduction
Parkinson’s disease and essential tremor are among the most common extrapyramidal diseases in
clinical practice. It is imperative to differentiate these nosologies since the curation tactics, prognosis, and
patient compliance depend on the exact diagnostics.
Parkinson’s disease is currently the second most common neurodegenerative disease. Additionally,
significant growth of Parkinson’s disease is observed in the last three decades. Both environmental
and genetic factors contribute to the rise of the number of patients suffering from this disease [3].
Parkinson’s disease is a slowly progressive neurodegenerative disease which causes impaired motor
function with slow movements, tremors, and gait and balance disorders. Various non-motor symptoms
are common for Parkinson’s disease [27]. They include impaired autonomic function with orthostatic
SibDATA 2021: The 2nd Siberian Scientific Workshop on Data Analysis Technologies with Applications 2021, June 25, 2021,
Krasnoyarsk, Russia
EMAIL: mdorrer@mail.ru (M. Dorrer); tsuprikova.mary.maria@yandex.ru (M. Tunik); msad@icm.krasn.ru (M. Sadovsky)
ORCID: 0000−0002−4063−4951 (V. Abramov); 0000−0002−2063−2297 (A. Alekhina); 0000−0002−8598−8331 (M. Dorrer);
0000−0003−0194−1219 (M. Tunik); 0000−0003−3979−1172 (X. Tutsenko); 0000−0001−9732−7920 (A. Khoroshavina);
0000−0002−1807−0715 (M. Sadovsky)
© 2021 Copyright for this paper by its authors.
Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
CEUR Workshop Proceedings (CEUR-WS.org)
hypotension, constipation and urinary disturbances, various sleep disturbances, and a wide range of
neuropsychiatric symptoms [28].
Parkinson’s disease is detected primarily through clinical practice; however, it should be
confirmed by relevant lab studies. The gold standard for Parkinson’s disease diagnostics includes the
autopsy confirming the loss of neurons in the substantia nigra and the occurrence of Lewy bodies as
pathological markers of the disease [14]. However, there are still no methods free from failures to
diagnose Parkinson’s disease in vivo. It heavily complicates the early detection of Parkinson’s
disease and the differential diagnosis of Parkinson’s disease and essential tremor. Sometimes even
very experienced clinicians find it challenging to understand and correctly interpret similar symptoms
of these diseases.
Essential tremor has completely different pathogenesis. It is also among the most common
movement disorders affecting both children and adults. In 2018, essential tremor was defined as an
isolated action tremor in the upper limbs on both sides. Tremor can also be present in other body
parts, usually in the neck or vocal cords. Patients exhibiting additional neurological symptoms are
now referred to as essential tremor plus. Patients with essential tremors may exhibit these different
clinical symptoms at a later stage of the disease [19]. Extra clinical manifestations associated with this
malfunction include cognitive impairment, psychiatric disorders, and hearing loss [26]. Essential
tremor often occurs years or decades before other symptoms appear and progressively worsen.
Commonly, essential tremors are supposed to be a familial case which is more frequent as compared to
sporadic ones [25].
While patients with essential tremors are known to exhibit the tremor in activity, those with
Parkinson’s disease show mainly resting tremor. Some patients exhibit standard features making a
trouble in diagnostic for a physician [30]. If a patient has both essential tremor and Parkinson’s
disease symptoms, then it becomes even more difficult for clinicians to make reliable diagnostics. A
number of papers report that essential tremor can later be accompanied with Parkinson’s disease.
Several epidemiological studies were carried out to support the idea that the combination of essential
tremor and Parkinson’s disease is not just coincidental. In a population-based study conducted in
Spain, 3 % of patients with essential tremor later developed Parkinson’s disease compared with 0.4 % in
the control group; this supports the observation that patients with essential tremor have a higher risk
of the development of Parkinson’s disease [1]. In the study of 678 patients, 6.1 % of patients with
essential tremor had Parkinson’s disease [16], while another study of patients with essential tremor,
reports on 20.8 % with concomitant Parkinson’s disease [9].
The first investigation involving 600 patients shows higher incidence of essential tremor in patients
with Parkinson’s disease (5.9 %), when compared to the healthy control (0.0 %) [29]. Retrospectively,
in the study of 350 patients with essential tremor more than 20.2 % of them showed at least three of
four main features of Parkinson’s disease [18].
The implementation of reliable highly sensitive and specific biomarkers is a hot topic for
clinicians, nowadays. For example, images as biomarkers are widely used to detect Parkinson’s
disease and approve clinical observations [17]. Magnetic resonance imaging in Parkinson’s disease
shows an increased iron content in the substantia nigra for Parkinson’s disease patients. One of the
most developed magnetic resonance imaging markers is iron loading using T2 / T2 relaxometry [24].
Using T2 and T2 imaging of substantia nigra in patients with Parkinson’s disease, one can measure
the variation of the relaxation time constants as a reference for increased iron concentration in PD
patients. The changes in the brain of the PD patients were detected using a powerful magnetic
resonance imaging scanner (3 Tesla scanner) with the 1.5 Tesla magnetic field which is 10 to 15 times
stronger than in open MRI scanners. These imaging techniques are more sensitive to the content of
ferritin or neuromelanin in glands. Thus, magnetic resonance imaging is now the standard method for
the examination of patients with Parkinson’s disease aimed at discriminating the secondary causes of
the malfunction and providing specific knowledge necessary for detecting neurodegenerative
diseases.
However, positron emission tomography, namely computed tomography with 18F- DOPA (a
fluorinated form of levodopa) is the method with the highest specificity to detect Parkinson’s
disease. 18F-DOPA is a radioactive tracer for positron emission tomography scanning. It reveals
the function of striatal neurons in terms of dopamine consumption and provides a biomarker for
Parkinson’s disease [21, 4, 23, 2].
Artificial neural networks are widely used to visualize positron-emission tomography data for the
disease diagnostics. The study in [5] reports that CNNs (convolution neural networks) trained on
images from Parkinson’s Progression Markers Initiative (PPMI, [13]) database show the excellent
results in recognition of Parkinson’s disease from magnetic resonance images. The overall accuracy of
95.29 %, an average recall of 0.943, an average accuracy of 0.927, an average specificity of 0.9430, a
f1-score of 0.936, and an area under the error curve (ROC-AUC) of 0.98 for both classes were
obtained.
The presence of signs of Parkinson’s disease using dopaminergic imaging methods SPECT
DaTscan was studied in [20]. Images were also taken from the PPMI database [13]. The resulting
convolutional neural network (VGG16) using transfer learning provided an accuracy of 95.2 %, a
sensitivity of 97.5 %, and a specificity of 90.9 % in diagnostics.
The study in [15] presents the approach to the classification of MRI images of the healthy and PD
patients using a deep convolution neural network, which is accompanied by the generation of
additional examples in the training set to improve the classification using generative adversarial networks
(Generative Adversarial Networks, GANs). The results illustrate that fine-tuning of the final CNN layers
improves the average classification accuracy up to 89.23 %.
The authors in [12] solved the problem of predicting the stage of disease in PD patients using the
functional images of their brain. The authors used a wide range of classification methods including
convolution neural networks to get the result. Additionally, transfer learning was used to incorporate the
pre-trained weights and VGG16 architecture into the model setup.
It was previously shown that the differential diagnosis of Parkinson’s disease and essential tremor
is essential for the proper treatment of neurological diseases. In some studies, artificial neural
networks are used to solve this problem. Thus, in [11] a new method is proposed to distinguish
parkinsonian tremor from essential tremor based on spectral analysis using an accelerometer and
surface EMG signals using neural networks.
The authors in [22] propose a new method for the differential diagnosis of parkinsonian tremor (PT)
and essential tremor (ET) analyzing both postural tremor and resting tremor. The long-term, short-
term memory (LSTM) neural network was used as a tool for solving the problem. The accuracy of the
tremor classification for the combined positions was 90 %.
2. Scope of the study
The above presented review of literature shows that artificial intelligence systems are highly effective
for the differential diagnosis of essential tremor and Parkinson’s disease. However, it is limited to the
study of the tremor in patients, solely. At the same time, when diagnosing Parkinson’s disease, good
results are shown by the methods involving computer vision (in particular, convolution neural networks)
for processing images obtained during magnetic resonance imaging of patients.
This research opens a series of articles devoted to the study of a group of patients with
Parkinson’s disease and essential tremor and clinically unclear cases of patient symptomatology by
magnetic resonance imaging of the brain and positron emission tomography, namely, computed
tomography as a method of differential diagnosis. It is assumed that the result will be a tool to help
doctors to clarify the diagnosis of obscure patients.
Here, we present the solution of the problem of preprocessing of magnetic resonance tomography
stored in grayscale monochrome images to improve the quality of their processing by neural
networks in the learning process and in the differential diagnosis of Parkinson’s disease and essential
tremor by neural networks.
The preprocessing consisted in solving the following tasks:
1. Selection of the area corresponding directly to the brain tissue in a magnetic resonance image
(image segmentation);
2. Colorization of halftones of a black-and-white magnetic resonance image to increase the
likelihood of distinguishing features in the image by a neural network.
The transformed images are used as training samples to succeed in the second part of the study:
training of a convolution neural network (CNN) to solve the problem of differential diagnosis of
Parkinson’s disease and essential tremor based on tomography images. The segmentation reduces the
size of the image, thereby decreasing the dimensionality of the input signal of the neural network and
deliberately eliminating insignificant input signals: image points that are not related to brain tissues.
This approach is expected to speed up the training of t h e neural network and to increase the likelihood
of valid diagnostics.
3. Materials and methods
3.1. DeepLab v3 + architecture
To solve the segmentation problem, the DeepLab v3 + architecture was used. The efficiency of
the implementation of this architecture for the processing of magnetic resonance images is shown in
[8], thus supporting the efficiency for segmenting magnetic resonance imaging of the brain in
accordance with the objectives of the research. Furthermore, the implementation of DeepLab v3 +
instead of the custom image storage approach for processing the images yields a significant reduction
of the complexity of the procedure, retaining the quality of semantic segmentation problems.
The architecture of DeepLab v3 + inherits the version DeepLab v3. While the standard convolution
performs channel and spatial computations in one step, depth separable convolution splits the process
into two stages: the former is depth convolution and the latter is point convolution. The depth
convolution provides independent spatial convolution of signals for each input channel using only the
convolution filter, while the pointwise convolution yields a combination of the results of the depth
convolution [10, 7, 6].
3.2. Materials
A sample of 500 MRI images of patients was used to train the model. The sample was preliminary divided
into training and test sets in the proportion of 80 % vs. 20 %; four images are the test ones (see Table 1). The
black and white images were 512 × 512 in size. An example of an image markup and a mask is shown in Fig.1.
Table 1
Distribution of the segments in the samples; number of segments. IV stands for independent
verification
Training sample Test sample Total IV
400 400 100 100 500 500 3 3
Figure 1: A mark-up example. Left to right: Original Image, Color Mask, Training Mask
3.3. The metrics used in the study
The choice of metrics for the image segmentation problem is based on the classical approach set
forth; see, e. g. [31]. To measure the detection accuracy of an object, the mAP (mean average
precision) ranking quality metric is used. The average AP (average precision) for each of the N
classes of a training sample is the crucial parameter.
To calculate the efficiency of the trained model, the Intersection over Union (IoU) metric was also
used, allowing the evaluation of the quality of the algorithms which are certain to identify rectangles
in the target image relevant to the recognized objects. To obtain these metrics, several related metrics
were calculated, as indicated in Table 2.
Table 2
Metrics of the quality of training of the DeepLab v3 + neural network
mAP Precision Recall IoU F1-score TP FP FN
0.6131 0.87 0.88 67.76 0.88 92 14 12
3.4. Image post-processing
At the next step the model implements a segmented mask using the built-in methods of the
OpenCV library, a new image of 512 × 512 pixels is created with a black background and a clipped
area of the original image that falls under the mask. The next step in post-processing the image
involves counting the number of gray shades, dividing it into intervals, and further color clustering. For
a more accurate display of the clusters, it was decided to divide them into 49 intervals of t he gray-scale
values. This value of the number of intervals was chosen because during the division of the halftone
values of a black-and-white image into smaller intervals (by 56 intervals or more), color pseudo-
colorization added visually noticeable noise to the processed image. This situation made it difficult for
human experts to work with the image and to find the localized area in the image processed by magnetic
resonance imaging. If the division of the gray value was performed into wider intervals (into 7, 14, and
so on), then the areas of interest for the experts in the magnetic resonance image were colored as
insufficiently informative.
3.5. Learning Process
The neural network was trained on an NVidia GeForce 1060 video card for 5 hours, completing
200 training epochs and the training with an error function value of 0.19. The model is trained using
the logarithmic loss function (otherwise called the cross-entropy function) of the Softmax family. The
graph of the change in the value of the error function for the training and control samples is shown in
Fig. 2. As shown in Table 1, t he training was performed using 500 images, divided into training
and test samples in a ratio of 80 % to 20 %.
Figure 2: A figure caption is always placed below the illustration. Short captions are centered, while
long ones are justified. The macro button chooses the correct format automatically
4. Results and discussion
The values of the training quality metrics described above, as a result of calculating the test
cases, are shown in Table 2. Fig. 3 shows the stages of processing t h e images of magnetic
resonance imaging using an expert (first column) and the DeepLab v3 + neural network. The third
column shows the images colored using the OpenCV library.
The indicators shown in Table 2 and illustrated in Fig. 3 confirm that the trained DeepLab v3 +
neural network adequately selects the area corresponding to the brain tissue on the magnetic
resonance image. Thus, the tasks set in this research were fulfilled, i.e. it was necessary to obtain a
mechanism for preparing data for the neural network designed for the differential diagnosis of
Parkinson’s disease and essential tremor.
(a) (b) (c)
(d) (e) (f)
Figure 3: The results of processing the frames of magnetic resonance imaging by the
developed neural network
At the same time, the false detection rates (FP, TP) in Table 2 are not exactly equal to zero.
One of the images in Fig. 3 (bottom line) shows that the system segmented a part of the nasal sinus as
brain tissue, which is an erroneous decision and differs from the zone determined by the expert. Thus,
the system has room for improvement.
5. Conclusion
There are the following ways of further implementation of the proposed solutions allowing one to
solve the problems stated in Discussion. However, the combination of the trained network based on
the DeepLab v3 + architecture with the procedure for pseudocolorization of black and white images
using the functions of the OpenCV library makes it possible to implement a set of training samples for
the convolution neural networks which solve the problem of differential diagnosis based on magnetic
resonance images of Parkinson’s disease and essential tremor.
6. Acknowledgements
This work is done under the project “Feasibility of molecular genetic analysis for evaluation of the
risk of early development of neurodegenerative diseases”.
7. References
[1] J. Benito-Leon, E. Louis, F. Bermejo-Pareja, Risk of incident Parkinson’s disease and
parkinsonism in essential tremor: a population based study, Journal of Neurology, Neurosurgery
& Psychiatry 80(4) (2009) 423–425.
[2] W. D. Brown, M. Taylor, A. D. Roberts, T.R. Oakes, M. Schueller, J. E. Holden, L. Malischke,
O. T. DeJesus, R. J. Nickles, Fluorodopa pet shows the nondopaminergic as well as
dopaminergic destinations of levodopa, Neurology 53(6) (1999) 1212–1212.
[3] V. Cabreira, J. Massano, Doenca de Parkinson: Revisa˜o cl´ınica e atual- izac¸a˜o. Acta M´edica
Portuguesa 32(10) (2019).
[4] F. F. Calabria, E. Calabria, V. Gangemi, G. L. Cascini, Current status and future challenges of
brain 18 imaging with f-dopa pet for movement disorders, Hell J Nucl Med 19 (2016) 33–41.
[5] S. Chakraborty, S. Aich, , H. C. Kim, Detection of parkinson’s disease from 3t t1 weighted mri
scans using 3d convolutional neural network, Diagnostics 10(6) (2020) 402.
[6] L. C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam, Encoder-decoder with atrous separable
convolution for semantic image segmentation, in: Proceedings of the European conference on
computer vision (ECCV), 2018, pp. 801–818.
[7] F. Chollet, Deep learning with depthwise separable convolutions, in: Proceedings of the IEEE
conference on computer vision and pattern recognition, 2017, pp. 1251–1258.
[8] P. Furtado, Deep segmentation of abdominal organs from mri: off-the-shelf ar- chitectures and
improvements, in: Medical Imaging Processing, volume 11596, 2021, p. 115963.
[9] J. J. Geraghty, J. Jankovic, W. J. Zetusky, Association between essential tremor and Parkinson’s
disease. Annals of Neurology, Official Journal of the American Neurological Association and the
Child Neurology Society 17(4) (1985) 329–333.
[10] Y. Guo, Y. Li, L. Wang, T. Rosing, Depthwise convolution is all you need for learning multiple
visual domains, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 33,
2019, pp. 8368–8375.
[11] A. Hossen, M. Muthuraman, J. Raethjen, G. Deuschl, U. Heute, A neural net- work approach to
distinguish parkinsonian tremor from advanced essential tremor, in: Proceedings of the
International Conference on Soft Computing for Problem Solving (SocProS 2011) December 20-
22, 2012. pp. 1051–1060.
[12] G. H. Huang, C. H. Lin, Y. R. Cai, T. B. Chen, S. Y. Hsu, N. H. Lu, H. Y. Chen, Y. C. Wu,
Multiclass machine learning classification of functional brain images for parkinson’s disease
stage prediction. Statistical Analysis and Data Mining, The ASA Data Science Journal 13(5)
(2020) 508–523.
[13] S. Iddi, D. Li, P. S. Aisen, M. S. Rafii, I. Litvan, W. K. Thompson, Donohue, M.C.: Estimating
the evolution of disease in the Parkinson’s progression markers initiative, Neurodegenerative
Diseases 18(4) (2018) 173–190.
[14] K. A. Jellinger, Neuropathology of sporadic Parkinson’s disease: evaluation and changes of
concepts, Movement disorders 27(1) (2012) 8–30.
[15] S. Kaur, H. Aggarwal, R. Rani, Diagnosis of Parkinson’s disease using deep cnn with transfer
learning and data augmentation, Multimedia Tools and Applications 80(7) (2021) 10113–10139.
[16] W.C. Koller, K. Busenbark, K. Miner, E.T.S. Group, The relationship of essential tremor to other
movement disorders: report on 678 patients, Annals of neurology 35(6) (1994) 717–723.
[17] S. Lotankar, K. S. Prabhavalkar, L. K. Bhatt, Biomarkers for Parkinson’s disease: recent
advancement, Neuroscience bulletin 33(5) (2017) 585–597.
[18] J. S. Lou, J. Jankovic, Essential tremor: clinical correlates in 350 patients, Neurology 41 (1991)
234–234.
[19] E. D. Louis, M. Bares, J. Benito-Leon, S. Fahn, S. J. Frucht, J. Jankovic, W. G. Ondo, P. K. Pal,
E. K. Tan, Essential tremor-plus: a controversial new concept, The Lancet Neurology 19(3)
(2020) 266–270.
[20] P. R. Magesh, R. D. Myloth, R. J. Tom, An explainable machine learning model for early
detection of Parkinson’s disease using lime on datscan imagery, Computers in Biology and
Medicine 126 (2020) 104041.
[21] F. Niccolini, M. Politis, A systematic review of lessons learned from pet molecular imaging
research in atypical parkinsonism, European journal of nuclear medicine and molecular imaging
43(12), (2016) 2244–2254.
[22] A. B. Oktay, A. Kocer, Differential diagnosis of parkinson and essential tremor with
convolutional lstm networks, Biomedical Signal Processing and Control 56 (2020) 101683.
[23] A. R. Pikstra, A. van der Hoorn, K. L. Leenders, B. M. de Jong, Relation of 18- f-dopa pet with
hypokinesia-rigidity, tremor and freezing in Parkinson’s disease, NeuroImage: Clinical 11
(2016) 68–72.
[24] N. Pyatigorskaya, C.Gallea, D. Garcia-Lorenzo, M.Vidailhet, S. Lehericy, A review of the use
of magnetic resonance imaging in parkinson’s disease, Therapeutic advances in neurological
disorders 7(4) (2014) 206–220.
[25] S. G. Reich, Essential tremor, The Medical Clinics of North America 103(2) (2019) 351–356.
[26] V. Shanker, Essential tremor: diagnosis and management. BMJ 366 (2019).
[27] S. Sharma, C. S. Moon, A. Khogali, A. Haidous, A. Chabenne, C. Ojo, M. Jelebinkov, Y. Kurdi,
M. Ebadi, Biomarkers in Parkinson’s disease (recent update), Neurochemistry international 63(3)
(2013) 201–229.
[28] S. Sveinbjornsdottir, The clinical symptoms of Parkinson’s disease, Journal of neurochemistry
139, (2016) 318–324.
[29] E. K. Tan, S. S. Lee, S. Y. Lum, Evidence of increased odds of essential tremor in Parkinson’s
disease, Movement disorders: official journal of the Movement Disorder Society 23(7) (2008)
993–997.
[30] A.Tarakad, J. Jankovic, Essential tremor and Parkinson’s disease: exploring the relationship,
Tremor and Other Hyperkinetic Movements 8 (2018).
[31] W. Van Gansbeke, S. Vandenhende, S. Georgoulis, L.Van Gool, Unsupervised semantic
segmentation by contrasting object mask proposals, arXiv preprint arXiv:2102.06191, 2021.