Application of Convolution Neural Networks of the DeepLabv3 Architecture for Preprocessing of Magnetic Resonance Imaging for the Differential Diagnosis of Parkinson’s Disease and Essential Tremors Vladislav Abramov1 Anna Alekhina2, Mikhail Dorrer2, Maria Tunik3, Xenia Tutsenko3, Alina Khoroshavina3 and Michael Sadovsky1,3,4 1 Siberian research & clinical center of FMBA of Russia, Krasnoyarsk, Russia 2 Reshetnev Siberian State University of Science and Technology, Krasnoyarsk, Russia 3 V.F. Voino-Yasenetsky Krasnoyarsk State Medical University, Krasnoyarsk, Russia 4 Institute of Computational Modeling of the Siberian Branch of the Russian Academy of Sciences, Krasnoyarsk, Russia Abstract Preprocessing of data for implementing an artificial intel- ligence system solving the problem of differential diagnosis of Parkin- son’s disease and essential tremor based on the data of nuclear magnetic resonance imaging is discussed. A database of 500 images of magnetic resonance imaging and positron emission tomography of patients with Parkinson’s disease, essential tremor, and unclear diagnoses is used. For preprocessing a neural network of the DeepLab v3 + architecture was used. The accuracy of preprocessing “raw” images reached 87 %. Furthermore, the proposed solution adequately highlights the area corresponding to the brain tissue on the magnetic resonance tomography. Therefore, the solution provides a feasible mechanism to prepare data for a neural network implemented for the differential diagnosis of Parkinson’s disease and essential tremor. Keywords 1 Parkinson’s disease, essential tremor, diagnostics, magnetic resonance imaging, computer vision, convolution neural net-works 1. Introduction Parkinson’s disease and essential tremor are among the most common extrapyramidal diseases in clinical practice. It is imperative to differentiate these nosologies since the curation tactics, prognosis, and patient compliance depend on the exact diagnostics. Parkinson’s disease is currently the second most common neurodegenerative disease. Additionally, significant growth of Parkinson’s disease is observed in the last three decades. Both environmental and genetic factors contribute to the rise of the number of patients suffering from this disease [3]. Parkinson’s disease is a slowly progressive neurodegenerative disease which causes impaired motor function with slow movements, tremors, and gait and balance disorders. Various non-motor symptoms are common for Parkinson’s disease [27]. They include impaired autonomic function with orthostatic SibDATA 2021: The 2nd Siberian Scientific Workshop on Data Analysis Technologies with Applications 2021, June 25, 2021, Krasnoyarsk, Russia EMAIL: mdorrer@mail.ru (M. Dorrer); tsuprikova.mary.maria@yandex.ru (M. Tunik); msad@icm.krasn.ru (M. Sadovsky) ORCID: 0000−0002−4063−4951 (V. Abramov); 0000−0002−2063−2297 (A. Alekhina); 0000−0002−8598−8331 (M. Dorrer); 0000−0003−0194−1219 (M. Tunik); 0000−0003−3979−1172 (X. Tutsenko); 0000−0001−9732−7920 (A. Khoroshavina); 0000−0002−1807−0715 (M. Sadovsky) © 2021 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings (CEUR-WS.org) hypotension, constipation and urinary disturbances, various sleep disturbances, and a wide range of neuropsychiatric symptoms [28]. Parkinson’s disease is detected primarily through clinical practice; however, it should be confirmed by relevant lab studies. The gold standard for Parkinson’s disease diagnostics includes the autopsy confirming the loss of neurons in the substantia nigra and the occurrence of Lewy bodies as pathological markers of the disease [14]. However, there are still no methods free from failures to diagnose Parkinson’s disease in vivo. It heavily complicates the early detection of Parkinson’s disease and the differential diagnosis of Parkinson’s disease and essential tremor. Sometimes even very experienced clinicians find it challenging to understand and correctly interpret similar symptoms of these diseases. Essential tremor has completely different pathogenesis. It is also among the most common movement disorders affecting both children and adults. In 2018, essential tremor was defined as an isolated action tremor in the upper limbs on both sides. Tremor can also be present in other body parts, usually in the neck or vocal cords. Patients exhibiting additional neurological symptoms are now referred to as essential tremor plus. Patients with essential tremors may exhibit these different clinical symptoms at a later stage of the disease [19]. Extra clinical manifestations associated with this malfunction include cognitive impairment, psychiatric disorders, and hearing loss [26]. Essential tremor often occurs years or decades before other symptoms appear and progressively worsen. Commonly, essential tremors are supposed to be a familial case which is more frequent as compared to sporadic ones [25]. While patients with essential tremors are known to exhibit the tremor in activity, those with Parkinson’s disease show mainly resting tremor. Some patients exhibit standard features making a trouble in diagnostic for a physician [30]. If a patient has both essential tremor and Parkinson’s disease symptoms, then it becomes even more difficult for clinicians to make reliable diagnostics. A number of papers report that essential tremor can later be accompanied with Parkinson’s disease. Several epidemiological studies were carried out to support the idea that the combination of essential tremor and Parkinson’s disease is not just coincidental. In a population-based study conducted in Spain, 3 % of patients with essential tremor later developed Parkinson’s disease compared with 0.4 % in the control group; this supports the observation that patients with essential tremor have a higher risk of the development of Parkinson’s disease [1]. In the study of 678 patients, 6.1 % of patients with essential tremor had Parkinson’s disease [16], while another study of patients with essential tremor, reports on 20.8 % with concomitant Parkinson’s disease [9]. The first investigation involving 600 patients shows higher incidence of essential tremor in patients with Parkinson’s disease (5.9 %), when compared to the healthy control (0.0 %) [29]. Retrospectively, in the study of 350 patients with essential tremor more than 20.2 % of them showed at least three of four main features of Parkinson’s disease [18]. The implementation of reliable highly sensitive and specific biomarkers is a hot topic for clinicians, nowadays. For example, images as biomarkers are widely used to detect Parkinson’s disease and approve clinical observations [17]. Magnetic resonance imaging in Parkinson’s disease shows an increased iron content in the substantia nigra for Parkinson’s disease patients. One of the most developed magnetic resonance imaging markers is iron loading using T2 / T2 relaxometry [24]. Using T2 and T2 imaging of substantia nigra in patients with Parkinson’s disease, one can measure the variation of the relaxation time constants as a reference for increased iron concentration in PD patients. The changes in the brain of the PD patients were detected using a powerful magnetic resonance imaging scanner (3 Tesla scanner) with the 1.5 Tesla magnetic field which is 10 to 15 times stronger than in open MRI scanners. These imaging techniques are more sensitive to the content of ferritin or neuromelanin in glands. Thus, magnetic resonance imaging is now the standard method for the examination of patients with Parkinson’s disease aimed at discriminating the secondary causes of the malfunction and providing specific knowledge necessary for detecting neurodegenerative diseases. However, positron emission tomography, namely computed tomography with 18F- DOPA (a fluorinated form of levodopa) is the method with the highest specificity to detect Parkinson’s disease. 18F-DOPA is a radioactive tracer for positron emission tomography scanning. It reveals the function of striatal neurons in terms of dopamine consumption and provides a biomarker for Parkinson’s disease [21, 4, 23, 2]. Artificial neural networks are widely used to visualize positron-emission tomography data for the disease diagnostics. The study in [5] reports that CNNs (convolution neural networks) trained on images from Parkinson’s Progression Markers Initiative (PPMI, [13]) database show the excellent results in recognition of Parkinson’s disease from magnetic resonance images. The overall accuracy of 95.29 %, an average recall of 0.943, an average accuracy of 0.927, an average specificity of 0.9430, a f1-score of 0.936, and an area under the error curve (ROC-AUC) of 0.98 for both classes were obtained. The presence of signs of Parkinson’s disease using dopaminergic imaging methods SPECT DaTscan was studied in [20]. Images were also taken from the PPMI database [13]. The resulting convolutional neural network (VGG16) using transfer learning provided an accuracy of 95.2 %, a sensitivity of 97.5 %, and a specificity of 90.9 % in diagnostics. The study in [15] presents the approach to the classification of MRI images of the healthy and PD patients using a deep convolution neural network, which is accompanied by the generation of additional examples in the training set to improve the classification using generative adversarial networks (Generative Adversarial Networks, GANs). The results illustrate that fine-tuning of the final CNN layers improves the average classification accuracy up to 89.23 %. The authors in [12] solved the problem of predicting the stage of disease in PD patients using the functional images of their brain. The authors used a wide range of classification methods including convolution neural networks to get the result. Additionally, transfer learning was used to incorporate the pre-trained weights and VGG16 architecture into the model setup. It was previously shown that the differential diagnosis of Parkinson’s disease and essential tremor is essential for the proper treatment of neurological diseases. In some studies, artificial neural networks are used to solve this problem. Thus, in [11] a new method is proposed to distinguish parkinsonian tremor from essential tremor based on spectral analysis using an accelerometer and surface EMG signals using neural networks. The authors in [22] propose a new method for the differential diagnosis of parkinsonian tremor (PT) and essential tremor (ET) analyzing both postural tremor and resting tremor. The long-term, short- term memory (LSTM) neural network was used as a tool for solving the problem. The accuracy of the tremor classification for the combined positions was 90 %. 2. Scope of the study The above presented review of literature shows that artificial intelligence systems are highly effective for the differential diagnosis of essential tremor and Parkinson’s disease. However, it is limited to the study of the tremor in patients, solely. At the same time, when diagnosing Parkinson’s disease, good results are shown by the methods involving computer vision (in particular, convolution neural networks) for processing images obtained during magnetic resonance imaging of patients. This research opens a series of articles devoted to the study of a group of patients with Parkinson’s disease and essential tremor and clinically unclear cases of patient symptomatology by magnetic resonance imaging of the brain and positron emission tomography, namely, computed tomography as a method of differential diagnosis. It is assumed that the result will be a tool to help doctors to clarify the diagnosis of obscure patients. Here, we present the solution of the problem of preprocessing of magnetic resonance tomography stored in grayscale monochrome images to improve the quality of their processing by neural networks in the learning process and in the differential diagnosis of Parkinson’s disease and essential tremor by neural networks. The preprocessing consisted in solving the following tasks: 1. Selection of the area corresponding directly to the brain tissue in a magnetic resonance image (image segmentation); 2. Colorization of halftones of a black-and-white magnetic resonance image to increase the likelihood of distinguishing features in the image by a neural network. The transformed images are used as training samples to succeed in the second part of the study: training of a convolution neural network (CNN) to solve the problem of differential diagnosis of Parkinson’s disease and essential tremor based on tomography images. The segmentation reduces the size of the image, thereby decreasing the dimensionality of the input signal of the neural network and deliberately eliminating insignificant input signals: image points that are not related to brain tissues. This approach is expected to speed up the training of t h e neural network and to increase the likelihood of valid diagnostics. 3. Materials and methods 3.1. DeepLab v3 + architecture To solve the segmentation problem, the DeepLab v3 + architecture was used. The efficiency of the implementation of this architecture for the processing of magnetic resonance images is shown in [8], thus supporting the efficiency for segmenting magnetic resonance imaging of the brain in accordance with the objectives of the research. Furthermore, the implementation of DeepLab v3 + instead of the custom image storage approach for processing the images yields a significant reduction of the complexity of the procedure, retaining the quality of semantic segmentation problems. The architecture of DeepLab v3 + inherits the version DeepLab v3. While the standard convolution performs channel and spatial computations in one step, depth separable convolution splits the process into two stages: the former is depth convolution and the latter is point convolution. The depth convolution provides independent spatial convolution of signals for each input channel using only the convolution filter, while the pointwise convolution yields a combination of the results of the depth convolution [10, 7, 6]. 3.2. Materials A sample of 500 MRI images of patients was used to train the model. The sample was preliminary divided into training and test sets in the proportion of 80 % vs. 20 %; four images are the test ones (see Table 1). The black and white images were 512 × 512 in size. An example of an image markup and a mask is shown in Fig.1. Table 1 Distribution of the segments in the samples; number of segments. IV stands for independent verification Training sample Test sample Total IV 400 400 100 100 500 500 3 3 Figure 1: A mark-up example. Left to right: Original Image, Color Mask, Training Mask 3.3. The metrics used in the study The choice of metrics for the image segmentation problem is based on the classical approach set forth; see, e. g. [31]. To measure the detection accuracy of an object, the mAP (mean average precision) ranking quality metric is used. The average AP (average precision) for each of the N classes of a training sample is the crucial parameter. To calculate the efficiency of the trained model, the Intersection over Union (IoU) metric was also used, allowing the evaluation of the quality of the algorithms which are certain to identify rectangles in the target image relevant to the recognized objects. To obtain these metrics, several related metrics were calculated, as indicated in Table 2. Table 2 Metrics of the quality of training of the DeepLab v3 + neural network mAP Precision Recall IoU F1-score TP FP FN 0.6131 0.87 0.88 67.76 0.88 92 14 12 3.4. Image post-processing At the next step the model implements a segmented mask using the built-in methods of the OpenCV library, a new image of 512 × 512 pixels is created with a black background and a clipped area of the original image that falls under the mask. The next step in post-processing the image involves counting the number of gray shades, dividing it into intervals, and further color clustering. For a more accurate display of the clusters, it was decided to divide them into 49 intervals of t he gray-scale values. This value of the number of intervals was chosen because during the division of the halftone values of a black-and-white image into smaller intervals (by 56 intervals or more), color pseudo- colorization added visually noticeable noise to the processed image. This situation made it difficult for human experts to work with the image and to find the localized area in the image processed by magnetic resonance imaging. If the division of the gray value was performed into wider intervals (into 7, 14, and so on), then the areas of interest for the experts in the magnetic resonance image were colored as insufficiently informative. 3.5. Learning Process The neural network was trained on an NVidia GeForce 1060 video card for 5 hours, completing 200 training epochs and the training with an error function value of 0.19. The model is trained using the logarithmic loss function (otherwise called the cross-entropy function) of the Softmax family. The graph of the change in the value of the error function for the training and control samples is shown in Fig. 2. As shown in Table 1, t he training was performed using 500 images, divided into training and test samples in a ratio of 80 % to 20 %. Figure 2: A figure caption is always placed below the illustration. Short captions are centered, while long ones are justified. The macro button chooses the correct format automatically 4. Results and discussion The values of the training quality metrics described above, as a result of calculating the test cases, are shown in Table 2. Fig. 3 shows the stages of processing t h e images of magnetic resonance imaging using an expert (first column) and the DeepLab v3 + neural network. The third column shows the images colored using the OpenCV library. The indicators shown in Table 2 and illustrated in Fig. 3 confirm that the trained DeepLab v3 + neural network adequately selects the area corresponding to the brain tissue on the magnetic resonance image. Thus, the tasks set in this research were fulfilled, i.e. it was necessary to obtain a mechanism for preparing data for the neural network designed for the differential diagnosis of Parkinson’s disease and essential tremor. (a) (b) (c) (d) (e) (f) Figure 3: The results of processing the frames of magnetic resonance imaging by the developed neural network At the same time, the false detection rates (FP, TP) in Table 2 are not exactly equal to zero. One of the images in Fig. 3 (bottom line) shows that the system segmented a part of the nasal sinus as brain tissue, which is an erroneous decision and differs from the zone determined by the expert. Thus, the system has room for improvement. 5. Conclusion There are the following ways of further implementation of the proposed solutions allowing one to solve the problems stated in Discussion. However, the combination of the trained network based on the DeepLab v3 + architecture with the procedure for pseudocolorization of black and white images using the functions of the OpenCV library makes it possible to implement a set of training samples for the convolution neural networks which solve the problem of differential diagnosis based on magnetic resonance images of Parkinson’s disease and essential tremor. 6. Acknowledgements This work is done under the project “Feasibility of molecular genetic analysis for evaluation of the risk of early development of neurodegenerative diseases”. 7. References [1] J. Benito-Leon, E. Louis, F. Bermejo-Pareja, Risk of incident Parkinson’s disease and parkinsonism in essential tremor: a population based study, Journal of Neurology, Neurosurgery & Psychiatry 80(4) (2009) 423–425. [2] W. D. Brown, M. Taylor, A. D. Roberts, T.R. Oakes, M. Schueller, J. E. Holden, L. Malischke, O. T. DeJesus, R. J. Nickles, Fluorodopa pet shows the nondopaminergic as well as dopaminergic destinations of levodopa, Neurology 53(6) (1999) 1212–1212. [3] V. Cabreira, J. Massano, Doenca de Parkinson: Revisa˜o cl´ınica e atual- izac¸a˜o. Acta M´edica Portuguesa 32(10) (2019). [4] F. F. Calabria, E. Calabria, V. Gangemi, G. L. Cascini, Current status and future challenges of brain 18 imaging with f-dopa pet for movement disorders, Hell J Nucl Med 19 (2016) 33–41. [5] S. Chakraborty, S. Aich, , H. C. Kim, Detection of parkinson’s disease from 3t t1 weighted mri scans using 3d convolutional neural network, Diagnostics 10(6) (2020) 402. [6] L. C. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam, Encoder-decoder with atrous separable convolution for semantic image segmentation, in: Proceedings of the European conference on computer vision (ECCV), 2018, pp. 801–818. [7] F. Chollet, Deep learning with depthwise separable convolutions, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 1251–1258. [8] P. Furtado, Deep segmentation of abdominal organs from mri: off-the-shelf ar- chitectures and improvements, in: Medical Imaging Processing, volume 11596, 2021, p. 115963. [9] J. J. Geraghty, J. Jankovic, W. J. Zetusky, Association between essential tremor and Parkinson’s disease. Annals of Neurology, Official Journal of the American Neurological Association and the Child Neurology Society 17(4) (1985) 329–333. [10] Y. Guo, Y. Li, L. Wang, T. Rosing, Depthwise convolution is all you need for learning multiple visual domains, in: Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, 2019, pp. 8368–8375. [11] A. Hossen, M. Muthuraman, J. Raethjen, G. Deuschl, U. Heute, A neural net- work approach to distinguish parkinsonian tremor from advanced essential tremor, in: Proceedings of the International Conference on Soft Computing for Problem Solving (SocProS 2011) December 20- 22, 2012. pp. 1051–1060. [12] G. H. Huang, C. H. Lin, Y. R. Cai, T. B. Chen, S. Y. Hsu, N. H. Lu, H. Y. Chen, Y. C. Wu, Multiclass machine learning classification of functional brain images for parkinson’s disease stage prediction. Statistical Analysis and Data Mining, The ASA Data Science Journal 13(5) (2020) 508–523. [13] S. Iddi, D. Li, P. S. Aisen, M. S. Rafii, I. Litvan, W. K. Thompson, Donohue, M.C.: Estimating the evolution of disease in the Parkinson’s progression markers initiative, Neurodegenerative Diseases 18(4) (2018) 173–190. [14] K. A. Jellinger, Neuropathology of sporadic Parkinson’s disease: evaluation and changes of concepts, Movement disorders 27(1) (2012) 8–30. [15] S. Kaur, H. Aggarwal, R. Rani, Diagnosis of Parkinson’s disease using deep cnn with transfer learning and data augmentation, Multimedia Tools and Applications 80(7) (2021) 10113–10139. [16] W.C. Koller, K. Busenbark, K. Miner, E.T.S. Group, The relationship of essential tremor to other movement disorders: report on 678 patients, Annals of neurology 35(6) (1994) 717–723. [17] S. Lotankar, K. S. Prabhavalkar, L. K. Bhatt, Biomarkers for Parkinson’s disease: recent advancement, Neuroscience bulletin 33(5) (2017) 585–597. [18] J. S. Lou, J. Jankovic, Essential tremor: clinical correlates in 350 patients, Neurology 41 (1991) 234–234. [19] E. D. Louis, M. Bares, J. Benito-Leon, S. Fahn, S. J. Frucht, J. Jankovic, W. G. Ondo, P. K. Pal, E. K. Tan, Essential tremor-plus: a controversial new concept, The Lancet Neurology 19(3) (2020) 266–270. [20] P. R. Magesh, R. D. Myloth, R. J. Tom, An explainable machine learning model for early detection of Parkinson’s disease using lime on datscan imagery, Computers in Biology and Medicine 126 (2020) 104041. [21] F. Niccolini, M. Politis, A systematic review of lessons learned from pet molecular imaging research in atypical parkinsonism, European journal of nuclear medicine and molecular imaging 43(12), (2016) 2244–2254. [22] A. B. Oktay, A. Kocer, Differential diagnosis of parkinson and essential tremor with convolutional lstm networks, Biomedical Signal Processing and Control 56 (2020) 101683. [23] A. R. Pikstra, A. van der Hoorn, K. L. Leenders, B. M. de Jong, Relation of 18- f-dopa pet with hypokinesia-rigidity, tremor and freezing in Parkinson’s disease, NeuroImage: Clinical 11 (2016) 68–72. [24] N. Pyatigorskaya, C.Gallea, D. Garcia-Lorenzo, M.Vidailhet, S. Lehericy, A review of the use of magnetic resonance imaging in parkinson’s disease, Therapeutic advances in neurological disorders 7(4) (2014) 206–220. [25] S. G. Reich, Essential tremor, The Medical Clinics of North America 103(2) (2019) 351–356. [26] V. Shanker, Essential tremor: diagnosis and management. BMJ 366 (2019). [27] S. Sharma, C. S. Moon, A. Khogali, A. Haidous, A. Chabenne, C. Ojo, M. Jelebinkov, Y. Kurdi, M. Ebadi, Biomarkers in Parkinson’s disease (recent update), Neurochemistry international 63(3) (2013) 201–229. [28] S. Sveinbjornsdottir, The clinical symptoms of Parkinson’s disease, Journal of neurochemistry 139, (2016) 318–324. [29] E. K. Tan, S. S. Lee, S. Y. Lum, Evidence of increased odds of essential tremor in Parkinson’s disease, Movement disorders: official journal of the Movement Disorder Society 23(7) (2008) 993–997. [30] A.Tarakad, J. Jankovic, Essential tremor and Parkinson’s disease: exploring the relationship, Tremor and Other Hyperkinetic Movements 8 (2018). [31] W. Van Gansbeke, S. Vandenhende, S. Georgoulis, L.Van Gool, Unsupervised semantic segmentation by contrasting object mask proposals, arXiv preprint arXiv:2102.06191, 2021.