Predicting the Increase in Postoperative Motor Deficits in Patients with Supratentorial Gliomas Using Machine Learning Methods⋆ Alexandra Kosyrkova1[0000−0002−3019−5203] , Eugene 2[0000−0002−9891−8658] Ilyushin , Daniel Saada2[0000−0003−4959−8093] , Ramin 1[0000−0001−6384−7960] Afandiev , Alexander Baev1[0000−0003−4908−0534] , Eudard 1[0000−0002−4803−6948] Pogosbekyan , Vladimir Okhlopkov1[0000−0001−8911−2372] , 1[0000−0003−1442−5993] Gleb Danilov , Artem Batalov1[0000−0002−8924−7346] , Igor 1[0000−0003−0326−7942] Pronin , Natalia Zakharova1[0000−0002−0516−3613] , Anna 1[0000−0003−3595−2696] Ogurtsova , Alexander Kravchuck1[0000−0002−3112−8256] , 1[0000−0003−0374−7970] David Pitskhelauri , Alexander Potapov1[0000−0001−8343−3511] , and Sergey Goryaynov1[0000−0002−6480−3270] 1 “N. N. Burdenko National Medical Research Center of Neurosurgery” of the Ministry of Health of the Russian Federation akosyrkova@nsi.ru 2 Lomonosov Moscow State University eugene.ilyushin@gmail.com Abstract. Surgery of glial tumors of the brain located in the motor areas vicinity is associated with a high risk of increasing neurological deficits. Motor deficit affects overall survival in this group of patients. Nowadays, no method allows for an objective preoperative data-based prognosis of the risk of neurological impairment in each particular case. Objective: develop a convolution neuronal network that can predict mo- tor worsening in patients with supranational gliomas using the preoper- ative MRI-data. Materials and methods: the study included 527 patients aged 18 years and older with newly identified supratentorial gliomas. All patients underwent preoperative MRI and tumor removal based on Burdenko National Center of neurosurgery in 2013-2019. Data on motor status dynamics after surgery for these patients were obtained from the electronic medical records using the original semiautomatic algorithm for natural language processing. The T2FLAIR mode is used for train- ing our model. The model demonstrates the following metrics of quality: accuracy 91%, sensitivity 94%, specificity 89%, ROC AUC 91%, and F1 92%. Thus, machine learning methods predict the motor worsening with relatively high accuracy in patients with supratentorial gliomas at the preoperative stage, based on brain MRI data. Keywords: Glioma · Machine learning · Corticospinal tract · Convolu- tional neural network · Paresis. ⋆ Supported by the RFBR, Project №19-29-0115. Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). 120 1 Introduction Surgery of the brain’s glial tumors located in the motor areas vicinity is a truly existing problem provoking several contradictions among neurosurgeons. The surgery in this area is associated with a high risk of transitory and persistent neurological disability [11, 28], which may interfere with the following treatment process (chemo or radiotherapy) in this group of patients, and as such, may directly affect not only the quality of life but also the overall survival. Patients and their relatives should be preoperatively informed on a possible neurological deficit that may develop, and it is them to decide whether to undergo surgery or refuse it. Today, this type of information is subjectively provided by a physician, based on personal experience and feelings. Meanwhile, no method allows for an objective preoperative data-based prognosis of the risk of neurological impair- ment in each particular case. Simultaneously, machine learning methods that were widely used in various spheres of our life and were found to be especially useful in predicting a neurological disorder have a limited application in neu- rosurgery. The PubMed search by the keywords “glioma machine learning” has resulted in 286 publications by May 1st, 2020. We found no paper describing ma- chine learning methods in predicting motor deficit development or aggravation in patients with glial tumors. 2 Objective We aim to develop a convolution neuronal network (CNN) for predicting the risk of motor deficit development or aggravation based on preoperative MRI-data in patients with supratentorial brain tumors in the early postoperative period. 3 Materials and Methods 3.1 Inclusion Criteria Patients aged 18 years and older, who underwent microsurgical brain tumor removal at N.N. Burdenko National Medical Research Center of Neurosurgery (the Center) from 2013 to 2020 entered this study. In all cases, there were verified primary gliomas of different grades located supratentorial. Preoperative MRI (T1, T2, T2 FLAIR, DWI, T1+C) was performed in all patients at the Center not earlier than one month before surgery. 3.2 MRI Data Collection Patient selection, defined by the inclusion criteria, was carried out by three experienced radiologists based on the database of patients examined in the de- partment of X-ray and radio-isotope methods of diagnosis at the Center. The database has been stored since 2013 and contains details on more than 4500 121 patients with brain lesions of different histological structures and locations, pri- marily or secondarily operated. It contains DISCOM image files from the MRI studies performed on 1,5 - 3 Tesla MRI scanners (GE Signa HDxt 3.0T, GE Signa HDxt 1.5T, GE Optima 450w 1.5T (GE Healthcare, Milwaukee, USA) using 8- channel imaging and an electronic list of patients (an Excel table) providing large amounts of primary data (patient name, surname, personal identification num- ber, age, case record No, date of preoperative MRI, date, and type of surgery, and morphological diagnosis). The majority (70%) of patients had morphologi- cally verified glial tumors of various locations. Other lesions like ependymomas, meningiomas, neurinomas were excluded from the trial, as well as patients who had no microsurgical tumor removal. To understand whether the rest of the pa- tients met the inclusion criteria, their manually traced MRI data were estimated by the Centre’s radiologists. Finally, 527 of the full 4500 MRI case recordings were selected to enter the study group. All personal data were anonymized, and each patient got a personal identification number. 3.3 Data on Patient Motor Status Before Surgery and in the Early Postoperative Stage Data on patients’ motor status were obtained via analysis of electronic case records, which traditionally included information about the muscle strength from the upper and lower extremities assessed by a 6-score grading scale (0-5) before the operation and in the early postoperative period. Considering the large size of the trial, the problem was solved using the original semiautomatic algorithm for natural language processing (Algorithm) developed at the Center. The Algo- rithm made up 91%, the logistic regression accuracy - 85%. All markup results were additionally estimated by an expert using the special support program, thus allowing sorting out the database fields of interest with their following check-up. Simultaneously, an expert concluded paresis: whether it was present or absent after surgery and its dynamics by the date of discharge from the neurosurgical clinic. According to the performed analysis, pre-surgical paresis was marked in 23% of cases. Its aggravation or muscle weakness in the postoperative period was observed in 14.3% of cases. 3.4 The Dataset Obtained Thus, the dataset was formed comprising MRI datasets in different pulse se- quences (T1, T2, T2 FLAIR, DWI, T1+C) in 527 patients, and data on the motor status dynamics in the early postoperative period for this group of pa- tients. The information about sex, age, histological diagnosis, and tumor location was not used for CNN learning; it is listed in the Table 1. It should be noticed that patients operated in all departments of the Center (except for pediatric and spinal ones) were enrolled in this study. Different sur- gical techniques were carried out by more than 20 surgeons, thus making this study group universe. 122 Table 1. The main characteristics of the group of patients selected for this study. Parameter Value total number of patients 527 mean age 39, 8 ± 17, 9 years women/men 50,4%/49,6% histology LGG – 25%, HGG – 75 % frontal lobe – 37,6%, parietal lobe -– 25,5%, localization temporal lobe – 18%, occipital lobe – 13,5%, more than one lobe – 5,4% 3.5 Neural Network for Predicting the Development or Growth of Paresis in the Early Postoperative Period Statement of the Problem. It is necessary to develop a method for predicting the increase in pyramidal deficiency in the postoperative period in patients with supratentorial glial brain tumors located near the motor zones of the cortex and pyramidal tract, according to MRI studies of the patient’s brain at a level that meets the specified quality criteria and is binary. Formal Definition of the Problem. Let X be the set of preoperative MRI slices of patients, and Y be the set of classes to which the elements of X may belong, in our case Y ∈ {0, 1}. It is necessary to develop an algorithm A that will allow classifying an arbitrary slice x ∈ X: A: X → Y (1) Data Preparation. The source dataset of different modalities X was the results of MR-research of the brain carried out on 527 patients who were operated on with a diagnosis of a supratentorial glial tumor. The data set contained informa- tion on the increase in the pyramidal deficit in the postoperative period for each research. Each research from the dataset consisted of MRI images performed in the following modes: T1, T2, T2 FLAIR, DWI, T1+C. The data preparation stage consisted of the following steps: 1. Markup. Radiologists match each slice, which was performed in a T2 FLAIR regime, with a binary label. If a tumor was present on the slice, the label was equal to 1, otherwise 0. 2. Converting. Marked slices were converted from DICOM format to PNG for- mat. 3. Normalization. All data were centered and normalized (Figure 1). 123 Fig. 1. Graphical representation of the data preparation process for CNN learning. Model Architecture. In the course of work on the project, our architecture of an artificial neural network (ANN) ”Shallow” was developed (Figure 2). Also, experiments were carried out using popular ANNs pre-trained on the ImageNet dataset, such as VGG16, VGG19, Inception v3, followed by their fine-tuning on the MRI data. Description of ”Shallow” ANN: 1. An input layer that takes into account the two-dimensional topology of the image and accepts a normalized, black and white image of 100 × 100 pixels as input. 2. A convolutional layer contains 32 filters with a 3 × 3 kernel. 3. Activation layer using ReLU f (x) = max(0, x) (2) 4. Average pooling layer. The pool size is 2 × 2. 5. Dropout layer with a rate equal to 0.2. 6. A convolutional layer contains 64 filters with a 3 × 3 kernel. 7. Activation layer using ReLU. 8. Average pooling layer. The pool size is 2 × 2. 9. Dropout layer with a rate equal to 0.2. 10. A convolutional layer contains 128 filters with a 3 × 3 kernel. 11. Activation layer using ReLU. 12. Average pooling layer. The pool size is 2 × 2. 13. Dropout layer with a rate equal to 0.2. 14. Flatten layer. 15. Dense layer with 128 units. 124 16. Activation layer using ReLU. 17. Dense layer with 1 unit. 18. Activation layer using a sigmoid function 1 f (x) = (3) 1 + ϵ( − x) To prevent overfitting of the model, in addition to disabling random neurons, L2-regularization is used. Fig. 2. Graphical representation of the Shallow’s CNN architecture developed for this research. 125 Loss Function. The binary cross-entropy was chosen as the loss function: L(y, ŷ) = −y log ŷ − (1 − y) log(1 − ŷ), (4) where y – the class the slice belongs to, ŷ – predicted class. Augmentations. In addition to common augmentations such as random hori- zontal or vertical flips, scaling and rotation, it was suggested to use adversarial training. Adversarial examples are images with addition of specially generated noise, which is visually not distinguishable by the human eye, but causes the model to make incorrect predictions. We used Fast Gradient Sign Method [12] and Carlini-Wagner L2 algorithm [5] to generate two adversarial examples for each sample in the training set. The model will be evaluated separately with the use of adversarial examples and without them. Models Evaluating. For models evaluating, we used the following metrics: – Accuracy = T P +TTN P +T N +F P +F N – Sensitivity = T PT+F P N – Specif icity = T NT+F N P R1 – ROCAU C = 0 T P R dF P R, where T P R = T PT+F P FP N , F P R = T N +F P – F1 = 2P recision×Recall P recision+Recall , P recision = T P/(T P + F P ), Recall = T P/(T P + F N ), where TP – true positive, TN – true negative, FP – false positive, FN – false negative. Training and Evaluating. Model training and evaluating were carried out on the following equipment: – Intel Core i9-9900KF CPU 3.6 – RAM 32Gb – GPU NVIDIA 2080Ti 11Gb As a basic framework for developing the model, we used TensorFlow 2.0. The prepared dataset was split into three parts. 70% of the total number of slices was included in the training set, 20% in the validation set, and 10% in the test set. The resulting datasets are class-balanced. The batch size was empirically chosen equal to 64, the number of epochs was 70. AdamOptimizer was used as an optimizer, and the learning rate coefficient was chosen as a piecewise constant function with an initial value of 10−4 , decreasing dynamically. After each epoch, the model was validated on 10 batches of 64 elements from the validation dataset. After training, the quality of the model was evaluated in an automatic mode on test data. The model was then manually verified by a neurosurgeon who used his dataset of MRI scans of 50 patients that were not present in the original dataset. 126 Results. The results of the experiments are presented in Table 2. According to these results, the best model on the test sample showed Accuracy 91%, Sensitiv- ity 94%, Specificity 89%, ROC AUC 91%, and F1 92% in predicting the increase in hemiparesis in the early postoperative period in patients with supratentorial gliomas of the brain. Table 2. Results of experimental studies in determining the increase in motor deficits for the developed CNN Shallow, as well as for other popular pre-trained on the Ima- geNet data set CNNs, such as VGG16, and Inception v3. Model Accuracy Sensitivity Specificity ROC AUC F1-score Shallow 0.82 0.87 0.76 0.82 0.83 Shallow+adv.ex. 0.91 0.94 0.89 0.91 0.92 VGG16 0.71 0.69 0.73 0.71 0.71 Inception v3 0.50 0.44 0.57 0.51 0.48 Figure 3 shows examples of preoperative MRI data, based on which the CNN correctly predicted the increase in motor deficits in the postoperative period (a- b), the absence of deterioration in motor status after surgery (c), and made a mistake in predicting the increase in motor deficits after surgery(d). Fig. 3. a-b – MRI-data based on which the CNN (Shallow) correctly predicted motor worsening in the postoperative period; c – MRI-data based on which the CNN (Shallow) correctly predicted the absence of motor worsening in the postoperative period; d – MRI-data based on which the CNN (Shallow) made a mistake in predicting motor worsening in the postoperative period. 127 4 Discussion Methods of machine learning are of limited use in neurosurgery, and in brain glioma surgery, in particular. The typical task spectrum that may be solved using machine learning methods is presented in Table 3. Table 3. Main directions of application of machine learning methods in brain glioma surgery according to the literature analysis. Task Examples of publications determining the genetic Zhao J2020 [35], Chen SC2019 [7], Matsui Y2019 [20], glioma profile Kocak B2019 [17], Aliotta E2019 [1], Ozturk-Isik E2019 [24], Sun Z2019 [30], Huang RY2020 [14] defining grades of brain Zhuge Y2020 [36], Rathore S2020 [27], Zhang Z2020 [34], gliomas Cao H2002 [4], Nakamoto T2019 [22], Park YW2019 [25], akahashi S 2019 [31] defining grades of gliomas Hu LS2020 [13], Gates EDH2020 [10] based on intratumoral heterogeneity differential diagnosis with Jianhua Qin 2019 [26], Kaplan K2020 [16] other intracranial tumors differential diagnosis with Elshafeey N 2019 [9], Bacchi S2019 [3] post-radiation changes prognosis of the overall Mizutani T2019 [21], Jang K2020 [15], Choi YS2020 [8] patient survival [26], Chang Y2019 [6] automatic segmentation Shusharina N2020 [29], Wu Y2019 [33], Krivov of tumors E2018 [18] early detection of gliomas Amin J2019 [2] prognostic prospects of Neves BJ2020 [23], Wu S2020 [32] chemotherapy We failed to discover publications devoted to machine learning in pyrami- dal insufficiency prognosis in patients with brain gliomas. As in the majority of papers, our CNN has analyzed MR imaging data of patients. Our literature survey analysis showed that most commonly conventional MR imaging (1, 2, 1+C, FLAIR) [17, 30, 22, 25, 8, 6, 33] was used as input data, rarely DWI [35, 31, 3], DTI [1] and MR perfusion [10, 26, 9] were additionally used. Besides, there were some papers reporting on the use of proton MR-spectroscopy [24], kurto- sis MRI [31], PET-CT [35, 20]. Alongside with neuroimaging data, some other graphic information was applied (but considerably rarely): photos of histological glioma slides [27], spectroscopic 5-ALA accumulation curves [19], as well as text data: results of RNA sequencing [7], clinical data (such as sex, age, extent of re- section etc.) [35, 20]. It stresses, that methods of machine learning can effectively utilize different types of information (multimodal data), with the best sensitivity and specificity effect being reached by their rational combination. Thus, multi- modal data permitted increasing sensitivity and specificity from 83% to 90% 128 and from 82% to 90%, respectively [35], and accuracy from 73, 14% to 91,48%, sensitivity from 72,84% to 93,47%, specificity from 74,05% to 85,36% [27]. It is also true when using different modalities of MR imaging or other additional methods of neuroimaging. Thus, Matsui Y et al. reports the following accuracy for different neuroimaging modalities: 58,5% - for MRI only, 60,4% - for MRI and PET-CT, 59,4%, – for MRI and CT, and 96% - for combination of MRI, PET- CT and CT [20]. During the first stage of study we have been using monomodal data - T2 FLAIR impulse sequences - as the most sensitive mode for visualizing glial tumors. In the future, we plan to include other MRI modalities and also the clinical details allow us to improve our neural network results significantly. As for machine learning instruments, the CNN and support vector machines have proved to be the best ones, probably due to the input data’s character (MR – imaging). Thus, Kocak B et al., in their study of the genetic glioma profile based on MRI data have compared 5 different methods of machine learning [17]. Av- erage AUC and precision ranged from 0,769 to 0,869 and from 80,1% to 84%, correspondingly. Moreover, the neural network showed the best results: AUC – 0,869 and precision – 83,8%. Similar outcomes were achieved by Chang Y et al. in 2019 [6]. In 2020 Zhuge Y et al. generated the automatic glioma grading scale based on the preoperative MRI of deep neural networks, thus achieving the sensitivity of 93,5% and specificity of 97,2% which may play a crucial role in advising the adjuvant therapy without biopsy and histological verification [36]. Generally, all recent publications report good data of machine learning meth- ods on accuracy, sensitivity and specificity: accuracy varied from 79,4% [25] to 98% [34], sensitivity – from 76,92% [24] to 98% [34], specificity – from 70% [25] to 100% [34]. These statistical measures correspond to the metrics obtained by machine learning in our study: accuracy was 91%, sensitivity - 94%, specificity - 89%, thus demonstrating that our neural network maybe useful in predicting motor deficit development or aggravation or in the early postoperative period, and also corresponds to the international level. 5 Conclusions Methods of machine learning allow us to predict the pyramidal symptom aggra- vation with comparatively high accuracy after microsurgical resection based on the preoperative brain MRI data in patients with primary supratentorial gliomas. The neural convolutional network created during the study showed the following metrics: accuracy - 91%, sensitivity - 94%, specificity - 89%. It gives objective information about the risk of neurological deficit, thus being a significant factor in informing patients at the pre-surgical stage and deciding about the treatment tactics. Using machine learning methods for prediction, the motor worsening af- ter surgery is an auspicious tool. This requires further study with the inclusion of input data of various modalities and increasing patients’ number. Acknowledgments. With grant RFBR support №19-29-01154 “Predicting of pyramidal symptoms and its reversibility in patients with supratentorial glial 129 tumors located near the motor areas, using the knowledge transfer method and deep neural networks based on multifactor analysis of digital data of different modality”. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results. References 1. Aliotta, E., Nourzadeh, H., Batchala, P., Schiff, D., Lopes, M., Druz- gal, J., Mukherjee, S., Patel, S.: Molecular subtype classification in lower- grade glioma with accelerated DTI . https://doi.org/10.3174/ajnr.A6162, http://www.ajnr.org/lookup/doi/10.3174/ajnr.A6162 2. Amin, J., Sharif, M., Raza, M., Saba, T., Anjum, M.A.: Brain tumor detection using statistical and machine learning method. 177, 69–79. https://doi.org/10.1016/j.cmpb.2019.05.015, https://linkinghub.elsevier.com/retrieve/pii/S0169260718313786 3. Bacchi, S., Zerner, T., Dongas, J., Asahina, A.T., Abou-Hamden, A., Otto, S., Oakden-Rayner, L., Patel, S.: Deep learning in the detec- tion of high-grade glioma recurrence using multiple MRI sequences: A pilot study. 70, 11–13. https://doi.org/10.1016/j.jocn.2019.10.003, https://linkinghub.elsevier.com/retrieve/pii/S0967586819304709 4. Cao, H., Erson-Omay, E.Z., Li, X., Günel, M., Moliterno, J., Fulbright, R.K.: A quantitative model based on clinically rele- vant MRI features differentiates lower grade gliomas and glioblas- toma. 30(6), 3073–3082. https://doi.org/10.1007/s00330-019-06632-8, http://link.springer.com/10.1007/s00330-019-06632-8 5. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks. (2017) 6. Chang, Y., Lafata, K., Sun, W., Wang, C., Chang, Z., Kirkpatrick, J.P., Yin, F.F.: An investigation of machine learning methods in delta-radiomics fea- ture analysis. 14(12), e0226348. https://doi.org/10.1371/journal.pone.0226348, https://dx.plos.org/10.1371/journal.pone.0226348 7. Chen, S.C.C., Lo, C.M., Wang, S.H., Su, E.C.Y.: RNA editing-based classification of diffuse gliomas: predicting isocitrate dehydrogenase mutation and chromo- some 1p/19q codeletion. 20, 659. https://doi.org/10.1186/s12859-019-3236-0, https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-3236- 0 8. Choi, Y.S., Ahn, S.S., Chang, J.H., Kang, S.G., Kim, E.H., Kim, S.H., Jain, R., Lee, S.K.: Machine learning and radiomic phenotyping of lower grade gliomas: improving survival prediction. 30(7), 3834–3842. https://doi.org/10.1007/s00330- 020-06737-5, http://link.springer.com/10.1007/s00330-020-06737-5 9. Elshafeey, N., Kotrotsou, A., Hassan, A., Elshafei, N., Hassan, I., Ahmed, S., Abrol, S., Agarwal, A., El Salek, K., Bergamaschi, S., Acharya, J., Moron, F.E., Law, M., Fuller, G.N., Huse, J.T., Zinn, P.O., Colen, R.R.: Multicenter study demonstrates radiomic features derived from magnetic resonance perfusion images identify pseu- doprogression in glioblastoma. 10(1), 3170. https://doi.org/10.1038/s41467-019- 11007-0, http://www.nature.com/articles/s41467-019-11007-0 10. Gates, E., Lin, J., Weinberg, J., Prabhu, S., Hamilton, J., Hazle, J., Fuller, G., Baladandayuthapani, V., Fuentes, D., 130 Schellingerhout, D.: Imaging-based algorithm for the local grad- ing of glioma. 41(3), 400–407. https://doi.org/10.3174/ajnr.A6405, http://www.ajnr.org/lookup/doi/10.3174/ajnr.A6405 11. González-Darder, J.M., González-López, P., Talamantes, F., Quilis, V., Cortés, V., Garcı́a-March, G., Roldán, P.: Multimodal navigation in the functional mi- crosurgical resection of intrinsic brain tumors located in eloquent motor areas: role of tractography 28(2), E5. https://doi.org/10.3171/2009.11.FOCUS09234, https://thejns.org/view/journals/neurosurg-focus/28/2/article-pE5.xml 12. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples (2015) 13. Hu, L.S., Hawkins-Daarud, A., Wang, L., Li, J., Swan- son, K.R.: Imaging of intratumoral heterogeneity in high-grade glioma. 477, 97–106. https://doi.org/10.1016/j.canlet.2020.02.025, https://linkinghub.elsevier.com/retrieve/pii/S030438352030094X 14. Huang, R.Y., Guenette, J.P.: Non-invasive diagnosis of h3 k27m mu- tant midline glioma. 22(3), 309–310. https://doi.org/10.1093/neuonc/noz240, https://academic.oup.com/neuro-oncology/article/22/3/309/5681667 15. Jang, K., Russo, C., Di Ieva, A.: Radiomics in gliomas: clini- cal implications of computational modeling and fractal-based anal- ysis. 62(7), 771–790. https://doi.org/10.1007/s00234-020-02403-1, http://link.springer.com/10.1007/s00234-020-02403-1 16. Kaplan, K., Kaya, Y., Kuncan, M., Ertunç, H.M.: Brain tumor clas- sification using modified local binary patterns (LBP) feature extrac- tion methods. 139, 109696. https://doi.org/10.1016/j.mehy.2020.109696, https://linkinghub.elsevier.com/retrieve/pii/S0306987720302632 17. Kocak, B., Durmaz, E.S., Ates, E., Sel, I., Turgut Gunes, S., Kaya, O.K., Zeynalova, A., Kilickesmez, O.: Radiogenomics of lower-grade gliomas: ma- chine learning–based MRI texture analysis for predicting 1p/19q codele- tion status. 30(2), 877–886. https://doi.org/10.1007/s00330-019-06492-2, http://link.springer.com/10.1007/s00330-019-06492-2 18. Krivov, E., Kostjuchenko, V., Dalechina, A., Shirokikh, B., Makarchuk, G., Denisenko, A., Golanov, A., Belyaev, M.: Tumor delineation for brain radio- surgery by a ConvNet and non-uniform patch generation. In: Bai, W., Sanroma, G., Wu, G., Munsell, B.C., Zhan, Y., Coupé, P. (eds.) Patch-Based Techniques in Medical Imaging, vol. 11075, pp. 122–129. Springer International Publishing. https://doi.org/10.1007/978-3-030-00500-9 14 19. Leclerc, P., Ray, C., Mahieu-Williame, L., Alston, L., Frindel, C., Brevet, P.F., Meyronet, D., Guyotat, J., Montcel, B., Rousseau, D.: Machine learning- based prediction of glioma margin from 5-ALA induced PpIX fluores- cence spectroscopy. 10(1), 1462. https://doi.org/10.1038/s41598-020-58299-7, http://www.nature.com/articles/s41598-020-58299-7 20. Matsui, Y., Maruyama, T., Nitta, M., Saito, T., Tsuzuki, S., Tamura, M., Kusuda, K., Fukuya, Y., Asano, H., Kawamata, T., Masamune, K., Muragaki, Y.: Prediction of lower-grade glioma molecular subtypes using deep learning. 146(2), 321–327. https://doi.org/10.1007/s11060-019-03376-9, http://link.springer.com/10.1007/s11060-019-03376-9 21. Mizutani, T., Magome, T., Igaki, H., Haga, A., Nawa, K., Sekiya, N., Nakagawa, K.: Optimization of treatment strategy by using a ma- chine learning model to predict survival time of patients with malignant glioma after radiotherapy. 60(6), 818–824. https://doi.org/10.1093/jrr/rrz066, https://academic.oup.com/jrr/article/60/6/818/5607843 131 22. Nakamoto, T., Takahashi, W., Haga, A., Takahashi, S., Kiryu, S., Nawa, K., Ohta, T., Ozaki, S., Nozawa, Y., Tanaka, S., Mukasa, A., Naka- gawa, K.: Prediction of malignant glioma grades using contrast-enhanced t1-weighted and t2-weighted magnetic resonance images based on a ra- diomic analysis. 9(1), 19411. https://doi.org/10.1038/s41598-019-55922-0, http://www.nature.com/articles/s41598-019-55922-0 23. Neves, B.J., Agnes, J.P., Gomes, M.d.N., Henriques Donza, M.R., Gonçalves, R.M., Delgobo, M., Ribeiro de Souza Neto, L., Senger, M.R., Silva- Junior, F.P., Ferreira, S.B., Zanotto-Filho, A., Andrade, C.H.: Efficient identification of novel anti-glioma lead compounds by machine learn- ing models. 189, 111981. https://doi.org/10.1016/j.ejmech.2019.111981, https://linkinghub.elsevier.com/retrieve/pii/S022352341931133X 24. Ozturk-Isik, E., Cengiz, S., Ozcan, A., Yakicier, C., Ersen Danyeli, A., Pamir, M.N., Özduman, K., Dincer, A.: Identification of IDH and TERTp mutation status using h-MRS in 112 hemispheric dif- fuse gliomas. 51(6), 1799–1809. https://doi.org/10.1002/jmri.26964, https://onlinelibrary.wiley.com/doi/abs/10.1002/jmri.26964 25. Park, Y.W., Choi, Y.S., Ahn, S.S., Chang, J.H., Kim, S.H., Lee, S.K.: Radiomics MRI phenotyping with machine learning to pre- dict the grade of lower-grade gliomas: A study focused on nonen- hancing tumors. 20(9), 1381. https://doi.org/10.3348/kjr.2018.0814, https://www.kjronline.org/DOIx.php?id=10.3348/kjr.2018.0814 26. Qin, J., Li, Y., Liang, D., Zhang, Y., Yao, W.: Histogram analysis of ab- solute cerebral blood volume map can distinguish glioblastoma from solitary brain metastasis. 98(42), e17515. https://doi.org/10.1097/MD.0000000000017515, http://journals.lww.com/00005792-201910180-00033 27. Rathore, S., Niazi, T., Iftikhar, M.A., Chaddad, A.: Glioma grading via analysis of digital pathology images using machine learning. 12(3), 578. https://doi.org/10.3390/cancers12030578, https://www.mdpi.com/2072- 6694/12/3/578 28. Rossi, M., Conti Nibali, M., Viganò, L., Puglisi, G., Howells, H., Gay, L., Sciortino, T., Leonetti, A., Riva, M., Fornia, L., Cerri, G., Bello, L.: Resec- tion of tumors within the primary motor cortex using high-frequency stimula- tion: oncological and functional efficiency of this versatile approach based on clinical conditions. 133(3), 642–654. https://doi.org/10.3171/2019.5.JNS19453, https://thejns.org/view/journals/j-neurosurg/133/3/article-p642.xml 29. Shusharina, N., Söderberg, J., Edmunds, D., Löfman, F., Shih, H., Bortfeld, T.: Automated delineation of the clinical target vol- ume using anatomically constrained 3d expansion of the gross tu- mor volume. 146, 37–43. https://doi.org/10.1016/j.radonc.2020.01.028, https://linkinghub.elsevier.com/retrieve/pii/S0167814020300475 30. Sun, Z., Li, Y., Wang, Y., Fan, X., Xu, K., Wang, K., Li, S., Zhang, Z., Jiang, T., Liu, X.: Radiogenomic analysis of vascular endothelial growth factor in pa- tients with diffuse gliomas. 19(1), 68. https://doi.org/10.1186/s40644-019-0256- y, https://cancerimagingjournal.biomedcentral.com/articles/10.1186/s40644-019- 0256-y 31. Takahashi, S., Takahashi, W., Tanaka, S., Haga, A., Nakamoto, T., Suzuki, Y., Mukasa, A., Takayanagi, S., Kitagawa, Y., Hana, T., Nejo, T., Nomura, M., Nakagawa, K., Saito, N.: Radiomics analysis for glioma malignancy evaluation using diffusion kurtosis and tensor 132 imaging. 105(4), 784–791. https://doi.org/10.1016/j.ijrobp.2019.07.011, https://linkinghub.elsevier.com/retrieve/pii/S0360301619334935 32. Wu, S., Calero-Pérez, P., Villamañan, L., Arias-Ramos, N., Pumarola, M., Ortega-Martorell, S., Julià-Sapé, M., Arús, C., Candiota, A.P.: Anti-tumour immune response in GL261 glioblastoma generated by temozolomide immune-enhancing metronomic schedule monitored with MRSI-based nosological images. 33(4). https://doi.org/10.1002/nbm.4229, https://onlinelibrary.wiley.com/doi/abs/10.1002/nbm.4229 33. Wu, Y., Zhao, Z., Wu, W., Lin, Y., Wang, M.: Automatic glioma segmentation based on adaptive superpixel. 19(1), 73, https://bmcmedimaging.biomedcentral.com/articles/10.1186/s12880-019-0369-6 34. Zhang, Z., Xiao, J., Wu, S., Lv, F., Gong, J., Jiang, L., Yu, R., Luo, T.: Deep con- volutional radiomic features on diffusion tensor images for classification of glioma grades. 33(4), 826–837, http://link.springer.com/10.1007/s10278-020-00322-4 35. Zhao, J., Huang, Y., Song, Y., Xie, D., Hu, M., Qiu, H., Chu, J.: Diagnos- tic accuracy and potential covariates for machine learning to identify IDH mu- tations in glioma patients: evidence from a meta-analysis. 30(8), 4664–4674, http://link.springer.com/10.1007/s00330-020-06717-9 36. Zhuge, Y., Ning, H., Mathen, P., Cheng, J.Y., Krauze, A.V., Cam- phausen, K., Miller, R.W.: Automated glioma grading on conventional MRI images using deep convolutional neural networks. 47(7), 3044–3053, https://onlinelibrary.wiley.com/doi/abs/10.1002/mp.14168 133