Introduction

Surgery of the brain's glial tumors located in the motor areas vicinity is a truly existing problem provoking several contradictions among neurosurgeons. The surgery in this area is associated with a high risk of transitory and persistent neurological disability [11,28], which may interfere with the following treatment process (chemo or radiotherapy) in this group of patients, and as such, may directly affect not only the quality of life but also the overall survival. Patients and their relatives should be preoperatively informed on a possible neurological deficit that may develop, and it is them to decide whether to undergo surgery or refuse it. Today, this type of information is subjectively provided by a physician, based on personal experience and feelings. Meanwhile, no method allows for an objective preoperative data-based prognosis of the risk of neurological impairment in each particular case. Simultaneously, machine learning methods that were widely used in various spheres of our life and were found to be especially useful in predicting a neurological disorder have a limited application in neurosurgery. The PubMed search by the keywords "glioma machine learning" has resulted in 286 publications by May 1st, 2020. We found no paper describing machine learning methods in predicting motor deficit development or aggravation in patients with glial tumors.

Objective

We aim to develop a convolution neuronal network (CNN) for predicting the risk of motor deficit development or aggravation based on preoperative MRI-data in patients with supratentorial brain tumors in the early postoperative period.

Materials and Methods

Inclusion Criteria

Patients aged 18 years and older, who underwent microsurgical brain tumor removal at N.N. Burdenko National Medical Research Center of Neurosurgery (the Center) from 2013 to 2020 entered this study. In all cases, there were verified primary gliomas of different grades located supratentorial. Preoperative MRI (T1, T2, T2 FLAIR, DWI, T1+C) was performed in all patients at the Center not earlier than one month before surgery.

MRI Data Collection

Patient selection, defined by the inclusion criteria, was carried out by three experienced radiologists based on the database of patients examined in the department of X-ray and radio-isotope methods of diagnosis at the Center. The database has been stored since 2013 and contains details on more than 4500 patients with brain lesions of different histological structures and locations, primarily or secondarily operated. It contains DISCOM image files from the MRI studies performed on 1,5 -3 Tesla MRI scanners (GE Signa HDxt 3.0T, GE Signa HDxt 1.5T, GE Optima 450w 1.5T (GE Healthcare, Milwaukee, USA) using 8channel imaging and an electronic list of patients (an Excel table) providing large amounts of primary data (patient name, surname, personal identification number, age, case record No, date of preoperative MRI, date, and type of surgery, and morphological diagnosis). The majority (70%) of patients had morphologically verified glial tumors of various locations. Other lesions like ependymomas, meningiomas, neurinomas were excluded from the trial, as well as patients who had no microsurgical tumor removal. To understand whether the rest of the patients met the inclusion criteria, their manually traced MRI data were estimated by the Centre's radiologists. Finally, 527 of the full 4500 MRI case recordings were selected to enter the study group. All personal data were anonymized, and each patient got a personal identification number.

Data on Patient Motor Status Before Surgery and in the Early Postoperative Stage

Data on patients' motor status were obtained via analysis of electronic case records, which traditionally included information about the muscle strength from the upper and lower extremities assessed by a 6-score grading scale (0-5) before the operation and in the early postoperative period. Considering the large size of the trial, the problem was solved using the original semiautomatic algorithm for natural language processing (Algorithm) developed at the Center. The Algorithm made up 91%, the logistic regression accuracy -85%. All markup results were additionally estimated by an expert using the special support program, thus allowing sorting out the database fields of interest with their following check-up.

Simultaneously, an expert concluded paresis: whether it was present or absent after surgery and its dynamics by the date of discharge from the neurosurgical clinic. According to the performed analysis, pre-surgical paresis was marked in 23% of cases. Its aggravation or muscle weakness in the postoperative period was observed in 14.3% of cases.

The Dataset Obtained

Thus, the dataset was formed comprising MRI datasets in different pulse sequences (T1, T2, T2 FLAIR, DWI, T1+C) in 527 patients, and data on the motor status dynamics in the early postoperative period for this group of patients. The information about sex, age, histological diagnosis, and tumor location was not used for CNN learning; it is listed in the Table 1.

It should be noticed that patients operated in all departments of the Center (except for pediatric and spinal ones) were enrolled in this study. Different surgical techniques were carried out by more than 20 surgeons, thus making this study group universe. LGG -25%, HGG -75 % localization frontal lobe -37,6%, parietal lobe --25,5%, temporal lobe -18%, occipital lobe -13,5%, more than one lobe -5,4%

Neural Network for Predicting the Development or Growth of Paresis in the Early Postoperative Period

Statement of the Problem. It is necessary to develop a method for predicting the increase in pyramidal deficiency in the postoperative period in patients with supratentorial glial brain tumors located near the motor zones of the cortex and pyramidal tract, according to MRI studies of the patient's brain at a level that meets the specified quality criteria and is binary.

Formal Definition of the Problem. Let X be the set of preoperative MRI slices of patients, and Y be the set of classes to which the elements of X may belong, in our case Y ∈ {0, 1}. It is necessary to develop an algorithm A that will allow classifying an arbitrary slice x ∈ X:

A: X → Y(1)

Data Preparation. The source dataset of different modalities X was the results of MR-research of the brain carried out on 527 patients who were operated on with a diagnosis of a supratentorial glial tumor. The data set contained information on the increase in the pyramidal deficit in the postoperative period for each research. Each research from the dataset consisted of MRI images performed in the following modes: T1, T2, T2 FLAIR, DWI, T1+C.

The data preparation stage consisted of the following steps:

1. Markup. Radiologists match each slice, which was performed in a T2 FLAIR regime, with a binary label. If a tumor was present on the slice, the label was equal to 1, otherwise 0. 2. Converting. Marked slices were converted from DICOM format to PNG format. 3. Normalization. All data were centered and normalized (Figure 1). Model Architecture. In the course of work on the project, our architecture of an artificial neural network (ANN) "Shallow" was developed (Figure 2). Also, experiments were carried out using popular ANNs pre-trained on the ImageNet dataset, such as VGG16, VGG19, Inception v3, followed by their fine-tuning on the MRI data.

Description of "Shallow" ANN:

1. An input layer that takes into account the two-dimensional topology of the image and accepts a normalized, black and white image of 100 × 100 pixels as input. 2. A convolutional layer contains 32 filters with a 3 × 3 kernel. 3. Activation layer using ReLU 16. Activation layer using ReLU. 17. Dense layer with 1 unit. 18. Activation layer using a sigmoid function

f (x) = max(0, x)(2) 4.f (x) = 1 1 + ϵ ( − x)(3)

To prevent overfitting of the model, in addition to disabling random neurons, L2-regularization is used. Loss Function. The binary cross-entropy was chosen as the loss function:

L(y, ŷ) = −y log ŷ − (1 − y) log(1 − ŷ),(4)

where y -the class the slice belongs to, ŷ -predicted class.

Augmentations. In addition to common augmentations such as random horizontal or vertical flips, scaling and rotation, it was suggested to use adversarial training. Adversarial examples are images with addition of specially generated noise, which is visually not distinguishable by the human eye, but causes the model to make incorrect predictions. We used Fast Gradient Sign Method [12] and Carlini-Wagner L2 algorithm [5] to generate two adversarial examples for each sample in the training set. The model will be evaluated separately with the use of adversarial examples and without them.

Models Evaluating. For models evaluating, we used the following metrics:

-Accuracy = Training and Evaluating. Model training and evaluating were carried out on the following equipment:

-Intel Core i9-9900KF CPU 3.6 -RAM 32Gb -GPU NVIDIA 2080Ti 11Gb

As a basic framework for developing the model, we used TensorFlow 2.0. The prepared dataset was split into three parts. 70% of the total number of slices was included in the training set, 20% in the validation set, and 10% in the test set. The resulting datasets are class-balanced. The batch size was empirically chosen equal to 64, the number of epochs was 70. AdamOptimizer was used as an optimizer, and the learning rate coefficient was chosen as a piecewise constant function with an initial value of 10 −4 , decreasing dynamically. After each epoch, the model was validated on 10 batches of 64 elements from the validation dataset. After training, the quality of the model was evaluated in an automatic mode on test data. The model was then manually verified by a neurosurgeon who used his dataset of MRI scans of 50 patients that were not present in the original dataset.

Results. The results of the experiments are presented in Table 2. According to these results, the best model on the test sample showed Accuracy 91%, Sensitivity 94%, Specificity 89%, ROC AUC 91%, and F1 92% in predicting the increase in hemiparesis in the early postoperative period in patients with supratentorial gliomas of the brain. Figure 3 shows examples of preoperative MRI data, based on which the CNN correctly predicted the increase in motor deficits in the postoperative period (ab), the absence of deterioration in motor status after surgery (c), and made a mistake in predicting the increase in motor deficits after surgery(d).

Discussion

Methods of machine learning are of limited use in neurosurgery, and in brain glioma surgery, in particular. The typical task spectrum that may be solved using machine learning methods is presented in Table 3.

Table 3. Main directions of application of machine learning methods in brain glioma surgery according to the literature analysis.

Task

Examples of publications determining the genetic glioma profile Zhao J2020 [35], Chen SC2019 [7], Matsui Y2019 [20], Kocak B2019 [17], Aliotta [1], Ozturk-Isik E2019 [24], Sun Z2019 [30], Huang RY2020 [14] defining grades of brain gliomas Zhuge Y2020 [36], Rathore S2020 [27], Zhang Z2020 [34], Cao H2002 [4], Nakamoto T2019 [22], Park YW2019 [25], akahashi S 2019 [31] defining grades of gliomas based on intratumoral heterogeneity Hu LS2020 [13], Gates EDH2020 [10] differential diagnosis with other intracranial tumors Jianhua Qin 2019 [26], Kaplan K2020 [16] differential diagnosis with post-radiation changes Elshafeey N 2019 [9], Bacchi S2019 [3] prognosis of the overall patient survival Mizutani T2019 [21], Jang K2020 [15], Choi YS2020 [8] [26], Chang Y2019 [6] automatic segmentation of tumors Shusharina N2020 [29], Wu Y2019 [33], Krivov E2018 [18] early detection of gliomas Amin J2019 [2] prognostic prospects of chemotherapy Neves BJ2020 [23], Wu S2020 [32] We failed to discover publications devoted to machine learning in pyramidal insufficiency prognosis in patients with brain gliomas. As in the majority of papers, our CNN has analyzed MR imaging data of patients. Our literature survey analysis showed that most commonly conventional MR imaging (1, 2, 1+C, FLAIR) [17,30,22,25,8,6,33] was used as input data, rarely DWI [35,31,3], DTI [1] and MR perfusion [10,26,9] were additionally used. Besides, there were some papers reporting on the use of proton MR-spectroscopy [24], kurtosis MRI [31], PET-CT [35,20]. Alongside with neuroimaging data, some other graphic information was applied (but considerably rarely): photos of histological glioma slides [27], spectroscopic 5-ALA accumulation curves [19], as well as text data: results of RNA sequencing [7], clinical data (such as sex, age, extent of resection etc.) [35,20]. It stresses, that methods of machine learning can effectively utilize different types of information (multimodal data), with the best sensitivity and specificity effect being reached by their rational combination. Thus, multimodal data permitted increasing sensitivity and specificity from 83% to 90% and from 82% to 90%, respectively [35], and accuracy from 73, 14% to 91,48%, sensitivity from 72,84% to 93,47%, specificity from 74,05% to 85,36% [27]. It is also true when using different modalities of MR imaging or other additional methods of neuroimaging. Thus, Matsui Y et al. reports the following accuracy for different neuroimaging modalities: 58,5% -for MRI only, 60,4% -for MRI and PET-CT, 59,4%, -for MRI and CT, and 96% -for combination of MRI, PET-CT and CT [20]. During the first stage of study we have been using monomodal data -T2 FLAIR impulse sequences -as the most sensitive mode for visualizing glial In the future, we plan to include other MRI modalities and also the clinical details allow us to improve our neural network results significantly. As for machine learning instruments, the CNN and support vector machines have proved to be the best ones, probably due to the input data's character (MRimaging). Thus, Kocak B et al., in their study of the genetic glioma profile based on MRI data have compared 5 different methods of machine learning [17]. Average AUC and precision ranged from 0,769 to 0,869 and from 80,1% to 84%, correspondingly. Moreover, the neural network showed the best results: AUC -0,869 and precision -83,8%. Similar outcomes were achieved by Chang Y et al. in 2019 [6]. In 2020 Zhuge Y et al. generated the automatic glioma grading scale based on the preoperative MRI of deep neural networks, thus achieving the sensitivity of 93,5% and specificity of 97,2% which may play a crucial role in advising the adjuvant therapy without biopsy and histological verification [36]. Generally, all recent publications report good data of machine learning methods on accuracy, sensitivity and specificity: accuracy varied from 79,4% [25] to 98% [34], sensitivity -from 76,92% [24] to 98% [34], specificity -from 70% [25] to 100% [34]. These statistical measures correspond to the metrics obtained by machine learning in our study: accuracy was 91%, sensitivity -94%, specificity -89%, thus demonstrating that our neural network maybe useful in predicting motor deficit development or aggravation or in the early postoperative period, and also corresponds to the international level.

Conclusions

Methods of machine learning allow us to predict the pyramidal symptom aggravation with comparatively high accuracy after microsurgical resection based on the preoperative brain MRI data in patients with primary supratentorial gliomas. The neural convolutional network created during the study showed the following metrics: accuracy -91%, sensitivity -94%, specificity -89%. It gives objective information about the risk of neurological deficit, thus being a significant factor in informing patients at the pre-surgical stage and deciding about the treatment tactics. Using machine learning methods for prediction, the motor worsening after surgery is an auspicious tool. This requires further study with the inclusion of input data of various modalities and increasing patients' number. tumors located near the motor areas, using the knowledge transfer method and deep neural networks based on multifactor analysis of digital data of different modality". The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.