Surgery of glial tumors of the brain located in the motor areas vicinity is associated with a high risk of increasing neurological deficits. Motor deficit afects overall survival in this group of patients. Nowadays, no method allows for an objective preoperative data-based prognosis of the risk of neurological impairment in each particular case. Objective: develop a convolution neuronal network that can predict motor worsening in patients with supranational gliomas using the preoperative MRI-data. Materials and methods: the study included 527 patients aged 18 years and older with newly identified supratentorial gliomas. All patients underwent preoperative MRI and tumor removal based on Burdenko National Center of neurosurgery in 2013-2019. Data on motor status dynamics after surgery for these patients were obtained from the electronic medical records using the original semiautomatic algorithm for natural language processing. The T2FLAIR mode is used for training our model. The model demonstrates the following metrics of quality: accuracy 91%, sensitivity 94%, specificity 89%, ROC AUC 91%, and F1 92%. Thus, machine learning methods predict the motor worsening with relatively high accuracy in patients with supratentorial gliomas at the preoperative stage, based on brain MRI data.
Surgery of the brain’s glial tumors located in the motor areas vicinity is a truly existing problem provoking several contradictions among neurosurgeons. The surgery in this area is associated with a high risk of transitory and persistent neurological disability [11, 28], which may interfere with the following treatment process (chemo or radiotherapy) in this group of patients, and as such, may directly afect not only the quality of life but also the overall survival. Patients and their relatives should be preoperatively informed on a possible neurological deficit that may develop, and it is them to decide whether to undergo surgery or refuse it. Today, this type of information is subjectively provided by a physician, based on personal experience and feelings. Meanwhile, no method allows for an objective preoperative data-based prognosis of the risk of neurological impairment in each particular case. Simultaneously, machine learning methods that were widely used in various spheres of our life and were found to be especially useful in predicting a neurological disorder have a limited application in neurosurgery. The PubMed search by the keywords “glioma machine learning” has resulted in 286 publications by May 1st, 2020. We found no paper describing machine learning methods in predicting motor deficit development or aggravation in patients with glial tumors. 2
We aim to develop a convolution neuronal network (CNN) for predicting the risk of motor deficit development or aggravation based on preoperative MRI-data in patients with supratentorial brain tumors in the early postoperative period. 3 3.1
Inclusion Criteria Patients aged 18 years and older, who underwent microsurgical brain tumor removal at N.N. Burdenko National Medical Research Center of Neurosurgery (the Center) from 2013 to 2020 entered this study. In all cases, there were verified primary gliomas of diferent grades located supratentorial. Preoperative MRI (T1, T2, T2 FLAIR, DWI, T1+C) was performed in all patients at the Center not earlier than one month before surgery. 3.2
MRI Data Collection Patient selection, defined by the inclusion criteria, was carried out by three experienced radiologists based on the database of patients examined in the department of X-ray and radio-isotope methods of diagnosis at the Center. The database has been stored since 2013 and contains details on more than 4500 patients with brain lesions of diferent histological structures and locations, primarily or secondarily operated. It contains DISCOM image files from the MRI studies performed on 1,5 - 3 Tesla MRI scanners (GE Signa HDxt 3.0T, GE Signa HDxt 1.5T, GE Optima 450w 1.5T (GE Healthcare, Milwaukee, USA) using 8channel imaging and an electronic list of patients (an Excel table) providing large amounts of primary data (patient name, surname, personal identification number, age, case record No, date of preoperative MRI, date, and type of surgery, and morphological diagnosis). The majority (70%) of patients had morphologically verified glial tumors of various locations. Other lesions like ependymomas, meningiomas, neurinomas were excluded from the trial, as well as patients who had no microsurgical tumor removal. To understand whether the rest of the patients met the inclusion criteria, their manually traced MRI data were estimated by the Centre’s radiologists. Finally, 527 of the full 4500 MRI case recordings were selected to enter the study group. All personal data were anonymized, and each patient got a personal identification number. 3.3
Data on Patient Motor Status Before Surgery and in the Early
Postoperative Stage
Data on patients’ motor status were obtained via analysis of electronic case
records, which traditionally included information about the muscle strength from
the upper and lower extremities assessed by a 6-score grading scale (
The Dataset Obtained Thus, the dataset was formed comprising MRI datasets in diferent pulse sequences (T1, T2, T2 FLAIR, DWI, T1+C) in 527 patients, and data on the motor status dynamics in the early postoperative period for this group of patients. The information about sex, age, histological diagnosis, and tumor location was not used for CNN learning; it is listed in the Table 1.
It should be noticed that patients operated in all departments of the Center (except for pediatric and spinal ones) were enrolled in this study. Diferent surgical techniques were carried out by more than 20 surgeons, thus making this study group universe. Statement of the Problem. It is necessary to develop a method for predicting the increase in pyramidal deficiency in the postoperative period in patients with supratentorial glial brain tumors located near the motor zones of the cortex and pyramidal tract, according to MRI studies of the patient’s brain at a level that meets the specified quality criteria and is binary.
Formal Definition of the Problem. Let X be the set of preoperative MRI slices of patients, and Y be the set of classes to which the elements of X may belong, in our case Y ∈ {0, 1}. It is necessary to develop an algorithm A that will allow classifying an arbitrary slice x ∈ X:
A: X → Y
(
The data preparation stage consisted of the following steps: 1. Markup. Radiologists match each slice, which was performed in a T2 FLAIR regime, with a binary label. If a tumor was present on the slice, the label was equal to 1, otherwise 0. 2. Converting. Marked slices were converted from DICOM format to PNG format. 3. Normalization. All data were centered and normalized (Figure 1). Model Architecture. In the course of work on the project, our architecture of an artificial neural network (ANN) ”Shallow” was developed (Figure 2). Also, experiments were carried out using popular ANNs pre-trained on the ImageNet dataset, such as VGG16, VGG19, Inception v3, followed by their fine-tuning on the MRI data.
Description of ”Shallow” ANN:
1. An input layer that takes into account the two-dimensional topology of the
image and accepts a normalized, black and white image of 100 × 100 pixels
as input.
2. A convolutional layer contains 32 filters with a 3 × 3 kernel.
3. Activation layer using ReLU
f (x) = max(0, x)
(
1
1 + ϵ ( − x)
(
To prevent overfitting of the model, in addition to disabling random neurons,
L2-regularization is used.
Loss Function. The binary cross-entropy was chosen as the loss function:
L(y, yˆ) = −y log yˆ − (1 − y) log(1 − yˆ),
(
Augmentations. In addition to common augmentations such as random horizontal or vertical flips, scaling and rotation, it was suggested to use adversarial training. Adversarial examples are images with addition of specially generated noise, which is visually not distinguishable by the human eye, but causes the model to make incorrect predictions. We used Fast Gradient Sign Method [12] and Carlini-Wagner L2 algorithm [5] to generate two adversarial examples for each sample in the training set. The model will be evaluated separately with the use of adversarial examples and without them.
Models Evaluating. For models evaluating, we used the following metrics:
T P +T N – Accuracy = T P +T N+F P +F N
T P – Sensitivity = T P +F N
T N – Specif icity = T N+F P – ROCAU C = R01 T P R dF P R, where T P R = T P +F N , F P R = T N+F P T P F P – F 1 = 2 PP rreecciissiioonn×R+eRceaclalll , P recision = T P/(T P + F P ), Recall = T P/(T P + F N ), where TP – true positive, TN – true negative, FP – false positive, FN – false negative.
Training and Evaluating. Model training and evaluating were carried out on the following equipment: – Intel Core i9-9900KF CPU 3.6 – RAM 32Gb – GPU NVIDIA 2080Ti 11Gb
As a basic framework for developing the model, we used TensorFlow 2.0. The prepared dataset was split into three parts. 70% of the total number of slices was included in the training set, 20% in the validation set, and 10% in the test set. The resulting datasets are class-balanced. The batch size was empirically chosen equal to 64, the number of epochs was 70. AdamOptimizer was used as an optimizer, and the learning rate coeficient was chosen as a piecewise constant function with an initial value of 10−4 , decreasing dynamically. After each epoch, the model was validated on 10 batches of 64 elements from the validation dataset. After training, the quality of the model was evaluated in an automatic mode on test data. The model was then manually verified by a neurosurgeon who used his dataset of MRI scans of 50 patients that were not present in the original dataset. Results. The results of the experiments are presented in Table 2. According to these results, the best model on the test sample showed Accuracy 91%, Sensitivity 94%, Specificity 89%, ROC AUC 91%, and F1 92% in predicting the increase in hemiparesis in the early postoperative period in patients with supratentorial gliomas of the brain.
Figure 3 shows examples of preoperative MRI data, based on which the CNN correctly predicted the increase in motor decfiits in the postoperative period (ab), the absence of deterioration in motor status after surgery (c), and made a mistake in predicting the increase in motor deficits after surgery(d).
Methods of machine learning are of limited use in neurosurgery, and in brain glioma surgery, in particular. The typical task spectrum that may be solved using machine learning methods is presented in Table 3.
We failed to discover publications devoted to machine learning in pyramidal insuficiency prognosis in patients with brain gliomas. As in the majority of papers, our CNN has analyzed MR imaging data of patients. Our literature survey analysis showed that most commonly conventional MR imaging (1, 2, 1+C, FLAIR) [17, 30, 22, 25, 8, 6, 33] was used as input data, rarely DWI [35, 31, 3], DTI [1] and MR perfusion [10, 26, 9] were additionally used. Besides, there were some papers reporting on the use of proton MR-spectroscopy [24], kurtosis MRI [31], PET-CT [35, 20]. Alongside with neuroimaging data, some other graphic information was applied (but considerably rarely): photos of histological glioma slides [27], spectroscopic 5-ALA accumulation curves [19], as well as text data: results of RNA sequencing [7], clinical data (such as sex, age, extent of resection etc.) [35, 20]. It stresses, that methods of machine learning can efectively utilize diferent types of information (multimodal data), with the best sensitivity and specificity efect being reached by their rational combination. Thus, multimodal data permitted increasing sensitivity and specificity from 83% to 90% and from 82% to 90%, respectively [35], and accuracy from 73, 14% to 91,48%, sensitivity from 72,84% to 93,47%, specificity from 74,05% to 85,36% [27]. It is also true when using diferent modalities of MR imaging or other additional methods of neuroimaging. Thus, Matsui Y et al. reports the following accuracy for diferent neuroimaging modalities: 58,5% - for MRI only, 60,4% - for MRI and PET-CT, 59,4%, – for MRI and CT, and 96% - for combination of MRI, PETCT and CT [20]. During the first stage of study we have been using monomodal data - T2 FLAIR impulse sequences - as the most sensitive mode for visualizing glial tumors. In the future, we plan to include other MRI modalities and also the clinical details allow us to improve our neural network results significantly. As for machine learning instruments, the CNN and support vector machines have proved to be the best ones, probably due to the input data’s character (MR – imaging). Thus, Kocak B et al., in their study of the genetic glioma profile based on MRI data have compared 5 diferent methods of machine learning [17]. Average AUC and precision ranged from 0,769 to 0,869 and from 80,1% to 84%, correspondingly. Moreover, the neural network showed the best results: AUC – 0,869 and precision – 83,8%. Similar outcomes were achieved by Chang Y et al. in 2019 [6]. In 2020 Zhuge Y et al. generated the automatic glioma grading scale based on the preoperative MRI of deep neural networks, thus achieving the sensitivity of 93,5% and specificity of 97,2% which may play a crucial role in advising the adjuvant therapy without biopsy and histological verification [36]. Generally, all recent publications report good data of machine learning methods on accuracy, sensitivity and specificity: accuracy varied from 79,4% [25] to 98% [34], sensitivity – from 76,92% [24] to 98% [34], specificity – from 70% [25] to 100% [34]. These statistical measures correspond to the metrics obtained by machine learning in our study: accuracy was 91%, sensitivity - 94%, specificity - 89%, thus demonstrating that our neural network maybe useful in predicting motor deficit development or aggravation or in the early postoperative period, and also corresponds to the international level. 5
Methods of machine learning allow us to predict the pyramidal symptom
aggravation with comparatively high accuracy after microsurgical resection based on
the preoperative brain MRI data in patients with primary supratentorial gliomas.
The neural convolutional network created during the study showed the following
metrics: accuracy - 91%, sensitivity - 94%, specificity - 89%. It gives objective
information about the risk of neurological deficit, thus being a significant factor
in informing patients at the pre-surgical stage and deciding about the treatment
tactics. Using machine learning methods for prediction, the motor worsening
after surgery is an auspicious tool. This requires further study with the inclusion
of input data of various modalities and increasing patients’ number.
Acknowledgments. With grant RFBR support №19-29-01154 “Predicting of
pyramidal symptoms and its reversibility in patients with supratentorial glial
tumors located near the motor areas, using the knowledge transfer method and
deep neural networks based on multifactor analysis of digital data of diferent
modality”. The funders had no role in the design of the study; in the collection,
analyses, or interpretation of data; in the writing of the manuscript, or in the
decision to publish the results.
Schellingerhout, D.: Imaging-based algorithm for the local
grading of glioma. 41(