=Paper=
{{Paper
|id=Vol-3036/paper09
|storemode=property
|title=Predicting the Increase in Postoperative Motor Deficits in Patients with Supratentorial Gliomas Using Machine Learning Methods
|pdfUrl=https://ceur-ws.org/Vol-3036/paper09.pdf
|volume=Vol-3036
|authors=Alexandra Kosyrkova,Eugene Ilyushin,Daniel Saada,Ramin Afandiev,Alexander Baev,Eudard Pogosbekyan,Vladimir Okhlopkov,Gleb Danilov,Artem Batalov,Igor Pronin,Natalia Zakharova,Anna Ogurtsova,Alexander Kravchuck,David Pitskhelauri,Alexander Potapov,Sergey Goryaynov
|dblpUrl=https://dblp.org/rec/conf/rcdl/KosyrkovaISABPO21
}}
==Predicting the Increase in Postoperative Motor Deficits in Patients with Supratentorial Gliomas Using Machine Learning Methods ==
https://ceur-ws.org/Vol-3036/paper09.pdf
Predicting the Increase in Postoperative Motor
Deficits in Patients with Supratentorial Gliomas
Using Machine Learning Methods⋆
Alexandra Kosyrkova1[0000−0002−3019−5203] , Eugene
2[0000−0002−9891−8658]
Ilyushin , Daniel Saada2[0000−0003−4959−8093] , Ramin
1[0000−0001−6384−7960]
Afandiev , Alexander Baev1[0000−0003−4908−0534] , Eudard
1[0000−0002−4803−6948]
Pogosbekyan , Vladimir Okhlopkov1[0000−0001−8911−2372] ,
1[0000−0003−1442−5993]
Gleb Danilov , Artem Batalov1[0000−0002−8924−7346] , Igor
1[0000−0003−0326−7942]
Pronin , Natalia Zakharova1[0000−0002−0516−3613] , Anna
1[0000−0003−3595−2696]
Ogurtsova , Alexander Kravchuck1[0000−0002−3112−8256] ,
1[0000−0003−0374−7970]
David Pitskhelauri , Alexander
Potapov1[0000−0001−8343−3511] , and Sergey Goryaynov1[0000−0002−6480−3270]
1
“N. N. Burdenko National Medical Research Center of Neurosurgery” of the
Ministry of Health of the Russian Federation akosyrkova@nsi.ru
2
Lomonosov Moscow State University eugene.ilyushin@gmail.com
Abstract. Surgery of glial tumors of the brain located in the motor
areas vicinity is associated with a high risk of increasing neurological
deficits. Motor deficit affects overall survival in this group of patients.
Nowadays, no method allows for an objective preoperative data-based
prognosis of the risk of neurological impairment in each particular case.
Objective: develop a convolution neuronal network that can predict mo-
tor worsening in patients with supranational gliomas using the preoper-
ative MRI-data. Materials and methods: the study included 527 patients
aged 18 years and older with newly identified supratentorial gliomas.
All patients underwent preoperative MRI and tumor removal based on
Burdenko National Center of neurosurgery in 2013-2019. Data on motor
status dynamics after surgery for these patients were obtained from the
electronic medical records using the original semiautomatic algorithm
for natural language processing. The T2FLAIR mode is used for train-
ing our model. The model demonstrates the following metrics of quality:
accuracy 91%, sensitivity 94%, specificity 89%, ROC AUC 91%, and F1
92%. Thus, machine learning methods predict the motor worsening with
relatively high accuracy in patients with supratentorial gliomas at the
preoperative stage, based on brain MRI data.
Keywords: Glioma · Machine learning · Corticospinal tract · Convolu-
tional neural network · Paresis.
⋆
Supported by the RFBR, Project №19-29-0115.
Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0).
120
1 Introduction
Surgery of the brain’s glial tumors located in the motor areas vicinity is a truly
existing problem provoking several contradictions among neurosurgeons. The
surgery in this area is associated with a high risk of transitory and persistent
neurological disability [11, 28], which may interfere with the following treatment
process (chemo or radiotherapy) in this group of patients, and as such, may
directly affect not only the quality of life but also the overall survival. Patients
and their relatives should be preoperatively informed on a possible neurological
deficit that may develop, and it is them to decide whether to undergo surgery or
refuse it. Today, this type of information is subjectively provided by a physician,
based on personal experience and feelings. Meanwhile, no method allows for an
objective preoperative data-based prognosis of the risk of neurological impair-
ment in each particular case. Simultaneously, machine learning methods that
were widely used in various spheres of our life and were found to be especially
useful in predicting a neurological disorder have a limited application in neu-
rosurgery. The PubMed search by the keywords “glioma machine learning” has
resulted in 286 publications by May 1st, 2020. We found no paper describing ma-
chine learning methods in predicting motor deficit development or aggravation
in patients with glial tumors.
2 Objective
We aim to develop a convolution neuronal network (CNN) for predicting the risk
of motor deficit development or aggravation based on preoperative MRI-data in
patients with supratentorial brain tumors in the early postoperative period.
3 Materials and Methods
3.1 Inclusion Criteria
Patients aged 18 years and older, who underwent microsurgical brain tumor
removal at N.N. Burdenko National Medical Research Center of Neurosurgery
(the Center) from 2013 to 2020 entered this study. In all cases, there were verified
primary gliomas of different grades located supratentorial. Preoperative MRI
(T1, T2, T2 FLAIR, DWI, T1+C) was performed in all patients at the Center
not earlier than one month before surgery.
3.2 MRI Data Collection
Patient selection, defined by the inclusion criteria, was carried out by three
experienced radiologists based on the database of patients examined in the de-
partment of X-ray and radio-isotope methods of diagnosis at the Center. The
database has been stored since 2013 and contains details on more than 4500
121
patients with brain lesions of different histological structures and locations, pri-
marily or secondarily operated. It contains DISCOM image files from the MRI
studies performed on 1,5 - 3 Tesla MRI scanners (GE Signa HDxt 3.0T, GE Signa
HDxt 1.5T, GE Optima 450w 1.5T (GE Healthcare, Milwaukee, USA) using 8-
channel imaging and an electronic list of patients (an Excel table) providing large
amounts of primary data (patient name, surname, personal identification num-
ber, age, case record No, date of preoperative MRI, date, and type of surgery,
and morphological diagnosis). The majority (70%) of patients had morphologi-
cally verified glial tumors of various locations. Other lesions like ependymomas,
meningiomas, neurinomas were excluded from the trial, as well as patients who
had no microsurgical tumor removal. To understand whether the rest of the pa-
tients met the inclusion criteria, their manually traced MRI data were estimated
by the Centre’s radiologists. Finally, 527 of the full 4500 MRI case recordings
were selected to enter the study group. All personal data were anonymized, and
each patient got a personal identification number.
3.3 Data on Patient Motor Status Before Surgery and in the Early
Postoperative Stage
Data on patients’ motor status were obtained via analysis of electronic case
records, which traditionally included information about the muscle strength from
the upper and lower extremities assessed by a 6-score grading scale (0-5) before
the operation and in the early postoperative period. Considering the large size
of the trial, the problem was solved using the original semiautomatic algorithm
for natural language processing (Algorithm) developed at the Center. The Algo-
rithm made up 91%, the logistic regression accuracy - 85%. All markup results
were additionally estimated by an expert using the special support program, thus
allowing sorting out the database fields of interest with their following check-up.
Simultaneously, an expert concluded paresis: whether it was present or absent
after surgery and its dynamics by the date of discharge from the neurosurgical
clinic. According to the performed analysis, pre-surgical paresis was marked in
23% of cases. Its aggravation or muscle weakness in the postoperative period
was observed in 14.3% of cases.
3.4 The Dataset Obtained
Thus, the dataset was formed comprising MRI datasets in different pulse se-
quences (T1, T2, T2 FLAIR, DWI, T1+C) in 527 patients, and data on the
motor status dynamics in the early postoperative period for this group of pa-
tients. The information about sex, age, histological diagnosis, and tumor location
was not used for CNN learning; it is listed in the Table 1.
It should be noticed that patients operated in all departments of the Center
(except for pediatric and spinal ones) were enrolled in this study. Different sur-
gical techniques were carried out by more than 20 surgeons, thus making this
study group universe.
122
Table 1. The main characteristics of the group of patients selected for this study.
Parameter Value
total number of patients 527
mean age 39, 8 ± 17, 9 years
women/men 50,4%/49,6%
histology LGG – 25%, HGG – 75 %
frontal lobe – 37,6%,
parietal lobe -– 25,5%,
localization temporal lobe – 18%,
occipital lobe – 13,5%,
more than one lobe – 5,4%
3.5 Neural Network for Predicting the Development or Growth of
Paresis in the Early Postoperative Period
Statement of the Problem. It is necessary to develop a method for predicting
the increase in pyramidal deficiency in the postoperative period in patients with
supratentorial glial brain tumors located near the motor zones of the cortex and
pyramidal tract, according to MRI studies of the patient’s brain at a level that
meets the specified quality criteria and is binary.
Formal Definition of the Problem. Let X be the set of preoperative MRI
slices of patients, and Y be the set of classes to which the elements of X may
belong, in our case Y ∈ {0, 1}. It is necessary to develop an algorithm A that
will allow classifying an arbitrary slice x ∈ X:
A: X → Y (1)
Data Preparation. The source dataset of different modalities X was the results
of MR-research of the brain carried out on 527 patients who were operated on
with a diagnosis of a supratentorial glial tumor. The data set contained informa-
tion on the increase in the pyramidal deficit in the postoperative period for each
research. Each research from the dataset consisted of MRI images performed in
the following modes: T1, T2, T2 FLAIR, DWI, T1+C.
The data preparation stage consisted of the following steps:
1. Markup. Radiologists match each slice, which was performed in a T2 FLAIR
regime, with a binary label. If a tumor was present on the slice, the label
was equal to 1, otherwise 0.
2. Converting. Marked slices were converted from DICOM format to PNG for-
mat.
3. Normalization. All data were centered and normalized (Figure 1).
123
Fig. 1. Graphical representation of the data preparation process for CNN learning.
Model Architecture. In the course of work on the project, our architecture
of an artificial neural network (ANN) ”Shallow” was developed (Figure 2). Also,
experiments were carried out using popular ANNs pre-trained on the ImageNet
dataset, such as VGG16, VGG19, Inception v3, followed by their fine-tuning on
the MRI data.
Description of ”Shallow” ANN:
1. An input layer that takes into account the two-dimensional topology of the
image and accepts a normalized, black and white image of 100 × 100 pixels
as input.
2. A convolutional layer contains 32 filters with a 3 × 3 kernel.
3. Activation layer using ReLU
f (x) = max(0, x) (2)
4. Average pooling layer. The pool size is 2 × 2.
5. Dropout layer with a rate equal to 0.2.
6. A convolutional layer contains 64 filters with a 3 × 3 kernel.
7. Activation layer using ReLU.
8. Average pooling layer. The pool size is 2 × 2.
9. Dropout layer with a rate equal to 0.2.
10. A convolutional layer contains 128 filters with a 3 × 3 kernel.
11. Activation layer using ReLU.
12. Average pooling layer. The pool size is 2 × 2.
13. Dropout layer with a rate equal to 0.2.
14. Flatten layer.
15. Dense layer with 128 units.
124
16. Activation layer using ReLU.
17. Dense layer with 1 unit.
18. Activation layer using a sigmoid function
1
f (x) = (3)
1 + ϵ( − x)
To prevent overfitting of the model, in addition to disabling random neurons,
L2-regularization is used.
Fig. 2. Graphical representation of the Shallow’s CNN architecture developed for this
research.
125
Loss Function. The binary cross-entropy was chosen as the loss function:
L(y, ŷ) = −y log ŷ − (1 − y) log(1 − ŷ), (4)
where y – the class the slice belongs to, ŷ – predicted class.
Augmentations. In addition to common augmentations such as random hori-
zontal or vertical flips, scaling and rotation, it was suggested to use adversarial
training. Adversarial examples are images with addition of specially generated
noise, which is visually not distinguishable by the human eye, but causes the
model to make incorrect predictions. We used Fast Gradient Sign Method [12]
and Carlini-Wagner L2 algorithm [5] to generate two adversarial examples for
each sample in the training set. The model will be evaluated separately with the
use of adversarial examples and without them.
Models Evaluating. For models evaluating, we used the following metrics:
– Accuracy = T P +TTN
P +T N
+F P +F N
– Sensitivity = T PT+F
P
N
– Specif icity = T NT+F
N
P
R1
– ROCAU C = 0
T P R dF P R, where T P R = T PT+F
P FP
N , F P R = T N +F P
– F1 = 2P recision×Recall
P recision+Recall , P recision = T P/(T P + F P ), Recall = T P/(T P +
F N ), where TP – true positive, TN – true negative, FP – false positive,
FN – false negative.
Training and Evaluating. Model training and evaluating were carried out on
the following equipment:
– Intel Core i9-9900KF CPU 3.6
– RAM 32Gb
– GPU NVIDIA 2080Ti 11Gb
As a basic framework for developing the model, we used TensorFlow 2.0. The
prepared dataset was split into three parts. 70% of the total number of slices
was included in the training set, 20% in the validation set, and 10% in the test
set. The resulting datasets are class-balanced. The batch size was empirically
chosen equal to 64, the number of epochs was 70. AdamOptimizer was used as
an optimizer, and the learning rate coefficient was chosen as a piecewise constant
function with an initial value of 10−4 , decreasing dynamically. After each epoch,
the model was validated on 10 batches of 64 elements from the validation dataset.
After training, the quality of the model was evaluated in an automatic mode on
test data. The model was then manually verified by a neurosurgeon who used
his dataset of MRI scans of 50 patients that were not present in the original
dataset.
126
Results. The results of the experiments are presented in Table 2. According to
these results, the best model on the test sample showed Accuracy 91%, Sensitiv-
ity 94%, Specificity 89%, ROC AUC 91%, and F1 92% in predicting the increase
in hemiparesis in the early postoperative period in patients with supratentorial
gliomas of the brain.
Table 2. Results of experimental studies in determining the increase in motor deficits
for the developed CNN Shallow, as well as for other popular pre-trained on the Ima-
geNet data set CNNs, such as VGG16, and Inception v3.
Model Accuracy Sensitivity Specificity ROC AUC F1-score
Shallow 0.82 0.87 0.76 0.82 0.83
Shallow+adv.ex. 0.91 0.94 0.89 0.91 0.92
VGG16 0.71 0.69 0.73 0.71 0.71
Inception v3 0.50 0.44 0.57 0.51 0.48
Figure 3 shows examples of preoperative MRI data, based on which the CNN
correctly predicted the increase in motor deficits in the postoperative period (a-
b), the absence of deterioration in motor status after surgery (c), and made a
mistake in predicting the increase in motor deficits after surgery(d).
Fig. 3. a-b – MRI-data based on which the CNN (Shallow) correctly predicted motor
worsening in the postoperative period; c – MRI-data based on which the CNN (Shallow)
correctly predicted the absence of motor worsening in the postoperative period; d –
MRI-data based on which the CNN (Shallow) made a mistake in predicting motor
worsening in the postoperative period.
127
4 Discussion
Methods of machine learning are of limited use in neurosurgery, and in brain
glioma surgery, in particular. The typical task spectrum that may be solved
using machine learning methods is presented in Table 3.
Table 3. Main directions of application of machine learning methods in brain glioma
surgery according to the literature analysis.
Task Examples of publications
determining the genetic Zhao J2020 [35], Chen SC2019 [7], Matsui Y2019 [20],
glioma profile Kocak B2019 [17], Aliotta E2019 [1], Ozturk-Isik
E2019 [24], Sun Z2019 [30], Huang RY2020 [14]
defining grades of brain Zhuge Y2020 [36], Rathore S2020 [27], Zhang Z2020 [34],
gliomas Cao H2002 [4], Nakamoto T2019 [22], Park YW2019 [25],
akahashi S 2019 [31]
defining grades of gliomas Hu LS2020 [13], Gates EDH2020 [10]
based on intratumoral
heterogeneity
differential diagnosis with Jianhua Qin 2019 [26], Kaplan K2020 [16]
other intracranial tumors
differential diagnosis with Elshafeey N 2019 [9], Bacchi S2019 [3]
post-radiation changes
prognosis of the overall Mizutani T2019 [21], Jang K2020 [15], Choi YS2020 [8]
patient survival [26], Chang Y2019 [6]
automatic segmentation Shusharina N2020 [29], Wu Y2019 [33], Krivov
of tumors E2018 [18]
early detection of gliomas Amin J2019 [2]
prognostic prospects of Neves BJ2020 [23], Wu S2020 [32]
chemotherapy
We failed to discover publications devoted to machine learning in pyrami-
dal insufficiency prognosis in patients with brain gliomas. As in the majority
of papers, our CNN has analyzed MR imaging data of patients. Our literature
survey analysis showed that most commonly conventional MR imaging (1, 2,
1+C, FLAIR) [17, 30, 22, 25, 8, 6, 33] was used as input data, rarely DWI [35, 31,
3], DTI [1] and MR perfusion [10, 26, 9] were additionally used. Besides, there
were some papers reporting on the use of proton MR-spectroscopy [24], kurto-
sis MRI [31], PET-CT [35, 20]. Alongside with neuroimaging data, some other
graphic information was applied (but considerably rarely): photos of histological
glioma slides [27], spectroscopic 5-ALA accumulation curves [19], as well as text
data: results of RNA sequencing [7], clinical data (such as sex, age, extent of re-
section etc.) [35, 20]. It stresses, that methods of machine learning can effectively
utilize different types of information (multimodal data), with the best sensitivity
and specificity effect being reached by their rational combination. Thus, multi-
modal data permitted increasing sensitivity and specificity from 83% to 90%
128
and from 82% to 90%, respectively [35], and accuracy from 73, 14% to 91,48%,
sensitivity from 72,84% to 93,47%, specificity from 74,05% to 85,36% [27]. It
is also true when using different modalities of MR imaging or other additional
methods of neuroimaging. Thus, Matsui Y et al. reports the following accuracy
for different neuroimaging modalities: 58,5% - for MRI only, 60,4% - for MRI and
PET-CT, 59,4%, – for MRI and CT, and 96% - for combination of MRI, PET-
CT and CT [20]. During the first stage of study we have been using monomodal
data - T2 FLAIR impulse sequences - as the most sensitive mode for visualizing
glial tumors. In the future, we plan to include other MRI modalities and also the
clinical details allow us to improve our neural network results significantly. As
for machine learning instruments, the CNN and support vector machines have
proved to be the best ones, probably due to the input data’s character (MR –
imaging). Thus, Kocak B et al., in their study of the genetic glioma profile based
on MRI data have compared 5 different methods of machine learning [17]. Av-
erage AUC and precision ranged from 0,769 to 0,869 and from 80,1% to 84%,
correspondingly. Moreover, the neural network showed the best results: AUC –
0,869 and precision – 83,8%. Similar outcomes were achieved by Chang Y et
al. in 2019 [6]. In 2020 Zhuge Y et al. generated the automatic glioma grading
scale based on the preoperative MRI of deep neural networks, thus achieving
the sensitivity of 93,5% and specificity of 97,2% which may play a crucial role in
advising the adjuvant therapy without biopsy and histological verification [36].
Generally, all recent publications report good data of machine learning meth-
ods on accuracy, sensitivity and specificity: accuracy varied from 79,4% [25] to
98% [34], sensitivity – from 76,92% [24] to 98% [34], specificity – from 70% [25]
to 100% [34]. These statistical measures correspond to the metrics obtained by
machine learning in our study: accuracy was 91%, sensitivity - 94%, specificity
- 89%, thus demonstrating that our neural network maybe useful in predicting
motor deficit development or aggravation or in the early postoperative period,
and also corresponds to the international level.
5 Conclusions
Methods of machine learning allow us to predict the pyramidal symptom aggra-
vation with comparatively high accuracy after microsurgical resection based on
the preoperative brain MRI data in patients with primary supratentorial gliomas.
The neural convolutional network created during the study showed the following
metrics: accuracy - 91%, sensitivity - 94%, specificity - 89%. It gives objective
information about the risk of neurological deficit, thus being a significant factor
in informing patients at the pre-surgical stage and deciding about the treatment
tactics. Using machine learning methods for prediction, the motor worsening af-
ter surgery is an auspicious tool. This requires further study with the inclusion
of input data of various modalities and increasing patients’ number.
Acknowledgments. With grant RFBR support №19-29-01154 “Predicting of
pyramidal symptoms and its reversibility in patients with supratentorial glial
129
tumors located near the motor areas, using the knowledge transfer method and
deep neural networks based on multifactor analysis of digital data of different
modality”. The funders had no role in the design of the study; in the collection,
analyses, or interpretation of data; in the writing of the manuscript, or in the
decision to publish the results.
References
1. Aliotta, E., Nourzadeh, H., Batchala, P., Schiff, D., Lopes, M., Druz-
gal, J., Mukherjee, S., Patel, S.: Molecular subtype classification in lower-
grade glioma with accelerated DTI . https://doi.org/10.3174/ajnr.A6162,
http://www.ajnr.org/lookup/doi/10.3174/ajnr.A6162
2. Amin, J., Sharif, M., Raza, M., Saba, T., Anjum, M.A.:
Brain tumor detection using statistical and machine learning
method. 177, 69–79. https://doi.org/10.1016/j.cmpb.2019.05.015,
https://linkinghub.elsevier.com/retrieve/pii/S0169260718313786
3. Bacchi, S., Zerner, T., Dongas, J., Asahina, A.T., Abou-Hamden, A.,
Otto, S., Oakden-Rayner, L., Patel, S.: Deep learning in the detec-
tion of high-grade glioma recurrence using multiple MRI sequences:
A pilot study. 70, 11–13. https://doi.org/10.1016/j.jocn.2019.10.003,
https://linkinghub.elsevier.com/retrieve/pii/S0967586819304709
4. Cao, H., Erson-Omay, E.Z., Li, X., Günel, M., Moliterno, J.,
Fulbright, R.K.: A quantitative model based on clinically rele-
vant MRI features differentiates lower grade gliomas and glioblas-
toma. 30(6), 3073–3082. https://doi.org/10.1007/s00330-019-06632-8,
http://link.springer.com/10.1007/s00330-019-06632-8
5. Carlini, N., Wagner, D.: Towards evaluating the robustness of neural networks.
(2017)
6. Chang, Y., Lafata, K., Sun, W., Wang, C., Chang, Z., Kirkpatrick, J.P., Yin,
F.F.: An investigation of machine learning methods in delta-radiomics fea-
ture analysis. 14(12), e0226348. https://doi.org/10.1371/journal.pone.0226348,
https://dx.plos.org/10.1371/journal.pone.0226348
7. Chen, S.C.C., Lo, C.M., Wang, S.H., Su, E.C.Y.: RNA editing-based classification
of diffuse gliomas: predicting isocitrate dehydrogenase mutation and chromo-
some 1p/19q codeletion. 20, 659. https://doi.org/10.1186/s12859-019-3236-0,
https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-3236-
0
8. Choi, Y.S., Ahn, S.S., Chang, J.H., Kang, S.G., Kim, E.H., Kim, S.H., Jain, R.,
Lee, S.K.: Machine learning and radiomic phenotyping of lower grade gliomas:
improving survival prediction. 30(7), 3834–3842. https://doi.org/10.1007/s00330-
020-06737-5, http://link.springer.com/10.1007/s00330-020-06737-5
9. Elshafeey, N., Kotrotsou, A., Hassan, A., Elshafei, N., Hassan, I., Ahmed, S., Abrol,
S., Agarwal, A., El Salek, K., Bergamaschi, S., Acharya, J., Moron, F.E., Law, M.,
Fuller, G.N., Huse, J.T., Zinn, P.O., Colen, R.R.: Multicenter study demonstrates
radiomic features derived from magnetic resonance perfusion images identify pseu-
doprogression in glioblastoma. 10(1), 3170. https://doi.org/10.1038/s41467-019-
11007-0, http://www.nature.com/articles/s41467-019-11007-0
10. Gates, E., Lin, J., Weinberg, J., Prabhu, S., Hamilton, J.,
Hazle, J., Fuller, G., Baladandayuthapani, V., Fuentes, D.,
130
Schellingerhout, D.: Imaging-based algorithm for the local grad-
ing of glioma. 41(3), 400–407. https://doi.org/10.3174/ajnr.A6405,
http://www.ajnr.org/lookup/doi/10.3174/ajnr.A6405
11. González-Darder, J.M., González-López, P., Talamantes, F., Quilis, V., Cortés,
V., Garcı́a-March, G., Roldán, P.: Multimodal navigation in the functional mi-
crosurgical resection of intrinsic brain tumors located in eloquent motor areas:
role of tractography 28(2), E5. https://doi.org/10.3171/2009.11.FOCUS09234,
https://thejns.org/view/journals/neurosurg-focus/28/2/article-pE5.xml
12. Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial
examples (2015)
13. Hu, L.S., Hawkins-Daarud, A., Wang, L., Li, J., Swan-
son, K.R.: Imaging of intratumoral heterogeneity in high-grade
glioma. 477, 97–106. https://doi.org/10.1016/j.canlet.2020.02.025,
https://linkinghub.elsevier.com/retrieve/pii/S030438352030094X
14. Huang, R.Y., Guenette, J.P.: Non-invasive diagnosis of h3 k27m mu-
tant midline glioma. 22(3), 309–310. https://doi.org/10.1093/neuonc/noz240,
https://academic.oup.com/neuro-oncology/article/22/3/309/5681667
15. Jang, K., Russo, C., Di Ieva, A.: Radiomics in gliomas: clini-
cal implications of computational modeling and fractal-based anal-
ysis. 62(7), 771–790. https://doi.org/10.1007/s00234-020-02403-1,
http://link.springer.com/10.1007/s00234-020-02403-1
16. Kaplan, K., Kaya, Y., Kuncan, M., Ertunç, H.M.: Brain tumor clas-
sification using modified local binary patterns (LBP) feature extrac-
tion methods. 139, 109696. https://doi.org/10.1016/j.mehy.2020.109696,
https://linkinghub.elsevier.com/retrieve/pii/S0306987720302632
17. Kocak, B., Durmaz, E.S., Ates, E., Sel, I., Turgut Gunes, S., Kaya, O.K.,
Zeynalova, A., Kilickesmez, O.: Radiogenomics of lower-grade gliomas: ma-
chine learning–based MRI texture analysis for predicting 1p/19q codele-
tion status. 30(2), 877–886. https://doi.org/10.1007/s00330-019-06492-2,
http://link.springer.com/10.1007/s00330-019-06492-2
18. Krivov, E., Kostjuchenko, V., Dalechina, A., Shirokikh, B., Makarchuk, G.,
Denisenko, A., Golanov, A., Belyaev, M.: Tumor delineation for brain radio-
surgery by a ConvNet and non-uniform patch generation. In: Bai, W., Sanroma,
G., Wu, G., Munsell, B.C., Zhan, Y., Coupé, P. (eds.) Patch-Based Techniques
in Medical Imaging, vol. 11075, pp. 122–129. Springer International Publishing.
https://doi.org/10.1007/978-3-030-00500-9 14
19. Leclerc, P., Ray, C., Mahieu-Williame, L., Alston, L., Frindel, C., Brevet,
P.F., Meyronet, D., Guyotat, J., Montcel, B., Rousseau, D.: Machine learning-
based prediction of glioma margin from 5-ALA induced PpIX fluores-
cence spectroscopy. 10(1), 1462. https://doi.org/10.1038/s41598-020-58299-7,
http://www.nature.com/articles/s41598-020-58299-7
20. Matsui, Y., Maruyama, T., Nitta, M., Saito, T., Tsuzuki, S., Tamura,
M., Kusuda, K., Fukuya, Y., Asano, H., Kawamata, T., Masamune, K.,
Muragaki, Y.: Prediction of lower-grade glioma molecular subtypes using
deep learning. 146(2), 321–327. https://doi.org/10.1007/s11060-019-03376-9,
http://link.springer.com/10.1007/s11060-019-03376-9
21. Mizutani, T., Magome, T., Igaki, H., Haga, A., Nawa, K., Sekiya,
N., Nakagawa, K.: Optimization of treatment strategy by using a ma-
chine learning model to predict survival time of patients with malignant
glioma after radiotherapy. 60(6), 818–824. https://doi.org/10.1093/jrr/rrz066,
https://academic.oup.com/jrr/article/60/6/818/5607843
131
22. Nakamoto, T., Takahashi, W., Haga, A., Takahashi, S., Kiryu, S., Nawa,
K., Ohta, T., Ozaki, S., Nozawa, Y., Tanaka, S., Mukasa, A., Naka-
gawa, K.: Prediction of malignant glioma grades using contrast-enhanced
t1-weighted and t2-weighted magnetic resonance images based on a ra-
diomic analysis. 9(1), 19411. https://doi.org/10.1038/s41598-019-55922-0,
http://www.nature.com/articles/s41598-019-55922-0
23. Neves, B.J., Agnes, J.P., Gomes, M.d.N., Henriques Donza, M.R., Gonçalves,
R.M., Delgobo, M., Ribeiro de Souza Neto, L., Senger, M.R., Silva-
Junior, F.P., Ferreira, S.B., Zanotto-Filho, A., Andrade, C.H.: Efficient
identification of novel anti-glioma lead compounds by machine learn-
ing models. 189, 111981. https://doi.org/10.1016/j.ejmech.2019.111981,
https://linkinghub.elsevier.com/retrieve/pii/S022352341931133X
24. Ozturk-Isik, E., Cengiz, S., Ozcan, A., Yakicier, C., Ersen Danyeli,
A., Pamir, M.N., Özduman, K., Dincer, A.: Identification of IDH
and TERTp mutation status using h-MRS in 112 hemispheric dif-
fuse gliomas. 51(6), 1799–1809. https://doi.org/10.1002/jmri.26964,
https://onlinelibrary.wiley.com/doi/abs/10.1002/jmri.26964
25. Park, Y.W., Choi, Y.S., Ahn, S.S., Chang, J.H., Kim, S.H., Lee,
S.K.: Radiomics MRI phenotyping with machine learning to pre-
dict the grade of lower-grade gliomas: A study focused on nonen-
hancing tumors. 20(9), 1381. https://doi.org/10.3348/kjr.2018.0814,
https://www.kjronline.org/DOIx.php?id=10.3348/kjr.2018.0814
26. Qin, J., Li, Y., Liang, D., Zhang, Y., Yao, W.: Histogram analysis of ab-
solute cerebral blood volume map can distinguish glioblastoma from solitary
brain metastasis. 98(42), e17515. https://doi.org/10.1097/MD.0000000000017515,
http://journals.lww.com/00005792-201910180-00033
27. Rathore, S., Niazi, T., Iftikhar, M.A., Chaddad, A.: Glioma grading via
analysis of digital pathology images using machine learning. 12(3), 578.
https://doi.org/10.3390/cancers12030578, https://www.mdpi.com/2072-
6694/12/3/578
28. Rossi, M., Conti Nibali, M., Viganò, L., Puglisi, G., Howells, H., Gay, L.,
Sciortino, T., Leonetti, A., Riva, M., Fornia, L., Cerri, G., Bello, L.: Resec-
tion of tumors within the primary motor cortex using high-frequency stimula-
tion: oncological and functional efficiency of this versatile approach based on
clinical conditions. 133(3), 642–654. https://doi.org/10.3171/2019.5.JNS19453,
https://thejns.org/view/journals/j-neurosurg/133/3/article-p642.xml
29. Shusharina, N., Söderberg, J., Edmunds, D., Löfman, F., Shih,
H., Bortfeld, T.: Automated delineation of the clinical target vol-
ume using anatomically constrained 3d expansion of the gross tu-
mor volume. 146, 37–43. https://doi.org/10.1016/j.radonc.2020.01.028,
https://linkinghub.elsevier.com/retrieve/pii/S0167814020300475
30. Sun, Z., Li, Y., Wang, Y., Fan, X., Xu, K., Wang, K., Li, S., Zhang, Z., Jiang,
T., Liu, X.: Radiogenomic analysis of vascular endothelial growth factor in pa-
tients with diffuse gliomas. 19(1), 68. https://doi.org/10.1186/s40644-019-0256-
y, https://cancerimagingjournal.biomedcentral.com/articles/10.1186/s40644-019-
0256-y
31. Takahashi, S., Takahashi, W., Tanaka, S., Haga, A., Nakamoto, T.,
Suzuki, Y., Mukasa, A., Takayanagi, S., Kitagawa, Y., Hana, T.,
Nejo, T., Nomura, M., Nakagawa, K., Saito, N.: Radiomics analysis
for glioma malignancy evaluation using diffusion kurtosis and tensor
132
imaging. 105(4), 784–791. https://doi.org/10.1016/j.ijrobp.2019.07.011,
https://linkinghub.elsevier.com/retrieve/pii/S0360301619334935
32. Wu, S., Calero-Pérez, P., Villamañan, L., Arias-Ramos, N., Pumarola,
M., Ortega-Martorell, S., Julià-Sapé, M., Arús, C., Candiota, A.P.:
Anti-tumour immune response in GL261 glioblastoma generated by
temozolomide immune-enhancing metronomic schedule monitored with
MRSI-based nosological images. 33(4). https://doi.org/10.1002/nbm.4229,
https://onlinelibrary.wiley.com/doi/abs/10.1002/nbm.4229
33. Wu, Y., Zhao, Z., Wu, W., Lin, Y., Wang, M.: Automatic
glioma segmentation based on adaptive superpixel. 19(1), 73,
https://bmcmedimaging.biomedcentral.com/articles/10.1186/s12880-019-0369-6
34. Zhang, Z., Xiao, J., Wu, S., Lv, F., Gong, J., Jiang, L., Yu, R., Luo, T.: Deep con-
volutional radiomic features on diffusion tensor images for classification of glioma
grades. 33(4), 826–837, http://link.springer.com/10.1007/s10278-020-00322-4
35. Zhao, J., Huang, Y., Song, Y., Xie, D., Hu, M., Qiu, H., Chu, J.: Diagnos-
tic accuracy and potential covariates for machine learning to identify IDH mu-
tations in glioma patients: evidence from a meta-analysis. 30(8), 4664–4674,
http://link.springer.com/10.1007/s00330-020-06717-9
36. Zhuge, Y., Ning, H., Mathen, P., Cheng, J.Y., Krauze, A.V., Cam-
phausen, K., Miller, R.W.: Automated glioma grading on conventional
MRI images using deep convolutional neural networks. 47(7), 3044–3053,
https://onlinelibrary.wiley.com/doi/abs/10.1002/mp.14168
133