Convolutional Neural Networks (CNN) are the state of the art in domain-specific neural networks for image data. We describe MEDIGUI-ConvNet, an efective CNN-based autonomous analysis and diagnosis system for medical imaging aimed at Alzheimer's disease, which promises great impact. To bridge the gap between data/image scientists and domain experts, a graphical user interface (GUI) framework allows endusers to load their medical image datasets in pickle format while successfully operating basic operations like training, testing, or deployment of models. Users can construct CNN models based on their demands, including defining network architecture and hyperparameters. After training, MEDIGUI-ConvNet allows testing models, allowing users to compare accuracy and predictions to ground-truth classifications. MEDIGUI-ConvNet ofers a user-friendly solution for medical professionals and researchers to harness the power of deep learning for medical image analysis without the need for specialized programming expertise, with a promise for accelerating research and clinical applications in areas such as disease diagnosis, prognosis, and treatment planning.
Alzheimer’s disease (AD) is the most common cause of dementia, accounting for an estimated
60-80% of cases worldwide [
Convolutional Neural Networks (CNNs) are a type of artificial neural network (ANN) inspired
by the visual system of animals. The first development of CNNs is credited to Fukushima in the
1980s for character recognition [
2.1. Data
The dataset used for model development and evaluation consists of 33,984 MRI images
acquired from patients with AD and healthy controls. The images were acquired with
various scanners and have a resolution of 100x100 pixels that has been adapted to our
purposes. The original dataset is available at https://www.kaggle.com/datasets/uraninjo/
augmented-alzheimer-mri-dataset and is distributed under GNU Lesser General Public License
(https://www.gnu.org/licenses/lgpl-3.0.html)[
CNNs rely on two key principles: convolution and pooling. With the convolution, CNNs apply a convolution operation to the input data, which helps extract local and invariant features. With pooling, CNNs utilize pooling to reduce the dimensionality of the input data, improving computational eficiency and model robustness.
The proposed CNN architecture (Figure 1) is designed to classify magnetic resonance imaging (MRI) images of patients afected by Alzheimer’s Disease into four classes: Non-Demented, Moderate-Demented, Mild-Demented, and Very-Mild-Demented. The dataset consists of 33,984 MRI images with a resolution of 100x100 pixels, split into 27,187 images for training and 6,797 images for testing. The architecture comprises a series of convolutional (Conv2D) and pooling (MaxPooling2D) layers, followed by two fully connected (Dense) layers for final classification.
Table 1 summarizes the CNN architecture. The total number of trainable parameters in the CNN architecture is 3,952,572, trained with cross-entropy loss using the Adam algorithm. We have extended the methods by introducing a graphical interface to guide users through the management of CNN training, as well as the ability to test the model with test images. For the graphical interface, we opted to use ipywidgets to facilitate software integration within a Jupyter Notebook environment. For the implementation of the CNN, we utilized Python 3 along with TensorFlow and Keras libraries. Accessing the GUI is straightforward by simply importing the module in a Jupyter Notebook as a Python object. Furthermore, once the module is imported, users can also access individual functions through the object, bypassing the GUI entirely.
For this study, we employed the CNN architecture detailed in Table 1. The training set includes
27,187 MRI images. After 30 epochs, the Adam algorithm set with a learning rate of 0.001 and
two regularization parameters set to 0.001 yielded a validation accuracy of 96.2% with a loss
value of 0.19, while attaining a training accuracy of 99.5% with a training loss of 0.031. However,
as we will describe later, we made our CNN open to various customizations, thus allowing for a
wide range of experiments.
2.4. The GUI
MEDIGUI-Convnet presents itself with a multi-tab interface to provide an optimal interactive
experience [
The ’Model Testing’ tab in MEDIGUI-Convnet is designed for your convenience. It allows you to load the trained models with a simple click. Once you’ve chosen and loaded a model, you can access a dataset of images previously uploaded using a sliding control. A window on the right will display the MRI image along with some information, such as the image’s class and the prediction made by the model. The GUI always displays a Log window, which tracks all processes performed by the algorithm, ensuring transparency and traceability (Figure 4).
MEDIGUI-ConvNet automatically generates a plot containing predictions for the first 100 images of the testing dataset. Each prediction in the plot is associated with the figure number and the actual class to which the image belongs. This lets users quickly identify predictions and conduct in-depth pattern analysis by exploring the CNN filters. (Figure 5).
Within the Model Testing Tab, upon clicking the "Feature Mapping" checkbox, a series of plots displaying activations from all layers of the CNN can be generated. This analysis is particularly useful for identifying characteristic patterns of the various classes. Such an analysis enables physicians and researchers to identify morphological-degenerative aspects in brain tissue that can be correlated with disease progression (Figure 6).
Figure 7 partially represents the patterns identified using the Feature Mapping approach. It illustrates the information captured by filters from some CNN layers (max_pooling_0, conv2D_1, and conv2D_4) for MRI image number 58 from the test dataset. These images come from a test subset randomly sampled as 20% of the data set using the medigui.splitDataset function. Figure 7 also provides insight into the activation levels of the 512 units in the flatten layer and the two dense layers. Of particular note is the last dense layer, indicating the activity of the four output neurons. The plot clearly shows an activation of 1.0 for output neuron 1, which corresponds to the label associated with the Moderate Demented class. As per the representation convention, brighter colours around yellow correspond to higher activation levels, indicating what the CNN layers deem most relevant for classification purposes (Figure 7).
The CNN was evaluated on the independent test set of 6,797 MRI images. The classification performance was evaluated using the 1 score, particularly useful when classes are imbalanced or when a good balance between precision and recall is important. It is calculated as follows: × 1 = 2 × + (1)
The scikit-learn f1_score function was applied using the weighted averaging strategy to handle class imbalance. A multiple cross-validation technique was also employed to evaluate the model’s performance on the dataset by using the scikit-learn RepeatedKFold function.
By doing so, we could train and evaluate our model multiple times on diferent subsets of the data, providing a more reliable estimate of its performance. The dimensions of the input data were also extended to accommodate the model’s requirements, ensuring compatibility during training and evaluation. Finally, we computed the mean accuracy score and standard deviation across all folds to quantify the model’s performance and variability.
The pre-trained model (30 epochs with two regularization parameters set to 0.001) was validated using a 15-fold cross-validation.
The CNN demonstrated robust generalization capabilities, efectively classifying previously unseen MRI images into the appropriate categories of Alzheimer’s Disease severity. The test accuracy metric reached a very good value of 96.2%, highlighting the reliability of this CNNbased model in clinical applications. Furthermore, precision, recall, and 1-score metrics provide valuable insights into CNN’s ability to identify each class while accurately minimizing false positives and negatives. In our case 1-score was 0.96, indicating strong performance, good balance between precision and recall and, consequently, good classification model performance.
Analyzing the information contained within the filters and visualizing the activation levels of units in the dense layers using our tool, we observed that our system could highlight distinguishable patterns between images classified as Non-Demented compared to those classified as Moderate-Demented. Many of the insights appear redundant within the filters, and in some cases, "black-box" information is obtained due to the complexity of connections between the various layers. For instance, comparing the patterns highlighted at level 8, corresponding to the Conv2D_4 convolution layer reveals how the arrangement of patterns within the filters may difer between ND and MoD comparisons. However, directly correlating these arrangements with the obtained outcome requires further analysis. In contrast, comparing the activation levels of various units obtained from the plot of levels 10 and 11, corresponding to the flatten and the ifrst dense layer ofers a better understanding of a potential correlation with the activation of output layers (Figure 8 and 9).
A noteworthy finding is a clear diference between ND and MoD samples in the activations of neurons with indices ranging from 20 to 40 and 70 to 120, as observed in the dense layer 11. These neural units’ distinct activation strongly correlates with the output levels of dense layer 12, representing the entire CNN’s final output layer. These distinct activation patterns might arise from a series of alterations related to the widening of certain cerebral sulci detectable by MRI, albeit challenging to identify with the naked eye.
Our analysis suggests that the convolutional neural network (CNN) has the potential to detect subtle and intricate anatomical features linked to severe dementia, such as widened convoluted sulci. By examining the activation patterns within the CNN’s dense layers, we observed a significant correlation between these features and the level of dementia severity. However, further investigation is necessary to confirm and interpret these associations definitively. The CNN might also be adept at identifying other anatomical features or patterns of neuronal activation related to dementia. These findings could provide valuable insights into the neuroanatomical underpinnings of the disease. In conclusion, our study suggests that the CNN approach holds promise for uncovering neuroanatomical markers of dementia. In addition, our analysis suggests the possibility of identifying recurring patterns within the fully connected layer. These patterns could represent specific configurations of neuroanatomical features characteristic of diferent stages or subtypes of dementia. By exploring the activations and weights within the dense layer, we may uncover meaningful associations between these patterns and clinical manifestations of the disease. However, further research is warranted to investigate these hypotheses and elucidate the clinical relevance of the identified patterns. Overall, detecting recurring patterns within the dense layer holds promise for enhancing our understanding of the underlying neurobiology of dementia and may lead to more targeted diagnostic and therapeutic interventions.
Our model’s performance on the test data is pivotal, reflecting its capacity to generalize beyond the confines of the training set. By emphasizing the model’s overall accuracy and juxtaposing it against alternative methodologies, we glean valuable insights into the eficacy of our approach in the realm of Alzheimer’s Disease diagnosis. We want to test our software as the next step on magnetic resonance images of size 256x256. Navigating challenges encountered during training and testing phases, such as mitigating overfitting and addressing data imbalance, yields a nuanced understanding of our model’s performance. Delving into these challenges not only elucidates areas for refinement but also paves the way for future advancements in neuroimaging analysis.
Proposing innovative solutions or enhancements derived from identified challenges catalyzes pushing the boundaries of model performance. Whether fine-tuning the model architecture, exploring novel training paradigms, or enriching the dataset with diverse clinical data, these endeavours contribute to ongoing progress in Alzheimer’s Disease research and diagnostics.
In conclusion, evaluating our model’s performance transcends mere accuracy assessment; it necessitates a holistic examination of its strengths, weaknesses, and avenues for improvement. This comprehensive approach underscores our commitment to continual refinement and innovation, driving forward the integration of AI-powered CNNs in the clinical diagnosis of Alzheimer’s Disease. The robust performance of the trained CNN underscores its potential as a valuable tool for assisting clinicians in diagnosing and monitoring Alzheimer’s Disease progression. The MEDIGUI-ConvNet’s code is available via https://github.com/lucazammataro/MEDIGUI-ConvNet.git and it is released under GNU General Public License v3.0.