The application of Artificial Intelligence technologies in healthcare can enhance and optimize medical diagnosis, treatment, and patient care. Medical imaging, which involves Computer Vision to interpret and understand visual data, is one area of healthcare that shows great promise for AI, and it can lead to faster and more accurate diagnoses, such as detecting early signs of cancer or identifying abnormalities in the brain. This short paper provides an introduction to some of the activities of the Artificial Intelligence for Media and Humanities Laboratory of the ISTI-CNR that integrate AI and medical image analysis in healthcare. Specifically, the paper presents approaches that utilize 3D medical images to detect the behavior-variant of frontotemporal dementia, a neurodegenerative syndrome that can be diagnosed by analyzing brain scans. Furthermore, it illustrates some Deep Learning-based techniques for localizing and counting biological structures in microscopy images, such as cells and perineuronal nets. Lastly, the paper presents a practical and cost-efective AI-based tool for multi-species pupillometry (mice and humans), which has been validated in various scenarios.
Artificial Intelligence (AI) is rapidly transforming many puter algorithms to simulate intelligent behavior. When applied to healthcare, these technologies can be used to enhance and optimize medical diagnosis, treatment, and patient care. larly promising is medical imaging, which involves Computer Vision (CV). CV specifically focuses on teaching computers to interpret and understand visual data from the world around us. Medical imaging plays a crucial role in diagnosing and treating many medical conditions.
consuming and complex process, and even experienced experts can sometimes miss subtle changes that indicate disease. This is where Artificial Intelligence (AI) and
used to enhance and optimize medical imaging, enabling faster and more accurate diagnoses, such as detecting early signs of cancer or identifying structural abnormalities in the brain [1, 2, 3, 4]. In addition, AI and CV can help reduce the workload of radiologists and other medi(G. Amato)
0000-0001-5014-5089 (F. Carrara); 0000-0002-6985-0439 (L. Ciampi); 0000-0001-5781-7060 (M. D. Benedetto);
Specifically, in [ 5], we focused on several neural network architectures to detect, from brain scans, the behavior-variant of the frontotemporal dementia (bvFTD), a neurodegenerative syndrome whose clinical diagnosis remains a challenging task, especially in the early stage of the disease. Currently, the presence of frontal and anterior temporal lobe atrophies on magnetic resonance imaging (MRI) is part of the diagnostic criteria for bvFTD. However, MRI data processing is usually dependent on the acquisition device and mostly requires human-assisted crafting of feature extraction. Following the impressive improvements of deep architectures, in our study, we reported on bvFTD identification using various classes of artificial neural networks, and we presented the results achieved on classification accuracy and obliviousness on acquisition devices using extensive hyperparameter search. As shown in Figure 1, we demonstrated the stability and generalization of diferent deep networks based on the attention mechanism, where data intra-mixing confers models the ability to identify the disorder even on MRI data in inter-device settings, i.e., on data produced by diferent acquisition devices and without model fine-tuning.
In more recent times, we are also studying how brain anomaly detection techniques based on neural architectures, e.g., Generative Adversarial Networks (GANs) or Masked Auto Encoders (MAEs), can be used to enrich the diagnosis toolbox of medicians with nowadays standards and of-the-shelves equipment.
Detection and counting of biological structures in microscopy images is an analysis of considerable interest in biology and medicine. For instance, a viable cell count Figure 2: Some samples of the data used for counting cells in is a fundamental step in diagnosing several diseases, and microscopy images. From left to right, (i) the Modified Bone it can be exploited to assist in cytotoxicity estimation, Marrow (MBM) Cells [8, 9], a dataset of the human bone mari.e., the quality of being toxic to cells. To this end, in row tissues pertaining to 8 diferent patients; (ii) the Nuclei [6], we investigated several counting approaches that Cells dataset [10] comprising RGB microscopy H&E stained have been successfully exploited in the literature over histology images of colorectal adenocarcinomas; (iii) the VGG three public collections of microscopy images containing Cells dataset [11], a synthetic collection of fluorescence mimarked cells (see Figure 2 for some samples), assessing croscopy images emulating bacterial cells. not only their counting performance compared to several state-of-the-art methods but also their ability to localize the counted cells correctly. Our analysis showed that colorectal adenocarcinomas having a common size of counting errors do not always agree with the localization 500 × 500 × 3. The images refer to 9 diferent patients. performance, and relying only on the counting metrics They have been cropped from non-overlapping areas repcan lead to SOTA models producing incorrect cell lo- resenting a variety of tissue appearances from normal calization. Therefore, we suggest measuring the mean and malignant regions. Still, they also comprise areas average precision, or at least a grid average mean abso- with artifacts, over-staining, and failed autofocussing to lute error [7], to help practitioners develop better models simulate realistic outliers. Another peculiarity of this and guide users to choose the model most tailored to dataset is that the nuclei of the cells belong to four difertheir needs. ent categories, presenting diferent visual characteristics;
This dataset has been presented in [10] and comprises some experts have manually annotated them by putting 100 RGB microscopy H&E stained histology images of a dot over the centroids of each biological structure for a ... evaluate the number of perineuronal nets (PNNs) in mi- has been increasingly used in the assessment of varicroscopy images. Specifically, PNNs are extracellular ous neuropsychiatric disorders, including autism specmatrix aggregates surrounding the cell body of a large trum disorder, attention deficit hyperactivity disorder, number of neurons [12, 13], and their alterations are as- schizophrenia, and anxiety disorders. sociated with psychiatric disorders such as schizophrenia
[14]. In [15], we proposed a two-stage counting method- CNR), we developed cheap, practical, AI-based setups to ology for counting PNNs in weakly-labeled data settings. perform multi-species pupillometry (mice and humans)
and validated it in several scenarios [17]. stage, we adopted existing state-of-the-art solutions naIn [18], we studied Cyclin-dependent kinase-like 5 tively designed for detecting and counting cells trained (Cdkl5) deficiency disorder (CDD) — a severe neurodeover single-rater weakly-labeled data, i.e., containing velopmental disorder that causes early-onset seizures, annotation errors due to the dificulty in finding the corintellectual disability, motor, and social impairment. No rect patterns, even among experts. In the second stage, efective treatment is currently available, and medical using a small set of multi-rater data, i.e., data labeled management is only supportive. Recently, mouse models by multiple annotators, we defined a rescoring model of CDD have been developed, demonstrating that mice aimed at refining predictions of the previous stage, inlacking Cdkl5 exhibit autism-like phenotypes, hyperaccreasing the correlation between the scores assigned by tivity, and dysregulation of the arousal system, providthe model to the predictions and the raters’ agreement ing the possibility to use these features as translational on the sample labels. Finally, very recently, we presented biomarkers. In this study, pupillometry was used to asa comprehensive atlas of PNN distribution and colocalsess the integrity of the arousal system in CDD mice, and ization with parvalbumin (PV) cells for over 600 regions the results revealed a global defect in arousal modulation of adult mouse brains that ofers a novel resource for (see Figures 4 and 5). Therefore, pupillometry may prounderstanding the organizational principles of the brain vide an easy and valuable biomarker for the diagnosis extracellular matrix [16].
and monitoring of CDD.
that measures changes in pupil size in response to var
laboratory of the ISTI-CNR concerning Computer Vision approaches relying on Artificial Intelligence for medical image analysis. The proposed technologies can be used to enhance and optimize medical diagnosis, treatment, and patient care and represent valid tools exploitable by medical professionals. We described some approaches tackling the detection of the behavior-variant of frontotemporal dementia in 3D brain scans, some Deep Learning networks for counting biological structures, such as cells and PNNs, in microscopy images, and, finally, a cheap and practical AI-based tool to perform multi-species pupillometry (mice and humans).
tures with raters’ uncertainty, Medical Image Analysis 80 (2022) 102500. URL: https://doi. org/10.1016%2Fj.media.2022.102500. doi:10.1016/ j.media.2022.102500. [16] L. Lupori, V. Totaro, S. Cornuti, L. Ciampi, F. Carrara, E. Grilli, A. Viglione, F. Tozzi, E. Putignano, R. Mazziotti, et al., A comprehensive atlas of perineuronal net distribution and colocalization with parvalbumin in the adult mouse brain, bioRxiv (2023) 2023–01. [17] R. Mazziotti, F. Carrara, A. Viglione, L. Lupori, L. L.
Verde, A. Benedetto, G. Ricci, G. Sagona, G. Amato, T. Pizzorusso, Meye: web app for translational and real-time pupillometry, eneuro 8 (2021). [18] A. Viglione, G. Sagona, F. Carrara, G. Amato, V. Totaro, L. Lupori, E. Putignano, T. Pizzorusso, R. Mazziotti, Behavioral impulsivity is associated with pupillary alterations and hyperactivity in cdkl5 mutant mice, Human Molecular Genetics 31 (2022) 4107–4120.