Medical imaging modalities such as computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), and ultrasound (US) play a key role in providing healthcare professionals with detailed and exhaustive visual data of the human body. These imaging techniques generate significant amounts of data that require efficient analysis and interpretation. This is where Artificial Intelligence (AI) comes in. AI may emulate human cognitive processes in analyzing and understanding healthcare data. By focusing on the analysis of biomedical images using computational techniques such as object detection, segmentation and registration, AI has the potential to enhance diagnostic and prognostic accuracy by identifying patterns and correlations that may be difficult for humans to observe [1].
In the past, the use of AI in medicine was constrained by technological limitations until 1998, when the US Food and Drug Administration (FDA) approved the first computer-aided detection (CAD) system for mammography [2]. Since then, there has been exponential growth in the use of AI techniques in the medical field.
Today, hospitals are actively exploring AI solutions to support operational efforts aimed at improving cost efficiency, increasing diagnostic accuracy, and fostering greater patient satisfaction. However, it is important to strike a delicate balance between promoting the benefits of AI in clinical practice, which are evident, and addressing concerns about the transparency, trustworthiness, and potential bias of AI algorithms.
This paper summarises the ongoing activities of a multidisciplinary research group within the Signals and Images Lab of the Institute of Information Science and Technologies of the National Research Council of Italy. The group aims to explore the potential applications of AI in promoting and supporting health and well-being, while also addressing the challenges related to algorithms' explainability and transparency.
In the following, we provide a brief overview of the research conducted in the field of AI supporting clinical diagnostics. The primary focus is on medical imaging, given that radiology is expected to benefit most from recent advancements in AI.
Hepatic steatosis, characterized by the accumulation of fat within the liver, when coupled with inflammation, can contribute to the advancement of fibrosis towards cirrhosis and hepatocellular carcinoma [3]. Therefore, early detection and quantification of steatosis (via fat fraction assessment) are crucial tasks for predicting the disease progression. Magnetic Resonance Spectroscopy is the gold standard for the fat fraction assessment, while US imaging is commonly used to identify liver steatosis during screenings. Despite being non-invasive, US is highly operator-dependent [4].
In collaboration with a team from the IFC-CNR and Pisa University Hospital, we conducted a systematic comparison between three Deep Learning (DL) models in estimating, from US images, the fat fraction [5]. The compared models were the following: (i) a deterministic Convolutional Neural Network (CNN), similar to the one in [6], (ii) an MC Dropout CNN model, and (iii) a Bayesian CNN with probabilistic output.
In comparison to [6], the multi-center dataset increased up to 186 subjects.
Regression results showed good prediction performance for all architectures on the 5-fold test sets (Normalized RMSE 5.87%, 5.35%, and 5.82% for deterministic, MC Dropout, and Bayesian CNN, respectively). However, the introduction of uncertainty quantification (UQ), contributes to decreasing the percentage of mispredicted cases (from 32.4% for classical CNN to less than 9% for Bayesian one). Furthermore, the possibility of having access to information about the confidence with which the network produces its outputs is a great advantage, especially from the point of view of physicians who want to use neural networks as computer-aided diagnosis.
During the Coronavirus Disease 2019 (COVID-19) pandemic, High-Resolution Computed Tomography (HRCT) of the chest has been adopted as a method to visually identify two distinct abnormal pulmonary patterns: Ground Glass Opacity (GGO), characterized by increased attenuation and hazy density in lung lobes, and Consolidation, indicated by bilateral areas of lung tissue filled with fluid instead of air [7]. However, these patterns appear scattered with undefined contours and often lack contrast with surrounding healthy tissue. Consequently, the segmentation and quantification of pathological lung regions from HRCT data have proven to be very challenging.
In [8], we compared four state-of-the-art CNNs based on the encoder-decoder paradigm for the binary segmentation of COVID-19 infections (UNet [9], Attention-UNet [10], Recurrent-Residual UNet (R2-UNet) [11], R2-Attention UNet [12]), after training and testing them on 90 HRCT volumetric scans of COVID-19 patients. The images were collected from the database of the Pisa University Hospital (in the framework of the regional project "Optimised -An Optimised Path for the Data Flow and Clinical Management of COVID-19 Patients", funded by Tuscany Region).
We conducted a comparison between them to ascertain insights into the cognitive mechanisms that can drive a neural model towards optimal performance for this task, as well as to identify the optimal balance between the volume of data, time, and computational resources necessary. From the results of the analysis, it can be concluded that Attention-UNet outperforms the other models by achieving the best performance of 81.93%, in terms of 2D Dice score on the test set.
On top of the work [13], the cerebrospinal fluid of 21 subjects who received a clinical diagnosis of Alzheimer's disease (AD) as well as of 22 pathological controls has been collected and analysed by Raman Spectroscopy (RS). The aim of this research is to understand if the Raman spectra could be used to distinguish AD from controls, after a preprocessing procedure. We applied machine learning to a set of topological descriptors extracted from the spectra, achieving a high classification accuracy of 86% (the best performing combination is the Ridge classifier applied to the persistence landscapes vectorization).Our experimentation indicates that RS and topological analysis may be effective to confirm or disprove a clinical diagnosis of Alzheimer's disease. Also, it opens the way to possibly increasing and/or confirming the knowledge about the precise molecular events and biological pathways behind the Alzheimer's disease, e.g., by identifying the bands of the Raman spectrum relevant for AD detection.
Eosinophilic esophagitis (EoE) is a chronic disease characterized by esophageal symptoms and eosinophilic inflammation of the esophagus. Among patients with dysphagia, EoE and non-EoE patients should receive different therapies and therefore must be timely and correctly identified from the clinical history or by using more invasive procedures (endoscopic and/or histological information).
In [14], an RDF-based ML model was trained on a multicenter international database (273 EoE and 55 non-EoE dysphagia patients clinical and endoscopic data, collected from Guy's and St. Thomas' Hospital NHS Foundation Trust (GSTT, London, United Kingdom), Pisa Univ. Hospital (Pisa, Italy), and Padua Univ. Hospital (Padua, Italy)) to provide indications for the investigation of EoE in adults reporting dysphagia or to inform point-of-care decision-making for performing esophageal biopsies in adults with dysphagia.
The model was further evaluated on an independent cohort of 93 consecutive patients with dysphagia, result-ing in an AUC of 0.90 (using clinical data) and an AUC of 0.94 (using a combination of clinical and endoscopic data) The model, re-trained on the whole dataset, has been integrated into an open-access online tool (https: //webapplicationing.shinyapps.io/PointOfCare-EoE/).
AI algorithms are showing potential in improving the current protocol for grading various cancers, such as breast and prostate cancer. In the following sections, we provide a brief description of our research in this area.
Although imaging techniques are commonly used for breast cancer screening, biopsy is the only method available to categorize a breast lesion as benign or malignant. However, biopsies are invasive and costly procedures that can cause discomfort in patients [15]. Radiomic analysis of biomedical images shows promise in addressing various clinical challenges, such as early detection and classification of breast tumors.
In the P.I.N.K study [16], 66 women were enrolled. Their paired Automated Breast Volume Scanner (ABVS) and Digital Breast Tomosynthesis (DBT) images, annotated with cancerous lesions, populated the first ABVS+DBT dataset. This allowed for radiomic analysis to differentiate between malignant and benign breast cancer.
Three Machine Learning (ML) methods were employed: Random Decision Forests (RDF), Support Vector Machines (SVM), and Logistic Regression (Logit). They were trained and validated using an ad hoc nested LOO cross-validation procedure to ensure a minimally biased estimation of the model's generalization ability, even with a limited sample size. The study's main finding highlights the superior effectiveness of RDF model in accurately predicting tumor classification using radiomic features in both ABVS and DBT acquisitions. It achieved AUC-ROC values of 89.9% with a subset of 19 features.
Additionally, promising outcomes were achieved using solely textural radiomic features to train RDF model, with AUC-ROC values of 71.8% and 74.1% for ABVS and DBT, respectively. This suggests the potential for integrating virtual biopsy into routine medical practice.
Current methods for determining Prostate cancer (PCa) aggressiveness rely on biopsy, an invasive and uncomfort-able procedure. Multi-parametric Magnetic Resonance Imaging (mpMRI) is frequently employed to get an initial assessment of the tumor. To this end, numerous studies have explored ML/DL models for automatic PCa grading from mpMRI images [17]. However, developing accurate and generalizable DL models for medical imaging, where data is often scarce, presents a significant challenge. Few-shot learning (FSL) offers a promising solution, particularly since the advancements in meta-learning [18]. For this reason, we investigated FSL techniques for assessing PCa aggressiveness from mpMRI images. We proposed a two-step approach: a disentangled self-supervised learning (SSL) pre-training step for robust feature extraction, followed by meta-fine-tuning utilizing finer-grained classes and the coarser-grained ones in meta-testing for enhanced generalization [19]. Our approach achieved a mean AU-ROC of 0.821 for a 4-way (ISUP 2-5) 5-shot setting. We further explored enhancing FSL models performance by leveraging synthetic image generation, employing a Denoising Diffusion Probabilistic Model (DDPM).
Also, we proposed a new technique to discover and exploit causality signals from images via neural networks for classification purposes [20,21]. We model how the presence of a feature in one part of the image affects the appearance of others in different parts of the image. Our method consists of a convolutional backbone and a causality-factors extractor computing weights to enhance feature maps according to their causal influence in the scene. We evaluated our method on a dataset of prostate MRI images for cancer diagnosis and studied its effectiveness of our module both in fully-supervised and 1-shot learning. On the binary classification of cancer versus no-tumor cases, our method led to a maximum test accuracy of 0.72, representing a 5 % increase to the baseline [21]. On distinguishing ISUP grades in 1-shot learning, we obtained a 0.71 AUROC for the classification ISUP 2 vs. all the others, with 13 % increasing to the baseline [20]. Our attention-inspired improved the overall classification and produced more robust XAI predictions focusing on relevant parts of the image.
Raman Spectroscopy (RS) allows for the observation of changes in biochemical constituents (such as proteins, lipid structures, DNA, and vitamins) among different tissues by obtaining their biochemical maps. Recently, RS has been applied to chondrogenic tumor classification with excellent results [22]. Chondrogenic tumors are the second largest group of bone tumors worldwide. They are generally classified as primary chondrosarcomas when they occur in previously normal bone. Secondary chondrosarcomas result from the malignant transformation of a benign cartilaginous lesion and are classified into three grades: CS G1, CS G2 and CS G3. Enchondroma (EC) is a non-cancerous tumor. Distinguishing between EC and CS G1 is a critical issue for pathologists, as it generates many false positive and false negative diagnoses [24].
In [23] we showed that the combination of persistent homology and ML techniques can support the classification of Raman spectra extracted from cancerous tissues to achieve a reliable chondrosarcomas grading.
A total of 410 Raman spectra from 10 patients with primary chondrogenic tumors of the skeleton, treated at Azienda Ospedaliera Universitaria Pisana (Pisa), were used to train the machine learning models. Despite the small size of the experimental dataset, the results show that the method not only achieved high accuracy on previously unseen data samples; also such a methos can be easily integrated into a Raman spectroscopic system as an automatic tool to assist clinicians in grading tumors.
Radiotherapy is a commonly used treatment for prostate cancer (PCa). In recent years, there has been a surge of interest in leveraging ML methods to analyze radiomic features derived from multiparametric MRI (mpMRI) scans of PCa. However, little attention has been given to predicting radiation-induced toxicity [25] before starting radiotherapy. In the work carried out in the framework of the EU H2020 ProCAncer-I project [26], we aimed to predict radiotherapy-induced side effects, including both genito-urinary and rectal toxicity.
A RDF model was trained on radiomic features ex-tracted from 134 T2-weighted Magnetic Resonance Imaging (MRI) images of patients who underwent radiotherapy. The MRI scans were obtained from ProstateNet (https://prostatenet.eu), the repository designed within the framework of the EU H2020 ProCAncer-I project. Data regarding the presence and severity of rectal and urinary side effects after treatment were also included.
The results demonstrated that radiomics-based approaches can be effective in predicting radiotherapyinduced side effects, achieving an AUROC of 70.8%. Also, a set of simplified model variants was used to estimate epistemic uncertainty and provide a reliability score to complement the main model's prediction.
In this field, we investigated also the use of thermal imaging for stress discrimination [27,28], to the aim of detecting stress in adults under stress stimuli, and of assessing the efficacy of the hortotherapy for female adolescence affected by anorexia nervosa. Notably, we are moving to a more challenging task: deepen the understanding of thermal profiles in the newborn (possibly pre-term) in order to develop or improve new treatment techniques related to the maturation of the newborn thermo-regulation system. A study protocol, joint work with the lab NINA and the NICU of Santa Chiara Hospital in Pisa, is under review.
One open issue related to children's research concerns neonatal imitation (NI), namely the primitive ability of infants to mirror the actions of others [29]. The question of whether imitation is present from birth is of great importance as it can foster a deeper understanding of how it contributes to later developmental outcomes, which is crucial for the preterm newborn.
Computer vision methods may unobtrusively detect and analyze the most relevant facial features, thus providing clinicians (or parents, caregivers, etc.) with objective data about children's health status [30]. However, for infants, this is a challenging task, due to significant changes in their facial morphology compared to adults, and to the increased complexity in data collection caused by unpredictable variations in their facial poses [31].
In [32], we analyzed videos of 10 newborns (8 preterms, 2 at term, ≤ 4 weeks post-term equivalent age), performing tasks such as tongue protrusion and mouth opening, to classify open/closed mouths. The videos were analyzed at frame-level, for a total of 41000 labeled frames. In each frame, we identified mouth landmarks and cropped the images around the mouth, then we applied an image preprocessing pipeline (which included mouth orientation, resizing, brightness and contrast enhancement, see Figure 2) to improve classification performance. A CNN was trained using a ten-fold cross-validation, which resulted in highly reliable results: accuracy, precision, and recall over 92% on unseen data.
AI has a big potential to improve care and health systems, specially for diagnostic tasks, even if facing very important technical issues like unbalance dataset, data drift, heterogeneous acquisition protocols, and input data and annotations of variable quality. Also, future research should involve healthcare professionals and caregivers as designers and users, comply with health-related regulations, improve transparency and privacy, integrate with healthcare technological infrastructure, explain their decisions to the users, and establish evaluation metrics and design guidelines.
This publication is based upon the work carried out within the COST Action GoodBrother (CA19121), the EU H2020 projects ProCAncer-I (GA 952159) and FAITH (GA 101135932), the PAR FAS Tuscany Region NAVIGA-TOR, PRAMA and OPTIMISED.