AI-Driven Innovations in Healthcare: Bridging Imaging and Genomics for Advanced Disease Insights Carlo Adornetto1,** , Pierangela Bruno1 , Francesco Calimeri1 , Edoardo De Rose1 , Gianluigi Greco1 and Alessandro Quarta1,2 1 Department of Mathematics and Computer Science, University of Calabria, Via Pietro Bucci, Rende, Italy 2 Department of Computer, Control and Management Engineering "Antonio Ruberti", Sapienza University of Rome, Via Ariosto 25, Rome, Italy Abstract The application of Artificial Intelligence (AI) techniques for analyzing medical images and omics data is revolutionizing the healthcare industry by offering profound insights into various diseases. Achieving precise diagnoses and formulating effective treatment plans, however, demands intricate and multimodal analysis of complex, sensitive, and diverse medical datasets. Recent advancements in Machine Learning and Deep Learning have proven to be formidable in identifying and classifying specific diseases. This paper outlines the current projects undertaken by our research group in this innovative domain. Keywords Artificial Intelligence, Medical Imaging, Genomics, Deep Learning 1. Introduction and less resource-demanding systems [1, 2, 3]. In omics analysis, DL has excelled by exploring the vast The rapid advancement of technology and increased data arrays of biological molecules, aiding in disease under- availability have positioned Artificial Intelligence (AI) as standing and treatment customization across fields like a cornerstone in healthcare. AI significantly enhances genomics, transcriptomics, proteomics, and metabolomics. patient care, refines treatment protocols, and accelerates Advancements in high-throughput and next-generation the diagnosis of diverse health conditions. Notably, AI sequencing technologies have fueled significant progress has advanced medical imaging and omics analysis, re- in functional genomics, especially in understanding fining diagnostic accuracy and personalizing treatment cancer-related genomic factors [4]. strategies. Despite the potential, DL models often suffer from a lack Deep Learning (DL), a subset of AI, excels in analyzing of interpretability, a critical challenge in bioinformat- medical images. Its ability to autonomously identify crit- ics. The rise of Explainable Artificial Intelligence (XAI) ical features and yield accurate interpretations has made aims to enhance model transparency and improve fea- it essential for analyzing complex visual data in medical ture selection. Techniques like Shapely Additive exPla- imaging modalities such as X-rays, MRI, CT scans, PET, nations (SHAP) and Gradient-weighted Class Activation and ultrasound. These capabilities are crucial for diagnos- Mapping (Grad-CAM) have become pivotal in demystify- ing complex conditions like cancers, and cardiovascular ing the decisions of Neural Networks (NNs), providing and neurological disorders. clearer insights into their predictive mechanisms [5, 6]. However, the assembly of extensive datasets poses sig- This paper, following our previous work [7], outlines nificant challenges. To address this, Continual Learning our recent advancements in medical imaging and omics (CL) has emerged as a solution, enabling models to adapt data analysis, paving the way for an in-depth exploration through ongoing data streams, thus enhancing scalability of AI’s evolving role in healthcare. The forthcoming and application efficiency resulting in more sustainable sections discuss medical imaging in Section 2, and omics- scale data analysis in Section 3.1, concluding with a com- Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga- nized by CINI, May 29-30, 2024, Naples, Italy prehensive overview in Section 5. * Corresponding author. $ carlo.adornetto@unical.it (C. Adornetto); pierangela.bruno@unical.it (P. Bruno); francesco.calimeri@unical.it 2. Medical Imaging and AI (F. Calimeri); edoardo.derose@unical.it (E. D. Rose); gianluigi.greco@unical.it (G. Greco); 2.1. Vessel segmentation of alessandro.quarta@uniroma1.it (A. Quarta)  0000-0002-9734-1017 (C. Adornetto); 0000-0002-0832-0151 cine-angiography (P. Bruno); 0000-0002-0866-0834 (F. Calimeri); 0000-0002-0032-9434 The methodology adopted in this study systematically (E. D. Rose); 0000-0002-5799-6828 (G. Greco); 0000-0001-6319-2466 (A. Quarta) enhances the evaluation of vascular complexity in Pe- © 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License ripheral Arterial Occlusive Disease (PAOD) patients Attribution 4.0 International (CC BY 4.0). CEUR Workshop Proceedings http://ceur-ws.org ISSN 1613-0073 CEUR Workshop Proceedings (CEUR-WS.org) through the integration of advanced imaging segmenta- CEUR ceur-ws.org Workshop ISSN 1613-0073 Proceedings Figure 1: Outcomes from automatic segmentation on two distinct patients are depicted. The figure on the left, Fig. a, represents the patient with the lowest AUC value across the entire study group, whereas Fig. b on the right displays the patient with the highest AUC value. Each figure progresses from top to bottom, beginning with the stitched image (i.e., the grayscale input image), followed by the ground truth segmentation, and concluding with the automatically segmented image. tion and computational analysis. Utilizing deep learning classifying different tissue types, proving to be an invalu- techniques, this research successfully transforms cine- able tool in enhancing diagnostic accuracy and patient angiography videos into detailed static images, markedly care [9]. enhancing the clarity and reliability of vascular assess- ments. Furthermore, the adoption of fractal dimension as 2.2.1. Laryngeal Endoscopic Images a quantitative metric for vascular complexity introduces a novel, objective criterion to the field. This dual approach In this work, we present a novel approach using deep not only promises to mitigate the subjectivity inherent learning (DL) for performing semantic segmentation on in current diagnostic practices but also establishes a ro- laryngeal endoscopy images, building upon the foun- bust correlation with conventional clinical evaluations, dations laid by previous research [10, 11]. The dataset potentially revolutionizing PAOD management strate- utilized in this study includes 536 color images manually gies [8]. Incorporating advanced imaging segmentation segmented from in vivo laryngeal examinations, all at a and computational analysis, our method significantly re- resolution of 512×512 pixels, originating from two sepa- fines the assessment of vascular complexity in PAOD pa- rate surgical procedures. These images are categorized tients. Figure 1 vividly illustrates the segmented vascular into seven distinct groups: void, vocal folds, other tis- trees from cine-angiography, alongside their correspond- sue, glottal space, pathology, surgical tool, and intubation. ing fractal dimension analysis, showcasing the clarity Our model’s predictive capabilities were significantly and precision of our deep learning-based approach. The enhanced by leveraging the capabilities of rule-based study achieved significant findings, demonstrating that languages, especially Answer Set Programming (ASP). the deep learning-based segmentation method resulted Incorporating ASP allowed us to navigate the neural in an Area Under the Curve mean value of 0.77 ± 0.07, network’s (NN) decision-making with greater precision, with a range from 0.57 to 0.87. This method significantly applying penalties for inaccuracies grounded in well- improved the reliability of visual assessments of vascular established knowledge. Moreover, rule-based methods complexity, achieving an Inter-Class Correlation Coeffi- were applied to refine our model’s output, successfully cient (ICC) of 0.96 for segmented images, compared to rectifying minor mistakes, such as single pixels misla- 0.76 for video assessments. Additionally, the Fractal Di- beled, and adjusting misclassified categories that were mension (FD) analysis correlated well with clinical scores, inconsistent with medical guidelines. showing coefficients of 0.85 for manually segmented im- In summary, our approach has shown substantial ef- ages and 0.75 for automatically segmented images. fectiveness, attaining an average Intersection over Union (IoU) score above 0.7, a figure significantly improved by subsequent post-processing strategies. 2.2. Segmentation Semantic segmentation, a process that entails labeling each pixel of an image with a specific class, represents a major leap forward within the realm of medical imaging. This method has been widely adopted for its critical role in identifying tumors, recognizing various organs, and Figure 2: The proposed algorithm for selecting a subset of genes relevant to classify CLL patients. The input data is used to compute the genes pairwise correlation matrix (step 1), and the correlation matrix is clustered (step 2) to group similarly correlated genes. The clusters are then mapped to the original input data and transposed. AEs are trained for each cluster to select the most representative gene, reducing dimensionality (step 3). The genes are ranked with F-test, selecting a subset with the highest F-value (step 4). A neural network is trained with a selected set of genes to perform binary classification of the CLL patients (step 5). The best NNs architecture is determined through model selection, and the SHAP XAI method explains each gene’s importance in the predictions (step 6). 3. Engineered Data Encoding for tailored treatments, yet their analysis is complex due to three main reasons: (1) course of dimensionality: a Medical Advancements genomics dataset typically consists of a very large num- In this section, we delve into the innovative intersection ber of genes (features) for a small number of patients of feature engineering and medicine, focusing on manipu- (samples); (2) imbalanced classes: there is often a signifi- lating latent spaces to enable new AI-based solutions. We cant difference between the number of instances in each explore a series of our works in which we exploit suitably group of interest; (3) Noise sequencing data are typically defined latent spaces to design new gene selection algo- collected from multiple sources, different laboratories, rithms and Generative AI approaches. In the following, and sequencing tools resulting in noisy datasets difficult we will discuss a new algorithm for gene selection and its to analyze. application to Chronic Lymphocytic Leukemia (CLL), and We proposed a new algorithm for genomic-scale analysis, two new generative AI approaches used for automatic based on DL and XAI, whose aim is threefold: first, select report generation and inverse design of materials and the most meaningful genes for a regression/classifica- molecules. Our works not only showcase the potential tion problem; second, provide a more accurate prediction of latent spaces in enhancing precision and efficiency in model; third, quantify and evaluate the feature’s contri- medical research but also highlight their role in fostering bution to the predictions through XAI [12]. The proposed the development of novel therapeutic strategies, mark- algorithm is based on two main ideas: (1) recognize simi- ing a significant stride toward the future of personalized larly correlated features using clustered correlation ma- medicine. trix and then filter the redundant information for each group by using Autoencoders (AEs). In contrast with previous works, where AEs are used for dimensional- 3.1. AI for Omics Data Analysis ity reduction [13], we implemented a mechanism to still Functional genomics data, particularly GEP datasets, are work at the level of the original features. We hence pro- crucial in medical science for diagnosis, prevention, and vide a more treatable dataset in terms of dimensionality, without affecting interpretability; (2) we train NNs and fering from the non-uniqueness of the solution where, we iteratively select the most meaningful features using moreover, very different devices can share identical prop- a new ad-hoc defined XAI score. We eventually use the erties. Furthermore, the design spaces are likely high- set of selected features (from all the iterations) to train dimensional and subjected to feasibility constraints. and explain a final model. Most of the state-of-the-art DL methods for inverse de- We used a preliminary version of this algorithm (depicted sign share the idea of looking for the design solution by in Figure 2) for the GEP analysis of CLL patients. In our directly working at the level of the design space; indeed, work [14] we introduced the DeepSHAP Autoencoder they have been mainly conceived to deal with applica- Filter for Genes Selection (DSAF-GS), a deep learning tions where such a space is a low-dimensional space. By and explainable AI-based method for gene selection in departing from these approaches, a few works in the genomics-scale data analysis. Through the SHAP explain- literature have already advocated the benefits of map- able AI techniques, we identified key genes influencing ping the input space into a continuous latent space. This CLL prognosis with high accuracy. Our findings pave perspective influenced our work which proposes a neu- the way for more targeted bio-molecular research in CLL, ral network architecture, named GIDnet (Generative suggesting novel paths for investigating disease mecha- Inverse Design Network), where the suitable solutions nisms and therapy timing. are additionally constrained to the only feasible region of the latent design space, and an exploration algorithm is 3.2. Building and Exploring Meaningful used to end up with more accurate solutions [16]. A thor- ough experimental activity over several state-of-the-art Latent Spaces for Generative AI in benchmark datasets evidenced the superior performance Medicine of GIDnet for inverse design problems. Automatic Medical Report Generation via Latent In a promising future scenario, our approach can be Space Conditioning and Transformers built using GNNs to generate specific social networks, In this work, we explore the integration of artificial molecules, and topological representations starting from intelligence within healthcare, focusing on automatic the prior desired properties. Our generative approach, in- medical report generation. We introduce the VAE-GPT deed, demonstrated breakthrough performances in such architecture, combining Variational Autoencoder (VAE) scenarios where the design space is large, discrete, and and Generative Pre-trained Transformer (GPT) for constrained, taking into account such feasibility con- generating medical reports from images. The VAE learns straints during the design process itself. a latent representation of images, capturing underlying patterns, while the GPT uses this representation to 4. Other Research Activities generate coherent text. For the purpose the VAE is jointly trained with a pre-trained text generator (GPT) This research group has also engaged in a variety of and a tags predictor such that images belonging to the studies including the impact of a Nutrition Education same context (e.g. diseases) are placed in the same Program combined with physical activity on the Mediter- region of the latent space. Furthermore, we propose a ranean Diet adherence and inflammatory biomarkers in novel metric, Medical Embeddings Attention Distance adolescents, showing significant improvements [17]. Ad- (MEAD), to measure the semantic similarity between ditionally, they have examined the dynamics of opinion generated and reference reports. Our experiments diffusion within social networks, identifying effective demonstrate state-of-the-art performance in creating strategies based on centrality measures for influencing informative medical reports, highlighting the potential opinion adoption [18]. Furthermore, [19] have proposed of AI in enhancing diagnostic processes [15]. a neuro-symbolic AI approach for the compliance verifi- cation of electrical control panels in Industry 4.0, utilizing GIDnets: Generative Neural Networks for Solving a combination of deep learning and Answer Set Program- Inverse Design Problems via Latent Space Explo- ming to detect anomalies with limited data. In [20] de- ration veloped a Graph Neural Network model to assess lateral In fields such as Engineering, Molecular Biology, and spreading displacement in New Zealand, aiming to en- Physics, the design of technological tools and device hance earthquake impact predictions. In [21] is presented structures is progressively supported by Inverse Design a statistical framework to learn more effectively from al- methods, providing suggestions on crucial architectural gorithm validation challenges, specifically for medical choices based on the properties that these tools and de- image analysis in laparoscopic videos, identifying under- vices should exhibit. The inverse design problem aims exposure and motion as significant sources of errors. [22] at designing proper devices according to a set of desired introduced a deep learning framework using heatmaps properties and it is typically an ill-posed problem suf- for disease classification based on gene expression data, demonstrating its effectiveness in tumor classification. [3] E. De Rose, Continual learning: an approach via In [23] detailed a method for reducing and visualizing feature maps extrapolation, in: DC@AIxIA23 Doc- data for automatic diagnosis using gene expression and toral Consortium of AIxIA 2023 conference, CEUR clinical data, achieving high recall rates in diagnoses. Proceedings, volume 3537, Italian Association for Lastly, we also developed a system to improve the in- Artificial Intelligence, 2023, p. To be assigned. terpretability of automatic diagnosis by analyzing the [4] E. Alhenawi, R. Al-Sayyed, A. Hudaib, S. Mirjalili, internal decision-making processes of neural networks Feature selection methods on gene expression mi- [24]. croarray data for cancer classification: A systematic review, Computers in Biology and Medicine 140 (2022) 105051. 5. Conclusion [5] S. M. Lundberg, S.-I. Lee, A unified approach to interpreting model predictions, Advances in neural This work advances the application of Artificial Intelli- information processing systems 30 (2017). gence (AI) and Deep Learning (DL) in medical diagnostics [6] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, and genomics, demonstrating their transformative po- D. Parikh, D. Batra, Grad-cam: Visual explanations tential for enhancing diagnostic accuracy and enabling from deep networks via gradient-based localization, personalized medicine. By employing advanced imaging in: Proceedings of the IEEE international confer- segmentation, computational analysis, and introducing ence on computer vision, 2017, pp. 618–626. fractal dimension as a novel metric for vascular com- [7] C. Adornetto, P. Bruno, F. Calimeri, E. DE ROSE, plexity, we offer innovative solutions to the challenges G. Greco, et al., Artificial intelligence in medicine: in medical imaging and omics data analysis. Our find- From imaging to omics, in: CEUR WORKSHOP ings highlight the effectiveness of these methods in im- PROCEEDINGS, volume 3486, 2023, pp. 140–145. proving the reliability of medical assessments and the [8] P. Bruno, M. F. Spadea, S. Scaramuzzino, S. De Rosa, interpretability of complex data through Explainable Ar- C. Indolfi, G. Gargiulo, G. Giugliano, G. Esposito, tificial Intelligence (XAI) techniques. The integration of F. Calimeri, P. Zaffino, Assessing vascular complex- AI in healthcare, as illustrated by our research, promises ity of paod patients by deep learning-based segmen- to refine diagnostic processes, optimize treatment plans, tation and fractal dimension, Neural Computing and contribute significantly to the future of personalized and Applications 34 (2022) 22015–22022. patient care. [9] T. Dhamija, A. Gupta, S. Gupta, R. Katarya, G. Singh, Semantic segmentation in medical images through Acknowledgements transfused convolution and transformer networks, Applied Intelligence 53 (2023) 1132–1148. This work has been partially funded by PON “Ricerca e [10] M.-H. Laves, J. Bicker, L. A. Kahrs, T. Ortmaier, Innovazione” 2014-2020, CUP: H25F21001230004, and has A dataset of laryngeal endoscopic images with been carried out while Alessandro Quarta was enrolled comparative study on convolution neural network- in the Italian National Doctorate on Artificial Intelligence based semantic segmentation, International jour- run by Sapienza University of Rome with University of nal of computer assisted radiology and surgery 14 Calabria. (2019) 483–492. [11] P. Bruno, F. Calimeri, C. Marte, M. Manna, Com- bining deep learning and asp-based models for the References semantic segmentation of medical images, in: Rules and Reasoning: 5th International Joint Conference, [1] C. S. Lee, A. Y. Lee, Clinical applications of contin- RuleML+ RR 2021, Leuven, Belgium, September 13– ual learning machine learning, The Lancet Digital 15, 2021, Proceedings 5, Springer, 2021, pp. 95–110. Health 2 (2020) e279–e281. [12] C. Adornetto, G. Greco, A new deep learning and [2] A. Quarta, P. Bruno, F. Calimeri, Continual learn- xai-based algorithm for features selection in ge- ing for medical image classification, in: F. Calimeri, nomics, 2023. arXiv:2303.16914. M. Dragoni, F. Stella (Eds.), 1st AIxIA Workshop on [13] P. Danaee, R. Ghaeini, D. A. Hendrix, A deep learn- Artificial Intelligence For Healthcare (HC@AIxIA ing approach for cancer detection and relevant gene 2022) co-located with the 21st International Con- identification, in: Pacific symposium on biocom- ference of the Italian Association for Artificial In- puting 2017, World Scientific, 2017, pp. 219–229. telligence (AIxIA 2022), Udine, Italy, November [14] F. Morabito, C. Adornetto, P. Monti, A. Amaro, 30 2022, volume 3307 of CEUR Workshop Proceed- F. Reggiani, M. Colombo, Y. Rodriguez-Aldana, ings, CEUR-WS.org, 2022, pp. 67–76. URL: https: G. Tripepi, G. D’Arrigo, C. Vener, et al., Genes //ceur-ws.org/Vol-3307/paper7.pdf. selection using deep learning and explainable artifi- cial intelligence for chronic lymphocytic leukemia [24] P. Bruno, F. Calimeri, A. S. Kitanidis, E. De Momi, predicting the need and time to therapy, Frontiers Understanding automatic diagnosis and classifica- in Oncology 13 (2023). tion processes with data visualization, in: 2020 [15] C. Adornetto, A. Guzzo, A. Vasile, Automatic medi- IEEE international conference on human-machine cal report generation via latent space condition- systems (ICHMS), IEEE, 2020, pp. 1–6. ing and transformers, in: 2023 IEEE Intl Conf on Dependable, Autonomic and Secure Comput- ing, Intl Conf on Pervasive Intelligence and Com- puting, Intl Conf on Cloud and Big Data Comput- ing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech), IEEE, 2023, pp. 0428–0435. [16] C. Adornetto, G. Greco, Gidnets: generative neural networks for solving inverse design problems via latent space exploration, in: Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, 2023, pp. 3404–3413. [17] C. Morelli, E. Avolio, A. Galluccio, G. Caparello, E. Manes, S. Ferraro, A. Caruso, D. De Rose, I. Barone, C. Adornetto, et al., Nutrition educa- tion program and physical activity improve the adherence to the mediterranean diet: impact on inflammatory biomarker levels in healthy adoles- cents from the dimenu longitudinal study, Frontiers in Nutrition 8 (2021) 685247. [18] C. Adornetto, V. Fionda, G. Greco, On the effec- tiveness of compact strategies for opinion diffusion in social environments, in: ECAI 2023, IOS Press, 2023, pp. 11–18. [19] V. Barbara, M. Guarascio, N. Leone, G. Manco, A. Quarta, F. Ricca, E. Ritacco, Neuro-symbolic ai for compliance checking of electrical control pan- els, Theory and Practice of Logic Programming 23 (2023) 748–764. [20] M. Durante, G. Terremoto, C. Adornetto, G. Greco, E. Rathje, A new graph neural network (gnn) based model for the evaluation of lateral spread- ing displacement in new zealand, in: Proceed- ings of the 8th International Conference on Earth- quake Geotechnical Engineering (VIII ICEGE), Os- aka, Japan, 2024, pp. 7–10. [21] T. Roß, P. Bruno, A. Reinke, M. Wiesenfarth, L. Koeppel, P. M. Full, B. Pekdemir, P. Godau, D. Trofimova, F. Isensee, et al., Beyond rankings: Learning (more) from algorithm validation, Medical image analysis 86 (2023) 102765. [22] P. Bruno, F. Calimeri, Using heatmaps for deep learning based disease classification, in: 2019 IEEE conference on computational intelligence in bioin- formatics and computational biology (CIBCB), IEEE, 2019, pp. 1–7. [23] P. Bruno, F. Calimeri, A. S. Kitanidis, E. De Momi, Data reduction and data visualization for automatic diagnosis using gene expression and clinical data, Artificial Intelligence in Medicine 107 (2020) 101884.