Artificial Intelligence (AI) encompasses a variety of methods and algorithms that have found applications across numerous domains over time. The increasing complexity and abundance of data in healthcare have spurred investigations into utilizing AI techniques within the medical field, leading to promising avenues for fostering innovation, facilitating early diagnosis, and aiding in treatments. In this study, we provide a concise overview of several initiatives conducted within this realm at the University of Naples Federico II node of the CINI-AIIS Lab, highlighting their primary objectives and contributions.
Artificial intelligence (AI) mimics human intelligence in machines to carry out tasks involving abstraction and problem-solving. Among the various sectors influenced by AI, healthcare stands out as a highly promising field for its application. Indeed, the integration of AI in healthcare has the potential to aid both patients and healthcare professionals, revolutionizing patient care and administrative operations. Additionally, AI-driven platforms possess the capability to analyze patient data, highlight potential health issues, and enhance diagnostic precision for physicians, particularly in scenarios involving intricate medical histories or multiple conditions.
AI applications encompass a variety of technologies rather than being confined to a single one. Among these, Machine Learning (ML) stands out as a subset of AI, comprising algorithms that enable systems to autonomously learn and enhance their performance through experience. In the realm of healthcare, traditional ML finds widespread use, notably in precision medicine, where it predicts optimal treatment protocols based on patient attributes and contextual factors.
Deep Learning (DL) is a class of ML algorithms characterized by the use of Artificial Neural Networks (ANNs) that simulate the structure of the human brain. DL approaches have gained popularity in pattern recognition tasks, particularly in image processing, improving medical image analysis. Moreover, in recent years, there has been a notable surge in interest surrounding the application of Large Language Models (LLMs) within the medical domain. LLMs are advanced natural language processing (NLP) models trained on massive amounts of text data, capable of understanding, generating, and processing human language with remarkable accuracy and fluency.
In this paper, we will illustrate some of the projects exploiting AI techniques in the medical field carried out at the University of Naples Federico II node of the CINI-AIIS Lab, highlighting their innovative aspects and contributions.
medical recommendations sideration of their initial perspectives, thus facilitating the generation of revised reports. Ultimately, the internist physician consolidates all reports to produce a unified summary for the human specialist utilizing the system.
LLMs often overlook the phenomenon of LLM
hallucinations [
The Predict module exploits the wealth of informa- communicating with a virtual model, even when aware
tion embedded within Electronic Health Records (EHRs) of its non-human nature. The potential of a model to
derived from the MIMIC-III dataset [
The Interpret module is pivotal for elucidating the this emulation capability is coupled with a remarkable
decision-making process of the GNN model. It consists accuracy of responses. The fact that some studies are
in understanding the underlying rationales behind a de- beginning to show a preference for responses provided
cision. Interpretation serves as an indispensable tool by AI models like ChatGPT, in terms of accuracy, nuance,
bridging the intricate nature of AI models with the hu- and even empathy, opens up many interesting
possibiliman requisites for transparency and trustworthiness in ties for the future of medicine and healthcare. In a recent
decision-making processes. In our study, we evaluated study, some responses from ChatGPT in the medical field
the eficacy of the Integrated Gradients [
In the Explain module, the outcomes derived from the tems to improve doctors’ responses to patient inquiries previous phases undergo refinement through a pool of and lighten their workload.
LLM-based collaborative agents, operating in accordance However, this progress also presents significant risks. with a specific protocol. Initially, an internist physician Considering only the efect on patients, there is concern enlists a panel of specialists (e.g., cardiologists, nephrolo- that the widespread use of AI to provide psychological gists), tailored to the individual patient’s characteristics. support may contribute to exacerbating stigma towards Each specialist evaluates the patient’s characteristics, the certain categories of patients, especially in the psychiatric predictions from the GNN-based model, and the findings context, relegating them to "non-human" interactions. of the Interpret phase to generate a comprehensive as- Additionally, there is a risk of eroding trust in traditional sessment, elucidating the justification behind the model’s medicine and perceiving healthcare as impersonal, as prediction. Then, specialists review the reports of their well as potentially reducing the responsibility of human peers and engage in a discussion that may lead to recon- operators in expressing empathy and managing interactions with patients, prompting various reflections on the areas of overlap between chatbots and human operators. clinical applications, providing deep insights into patientIf the new possibilities ofered by AI in the medical field specific brain characteristics. Indeed, the network’s abilare manifold, it seems crucial to recognize its limitations ity to predict the subject’s age allows for the evaluation and ensure that these technologies are used ethically and of brain-PAD as a biomarker for a range of neurological under the supervision of qualified professionals. Technol- diseases, including Multiple Sclerosis, Alzheimer’s disogy, as a tool and given its current state of development, ease, and schizophrenia. Moreover, the decomposition of should integrate rather than replace human interaction brain MRI into its main components of shape and appearin the medical context, ensuring that values such as em- ance enables the assessment of how a specific subject pathy, clinical judgment, and professional ethics remain deviates from an age-specific standard template. When at the centre of healthcare. While the combination of hu- applied to a population of patients with neurological disman expertise and technological advancements promises eases, we expect that the deformation field will capture to significantly improve healthcare, its success will de- disease-specific characteristics such as atrophy, while the pend on the ability to efectively balance the skills of texture component may be useful for lesion detection. both. Finally, the ability to disentangle age-related features provides the architecture with generative capacity, allowing for longitudinal evaluation obtained by varying the 4. ASAD Project age-specific latent representation
used to model chronological age as a function of brain 5. Dementia Severity Assessment Magnetic Resonance Imaging (MRI) scans in healthy individuals with excellent results [9]. The extent to which with Incomplete Multimodal a person deviates from healthy brain-ageing trajecto- Data ries, expressed as the diference between predicted and chronological age (brain-predicted age diference, brain- Alzheimer’s disease (AD) is the most common cause of PAD), has been proposed as an index of structural brain dementia, afecting millions of elderly people around the health, sensitive to brain pathology in a wide spectrum world. AD is a neurodegenerative disorder, and early of neurological disorders [10]. However, the presence of detection is a key element to improve the quality of life subject-specific characteristics in diferent acquisitions of afected patients and their families. In clinical trials, necessitates capturing factors of variation within the tar- Magnetic Resonance Imaging (MRI) and Positron Emisget population. In this project, our aim is to propose sion Tomography (PET) are mostly used for the early an innovative DL-based model for age, shape, and ap- diagnosis of neurodegenerative disorders since they propearance disentanglement (ASAD) in brain MRI. This vide volumetric and metabolic function information of model will enable a more precise quantification of the the brain, respectively. impact of pathologies on the brain, significantly enhanc- There is the need of combining information from heteroing the analysis of the brain-PAD. We will employ an geneous and complementary sources, such as MRI and Autoencoder architecture consisting of an Encoder to PET, to evaluate the structural and metabolic characterdefine three latent representations for age, shape, and istics of the brain. This makes the Multimodal Learning appearance, respectively. Additionally, a Regressor will well suited in the case of dementia assessment. Techbe utilized to compute the predicted age, along with a set niques for multimodal data fusion can be categorized of three decoders: Da, Ds, and Dt. Specifically, Da and into early, intermediate, and late fusion. Early fusion inDs will extract the texture (appearance) and the deforma- tegrates multiple sources of data into a single features vection field (shape component), respectively, while Dt will tor, before being used as input to a learning model. Late consider the age-specific latent representation to provide fusion, also referred as decision-level fusion, combines an age-specific template. Following a similar approach as according to a given rule the decisions from multiple clasproposed in [11], the ASAD architecture will be trained sifiers, each trained on separate modalities. Intermediate to reconstruct the input from the disentangled compo- fusion, also named as joint learning, exploits the deep nents, using only data from healthy controls with varying neural networks to transform raw inputs into higherage ranges. This enables the network to model a healthy level and shared representations, which are constructed, brain-aging trajectory. Subsequently, the architecture for instance, by merging into a single layer, units coming will be applied to heterogeneous patient populations en- from multiple modality-specific paths. However, when compassing a wide spectrum of neurological disorders. working with a multimodal dataset in the medical field, it This application aims to detect disease-specific character- is not easy to have images of all the involved modalities, istics within the disentangled components. The disentan- belonging to the same patient. For each subject, paired glement of age, shape, and appearance may have several acquisition consists of images coming from all the diferent sources and collected at the same time or in a specific range. In a real scenario, patients may have incomplete acquisitions, in which some modalities are missed.
In the work proposed in [12], we conducted a systematic analysis of early, late and joint approaches in fusion for dementia severity assessment on the publicly available OASIS-3 dataset [13]. We focused on 3D Convolutional Neural Network (CNN) to exploit the volumetric features of the involved images, including in the training step strategies to handle a high imbalance and incomplete dataset. In particular, we analyzed the efects of the in- Figure 1: Use case: Data-centric approach for digital health complete dataset in each multimodal fusion technique, transformation and in the case of intermediate fusion, we proposed a Multiple Input - Multi Output 3D CNN whose training iteration changes according to the characteristics of the input volumes. To further assess the generalization ability of the implemented methodology, we are including the ADNI dataset [14], a study consisting of about 2500 subjects and focusing on the progression of mild cognitive impairment and early AD [15]. when making decisions, leading to better outcomes for patients. This requires three fundamental steps: integrate, open, innovate: use interoperability standards to integrate existing systems and data. Storing data in an open, vendor-neutral format will then enable ecosystems of vendors to innovate. Real use cases of data-centric architectures for healthcare, such as the one proposed at Karolinska University Hospital, have been already developed (see Figure 1). The adoption of standards like OpenEHR, FHIR, HL7, and ontologies like Snomed CT represent the technical foundations upon which this vision can be realized to achieve semantic and structural interoperability in personal health data, that is to ensure high data quality.
In the age of digital transformation, healthcare is rapidly evolving into a data-driven ecosystem. Imagine a healthcare system where patient records are seamlessly interconnected, diagnosis is made with unprecedented precision, and treatments are tailored in real time. Datacentric architectures are the key to unlocking this visionary healthcare landscape. The transition from a model- 7. UNet-based multi-class nuclei centric approach to AI to a data-centric one signifies a shift in emphasis when it comes to creating and imple- segmentation menting AI systems. Model-centric AI aims at producing the best model for a given dataset, whereas data-centric In recent years, the application of AI in Healthcare is AI aims at systematically and algorithmically producing increasingly stimulating researchers interests [16, 17]. the best dataset to feed a given ML model. Nuclei panoptic segmentation, i.e., the simultaneous de
Current challenges and limitations in health data gov- tection, segmentation, and classification of nuclear inernance have demonstrated the need for a real digital stances, is at the core of the automation of several tasks in transformation of healthcare, where decisions are made digital pathology, particularly in the analysis of routine based on data, whether it is patient history, laboratory Hematoxylin and Eosin (H&E) stained histology slides. results, or imaging data. AI algorithms can assist in iden- Distillation Framework. In our framework, we adopt tifying high-risk patients, predicting treatment outcomes, an ofline technique using HoVerNet as a pre-trained and enabling personalized medicine. Nevertheless, ensur- teacher network. Given that HoVerNet performs nuclei ing patient data security, AI algorithm accountability, and instance segmentation and classification through three transparency are crucial to address privacy, security, and branches, our distillation strategy is based on the idea of bias concerns. Emerging technologies, such as Extended combining all output branches of HoVerNet into a single reality (XR) and blockchains, are being already used for branch network. Note that we aim to train a student that improving patient care and guarantee the security and can replace only the HoVerNet backbone, not its postprivacy of the data. In this context, the data-centric man- processing steps, which we left unvaried. We employ ifesto serves as a beacon for the healthcare community. a single-branch UNet as our student model and join all Collaboration among clinicians, healthcare organizations, HoVerNet branch outputs into a single branch with a and technology vendors is indispensable in implement- number of output channels equal to the total number ing a data-centric approach to coalesce around a common of HoVerNet’s branches. In particular, we used a Mix vision, and to ensure that all relevant data is considered Vision Transformer (MixViT) as the backbone for UNet, resulting in the best combination based on our experi- fined by the absence of identifiable biochemical or strucments. Our loss is a linear combination of the student tural anomalies. These conditions afect almost 50% of loss between the student and the ground truth and the infants in their first year of life [20] distillation loss between the student and the teacher reg- This study examines the considerable impact of FGIDs ulated by parameter. on children, their families and healthcare systems, and In this work we used two datasets, namely PanNuke [18] highlights the historic challenge of identifying children at for training HoVer-UNet and CoNSeP [19] for validating risk due to unclear pathophysiology. The research aims results on external data. In the case of PanNuke dataset, to identify early-life risk factors for FGIDs [21], within our solution achieved comparable performance to HoV- the first year of life. Using a prospective observational erNet, demonstrating a significant advantage in terms of cohort design, the study enrolled term and preterm inprocessing speed. When the CoNSeP dataset is consid- fants from a tertiary care university hospital in Foggia, ered, the results showed that our solution outperforms Italy, between 1 January 2020 and 31 December 2022, HoVerNet in terms of panoptic quality, though it falls excluding infants with severe disease or major neonatal short in terms of F-score detection. Regarding classifica- complications. By using conventional statistical methods tion metrics, our solution outperforms HoVerNet across and Machine Learning (ML), this study identified birth neoplastic and epithelial nuclei; it is practically equal for weight, cord blood pH, and maternal age as significant miscellaneous and worse for inflammatory. Lastly, the predictors for FGIDs. A logistic regression predictive inference time is about three times lower. model also established an inverse relationship between
Figure 2 shows visual examples of the results of HoV- these variables and the occurrence of FGIDs. Using these erNet and HoVer-UNet compared with the CoNSeP refer- findings, the study created a ML-based predictive model ence standard. Overall, the similarity between the results and a practical, user-friendly web interface for risk assupports the practical efectiveness of our approach. sessment. This enables clinicians to identify infants at a higher risk for FGIDs. The approach marks a pioneering Ground Truth HoVerNet Ours Ground Truth HoVerNet Ours step in FGID risk prediction.
Neoplastic
Inflammatory
Epithelial
Miscellaneous
nificant challenge in pediatric healthcare due to their prevalence and impact on infants. FGIDs refer to a range of conditions, including infant colic, regurgitation, functional diarrhea, and functional constipation, that are de
Vancouver, BC, Canada, 2019, pp. 9240–9251. URL: [15] A. De Simone, C. Sansone, A multimodal deep https://proceedings.neurips.cc/paper/2019/hash/ learning based approach for alzheimer’s disease d80b7040b773199015de6d3b4293c8f-Abstract. diagnosis, in: International Conference on Image html. Analysis and Processing, Springer, 2023, pp. 131– [5] R. Bowman, O. Cooney, J. W. Newbold, A. Thieme, 139.
L. Clark, G. Doherty, B. Cowan, Exploring how [16] C. Tommasino, F. Merolla, C. Russo, S. Staibano, politeness impacts the user experience of chatbots A. M. Rinaldi, Histopathological image deep feature for mental health support, International Journal of representation for cbir in smart pacs, Journal of Human-Computer Studies 184 (2024) 103181. Digital Imaging 36 (2023) 2194–2209. [6] F. Amato, S. Marrone, V. Moscato, G. Piantadosi, [17] A. M. Rinaldi, C. Russo, C. Tommasino, Efects of A. Picariello, C. Sansone, et al., Chatbots meet color stain normalization in histopathology image ehealth: Automatizing healthcare., in: WAIAH@ retrieval using deep learning, in: 2022 IEEE InternaAI* IA, 2017, pp. 40–49. tional Symposium on Multimedia (ISM), IEEE, 2022, [7] F. Amato, S. Marrone, V. Moscato, G. Piantadosi, pp. 26–33.
A. Picariello, C. Sansone, Holmes: Ehealth in the [18] J. Gamper, N. A. Koohbanani, K. Benes, S. Graham, big data and deep learning era, Information 10 M. Jahanifar, S. A. Khurram, A. Azam, K. Hewitt, (2019) 34. N. Rajpoot, Pannuke dataset extension, insights and [8] J. W. Ayers, A. Poliak, M. Dredze, E. C. Leas, Z. Zhu, baselines, arXiv preprint arXiv:2003.10778 (2020).
J. B. Kelley, D. J. Faix, A. M. Goodman, C. A. [19] S. Graham, Q. D. Vu, S. E. A. Raza, A. Azam, Y. W. Longhurst, M. Hogarth, et al., Comparing physi- Tsang, J. T. Kwak, N. Rajpoot, Hover-net: Simulcian and artificial intelligence chatbot responses to taneous segmentation and classification of nuclei patient questions posted to a public social media in multi-tissue histology images, Medical image forum, JAMA internal medicine 183 (2023) 589–596. analysis 58 (2019) 101563. [9] M. Tanveer, M. Ganaie, I. Beheshti, T. Goel, N. Ah- [20] A. Chogle, C. A. Velasco-Benitez, I. J. Koppen, J. E. mad, K.-T. Lai, K. Huang, Y.-D. Zhang, J. Del Ser, Moreno, C. R. R. Hernández, M. Saps, A populationC.-T. Lin, Deep learning for brain age estimation: based study on the epidemiology of functional gasA systematic review, Information Fusion (2023). trointestinal disorders in young children, The Jour[10] J. H. Cole, R. P. Poudel, D. Tsagkrasoulis, M. W. nal of pediatrics 179 (2016) 139–143.
Caan, C. Steves, T. D. Spector, G. Montana, Predict- [21] D. Bi, H. Jiang, K. Yang, T. Guan, L. Hou, G. Shu, ing brain age with deep learning from raw imaging Neonatal risk factors for functional gastrointestinal data results in a reliable and heritable biomarker, disorders in preterm infants in the first year of life NeuroImage 163 (2017) 115–124. (2022). [11] Z. Shu, M. Sahasrabudhe, R. A. Guler, D. Samaras,
N. Paragios, I. Kokkinos, Deforming autoencoders: Unsupervised disentangling of shape and appearance, in: Proceedings of the European conference on computer vision (ECCV), 2018, pp. 650–665. [12] M. Gravina, A. García-Pedrero, C. Gonzalo-Martín,
C. Sansone, P. Soda, Multi input–multi output 3d cnn for dementia severity assessment with incomplete multimodal data, Artificial Intelligence in
Medicine 149 (2024) 102774. [13] P. J. LaMontagne, T. L. Benzinger, J. C. Morris,
S. Keefe, R. Hornbeck, C. Xiong, E. Grant, J. Hassenstab, K. Moulder, A. G. Vlassenko, et al., Oasis3: longitudinal neuroimaging, clinical, and cognitive dataset for normal aging and alzheimer disease,
MedRxiv (2019). [14] C. R. Jack Jr, M. A. Bernstein, N. C. Fox, P. Thompson, G. Alexander, D. Harvey, B. Borowski, P. J. Britson, J. L. Whitwell, C. Ward, et al., The alzheimer’s disease neuroimaging initiative (adni): Mri methods, Journal of Magnetic Resonance Imaging: An Oficial Journal of the International Society for Magnetic Resonance in Medicine 27 (2008) 685–691.