=Paper=
{{Paper
|id=Vol-3762/476
|storemode=property
|title=Advancing e-health with AI: Insights from our research experience in neuroimaging, acoustic signals, and vital parameter monitoring
|pdfUrl=https://ceur-ws.org/Vol-3762/476.pdf
|volume=Vol-3762
|authors=Gabriella Casalino,Giovanna Castellano,Gennaro Vessio,Gianluca Zaza
|dblpUrl=https://dblp.org/rec/conf/ital-ia/CasalinoCVZ24
}}
==Advancing e-health with AI: Insights from our research experience in neuroimaging, acoustic signals, and vital parameter monitoring==
https://ceur-ws.org/Vol-3762/476.pdf
Advancing e-health with AI: Insights from our research
experience in neuroimaging, acoustic signals, and vital
parameter monitoring
Gabriella Casalino, Giovanna Castellano∗ , Gennaro Vessio and Gianluca Zaza
Department of Computer Science, University of Bari Aldo Moro, Italy
Abstract
This contribution briefly describes the research being carried out in the Computational Intelligence Laboratory of the
Department of Computer Science, University of Bari Aldo Moro, in AI-based e-health. Our research encompasses a wide
array of methodologies and applications aimed at leveraging the capability of AI to empower the diagnosis, monitoring, and
treatment of various health conditions. Through multifaceted research that covers neuroimaging analysis, acoustic signal
processing, and vital parameter monitoring, our goal is to shed light on the potential of AI in enhancing healthcare services.
Keywords
artificial intelligence, explainability, e-health, neuroimaging, acoustic signals, vital parameters
1. Introduction 2.1. Alzheimer’s disease detection
Integrating Artificial Intelligence (AI) into healthcare is Dementia, with Alzheimer’s disease (AD) being its most
transforming how we diagnose, monitor, and care for common form, poses a significant global health challenge,
patients. At the Computational Intelligence Laboratory especially among the aging population. It currently af-
(CILab) of the Department of Computer Science, Uni- fects around 55 million people worldwide, predominantly
versity of Bari Aldo Moro, we are contributing to this in low- and middle-income countries, and this number is
transformation by applying AI to neuroimaging, acoustic expected to increase as the global population ages. Un-
signal analysis, and vital signs monitoring. Our work fortunately, effective cures remain elusive, with available
aims to address current healthcare challenges by devel- treatments focusing more on symptom management than
oping innovative and practical AI solutions. addressing the underlying causes. This underscores the
This paper presents an overview of our research efforts critical need for early and accurate diagnosis to improve
and achievements in these areas. By sharing our findings patient care.
and methodologies, we aim to highlight AI’s significant AI, mainly through advanced machine and deep learn-
impact on improving healthcare services and patient out- ing techniques, is increasingly recognized for its potential
comes. Our goal is to showcase our work and encourage to revolutionize the diagnosis of dementia, including AD.
ongoing innovation and dialogue in the rapidly evolving Neuroimaging techniques, such as Magnetic Resonance
field of AI in healthcare. Imaging (MRI) and amyloid Positron Emission Tomog-
raphy (PET) scans, have been identified as promising
tools for early detection. MRI provides detailed images
2. Neuroimaging of the brain, enabling the identification of brain atrophy
patterns characteristic of AD. At the same time, amy-
Neuroimaging is a pivotal area within our research port- loid PET scans offer insights into the pathophysiology of
folio, where the application of AI-driven algorithms plays the disease by detecting amyloid plaques in the brain, a
a crucial role in enabling the early and precise diagnosis hallmark of AD.
of neurological disorders. Our research has advanced the application of Convolu-
tional Neural Network (CNN) models for the automated
Ital-IA 2024: 4th National Conference on Artificial Intelligence, orga- diagnosis of AD, using the strengths of both MRI and PET
nized by CINI, May 29-30, 2024, Naples, Italy scans [1]. We examined these neuroimaging techniques’
∗
Corresponding author.
efficacy in uni-modal and multi-modal setups, under-
Envelope-Open gabriella.casalino@uniba.it (G. Casalino);
giovanna.castellano@uniba.it (G. Castellano); scoring the advantage of integrating data from diverse
gennaro.vessio@uniba.it (G. Vessio); gianluca.zaza@uniba.it modalities to refine diagnostic precision. Additionally,
(G. Zaza) we incorporated an explainable AI method to address
Orcid 0000-0003-0713-2260 (G. Casalino); 0000-0002-6489-8628 the demand for transparency in medical AI applications,
(G. Castellano); 0000-0002-0883-2691 (G. Vessio); offering insights into the AI-driven diagnostic process
0000-0003-3272-9739 (G. Zaza)
© 2024 Copyright for this paper by its authors. Use permitted under Creative Commons License and contributing to a deeper understanding of AD’s un-
Attribution 4.0 International (CC BY 4.0).
CEUR
ceur-ws.org
Workshop ISSN 1613-0073
Proceedings
Figure 1: Diagram illustrating the proposed multi-modal CNN architecture designed for AD detection, which simultaneously
processes 3D MRI and PET scan inputs for enhanced diagnostic accuracy.
derlying mechanisms. in an unsupervised manner. These representations were
Specifically, our investigation has yielded several key then used to train a supervised 3D CNN for AD detec-
insights, highlighting the potential of multi-modal imag- tion. This innovative strategy demonstrated encouraging
ing strategies. By analyzing the classification results from outcomes on the OASIS-3 dataset and lessened the de-
various model configurations on the OASIS-3 benchmark pendence on extensively annotated datasets, setting the
dataset, we discovered that models utilizing 3D inputs stage for more autonomous and quantitative AD detec-
consistently outperformed those using 2D inputs, likely tion in clinical practice. Our future endeavors will aim
due to the richer spatial information available in 3D scans. to assess this method across broader and more varied
Moreover, regardless of whether in 2D or 3D, MRI scans datasets to affirm its diagnostic validity further.
significantly surpassed amyloid PET scans in diagnostic
performance, emphasizing MRI’s inherent value in AD 2.2. Brain tumor segmentation
detection within our study. However, multi-modal strate-
gies, particularly our “fusion” model (shown in Fig. 1), Brain tumor segmentation from MRI scans is crucial for
demonstrated a clear advantage, achieving up to 95% ac- accurate diagnosis, treatment planning, and patient mon-
curacy. This underlines the complementary nature of itoring. Recent strides in deep learning, particularly with
MRI and PET scans in AD diagnosis. CNNs, have significantly advanced the automation and
Our adoption of the Grad-CAM technique further al- precision of tumor segmentation. Nevertheless, these
lowed us to pinpoint the brain regions most relevant models’ “black-box” nature raises challenges in explain-
for classification, offering valuable insights into the neu- ability—a vital aspect of clinician trust and decision-
roanatomical underpinnings of AD. This supports the making.
validity of our models and enhances our understanding Graph Neural Networks (GNNs) have recently gained
of AD’s neuropathology. attention as a novel approach to medical image analysis,
In another exploratory study [2], we delved into Diffu- offering an alternative that might bridge the gap between
sion Tensor Imaging (DTI), a sophisticated MRI technique accuracy and explainability. By conceptualizing brain im-
that assesses the integrity of white matter fiber tracts in ages as graphs—where nodes represent voxels or regions
the brain. We explored fractional anisotropy (FA), a DTI of interest, and edges depict spatial relationships—GNNs
metric that measures the uniformity of water diffusion use relational dependencies to achieve fine segmentation.
directionality, which exhibits notable changes in AD pa- This capability to capture both local and global contexts
tients, suggesting its utility as a diagnostic indicator. through message passing between nodes positions GNNs
We introduced a dual-stage deep learning approach, as a promising tool for achieving high precision in brain
combining unsupervised and supervised techniques. Ini- tumor segmentation and providing a pathway to model
tially, a 3D convolutional autoencoder was employed to understandability.
extract low-dimensional representations from FA images In a recent study [3], we analyzed GNN models for seg-
menting brain tumors, focusing on their explainability. shown promise in enhancing classification accuracy, even
Using GNNExplainer, we aimed to improve the trans- with limited labeled data [4].
parency of GNN models, making their decision-making We introduced a novel algorithm, the Dynamic In-
processes accessible and understandable to clinicians. cremental Semi-Supervised Fuzzy C-Means (DISSFCM),
Our exploration highlighted the effectiveness of GNNs designed to monitor BD states while considering the
in medical imaging. It also laid the foundation for future temporal acquisition of acoustic features. DISSFCM, an
research, suggesting potential synergies between GNNs extension of the Semi-Supervised Fuzzy C-Means (SS-
and CNNs, such as integrating GNNs with 3D U-Net FCM) algorithm, analyses data chunks sequentially in
architectures, to refine segmentation results further. In near real-time, maintaining historical data insights with-
addition, collaboration with medical experts to examine out extensive storage. It adapts to new information, refin-
critical features identified by GNNExplainer could further ing the classification through an increased cluster count
solidify the role of GNNs in clinical practice, combining representing the patient’s condition states. This method
accuracy and explainability in brain tumor management. has proven effective in predicting episodes of health and
illness with as little as 25% labeled data [5].
DISSFCM operates on labeled prototypes, summariz-
3. Acoustic signals ing data clusters for each segment. It generates member-
ship matrices, clarifying each data point’s cluster associ-
The analysis of vocal characteristics from speech sam-
ation and facilitating outcome explanation. Initially, we
ples is an effective approach to identifying conditions
applied visual analytics for interpretation [6], advancing
associated with mental diseases, notably bipolar disorder
to natural language explanations or linguistic summaries,
(BD). Our research focused on extracting acoustic fea-
which translate complex data relations into understand-
tures from patients with BD using a specialized mobile
able sentences [7]. For example, we could deduce that
application, developed at the Department of Affective Dis-
“Most calls in the state of hypomania have low loudness
orders, Institute of Psychiatry and Neurology in Warsaw,
compared to the state of euthymia”. This approach seg-
Poland, under the project “Smartphone-based diagnos-
ments acoustic features into semantic categories—loud-
tics of phase changes in the course of bipolar disorder”.
ness, pitch, spectrum, and voice quality—guided by psy-
BD manifests through fluctuating mood states, including
chiatric expertise. Our experiments have demonstrated
euthymia, depression, mixed states, and mania, tradi-
the practical application of linguistic summaries as infor-
tionally diagnosed through regular consultations using
mative granules for smartphone-based BD monitoring.
standard psychiatric tools like the Hamilton Depression
They offer clear, insightful linguistic descriptions, mak-
Rating Scale (HDRS) and the Young Mania Rating Scale
ing the complex data and sparse psychiatric evaluations
(YMRS). These instruments allow healthcare providers to
comprehensible.
detect symptoms and evaluate the intensity of depressive
and manic episodes, facilitating precise diagnoses and
the development of customized treatment strategies. 3.2. Explaining bipolar disorder states
Our research examined several critical dimensions of We designed a versatile, multi-task neural network to
data related to BD, specifically focusing on the impor- leverage the detailed symptom information captured dur-
tance of continuous monitoring to track temporal fluc- ing patient assessments. This network is trained to gen-
tuations. We addressed the challenge of missing labels erate several outputs, each aligning with the various
while also handling the uncertainty in labeling due to the levels of labels obtained from intermediate assessment
inherent ambiguity and variability in data classification. stages. These intermediate outputs fulfill dual roles: they
Moreover, we worked on generating readily understand- enhance the model’s overall predictive accuracy and pro-
able explanations of BD state classifications leveraging vide insights into classifying mid-level labels. Our ar-
the availability of multi-layered information. chitecture, designed to handle data with a hierarchical
class structure, is a crucial component of PLENARY (ex-
3.1. Monitoring bipolar disorder states Plaining bLack-box modEls in Natural lAnguage thRough
fuzzY linguistic summaries) [8]. PLENARY aims to cate-
The specialized application captures acoustic features
gorize tabular data across different class levels and render
daily, but patient assessments are less frequent, resulting
the model’s explanations into natural language, employ-
in a scarcity of labeled data. This gap leaves many acous-
ing fuzzy linguistic summaries for clarity.
tic features without clear annotations of the patient’s
In collaboration with a neuropsychiatrist, we identified
condition.
ten critical symptoms as intermediate labels, including
We explored semi-supervised learning algorithms to
anxiety, decreased activity, mood changes, disorganiza-
harness the geometric data properties and the prede-
tion, and sleep disorders, among others. The model’s
fined knowledge of patient states. These algorithms have
outcomes and explanations focus on the patient’s state
and these specific symptoms. For instance, we found that individuals from the discomfort of traditional contact-
“Among records that contribute positively to predicting based monitoring, making it a more convenient and user-
mania, most of them have spectral-related features at friendly option for daily home use.
low level” and “Among records that contribute against Our methodological pipeline (shown in Fig. 2(b)) ini-
predicting decreased activity, most of them have quality- tiates with the detection of the subject’s face, focusing
related features at low level”. specifically on the forehead as the region of interest (ROI)
Through rigorous experimental evaluation, we have for signal extraction. The rPPG signal is then processed
demonstrated that augmenting model explanations with using Independent Component Analysis (ICA) and Fast
fuzzy linguistic summarization—especially those derived Fourier Transform (FFT) to estimate HR and BR. At the
from SHAP analyses—significantly enhances understand- same time, SpO2 measurements are derived by apply-
ing of the model’s predictions. This approach effectively ing the Beer-Lambert law. For lip color detection, the
combines domain-specific knowledge with technical in- system identifies the lip ROI and determines the dom-
sight, providing a comprehensive and accessible explana- inant color using clustering methods. Our contactless
tion framework. approach has not only demonstrated measurement ac-
curacy within acceptable ranges for both stationary and
minimally moving subjects, but it has also shown supe-
4. Vital parameters rior performance compared to traditional contact devices,
instilling confidence in its reliability and accuracy.
Our research into vital parameter monitoring leverages
Further enhancements included the addition of new
AI to anticipate important diseases, equipping patients
ROIs and a face-tracking feature to accommodate head
and physicians with critical insights for preemptive
movements, improving usability on mobile devices [10,
healthcare management. Herein, we detail our efforts
11]. This comprehensive framework is adaptable to any
in remote vital parameter estimation and creating eX-
camera-equipped device, leading to the creation of a
plainable AI (XAI) models to support medical diagnosis.
smartphone application that facilitates easy, widespread
These models use vital signs data to aid medical profes-
monitoring of vital health parameters [12].
sionals in the early detection of cardiovascular diseases
and stress-related conditions.
4.2. Cardiovascular risk assessment
4.1. Contact-less monitoring of vital While traditional machine learning algorithms have sig-
parameters nificantly aided physicians in diagnosing symptoms early
to prevent disease progression, their often opaque nature
Our endeavors in vital parameter monitoring have been presents a challenge. These “black box” models deliver
concentrated on heart rate, breathing rate, blood oxy- accurate predictions but lack an intuitive explanation
gen saturation (SpO2 ), and systolic and diastolic blood for their results. This makes them less practical in fields
pressure—key indicators for cardiovascular health. Tra- where end-users are non-technical professionals, notably
ditional methods, like ECG, require direct skin contact, in healthcare.
often necessitating cumbersome wearable devices. To Our work concentrates on advancing XAI models,
overcome the limitations and discomfort of contact-based which are mainly aimed at supporting medical decisions
monitoring, advancements have been made toward de- in cardiovascular disease (CVD) assessment. CVDs are a
veloping photoplethysmography (PPG) techniques that primary global health concern, responsible for approx-
operate using camera-based systems. However, these can imately 17.9 million deaths annually,1 spanning condi-
be expensive and not user-friendly for daily home use. tions such as coronary heart disease and stroke. Given
Addressing these challenges, our lab has developed an the multifactorial causes of CVDs, including lifestyle and
innovative, cost-effective approach for monitoring car- genetic predispositions, early intervention and continu-
diovascular parameters that seamlessly integrates into ous monitoring of vital signs are crucial to prevention.
everyday living environments [9]. This system employs In our efforts, we have developed a fuzzy rule-based
a non-invasive, contactless device consisting of a trans- system to assist clinicians in evaluating cardiovascular
parent mirror equipped with a camera that identifies risks with greater interpretability [13]. This system uti-
the user’s face and uses remote photoplethysmography lizes IF-THEN rules, a natural language format that sim-
(rPPG) to analyze video frames. The prototype of the plifies understanding and application, incorporating pa-
smart mirror is shown in 2(a). This method calculates tient data like heart rate and blood oxygen saturation
vital parameters like blood oxygen saturation, heart rate, to estimate CVD risk. Developed in collaboration with
and breathing rate and includes a novel technique for au- medical experts, this model prioritizes accuracy while
tomatic lip color detection through clustering-based color
quantization. With this new method, we aim to relieve 1
https://www.who.int/health-topics/cardiovascular-diseases
(a) (b)
Figure 2: (a) Prototype of the smart mirror developed in our lab and (b) methodological pipelines for vital sign measurement.
ensuring user-friendly interpretability, offering a slight [2] G. Castellano, E. Lella, V. Longo, G. Placidi,
trade-off in precision for much greater transparency. M. Polsinelli, G. Vessio, Combining Unsupervised
To bridge the gap between data-driven precision and and Supervised Deep Learning for Alzheimer’s Dis-
expert intuition, we explored neuro-fuzzy systems, which ease Detection by Fractional Anisotropy Imaging,
automate the generation of fuzzy rule-based models in: 2023 IEEE 36th International Symposium on
from data, streamlining the otherwise manual and labor- Computer-Based Medical Systems (CBMS), IEEE,
intensive process of rule formation. Our research demon- 2023, pp. 511–516.
strates that models created through neuro-fuzzy sys- [3] D. Amendola, A. Basile, G. Castellano, G. Vessio,
tems maintain accuracy and significantly enhance in- G. Zaza, From Voxels to Insights: Exploring the
terpretability, outperforming manually designed models Effectiveness and Transparency of Graph Neural
in cardiovascular risk prediction [14]. Networks in Brain Tumor Segmentation, in: In-
Expanding beyond cardiovascular health, we have ap- ternational Joint Conference on Neural Networks
plied neuro-fuzzy systems to diagnose hypertension and (IJCNN 2024), IEEE, to appear.
stress, focusing on minimizing complexity for clearer [4] G. Casalino, G. Castellano, O. Hryniewicz, D. Leite,
understanding. We have balanced accuracy and inter- K. Opara, W. Radziszewska, K. Kaczmarek-Majer,
pretability by employing feature selection to refine the Semi–Supervised vs. Supervised Learning for Men-
number of relevant indicators and fuzzy rules, making tal Health Monitoring: A Case Study on Bipolar
these models highly practical for real-world medical ap- Disorder, International Journal of Applied Mathe-
plications [15]. matics and Computer Science 33 (2023) 419–428.
[5] G. Casalino, G. Castellano, F. Galetta, K. Kaczmarek-
Majer, Dynamic incremental semi-supervised fuzzy
Acknowledgments clustering for bipolar disorder episode prediction,
in: International Conference on Discovery Science,
G.C. and G.Z. acknowledge the support from the FAIR -
Springer, 2020, pp. 79–93.
Future AI Research (PE00000013) project, Spoke 6 - Sym-
[6] G. Casalino, G. Castellano, K. Kaczmarek-Majer,
biotic AI (CUP H97G22000210007), under the NRRP MUR
O. Hryniewicz, Intelligent analysis of data streams
program funded by NextGenerationEU. Ga.C. acknowl-
about phone calls for bipolar disorder monitoring,
edges funding from the European Union PON project
in: 2021 IEEE International Conference on Fuzzy
Ricerca e Innovazione 2014-2020, D.M. 1062/2021. All au-
Systems (FUZZ-IEEE), IEEE, 2021, pp. 1–6.
thors are members of the INdAM GNCS research group.
[7] K. Kaczmarek-Majer, G. Casalino, G. Castel-
Ga.C, G.C, and G.V. are members of the CITEL - Centro
lano, O. Hryniewicz, M. Dominiak, Explaining
Interdipartimentale della ricerca in Telemedicina, Univer-
smartphone-based acoustic data in bipolar disor-
sity of Bari Aldo Moro.
der: Semi-supervised fuzzy clustering and relative
linguistic summaries, Information Sciences 588
References (2022) 174–195.
[8] K. Kaczmarek-Majer, G. Casalino, G. Castellano,
[1] G. Castellano, A. Esposito, E. Lella, G. Montanaro, M. Dominiak, O. Hryniewicz, O. Kamińska, G. Ves-
G. Vessio, Automated detection of Alzheimer’s sio, N. Díaz-Rodríguez, PLENARY: Explaining
disease: a multi-modal approach with 3D MRI and black-box models in natural language through fuzzy
amyloid PET, Scientific Reports 14 (2024) 5210. linguistic summaries, Information Sciences 614
(2022) 374–399.
[9] G. Casalino, G. Castellano, V. Pasquadibisceglie,
G. Zaza, Contact-less real-time monitoring of car-
diovascular risk using video imaging and fuzzy in-
ference rules, Information 10 (2018) 9.
[10] G. Casalino, G. Castellano, G. Zaza, A mHealth so-
lution for contact-less self-monitoring of blood oxy-
gen saturation, in: 2020 IEEE Symposium on Com-
puters and Communications (ISCC), IEEE, 2020, pp.
1–7.
[11] G. Casalino, G. Castellano, G. Zaza, Evaluating the
robustness of a contact-less mHealth solution for
personal and remote monitoring of blood oxygen
saturation, Journal of Ambient Intelligence and
Humanized Computing (2023) 1–10.
[12] G. Casalino, G. Castellano, A. Nisio, V. Pasquadibis-
ceglie, G. Zaza, A mobile app for contactless
measurement of vital signs through remote Photo-
plethysmography, in: 2022 IEEE international con-
ference on systems, man, and cybernetics (SMC),
IEEE, 2022, pp. 2675–2680.
[13] G. Casalino, G. Castellano, C. Castiello,
V. Pasquadibisceglie, G. Zaza, A fuzzy rule-
based decision support system for cardiovascular
risk assessment, in: Fuzzy Logic and Applications:
WILF 2018, Springer, 2019, pp. 97–108.
[14] G. Casalino, G. Castellano, U. Kaymak, G. Zaza, Bal-
ancing accuracy and interpretability through neuro-
fuzzy models for cardiovascular risk assessment, in:
2021 IEEE Symposium Series on Computational In-
telligence (SSCI), IEEE, 2021, pp. 1–8.
[15] G. Casalino, G. Castellano, G. Zaza, Interpretable
Neuro-Fuzzy Models for Stress Prediction, in: Con-
ference of the European Society for Fuzzy Logic
and Technology, Springer, 2023, pp. 630–641.