Over the last decades, many prognostic models based on artificial intelligence techniques have been used to provide detailed predictions in healthcare. Unfortunately, the real-world observational data used to train and validate these models are almost always afected by biases that can strongly impact the outcomes validity: two examples are values missing not-at-random and selection bias. Addressing them is a key element in achieving transportability and in studying the causal relationships that are critical in clinical decision making, going beyond simpler statistical approaches based on probabilistic association. In this context, we propose a novel approach that combines selection diagrams, missingness graphs, causal discovery and prior knowledge into a single graphical model to estimate the cardiovascular risk of adolescent and young females who survived breast cancer. We learn this model from data comprising two diferent cohorts of patients. The resulting causal network model is validated by expert clinicians in terms of risk assessment, accuracy and explainability, and provides a prognostic model that outperforms competing machine learning methods.
An important application of artificial intelligence in healthcare is predicting a disease trajectory conditional on the patient’s history and a projected treatment strategy. In this context, we aim to develop a model that generalizes well: it must provide accurate predictions not only on the study cohort (the population from which the data have been collected) but also on the target cohort, the general population that it is designed for. The model’s validity, accuracy and usefulness in clinical practice are then of paramount relevance.
In order to establish validity in a rigorous causal framework, we must clearly identify the
scope of the study the data come from, as well as the applicability and limitations of the theory
we are studying. However, many other issues can impact model validity. Firstly, study cohorts
difer from target cohorts in both randomized controlled trials and clinical observational studies
due to their inclusion and exclusion criteria. Secondly, missingness patterns may introduce bias
in the model when the data are missing not at random (MNAR). Thirdly, information can be
fragmented across diferent data sets ( data sparsity) and experts (domain knowledge). Therefore,
it is important to understand the data selection mechanism and the merging process to ensure
the model validity and generalisability [
Our main contributions in this context are: • Developing the first causal network model for estimating the risk of cardiovascular diseases in adolescent and young adults that have been treated and survived breast cancer. • Developing and adapting a methodological approach to deal with data afected by selection bias and sufering by MNAR. • A thoughtful review of the model we developed and of its implications by domain experts (that is, expert physicians).
The rest of the paper is organized as follows. We start by introducing our research question and the topic of the study: cardio-oncology, the subfield of cardiology that aims at significantly reducing cardiovascular morbidity and mortality and at improving the quality of life in cancer survivors, here young patients (Section 2). We complete this introduction by providing an overview of the related work available in the literature (Section 3). We then describe the data (Section 4.1), the domain knowledge (Section 4.2) and the causal networks methodology (Section 5) that we use to design and develop our model. Finally, we describe our findings (Section 6), we contrast them with the available clinical and epidemiological knowledge and we assess model performance against that of other commonly-used machine learning approaches. We complete the paper by summarising our conclusions and presenting future work (Section 7).
The increasing number and life expectancy of cancer survivors in the last decades has
highlighted the acute and long-term cardio-toxic efects of diferent cancer therapies. The most
common cancer among adolescent and young-adult females (AYAs; 15–39 years at the first cancer
diagnosis) is breast cancer (BC), which is a rare disease with unique genetic and biological
features in this age class. AYA female patients are more likely to be genetically susceptible to
more aggressive form of BC than older females, and their diagnosis is often delayed because of
the lack of BC screening policies for their age group. While lower than in older females with
BC, AYAs BC survival rates are high and continuously increasing, thus making such patients
likely to be long-term survivors [
While cardiovascular diseases (CVDs) are well characterized in some groups of cancer survivors, only few studies describe CVDs in AYAs because of their rarity in young patients. Therefore, further evidence is needed to understand and prevent CVDs in AYAs with BC for helping clinicians to plan personalized and efective follow-up guidelines for these patients. Thus, with this work we are interested in answering to the question: “To what extent is it possible to predict and explain individual susceptibility to cardiotoxicity in AYA with BC?”,
Machine learning (ML) techniques have achieved remarkable results in predicting CVDs in
cancer patients [
4.1. Data
This project makes use of data coming from two separate cohorts:
• The population-based cohort (PBC): a retrospective cohort of about 1,500 AYAs BC
patients who completed cancer treatment, that is, that survived at least 1 year after the cancer
diagnosis. In this cohort, BC cases have been identified in population-based cancer
registries and the information from each BC patient has been linked to several administrative
data sets (hospital discharge records, outpatients and drug flows) for CVD follow-ups [
Around 3% of AYAs BC patients from the PBC had at least one CVD event during the follow-up
(mean follow-up time = 5 years). However, the Framingham score for women with the same
characteristics as those in the PBC ranges between 0 and 9, which translates to a predicted
risk of 0% to 1% of developing a hard coronary heart disease (such as myocardial infraction)
or of dying for a CVD event within 10 years [
The CBC consists of patients treated by a single institution (Fondazione IRCCS Istituto Nazionale dei Tumori di Milano) and is biased due to the patient selection mechanism in ways that are apparent from the baseline and treatment variables distribution. Around 8% of AYAs BC patients from the CBC have at least one major cardiac risk factor before treatment, compared to only 5% in the PBC. Moreover, there is an higher proportion of patients receiving neoadjuvant treatments (30% in the CBC vs 23% in the PBC), which are more likely to be prescribed when dealing with later tumor stages at diagnosis (with lymph nodes involvement, metastasis and high dimension tumors). These facts reflect a more severe case-mix of both baseline characteristics of the patients and of tumor aggressiveness in the CBC compared to the PBC, which is expected since the Institute is a referral expertise cancer center.
Furthermore, the PBC and the CBC were collected for diferent purposes, resulting in different missingness mechanisms. In the CBC, the purpose is patient care, so factors that may influence tumor prognosis (such as tumor grading, staging, etc.) are recorded in greater detail; whereas cancer registry data, which form the basis of the PBC, are collected for administrative, epidemiological monitoring and public health purposes. More details about the distribution of missing values by cohort type is reported in Table S1 in the Supplementary Material. their percentage of completeness in the cohort of origin.
In this section we elicit the clinical knowledge on cardio-oncology. To make it easier for the reader to understand how the domain knowledge has been translated from natural language to the structure of the causal network model, we report in square brackets the label of the corresponding nodes.
The description of the selection mechanism behind the selection node [cohort] has already been described in Section 4.1.
To decide which treatment to give to a young BC patient, clinicians rely on the most
wellknown prognostic factors for 5-year cancer survival [death_in_5y] [
The risk of CVDs after both neoadjuvant [_neo] and adjuvant [_adju] cancer treatments
is well known. For instance, anthracycline is frequently prescribed as a chemotherapy regimen
[chemo_], either alone or in combinations with other chemotherapical drugs. Its
cardiotoxic efects are well documented and largely attributable to the generation of free radicals:
it ultimately results in left-ventricular dysfunction, arrhythmias [cardiotoxicity] and, in
turn, heart failure in the most severe cases [
Moreover, 5 years of tamoxifen [hormons_], with or without ovarian suppression/ablation, is considered the standard hormone therapy in young women with hormone-receptor positive disease, which represent the vast majority in this age group. However, it has frequent chronic late efects like type-2 diabetes [t2db], hypertension [hypertension] and dyslipidemia [dyspidemia] which are all known to be major risk factors for long-term ischemic heart diseases [ischemic_heart_disease] [19].
Assessing validity is a crucial aspect of causal inference. Specifically, internal validity refers to generalizing the evidence from a study sample to the underlying study population; external validity refers to transporting the evidence to a diferent target population. The interest in the transportability of inference has increased exponentially in the last few years [20, 21] thanks to methodological breakthroughs in handling distribution shifts and selection bias. In this work, we leverage both missingness graphs [22, 23, 24] and selection diagrams [25] to learn a causal Bayesian network by combining observational data together with prior knowledge in the presence of selection bias and missingness bias.
Model-based approaches rely on the formal specification of a model representing the interactions we are interested in. In the case of causal inference, causal graphs are the de-facto standard for encoding causal relationships.
Definition 1 (Causal Graph). A causal graph = (V, E) [26] is a directed acyclic graph where for each directed edge (, ) ∈ E, is a direct cause of and is a direct efect of . The vertex set V can be split into two disjoint subsets V = O ∪ U, where O is the set of the fully observed variables (with no missing values) and U is the set of fully unobserved variables, also known as latent variables.
A causal graph allows researches to (i) decide if a consistent estimator exists for a casual efect and (ii) derive that estimator directly from the graph. Being able to correctly estimate causal efects is extremely important for policy making. For instance, it allows clinicians to assess the impact of a given drug on a disease from observational studies when randomized controlled trials are not available for a particular sub-population [27]. In more general terms, let be a causal graph, a treatment and an outcome. A consistent estimator for the causal efect of on is given by the do-operator: ( = | ( = )) = ∑︁ ( = | = , Z = z) (Z = z) z (1) if there exists a set Z that satisfies the back-door criterion for [28]. If no consistent estimator exists, confounding bias makes it impossible to estimate the causal efect correctly. Unbiased estimators of causal efects are theoretically possible even without a causal graph, but their assumptions are rarely satisfied in practical applications [ 29]. Therefore, causal graphs remain the tool of choice to achieve both explainability and consistency.
The missingness mechanism also plays an important role in causal modeling. Common pre-processing techniques that deal with missing values such as sample deletion and missing imputation are often inefective or even detrimental to causal efect estimation. For instance, [ 30, 31] have shown how missing-data handling has a significant impact on the clinical conclusions that can be drawn from experimental results . Therefore, modeling the reason why a given value is missing is crucial, especially for MNAR data.
The framework for modeling missingness mechanisms is detailed in Rubin’s foundational
work [
Definition 2 (Missingness Graph). A missingness graph ℳ = (V, E) [22] is a causal graph whose vertex set V is partitioned into five disjoint subsets O ∪ U ∪ M ∪ S ∪ R, where: M is the set of the partially observed variables, that is, the variables with at least one missing value; S is the set of the proxy variables, that is, the variables that are actually observed; R is the set of the missingness indicators.
Since missingness graphs are extended causal graphs, d-separation [
This makes it possible to verify whether a consistent estimator for the joint probability (X)
exists in case of MNAR. When the conditions in [
(RX = 0, X) ∈X ( = 0 | Π , RΠ = 0) (2) where is the missingness indicator for variable , Π is the parent set of and RX is the union of the for all the variables in X.
When the data are a collation of multiple data sets, their diferent selection criteria may induce
discrepancies in the distribution of some of the collected variables. In such cases, pooling the
data together without modeling the context from which observations come from could induce
inconsistent estimates [
Definition 3 (Selection Diagram). Let Π and Π * be two diferent populations with a common
underlying causal graph . A selection diagram extends the causal graph so that:
• E ⊂ E , that is, is a sub-graph of ;
• ∃S ̸= ∅, S ⊂ V ∧ S ̸⊂ V where ( → ) ∈ E if ̸= * , that is, the variables S
point to the variables V that difer in their value assignment in Π and Π * .
The variables S are usually called selection variables and allow us to pool together all the
observations while keeping track of their provenance. In related work on selection bias, these
variables are called context variables and identify the context in which V have been collected.
We refer the reader to [
Once a causal graph is designed, we need to connect it to the data . To do so, we rely
on Casual networks (CNs) [
Definition 4 (Causal Network). Let be a causal graph and let (X) be a global probability distribution with parameters Θ . A causal network = (, Θ) is a causal model where each variable of X is a vertex of and (X) factorizes into local probability distributions following: (X) = ∏︁ ( | Π )
∈X where Π is the parent set of the variable .
If we combine a missingness graph and a selection diagram into a single causal graph, we can recover from confounding bias, inconsistent estimators due to missing values and domain discrepancies with a single causal network.
Manually constructing the true causal graph is virtually impossible in real-world applications
where there is little to no control over the experimental setting. For instance, there could
be unobserved variables afecting the data generating mechanism or unknown interactions
between the observed variables. Causal discovery [
In this contribution, we take into account the impact of the missingness mechanism using the
Structural Expectation-Maximization (SEM) algorithm [
Literature validation. We validated the edges that are added by the SEM algorithm to the model using the medical literature to support the results relevance in this domain. For this purpose, we performed a literature review to identify publications that explain what we observe in the data.
Performance metrics. The combined data set comprising the PBC and CBC cohorts was split in a training set and a test set. As mentioned in Section 4.1, the CBC does not include information on the outcome variables, so we included it only in the training set. The ratio between training and test set was 70% (60% PBC + 10% CBC) / 30% (PBC). We considered further splitting the training set to obtain a validation set, but that would reduce the sets sample size to the point of making the analysis unfeasible.
We evaluated the CN we learned from the training set on the test set by estimating the probability of CVDs and the associated Area Under the receiver operating characteristic Curve (AUC). In particular, we contrasted the AUC obtained with 0, with the network learned from SEM (starting from 0), and with other standard ML methods. AUC represents the degree or measure of separability: it measures how much the model is capable of distinguishing between classes. The higher the AUC, the better the model is at predicting healthy individuals as healthy and individuals afected by CVDs as at risk.
Validation of SEM algorithm results. Figure 1 shows the network learned by SEM: edges
elicited from the domain knowledge are in black, while the new ones, that highlight new medical
insights, are in bold blue. The latter comprise:
• Edges directed from [cohort] to the three adjuvant treatments [radio_adju],
[chemo_adju], [hormons_adju] and to one neoadjuvant treatment [target_neo].
These edges can be explained by the selection mechanism: the treatments nodes act as
proxies of case severity [
These edges can be explained by other unstudied efects of oxidative stress and inflammation that induce CVDs but that are not mediated by cardiotoxicity (for instance, grade histology
cohort ki67 pt receptors vascular
pn death_in_5y radio_neo chemo_neo target_neo
hormons_neo surgery radio_adju chemo_adju hormons_adju t2db target_adju hypertension dyslipidemia cardiotoxicity ischemic_heart_disease
metabolomic and other unmeasured CV risk factors such as obesity) [
Model performance. The CN model based on the clinical prior knowledge illustrated in Section 4.2 showed a strong classification performance, with an AUC of about 83% in the test set. The new edges introduced by the SEM algorithm improved performance to about 88%.
It is important to clarify that results obtained from standard ML methods are not directly comparable to those obtained by the proposed approach, for several reasons. Firstly, standard ML methods are not designed to accommodate missing values, relying instead on pre-processing techniques such as sample deletion or imputation. Therefore, only cases coming from the PBC and with complete information on all variables, less than half of the available cases (≈ 800), can be used to train and validate ML models. Secondly, these models do not address the selection bias and lack of external validity. Finally, their results are not causally and clinically interpretable. Against these limitations, we can see in Table 1 that the proposed CN models outperforms all standard ML methods in terms of classification performance.
The CN shown in Figure 1 is the first causal network model for estimating the CVD risk in adolescents and young adult females who survived breast cancer. Thanks to the combination of several causal inference methods, including missingness graphs, selection diagrams and causal discovery algorithms, we were able to build a model that combines domain expert knowledge and multiple data sets. The proposed model is able to: • Deal with the uncertainty and biases that are intrinsic to real-world observational data and produce valid results, generalizing to other applications or populations; • Combine data coming from diferent data sources , which is especially relevant when dealing with small and understudied populations for which data are extremely sparse. • Incorporate the domain knowledge coming from experts, which has been shown to be fundamental in maximizing the model’s performance. • Provide interpretable causal recommendations to identify those patients that are at higher risk of developing a CVDs and what risk factors are involved, to help physicians in clinical decision making and patient follow-up management.
Moreover, the edges added by the causal discovery to the prior-knowledge-based causal network improved its performance and highlighted new arcs that open medical insights particularly valuable, especially for such an understudied population.
According to our experimental results, the proposed CN model outperforms standard supervised learning methods in terms of classification accuracy: this is unexpected since these methods are specifically trained to optimise it. The are two possible explanations for this phenomenon. Firstly, as mentioned in the results section, the data used to train the models are XGBoost: Extreme Gradient Boosting; AODE: Averaged One-Dependence Estimators. diferent: standard ML methods can use only cases coming from the PBC and with complete information on all variables, which means less than half of the cases used to train the CN models. Secondly, the CN models did not learn the model from scratch thanks to their ability to incorporate prior domain knowledge; that makes them more eficient at dealing with low sample sizes and missing-not-at-random issues.
Despite its relevance and strengths, this work has some limitations. First of all, we focused on the classification accuracy of a specific variable of interest instead of evaluating the CN as a whole. This allowed us to compare it with commonly-used ML supervised learning approaches using standard metrics such as AUC, arguing that the proposed CN is competitive in this respect. Having shown that, we are now ready to move beyond prediction into more advanced applications of causal inference which are impossible to carry out with traditional ML models. Moreover, while the arcs added by the SEM algorithm have been validated by a clinical expert and a literature review, a further review by a domain expert (that is, a cardio-oncologist) will be crucial to completely validate the final model.
In conclusion, we are working on further extending our model to include unmeasured variables, such as distant metastases; this is necessary to further address the need of a clinically relevant model, to help cardio-oncologists in their everyday practice.
Alice Bernasconi is funded by an AIRC 2020 project, grant number 24864, titled “pRedicting cardiOvascular diSeAses iN adolescent and young breast caNcer pAtients (ROSANNA)”.
Alessio Zanga is funded by F. Hofmann-La Roche Ltd.
This work was partially supported by the MUR under the grant “Dipartimenti di Eccellenza 2023-2027" of the Department of Informatics, Systems and Communication of the University of Milano-Bicocca, Italy. [18] J. An, M. S. Sheikh, Toxicology of trastuzumab: An insight into mechanisms of cardiotoxicity, Current Cancer Drug Targets 19 (2019) 400–407. doi:10.2174/ 1568009618666171129222159. [19] A. Christinat, S. D. Lascio, O. Pagani, Hormonal therapies in young breast cancer patients: when, what and for how long?, Journal of thoracic disease 5 Suppl 1 (2013) S36–46. doi:10.3978/j.issn.2072-1439.2013.05.25. [20] I. Degtiar, S. Rose, A Review of Generalizability and Transportability, Annual Review of Statistics and Its Application 10 (2023) 501–524. URL: https:// www.annualreviews.org/doi/10.1146/annurev-statistics-042522-103837. doi:10.1146/ annurev-statistics-042522-103837. [21] K. M. Esterling, D. Brady, E. Schwitzgebel, The Necessity of Construct and External Validity for Generalized Causal Claims, Technical Report 18, s.l., 2023. URL: http://hdl.handle.net/ 10419/268605. [22] K. Mohan, J. Pearl, Graphical Models for Processing Missing Data, Journal of the American Statistical Association 116 (2018) 1023–1037. URL: https://www.tandfonline.com/doi/full/ 10.1080/01621459.2021.1874961http://arxiv.org/abs/1801.03583. doi:10.1080/01621459. 2021.1874961. [23] R. Tu, K. Zhang, P. Ackermann, B. C. Bertilson, C. Glymour, H. Kjellström, C. Zhang, Causal
Discovery in the Presence of Missing Data (2018). URL: http://arxiv.org/abs/1807.04010. [24] Y. Liu, A. C. Constantinou, Greedy structure learning from data that contain systematic missing values, Machine Learning 111 (2022) 3867–3896. URL: https://link.springer.com/ article/10.1007/s10994-022-06195-8. doi:10.1007/S10994-022-06195-8/TABLES/9. [25] S. Lee, J. Correa, E. Bareinboim, General Transportability – Synthesizing Observations and Experiments from Heterogeneous Domains, Proceedings of the AAAI Conference on Artificial Intelligence 34 (2020) 10210–10217. URL: https://ojs.aaai.org/index.php/AAAI/ article/view/6582. doi:10.1609/aaai.v34i06.6582. [26] J. Pearl, Causal diagrams for empirical research, Biometrika 82 (1995) 669–688. doi:10.
1093/biomet/82.4.669. [27] C. Fernández-Loría, F. Provost, Causal Decision Making and Causal Efect Estimation Are Not the Same... and Why It Matters, the inaugural issue of the INFORMS Journal of Data Science (2021). URL: http://arxiv.org/abs/2104.04103. [28] J. Pearl, Causality: Models, Reasoning and Inference, 2nd ed., Cambridge University Press,
USA, 2009. [29] M. A. Hernán, J. M. Robins, Causal Inference: What If, Boca Raton: Chapman \& Hall/CRC, 2020. [30] M. R. Stavseth, T. Clausen, J. Røislien, How handling missing data may impact conclusions: A comparison of six diferent imputation methods for categorical questionnaire data, SAGE Open Medicine 7 (2019) 205031211882291. URL: https://doi.org/10.1177/2050312118822912. doi:10.1177/2050312118822912. [31] A. Zanga, A. Bernasconi, P. J. F. Lucas, H. Pijnenborg, C. Reijnen, M. Scutari, F. Stella, Causal Discovery with Missing Data in a Multicentric Clinical Study, in: Proceedings of the 21st International Conference of Artificial Intelligence in Medicine (AIME), volume 13897 LNAI, 2023, pp. 40–44. URL: https://link.springer.com/10.1007/978-3-031-34344-5_5. doi:10.1007/978-3-031-34344-5{\_}5.
PBC = Population-based cohort; CBC = Clinical-based Cohort; CVDs = Cardiovascular diseases. CBC