Incorporating real-world data (RWD) into clinical trials can enhance trial eficiency, diversity, and generalizability. This paper introduces the Framework for Research in Synthetic Control Arms (FRESCA), which uses a novel Recommender System combined with Equity Adjustment strategies to design and evaluate Representative Hybrid Clinical Trials (HCTs). FRESCA employs a novel matching algorithm through its recommendation system to select suitable patients from RWD while ensuring that the selected population is representative of the target demographic. This dual approach improves both patient selection and trial outcomes by balancing statistical appropriateness and equity. Simulations based on data from two existing randomized clinical trials (RCTs) show that using FRESCA to recommend patients from RWD and apply equity adjustments enhances internal validity and generalizability. Our analysis indicates that combining matching and equity adjustments yields more accurate treatment efect estimates and fair population representation, even with reduced RCT control group sizes. In contrast, using either method alone may result in biased outcomes. The flexibility of FRESCA to simulate various HCT scenarios makes it a valuable tool for advancing equitable and eficient clinical trial designs.
Enhancing the eficiency, diversity, and generalizability of clinical trials can be achieved by incorporating
real-world data (RWD) [
The health recommender system in FRESCA identifies patients from RWD who closely match the
characteristics of those in the trial, enhancing the statistical power and reducing variance without
extending the trial duration or increasing costs. However, integrating RWD with randomized control
trial (RCT) data is challenging due to diferences in their distributions [
Our research aims to develop methods for conducting equitable HCTs and to provide a framework
for evaluating them. In this context, equity means that the trial participants should represent a broader
target population [
FRESCA was developed to design and evaluate these methods [9]. Before FRESCA, existing methods for HCTs did not explicitly address equity in patient selection. FRESCA uses real clinical trial data to simulate hypothetical trials, applying its methods to scenarios based on real RCTs, such as Systolic Blood Pressure Intervention Trial (SPRINT) [10] and Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT) [11]. By integrating health recommender systems and equity adjustment techniques, FRESCA ensures that the selected patients not only meet statistical criteria but also represent the demographic characteristics of the target population. We define protected subgroups based on age, gender, and race/ethnicity, using NHANES data [12] to estimate the rates for these subgroups in a simulated target population. We will explore the use of additional protected attributes in future work. FRESCA includes five main functions: generating cohorts, simulating scenarios, calculating target subgroup rates, estimating treatment efects and equity, and providing a final assessment. It can evaluate any HCT method, including those that combine health recommender systems with equity adjustments. Detailed descriptions of the FRESCA framework and the trial configurations are provided in sections 2.4 and 2.5.
This paper makes several key contributions• It identifies equity issues in HCTs and proposes solutions to improve the representativeness of trial populations. • It introduces an enhanced FRESCA framework with a modular architecture that supports multitrial, multi-outcome, and multi-metric assessments for creating and evaluating HCTs. • It evaluates HCT methods that combine health recommender systems based on novel matching algorithm with equity adjustments, showing that the best results come from combining propensity score matching with IPF [13] weighting. • It demonstrates that using both health recommender systems and equity adjustments results in more equitable populations and precise estimates of the Population Hazard Ratio (PHR), even with smaller CC sizes. • It shows that variations in the sizes of treatment and control groups significantly afect the precision of treatment efect estimates, and that a balanced use of recommended synthetic controls is essential for accuracy.
There are several strategies for integrating synthetic control populations into trial populations. Most
approaches use propensity score matching to select suitable external controls. Some methods focus
on matching based on treatment propensity [
We define the problem using the potential outcomes framework from Neehal et al. [ 9]. Let represent the potential outcome for subject in sample under treatment , where = 0 is the target population, = 1 the RCT population, = 2 the RWD population, and = 3 an adjusted sample combining RCT and RWD data. The Sample Hazard Ratio (SHR) in the RCT is = (efect (11, 10)| = 1), where efect (11, 10) is the diference in treatment efects between treated and control groups. The Population Hazard Ratio (PHR) for the target population ( = 0) is defined as = (efect (01, 00)| = 0), representing the expected treatment efect in the target population. Two main issues arise: (1) Equity—the RCT may not represent the target population, leading to biased estimates ( ̸= ), and (2) Sample Size—insuficient patients in the control group may require synthetic controls (SCs) from RWD. To accurately estimate PHR, we use a health recommender system with propensity score matching to augment RCT data with SCs, and then perform appropriate equity adjustment, forming an "equity-adjusted" sample ( = 3). 2.2. Data We define the target population using the nationally representative hypertensive cohort from the National Health and Nutrition Examination Survey (NHANES) 2015-2016 [12]. Representativeness in the RCTs is assessed based on three protected attributes: Gender (Male, Female), Age Group (40-59, 59+), and Race/Ethnicity (Non-Hispanic Black, Non-Hispanic White, Non-Hispanic Asian, Hispanic, Other). These age groups align with the inclusion criteria of the SPRINT and ALLHAT hypertension studies. Target rates for each subgroup are calculated using survey-weighted analysis of US subjects aged 40+ with hypertension.
We use data from SPRINT and ALLHAT trials available from BioLINCC [18]. For ALLHAT, we focus on the Amlodipine vs. Clorthalidone group, as the results are similar for the Lisinopril vs. Clorthalidone group. After preprocessing, SPRINT includes 4,234 treated and 4,200 control patients, and ALLHAT includes 8,116 treated and 13,762 control patients. The primary outcome for SPRINT is a composite of major cardiovascular events, while for ALLHAT, the outcome is heart failure. Figure 1 illustrates how FRESCA divides the RCT data into Treatment Arm (TA), Concurrent Controls (CC), and External Controls (EC) cohorts. Table 1 presents the distribution of protected attributes in the RCT data and the Biased External Controls for both SPRINT and ALLHAT, showing their diferences from the target NHANES population.
The NHANES surveillance data and clinical trial data from BioLINCC were used with appropriate approvals: BioLINCC approved ALLHAT and SPRINT data use under case 123537, and NHANES data is freely available and exempt from human subjects research regulations per Rensselaer Polytechnic Institute IRB 1863.
For balancing distributions between synthetic control and trial populations through the recommender system, we employ propensity score matching using the “MatchIt” R package [19]. Iterative Proportional Fitting (IPF) via the “IPFR” R package [20] is used for equity adjustment and Biased EC Cohort formation.
Treatment efects are assessed using Cox’s Proportional Hazards Regression to estimate the Population
Hazard Ratio (PHR). The “ground-truth” Target PHR is estimated by equity adjustment on the entire
RCT dataset. For equity assessment, we use a variant of log disparity (LD)[
FRESCA integrates a health recommender system to select suitable synthetic controls from external data and combine them with RCT data, forming a hybrid control population. This framework uses propensity score matching for patient selection and equity adjustments to ensure accurate estimates of the Population Hazard Ratio (PHR) while maintaining representativeness for any target population. FRESCA provides tools to assess the efectiveness of these methods, as shown in Fig 1, and comprises ifve main functions. We demonstrate FRESCA’s application using the SPRINT and ALLHAT trials with NHANES as the target population, but the framework is flexible and can be adapted to any RCT, RWD, or target population.
In the Cohort Generation phase, FRESCA employs its health recommender system to generate three cohorts: TA (Treatment Arm), CC (Concurrent Controls), and EC (External Controls). The TA and CC cohorts are derived from RCT data, representing the treatment and control groups, respectively. The EC cohort is sourced from real-world data, providing a pool of synthetic controls recommended by the health recommender system to supplement the RCT. These cohorts collectively form the basis for subsequent analyses.
FRESCA facilitates the creation of diverse simulated trial scenarios to calculate adjusted PHRs and equity metrics. This involves two stages: first, generating unbiased samples from the TA and CC cohorts to simulate a randomized clinical trial; second, creating a Biased EC Sample from the EC cohort, reflecting biased real-world data as the source of synthetic controls. The health recommender system plays a key role in selecting these controls. Further details on simulation configurations are provided in section 2.5.
None 1.39 [1.36, 1.43] 0.89 [0.84, 0.94]† PNrCopMenasticthyinMgatching + IPF Sampling 1.38 [1.36, 1.41] 11..4423 [[11..3372,, 11..4489]] 00..8073[[00..8012,,00..9044]]† Propensity Matching + IPF Weighting 1.39 [1.33, 1.46] 0.04 [0.03, 0.05] None 0.75 [0.73, 0.78] 0.91 [0.86, 0.97]† PNrCopMenasticthyinMgatching + IPF Sampling 0.79 [0.77, 0.82] 00..7745 [[00..7627,, 00..7874]] 00..8091[[00..8040,,00..9061]]† Propensity Matching + IPF Weighting 0.78 [0.74, 0.81] 0.04 [0.03, 0.05]
The target rates for each subgroup are calculated using a survey-weighted analysis of the desired target population from NHANES (e.g., adults with hypertension).
Once the scenario samples are created and target subgroup rates are determined, the next step is to construct an equity-adjusted HCT population and estimate the treatment efect. FRESCA utilizes a health recommender system based on propensity score matching to recommend suitable synthetic controls (SCs) from the Biased EC Sample. A binary logistic regression model, incorporating TA, CC, and Biased EC samples, generates propensity scores to select SCs, thereby forming a matched Hybrid Control Arm (HCA). Equity adjustments are then applied to both the TA and HCA cohorts using the Iterative Proportional Fitting (IPF) technique to better align them with the target population. This process results in specific weight vectors, _ for the TA and _ for HCA. Unlike the previously used approach [9], where these weights were used to generate random samples, we now directly compute the weighted and equity-adjusted treatment efect and equity value (LD) using these weights.
In the Assessment phase, FRESCA evaluates various HCT construction methods by combining the
health recommender system for propensity matching with two types of IPF-based equity adjustments:
weighted and sampling. Baseline scenarios without any adjustments or inclusion of SC are also evaluated
and compared with the NC Matching technique [14], with results summarized in Table 2. To assess the
precision of PHR estimations, FRESCA compares them with a “ground-truth” target PHR, derived from
equity adjustments applied to the complete RCT dataset (e.g., SPRINT/ALLHAT) using all treated and
control subjects. The data set is divided into treated and control cohorts, bootstrapped to match the sizes,
and adjusted to align with the NHANES population. The target PHR is calculated as an average across
all bootstrap samples and scenarios. Equity is evaluated by checking if the Cohort-Target Disparity
(CTD) falls within the [0, 0.22] range, adhering to the 80% rule [
FRESCA simulates various clinical trial scenarios to evaluate the efects of diferent trial design parameters on outcomes using a health recommender system for patient selection. 2.5.1. ALLHAT For ALLHAT, FRESCA creates a Concurrent Control (CC) Cohort by selecting 4,000 individuals from the ALLHAT trial’s original control group, leaving 9,762 in the External Control (EC) Cohort. The Treatment Arm (TA) remains unchanged with 8,116 participants. During simulation, biases are introduced in the EC Cohort using the Iterative Proportional Fitting (IPF) method to reflect biased subgroup rates for age, gender, and race, as well as smoker status, depression history, and HDLC history (Table 1). FRESCA explores diferent experimental scenarios with varying sample sizes for TA ( =4000) and CC ( = 0, 500, 1000, 2000, 4000), conducting 50 bootstrap simulations for each scenario. The mean Population Hazard Ratio (PHR) and Cohort-Target Disparity (CTD) are calculated for each scenario, along with a 95% Confidence Interval. 2.5.2. SPRINT For SPRINT, FRESCA selects a CC Cohort of 2,000 from the control group (total N=4200), leaving 2,200 in the EC Cohort, with the TA consisting of 4,234 participants. IPF is used to adjust for biases in three protected attributes (Table 1) and additional factors such as Framingham Risk Score and Cardiovascular Disease (CVD) History. Simulations use a TA sample size ( =2000) and vary CC sample sizes ( =0, 500, 1000, 1500, 2000), running 50 bootstrap simulations per scenario. Results include mean PHR, CTD, and 95% Confidence Intervals, as reported in the final assessment.
PHR (ALLHAT) − Only CC
PHR (ALLHAT) − Propensity Matched
PHR (ALLHAT) − Equity Adjusted
PHR (ALLHAT) − Both 4000 2000 ze iS1000 C C 500 4000 2000 ze iS1000 C C 500 0 Nodata geeonqabuER i lit 0 Nodata 1.0
Our study evaluates methods for creating HCTs using two metrics as described above: PHR and CTD. We analyze methods that combine propensity score matching with Iterative Proportional Fitting (IPF) equity-adjustment methods (weighting or random sampling) and compare them to two baseline scenarios: (i) no adjustments applied to the CC population, and (ii) SC added to trial via NC Matching [14] algorithm. The results for the ALLHAT and SPRINT trials are detailed in Table 2. Our analysis reveals two key findings: (i) Necessity of both Propensity and Equity Adjustments: PHR estimates without equity adjustments (either by IPF weighting or sampling) are typically inequitable as evident by CTD values, and they may produce biased treatment efects, as evident in high CTD values, and (ii) Superiority of Weighting over Sampling: Using IPF Weighting for equity adjustment improves the accuracy and consistency of PHR estimates compared to IPF Sampling. These findings, particularly the need for comprehensive adjustments and the efectiveness of sample weighting, have been observed consistently across multiple trials and outcomes, demonstrating the robustness of the FRESCA framework.
We studied the efect of varying CC population sizes on the estimated PHR and CTD in HCT. We examined four methods: (i) no adjustments, (ii) only propensity matching, (iii) only equity adjustment, and (iv) both propensity and equity adjustments. Fig 2 shows the results on ALLHAT for CC sizes varying from 0 to 4000. Two main findings emerged - (i) Benefits of Synthetic Controls for Limited Data: In scenarios with smaller CC sizes, missing subgroups were compensated by incorporating SC. This strategy, especially with both propensity and equity adjustments, yielded accurate estimates, and (ii) PHR Accuracy and Acceptable Equity with Reduced CC Population Size: Reducing the CC population size by 50% (from 4000 to 2000) still produced PHR estimates close to the target PHR. However, a larger CC size is preferable for lower estimation variance, indicating a trade-of for trial designers. These patterns were consistent across both SPRINT and ALLHAT trials, afirming the robustness of our findings. We only show results for weighting equity adjustment for the ALLHAT trial for brevity, and additional results are available in the Supplementary.
We analyzed factors afecting the accuracy of PHR estimates in ALLHAT using a linear model with seven predictors. Bias in PHR estimation was quantified as the squared deviation from the target PHR. Predictors included the size of the treatment arm (TA Size), the control group size (CC Size), CohortTarget Disparity (CTD), and Cohort-RCT Disparity (CRD), which measures the distribution diferences between control populations and the RCT population. Controls were categorized by adjustment type: propensity, equity, both, or none. Results in Table 3 showed that TA Size, CC Size, and CRD significantly predicted PHR bias. Key findings include: (i) Larger CC Size Reduces Bias: Increasing CC size lowers bias, favoring a larger control group directly recruited over synthetic controls; (ii) Impact of Adjustments: "Only Equity" and "Propensity and Equity" adjustments reduce bias compared to the "Only CC" category, while "Only Propensity" adjustments increase bias, highlighting the importance of equity adjustments for accurate PHR estimates.
We examined the balance between TA and CC population sizes when supplemented by SC in clinical trials with a fixed recruitment size. Using ALLHAT and SPRINT trial data, we maintained a total recruited participant cap of 4000, varying TA and CC sizes with corresponding SC adjustments. Four PHR (ALLHAT)
PHR (SPRINT) 3000 e2000 z i S C S1000 0 3000 e2000 z i S C S1000
0 1.0 1.2 Effec1t.4Size 1.6 1.8 0.6 0.7 1.0
1.1 0.8 0.9 Effect Size scenarios were considered with TA sizes of 3500, 3000, 2500, 2000 and CC sizes of 500, 1000, 1500, 2000, inversely adjusting SC sizes 3000, 2000, 1000, 0. The PHR estimates from these scenarios are shown in Figure 3. Key findings include: (i) The variance of PHR estimates increases with SC size, afecting the stability of treatment efect estimation, and (ii) The PHR estimate can significantly shift with a highly imbalanced ratio of real to synthetic data; especially observed in some scenarios with substantially high SC size. This investigation therefore highlights the importance of carefully balancing the ratio of CC and SC patients in HC to ensure accurate treatment efect estimates and avoiding erroneous conclusions about a trial’s eficacy.
FRESCA ofers a major advancement in equitable HCT methods and serves as a valuable tool for future research. It creates realistic HCT scenarios, using a health recommender system for propensity score matching and equity adjustments to provide more precise and equitable PHR estimates. Our simulations suggest that fewer patients may be needed to achieve results similar to full trials, but further research is required to determine the optimal balance of synthetic and concurrent controls in fixed-size trials. Future work will involve testing FRESCA with more realistic EHR data, with additional protected attributes and exploring the optimal size for CC recruitment during RCT design. Additionally, developing strategies that integrate matching and equity adjustments in a single step could enhance eficiency and reduce variance. These areas present opportunities for further refinement, making FRESCA a significant step forward in hybrid clinical research with potential for ongoing improvement.
This work was partially funded by IBM Research. This manuscript was prepared using SPRINT and ALLHAT study research materials obtained from the NHLBI Biologic Specimen and Data Repository Information Coordinating Center and does not necessarily reflect the opinions or views of SPRINT, ALLHAT or NHLBI. [7] A. Y. Ling, M. E. Montez-Rath, P. Carita, K. J. Chandross, L. Lucats, Z. Meng, B. Sebastien, K. Kapphahn, M. Desai, An overview of current methods for real-world applications to generalize or transport clinical trial findings to target populations of interest, Epidemiology 34 (2023) 627–636. doi:10.1097/EDE.0000000000001633. [8] J. Petkovic, J. Jull, M. Yoganathan, O. Dewidar, S. Baird, J. M. Grimshaw, K. A. Johansson, E. Kristjansson, J. McGowan, D. Moher, et al., Reporting of health equity considerations in cluster and individually randomized trials, Trials 21 (2020) 1–12. doi:10.1186/s13063-020-4223-5. [9] N. Neehal, V. Anand, K. P. Bennett, Framework for research in equitable synthetic control arms, in: AMIA Annual Symposium Proceedings, volume 2023, American Medical Informatics Association, 2023, p. 530. [10] J. T. Wright Jr, et al., A randomized trial of intensive versus standard blood-pressure control, New
England Journal of Medicine 373 (2015) 2103–2116. doi:10.1056/NEJMoa1511939. [11] C. D. Furberg, et al., Major outcomes in high-risk hypertensive patients randomized to angiotensinconverting enzyme inhibitor or calcium channel blocker vs diuretic: The antihypertensive and lipid-lowering treatment to prevent heart attack trial (allhat), JAMA 288 (2002) 2981–2997. doi:10. 1001/jama.288.23.2981. [12] Centers for Disease Control and Prevention, National health and nutrition examination survey (nhanes) data, U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, 2023. URL: https://www.cdc.gov/nchs/nhanes/index.htm, accessed on: October 2023. [13] W. E. Deming, F. F. Stephan, On a least squares adjustment of a sampled frequency table when the expected marginal totals are known, The Annals of Mathematical Statistics 11 (1940) 427–444. doi:10.1214/aoms/1177731829. [14] J. Yuan, J. Liu, R. Zhu, Y. Lu, U. Palm, Design of randomized controlled confirmatory trials using historical control data to augment sample size for concurrent controls, Journal of Biopharmaceutical Statistics 29 (2019) 558–573. doi:10.1080/10543406.2018.1559853. [15] Y. Liu, B. Lu, J. Foster, Y. Zhang, Z. J. Zhong, M.-H. Chen, P. Sun, Matching design for augmenting the control arm of a randomized controlled trial using real-world data, Journal of Biopharmaceutical Statistics 32 (2022) 124–140. doi:10.1080/10543406.2021.2011900. [16] K. Viele, et al., Use of historical control data for assessing treatment efects in clinical trials,
Pharmaceutical statistics 13 (2014) 41–54. doi:10.1002/pst.1589. [17] X. Pang, et al., A bayesian alternative to synthetic control for comparative case studies, Political
Analysis 30 (2022) 269–288. doi:10.1017/pan.2021.22. [18] National Heart, Lung, and Blood Institute, Biologic specimen and data repository information coordinating center, https://biolincc.nhlbi.nih.gov/home/, 2024. Accessed September 22, 2024. [19] D. E. Ho, K. Imai, G. King, E. A. Stuart, Matchit: Nonparametric preprocessing for parametric causal inference, Journal of Statistical Software 42 (2011) 1–28. doi:10.18637/jss.v042.i08. [20] K. Ward, IPFR: List Balancing for Reweighting and Population Synthesis, 2020. URL: https://CRAN.
R-project.org/package=ipfr, r package version 1.0.2.