Robust Comparative Effectiveness Without Head-to-Head Trials: Leveraging Primary Clinical Trial Data for Indirect Comparisons

Presenter: James Signorovitch, Analysis Group Inc.

Abstract

Rationale: Comparative effectiveness research often requires the comparison of alternative therapies without the benefit of a head-to-head randomized trial. For example, we may wish to compare drug A vs. drug B when randomized trials have only compared each to placebo. Existing approaches to this problem, such as meta-regression and mixed treatment comparison meta-analysis, require data from many trials and strong assumptions about the consistency of treatment effects across trials. In particular, these methods can be biased by differences in patient populations across trials. Furthermore, health care decision makers must evaluate the comparative effectiveness of new treatments as they enter the market, when few clinical trials have been conducted. In this important setting, with only one or two trials for each treatment, the limitations of traditional methods for indirect comparison become prohibitive. To compare treatments in this scenario, we propose leveraging patient-level clinical trial data when it is available for one of the alternative treatments of interest.

Objective: The goal of this presentation is to describe and evaluate a novel statistical method for making robust indirect comparisons between therapies when patient-level clinical trial data are available for only one treatment of interest.

Methods: We consider an indirect comparison of drug A vs. drug B when patient-level data are available from a randomized trial of drug A vs. placebo but only published summary results are available from a trial of drug B vs. placebo. Our general approach is to re-weight individual patients from the drug A trial such that their baseline characteristics and placebo arm outcomes match those reported for the drug B trial. Under such a re-weighting, the mean outcome on the drug A arm provides a projection of drug A’s effects into the drug B trial population. Since there are many such re-weightings, a robust comparison to drug B is provided by identifying the range of all plausible projections for drug A. If the published mean outcome for drug B falls within the projected range for drug A, then no differences in outcomes can be conclusively identified. However if the published mean outcome for drug B falls outside the projected range for drug A, we have strong evidence of a true difference between drugs and can identify the superior treatment. Statistical comparisons between projected and observed outcomes are obtained by bootstrapping. To evaluate the proposed methodology, simulations were conducted under a range of scenarios, varying the sample size, the degree of baseline differences between trials and the strength of associations between baseline characteristics and outcomes.

Results: Analyses of 1,000 replicates of each simulation scenario illustrate that the range of projected drug A outcomes reliably covers the true mean outcome over a wide range of scenarios. Simulations under extreme scenarios provide criteria for judging whether the proposed methods are appropriate for a given application.

Conclusion: In the absence of a head-to-head randomized trial comparing two treatments of interest, robust comparisons of clinical effectiveness can be made by leveraging patient-level data for one of the treatments.