Methods for Indirect Comparisons in Outcomes Research

Comparative effectiveness research (CER) has the potential to improve health outcomes while controlling costs. But a growing focus on CER has created practical challenges for pharmaceutical, biotechnology, and medical device makers aiming to demonstrate the value of their products.

How can growing customer expectations for comparative evidence be satisfied for new products with limited clinical trial data? And how can CER, which has traditionally focused on the average effects of health interventions across broad populations, inform care for individual patients?

CER aims to inform the choices of health care decision makers with the best possible information about potential treatments. With innovation increasing the number and diversity of effective therapies, the need for comparative evidence has never been greater. Yet the availability of reliable comparative evidence is often severely limited.

In the hierarchy of comparative evidence, head-to-head randomized trials are considered the gold standard. However, randomized trials are rarely available for all of the treatment comparisons that physicians and payers need to make decisions, especially as new treatments enter a market.

While studies comparing real-world treatment outcomes can fill this need over time, such studies, by their very nature, require long-duration real world use and therefore are not immediately possible for new treatments. This systemic lag before CER evidence becomes available means that many of the critical post-launch decisions for a product, such as price, formulary position, and physician utilization, must be made without key information on the product’s strengths and weaknesses.

An additional challenge for pharmaceutical firms has been identifying high-value patient populations for new treatments. There is growing recognition that developing blockbusters for large markets like hypertension or diabetes is becoming increasingly challenging as these markets become saturated with effective generics. Furthermore, payers often implement cost-containment measures to limit drug utilization, frequently to a smaller market than is supported by a drug’s label. In this environment, identifying and supporting a possibly smaller market for a drug, but one in which its value proposition is enhanced for individual patients, becomes an attractive strategy.

An Analysis Group team has been collaborating with the Harvard School of Public Health’s Biostatistics Department to apply and develop methods for filling gaps in CER. This work has been presented at a number of conferences and published in clinical and health economics journals, including a 2011 symposium on comparative effectiveness research methods sponsored by the Agency for Healthcare Research and Quality (AHRQ); the last three annual meetings of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR); and "Comparative Effectiveness Research without Head-to-Head Trials," PharmacoEconomics, Vol. 28, No. 10, 2010.

Getting More CER Value from Existing Data

Over the past several years, comparisons of treatment outcomes across separate randomized trials have become an important tool for CER. Where there are no head-to-head trials of two treatments, these so-called indirect comparisons across separate trials may provide the only evidence for decision makers. However, an indirect comparison is only as reliable as the trials are similar. In particular, crosstrial differences in patient populations, such as greater disease severity in one trial versus another, can lead to incorrect comparisons of treatment outcomes. The differences between trial populations can limit the reliability of indirect comparisons and, therefore, their value for decision makers.

To address this limitation, we have developed a new method for matching patient populations across separate trials, in which we leverage the complete individual patient data from one trial (see Figure 1). With these complete data, we can then adjust the patient population on key dimensions (average age, disease severity, or prior treatment profile, for example) to match that of a published trial for which we have access only to summary results. After matching, the outcomes of different treatments can reliably be compared across the balanced trial populations (see Figure 2).

Figure 1. Reweighting Patients Based on Gender Differences

Using robust statistical methodology, patients in Trial B can be reweighted to match the baseline characteristics of Trial A.

Figure 2. Comparison of Outcomes between Matched Trials

Since publishing this methodology in 2010, we have used it to provide timely CER for new therapies in oncology, diabetes, and other treatment areas. Individual patient data from clinical trials have been a largely untapped resource for CER. With the advent of new research methodologies, we expect its use to grow in coming years.

Moving from Treating Diseases to Treating Individuals

At each stage of a therapeutic product’s life cycle, the preferred strategy is often to serve the largest possible patient population. However, in today’s competitive and cost-conscious health care environment, a product’s profile may not support this goal, due to limited average efficacy versus other effective therapies, or concerns about average cost-effectiveness or benefit-risk profiles. In these cases, it can be helpful to identify subpopulations of patients who receive the greatest value from a product.

To that end, working closely with Harvard biostatistician Professor Lee-Jen Wei, Analysis Group has applied new methodologies to identify the optimal treatment choice for each patient based on a large number of individual characteristics in addition to disease status. This approach has proved useful for identifying high-value subpopulations in early-phase trials that can be confirmed in later-phase trials.

Figure 3. The Tradeoff between Drug Efficacy and Subpopulation Size 

The ideal subpopulation is the largest subpopulation with acceptable efficacy (or cost-effectiveness, or benefit-risk, etc.), and will lie on the efficiency frontier. Inefficient subpopulations limit the population size more than is needed and should be avoided.

Currently, when the cost-effectiveness of a drug is limited for the average patient, payers may restrict access or reimbursement to a targeted subpopulation. However, the subpopulation is usually not chosen based on a systematic evaluation of the costs and benefits. The individualized medicine approach developed by Professor Wei and Analysis Group addresses this shortcoming and has been used to identify the largest subpopulation in which a drug’s efficacy and cost effectiveness are acceptable to clinicians and payers (see Figure 3). For later-stage products, this approach has also been used to identify subpopulations that have a particularly poor benefit-risk profile with a traditional generic first-line agent, and who should have less-restricted access to newer drugs as a first-line therapy. ■