Methodological challenges identified for the evaluation of clinical effectiveness in the context of accelerated regulatory approval:an overview

BACKGROUND: Regulatory authorities are approving innovative therapies with limited evidence. Whilst this level of data is sufficient for the regulator to establish an acceptable risk-benefit balance, it is problematic for downstream health technology assessment, where assessment of cost-effectiveness requires reliable estimates of effectiveness relative to existing clinical practice. Some key issues associated with a limited evidence base include using data, from non-randomised studies, from small single-arm trials, or from single-centre trials; and using surrogate endpoints. METHODS: We examined these methodological challenges through a pragmatic review of the available literature. RESULTS: Methods to adjust non-randomised studies for confounding are imperfect. The relative treatment effect generated from single-arm trials is uncertain and may be optimistic. Single-centre trial results may not be generalisable. Surrogate endpoints, on average, overestimate treatment effects. Current methods for analysing such data are limited and effectiveness claims based on these sub-optimal forms of evidence are likely to be subject to significant uncertainty. CONCLUSIONS: Assessments of cost-effectiveness, based on the modelling of such data, are likely to be subject to considerable uncertainty. This uncertainty must not be underestimated by decision makers: methods for its quantification are required and schemes to protect payers from the cost of uncertainty should be implemented.