Recommendations on multiple-testing adjustment in multi-arm trials with correlated hypotheses

Multi-arm clinical trials assessing multiple experimental treatments against a shared control group offer efficiency advantages over independent trials through assessing an increased number of hypotheses in a single study. Published opinion differs on the requirement for multiple testing adjustment to control the chance of at least one false-positive outcome within the family of hypotheses, known as the familywise type-I error rate (FWER). In this talk we explore the reasons why opinions differ, seeking to qualify the effect on error rates of different trial contexts and, in particular, quantitatively explore the impact of the correlation between the hypotheses in a family due to sharing control data. Building on previous work1, we demonstrate that this correlation in fact leads to a reduction in the FWER, although the probability of more than one false positive outcome across the hypotheses is increased. Standard FWER adjustment methods do not control for the increased error across more than one outcome, therefore FWER adjustment is not recommended due to sharing control data. However, if, in the context of the trial, multiple positive outcomes contribute to a single recommendation and could advance an experimental therapy, the increased chance of concluding superiority due to testing more hypotheses would be unacceptable and should be adjusted for. A stringent critical value adjustment is proposed to maintain equivalent evidence of superiority in two correlated hypotheses to that obtained within independent trials. For competing experimental therapies, the correlation between hypotheses can be advantageous as it eliminates bias due to the experimental therapies being compared to different control populations. The talk concludes with a summary of recommendations for multiple testing adjustments in multi-arm trials.