Sample size calculations for pilot cluster-randomised controlled trial: a review 2010–2020

Background

There is no recommended guidance on appropriate sample sizes for pilot cluster-randomised controlled trials (cRCTs). Pilot trials should not aim to demonstrate efficacy, and achieving power should not be used to justify the sample size. However, the CONSORT extension for pilot trials states that some justification for their sample size should be given.

We conducted a review to understand the choices and justifications of pilot cRCT sample sizes and their trends over time, and to explore apparent changes following the publication of CONSORT extensions for cluster trials and for pilot and feasibility trials.

Methods

We searched PubMed and Web of Science for pilot cRCTs. The search took place on 01/10/2020 and was restricted to papers published on or after 01/01/2010. Identification of papers was based around a search for the terms ‘pilot’ or ‘feasibility’ in the title and abstract/topic. The primary interest in the review was in the planned sample size in terms of clusters per-arm. We also examined participants per-arm and enrolled sample size. Analyses were descriptive or displayed graphically.

Results

Our search returned 3090 records. After removing exact duplicates, aggregating records into unique studies and excluding ineligible studies, we identified 170 pilot or feasibility cRCTs. The median sample size was four clusters per arm. Stratification showed this to be fairly consistent, regardless of the justification given, whether a formal analysis was planned, whether the intention was to estimate the Intra-Cluster Correlation, cluster type, general medical area, funding type, and over time.

Conclusion

Average sample sizes for cRCTs have remained strikingly constant over the period 2010–2020 and across several key features of studies; they do not appear to be meaningfully impacted by the stated study aims or sample size justifications. This is despite the fact that the reported main aims for pilot cRCTs, and justifications for their sample sizes, do appear to have changed during this time.

Given that aims and justifications appear to have changed, but typical sample sizes have not, it is possible that some researchers choose sample size for pilot cRCTs primarily out of convenience or practical reasons, whilst stating other justifications.