Research and policy impact of trials published by the UK National Institute of Health Research (2006-2015)

Objectives: Health technology assessment aims to inform and support healthcare decision making, and trials are part of that process. The purpose of this study was to measure the impact of a sample of trials in a meaningful but robust fashion.

Methods: All randomized controlled trials funded and published by the UK National Institute of Health Research in the Health Technology Assessment journal series and other peer-reviewed journals were identified for 2006 to 2015. Citation analysis was performed for all trials, and quantitative content analysis was undertaken on a purposive sample to determine whether impact could be categorized as “instrumental” (ie, having a clear influence on key research and policy publications).

Results: The search identified 133 relevant trials. The citation rate per trial was 102.97. Of the 133 trials, 129 (98%) were cited in 1 or more systematic reviews or meta-analyses (mean per trial = 7.18, range = 0-44). Where they were cited, the trials were used in some form of synthesis 63% of the time. Ninety-one of the 133 (68%) trials were found to be cited in 1 or more guidance or policy document (mean per trial = 2.75, range = 0-26) and had an instrumental influence 41% of the time. The publication of these trials’ results in journals other than the Health Technology Assessment journal appears to enhance the discoverability of the trial data. Altmetric.com proved to be very useful in identifying unique policy and guidance documents.

Conclusion: These trials have impressive citation rates, and a sizeable proportion are certainly being used in key publications in a genuinely instrumental manner.