Bramley, P. orcid.org/0000-0001-6209-6640, Hulman, J. and Wanstall, H. (2024) Risk of bias and problematic trials: characterising the research integrity of trials submitted to Anaesthesia. Anaesthesia, 79 (12). pp. 1309-1316. ISSN 0003-2409
Abstract
Background
There is some evidence for systematic biases and failures of research integrity in the anaesthesia literature. However, the features of problematic trials and effect of editorial selection on these issues have not been well quantified.
Methods
We analysed 209 randomised controlled trials submitted to Anaesthesia between 8 March 2019 and 31 March 2020. We evaluated the submitted manuscript, registry data and the results of investigations into the integrity of the trial undertaken at the time of submission. Trials were labelled ‘concerning’ if failures of research integrity were found, and ‘problematic’ if identified issues would have warranted retraction if they had been found after publication. We investigated how ‘problematic’ trials were detected, the distribution of p values and the risk of outcome reporting bias and p-hacking. We also investigated whether there were any factors that differed in problematic trials.
Results
We found that false data was the most common reason for a trial to be labelled as ‘concerning’, which occurred in 51/62 (82%) cases. We also found that while 195/209 (93%) trials were preregistered, we found adequate registration for only 166/209 (79%) primary outcomes, 100/209 (48%) secondary outcomes and 11/209 (5%) analysis plans. We also found evidence for a step decrease in the frequency of p values > 0.05 compared with p values < 0.05. ‘Problematic’ trials were all single-centre and appeared to have fewer authors (incident risk ratio (95%CI) 0.8 (0.7–0.9)), but could not otherwise be distinguished reliably from other trials.
Conclusions
Identification of ‘problematic’ trials is frequently dependent on individual patient data, which is often unavailable after publication. Additionally, there is evidence of a risk of outcome reporting bias and p-hacking in submitted trials. Implementation of alternative research and editorial practices could reduce the risk of bias and make identification of problematic trials easier.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2024 The Author(s). Anaesthesia published by John Wiley & Sons Ltd on behalf of Association of Anaesthetists. This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited (http://creativecommons.org/licenses/by/4.0/) |
Keywords: | publication bias; questionable research practices; randomised controlled trials; research integrity |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Medicine, Dentistry and Health (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 21 Aug 2024 09:03 |
Last Modified: | 25 Feb 2025 10:00 |
Status: | Published |
Publisher: | Wiley |
Refereed: | Yes |
Identification Number: | 10.1111/anae.16411 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:216255 |