Crocker, T orcid.org/0000-0001-7450-3143, Lam, N orcid.org/0000-0001-8591-444X, Jordão, M orcid.org/0000-0003-2108-2677 et al. (6 more authors) (2023) Risk-of-bias assessment using RoB2 was useful but challenging and resource-intensive: observations from a systematic review. Journal of Clinical Epidemiology, 161. pp. 39-45. ISSN 0895-4356
Abstract
Objective
To report our experience using Version 2 of the Cochrane risk-of-bias tool for randomised trials (RoB2).
Study design and setting
Two reviewers independently applied RoB2 to results of interest in a large systematic review of complex interventions and reached consensus. We recorded time taken, and noted and discussed our difficulties using the tool, and the resolutions we adopted. We explored time taken with regression analysis and summarised our experience of implementing the tool.
Results
We assessed risk of bias in 860 results of interest in 113 studies. Staff resource averaged 358 minutes per study (SD 183). Number of results (β=22) and reports (β=14) per study and experience of the team (β=-6) significantly affected assessment time. To implement the tool consistently we developed cut-points for missingness and considerations of balance regarding missingness, assumed some concerns with intervention deviations unless otherwise prevented or investigated, some concerns with measurements from unblinded self-reporting participants, and judged low risk of selection for certain dichotomous outcomes despite the absence of an analysis plan.
Conclusion
The RoB2 tool and guidance are useful but resource-intensive and challenging to implement. Critical appraisal tools and reporting guidelines should detail risk-of-bias implementation. Improved guidance focussing on implementation could assist reviewers.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2023, Elsevier. This is an author produced version of an article published in Journal of Clinical Epidemiology. Uploaded in accordance with the publisher's self-archiving policy. This is an open access article under the CC BY license (http://creativeco mmons.org/licenses/by/4.0/). |
Keywords: | RoB2; risk of bias; process duration; research methods; certainty assessment; systematic reviews |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Medicine and Health (Leeds) > School of Medicine (Leeds) > Leeds Institute of Health Sciences (Leeds) > Academic Unit of Elderly Care and Rehabilitation (Leeds) |
Funding Information: | Funder Grant number NIHR National Inst Health Research NIHR128862 |
Depositing User: | Symplectic Publications |
Date Deposited: | 03 Jul 2023 15:23 |
Last Modified: | 23 Jan 2025 12:14 |
Published Version: | https://www.jclinepi.com/article/S0895-4356(23)001... |
Status: | Published |
Publisher: | Elsevier |
Identification Number: | 10.1016/j.jclinepi.2023.06.015 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:200718 |
Downloads
Filename: ROB2 experience no fields revised final no marks (1).pdf
Licence: CC-BY-NC-ND 4.0
Filename: 1-s2.0-S0895435623001634-main.pdf
Licence: CC-BY 4.0