Davies, C. orcid.org/0000-0001-9347-7905 and Ingram, H. (2025) Sceptics and champions: participant insights on the use of partial randomization to allocate research culture funding. Research Evaluation, 34 (1). rvaf006. ISSN 0958-2029
Abstract
As part of the shift towards a more equitable research culture, funders are reconsidering traditional approaches to peer review. In doing so, they seek to minimize bias towards certain research ideas and researcher profiles, to ensure greater inclusion of disadvantaged groups, to improve review quality, to reduce burden, and to enable more transformative research. This paper presents the rationale and findings from a trial of partially randomized funding allocation (PRA) used to internally distribute Research England’s Enhancing Research Culture fund at the University of Leeds. Of 26 eligible applications, six fell in the upper midfield on quality assessment and were randomized. Of this subset, one received the available funding. Qualitative data from applicants, reviewers, and moderators in the trial suggest modest gains regarding the reduction or perception of bias and efficiency of peer review. There were variable benefits of the feedback that PRA provides. A range of concerns emerged about PRA’s perceived fairness, bluntness, and threat to quality. Based on the behavioural responses we captured, we present strategic and operational recommendations for professionals working in research evaluation who are interested in adopting PRA as a way of improving research culture. For example, we highlight the importance of (1) sharing data on the role of chance in traditional peer review, and (2) considering the benefits and risks of PRA at both group and individual applicant levels.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © The Author(s) 2025. This is an open access article under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited. |
Keywords: | grant peer review; research funding allocation; partial randomization; modified lottery; research culture |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Arts, Humanities and Cultures (Leeds) > School of Languages Cultures & Societies (Leeds) > Linguistics & Phonetics (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 17 Jan 2025 12:00 |
Last Modified: | 18 Feb 2025 15:40 |
Status: | Published |
Publisher: | Oxford University Press |
Identification Number: | 10.1093/reseval/rvaf006 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:221927 |