“Let the community decide”? The vision and reality of soundness-only peer review in open-access mega-journals

Purpose: The aim of this research is to better understand the theory and practice of peer review in open-access mega-journals (OAMJs). Mega-journals typically operate a “soundness only” review policy aiming to evaluate only the rigour of an article, not the novelty or significance of the research or its relevance to a particular community, with these elements being left for “the community to decide” post-publication. Design/methodology/approach: The paper reports the results of interviews with 31 senior publishers and editors representing 16 different organisations, including 10 that publish an OAMJ. Thematic Analysis was carried out on the data and an analytical model developed to explicate their significance. Findings: Findings suggest that in reality criteria beyond technical or scientific soundness can and do influence editorial decisions. Deviations from the original OAMJ model are both publisher-supported (in the form of requirements for an article to be ‘worthy’ of publication) and practice-driven (in the form of some reviewers and editors applying traditional peer review criteria to mega-journal submissions). Also publishers believe post-publication evaluation of novelty, significance, and relevance remains problematic. Originality/value: The study is based on unprecedented access to senior publishers and editors, allowing insight into their strategic and operational priorities. The paper is the first to report in-depth qualitative data relating specifically to soundness-only peer review for OAMJs, shedding new light on the mega-journal phenomenon, and helping inform discussion on its future role in scholarly communication. The paper proposes a new model for understanding the mega-journal approach to quality assurance, and how it is different from traditional peer review.