Background
The COVID-19 pandemic demanded evidence synthesis approaches that could keep pace with rapidly evolving science and urgent policy needs. Rapid reviews and living evidence syntheses (LES) emerged as critical methodological responses, yet limited guidance exists for teams undertaking such work, particularly regarding the operational challenges of sustained production, multi-team coordination, and the transition from rapid reviews to living formats under emergency conditions.
Objectives
This paper aims to document and appraise methodological decisions made across each component of the synthesis process and their rationale; identify the contextual factors, coordination mechanisms, and infrastructure that enabled or constrained synthesis implementation under emergency conditions; and derive practical recommendations for evidence synthesis teams, funders and commissioners, and methodologists.
Methods
We present a structured reflective process evaluation of our experience producing three rapid LESs on COVID-19 vaccine effectiveness as part of COVID-END in Canada, addressing vaccine effectiveness in adults (short-term and long-term) and children/adolescents, collectively producing 79 versions over approximately two years. Drawing on a Context–Mechanism–Outcome (CMO) lens, we examine eight core methodological processes, each time highlighting: the methodological decision and its rationale; the challenges encountered; and the lessons learned.
Findings
Key insights include: decision-maker engagement played a central role in methodological decisions, directly determining scope boundaries, update frequency, and presentation formats; the multi-team coordination structure was essential to quality under streamlined methods, with cross-team verification partially offsetting the validity risks of single-reviewer screening; choosing between narrative synthesis (compared to meta-analytic approaches) in two of three products reflected capacity constraints during emergency production, a distinction with implications for how such adaptations are evaluated against existing guidance; automation produced considerable efficiency gains but required added development capacity and resources, demonstrating that teams should plan and budget for automation from the outset rather than absorbing its demands reactively during production; and coverage was acceptable for decision timelines. External cross-checks identified five studies not captured by our searches across the entire project, of which only two were ultimately eligible for synthesis. Emergency production meant that key process records, including change logs, per-cycle search records, and pilot screening agreement data, were not systematically captured. Future teams should plan to assemble a minimal transparency package from the outset. A minimum viable LES framework is proposed comprising minimum infrastructure, minimum methods, and optional enhancers.
Conclusions
The successful implementation of rapid LES requires pre-existing or rapidly mobilisable infrastructure, sustained and explicitly resourced decision-maker engagement, transparent documentation of methodological adaptations across update cycles, effective coordination in multi-team contexts, and realistic sustainability planning. These lessons, derived from a specific emergency context, offer transferable guidance for evidence synthesis teams preparing for future health emergencies and for standards bodies considering how existing guidance accommodates emergency and living synthesis conditions.
CORE (COnnecting REpositories)
CORE (COnnecting REpositories)