Looking beyond the forest: using harvest plots, gap analysis and expert consultations to assess effectiveness, engage stakeholders and inform policy

We describe a combination of methods for assessing the effectiveness of complex interventions, especially where substantial heterogeneity with regard to the population, intervention, comparison, outcomes and study design of interest is expected. We applied these methods in a recent systematic review of the effectiveness of reinforced home-based palliative care (rHBPC) interventions, which included home-based care with an additional and explicit component of lay caregiver support. We first summarized the identified evidence, deemed inappropriate for statistical pooling, graphically by creating harvest plots. Though very useful as a tool for summary and presentation of overall effectiveness, such graphical summary approaches may obscure relevant differences between studies. Thus, we then employed a gap analysis and conducted expert consultations to look beyond the aggregate level at how the identified evidence of effectiveness may be explained. The goal of these supplemental methods was to step outside of the conventional systematic review and explore this heterogeneity from a broader perspective, based on the experience of palliative care researchers and practitioners. The gap analysis and expert consultations provided valuable input into possible underlying explanations in the evidence, which could be helpful in the further adaptation and testing of existing rHBPC interventions or the development and evaluation of new ones. We feel that such a combination of methods could prove accessible, understandable and useful in informing decisions, and could thus help increase the relevance of systematic reviews to the decision-making process.