The Gnat and the Bull: Do Climate Outlook Forums Make a Difference?

Little has been documented about the benefits and impacts of the recent growth in climate services, despite a growing call to justify their value and stimulate investment. Regional Climate Outlook Forums (RCOFs), an integral part of the public and private enterprise of climate services, have been implemented over the last 20 years with the objectives of producing and disseminating seasonal climate forecasts to inform improved climate risk management and adaptation. In proposing guidance on how to measure the success of RCOFs, we offer three broad evaluative categories that are based on the primary stated goals of the RCOFs: 1) quality of the climate information used and developed at RCOFs; 2) legitimacy of RCOF processes focused on consensus forecasts, broad user engagement, and capacity building; and 3) usability of the climate information produced at RCOFs. Evaluating the quality of information relies largely on quantitative measures and statistical techniques that are standardized and transferrable, but assessing the RCOF processes and perceived usability of RCOF products will necessitate a combination of quantitative and qualitative social science methods that are sensitive to highly variable regional contexts. As RCOFs have taken up different formats and procedures to adapt to diverse institutional and political settings and varied technical and scientific capacities, objective evaluation methods adopted should align with the goals and intent of the evaluation and be performed in a participatory, coproduction manner where producers and users of climate services together design the evaluation metrics and processes. To fully capture the potential benefits of the RCOFs, it may be necessary to adjust or recalibrate the goals of these forums to better fit the evolving landscape of climate services development, needs, and provision.