Background
Blood transfusion is a common but costly treatment. Repeated national audits in the UK suggest that up to one-fifth of transfusions are unnecessary when judged against recommendations for good clinical practice. Audit and feedback seeks to improve patient care and outcomes by comparing clinical care against explicit standards. It is widely used internationally in quality improvement. Audit and feedback generally has modest but variable effects on patient care. A considerable scope exists to improve the impact that audit and feedback has, particularly through head-to-head trials comparing different ways of delivering feedback.
Objectives
The AFFINITIE (Development & Evaluation of Audit and Feedback INterventions to Increase evidence-based Transfusion practIcE) programme aimed to design and evaluate enhanced feedback interventions, within a national blood transfusion audit programme, to promote evidence-based guidance and reduce the unnecessary use of blood components. We developed, piloted and refined two feedback interventions, ‘enhanced content’ and ‘enhanced follow-on’ (workstream 1), evaluated the effectiveness and cost-effectiveness of the two feedback interventions compared with standard feedback practice (workstream 2), examined intervention fidelity and contextual influences (workstream 3) and developed general implementation recommendations and tools for other audit and feedback programmes (workstream 4).
Design
Interviews, observations and documentary analysis in four purposively sampled hospitals explored contemporary practice and opportunities for strengthening feedback. We developed two interventions: ‘enhanced content’, to improve the clarity and utility of feedback reports, and ‘enhanced follow-on’, to help hospital staff with action-planning (workstream 1). We conducted two linked 2 × 2 factorial cross-sectional cluster-randomised trials within transfusion audits for major surgery and haematological oncology, respectively (workstream 2). We randomised hospital clusters (the organisational level at which hospital transfusion teams operate) to enhanced or standard content or enhanced or standard follow-on. Outcome assessment was masked to assignment. Decision-analytic modelling evaluated the costs, benefits and cost-effectiveness of the feedback interventions in both trials from the perspective of the NHS. A parallel process evaluation used semistructured interviews, documentary analyses and web analytics to assess the fidelity of delivery, receipt and enactment and to identify contextual influences (workstream 3). We explored ways of improving the impact of national audits with their representatives (workstream 4).
Setting and participants
All NHS hospital trusts and health boards participating in the National Comparative Audit of Blood Transfusions were invited to take part. Among 189 hospital trusts and health boards screened, 152 hospital clusters participated in the surgical audit. Among 187 hospital trusts and health boards screened, 141 hospital clusters participated in the haematology audit.
Interventions
‘Enhanced content’ aimed to ensure that the content and format of feedback reports were consistent with behaviour change theory and evidence. ‘Enhanced follow-on’ comprised a web-based toolkit and telephone support to facilitate local dissemination, planning and response to feedback.
Main outcome measures
Proportions of acceptable transfusions, based on existing evidence and guidance and algorithmically derived from national audit data.
Data sources
Trial primary outcomes were derived from manually collected, patient-level audit data. Secondary outcomes included routinely collected data for blood transfusion.
Results
With regard to the transfusions in the major surgery audit, 135 (89%) hospital clusters participated from 152 invited. We randomised 69 and 66 clusters to enhanced and standard content, respectively, and 68 and 67 clusters to enhanced and standard follow-on, respectively. We analysed a total of 2222 patient outcomes at 12 months in 54 and 58 (enhanced and standard content, respectively) and 54 and 58 (enhanced and standard follow-on, respectively) hospital clusters. With regard to the haematology audit, 134 hospital clusters (95%) participated from 141 invited. We randomised 66 and 68 clusters to enhanced and standard content, respectively, and 67 clusters to both enhanced and standard follow-on. We analysed a total of 3859 patient outcomes at 12 months in 61 and 61 (enhanced and standard content, respectively) and 63 and 59 (enhanced and standard follow-on) hospital clusters. We found no effect of either of the enhanced feedback interventions in either trial across all outcomes. Incremental enhanced intervention costs ranged from £18 to £248 per site. The enhanced feedback interventions were dominated by the standard intervention in cost-effectiveness analyses. The interventions were delivered as designed and intended, but subsequent local engagement was low. Although the enhancements were generally acceptable, doubts about the credibility of the blood transfusion audits undermined the case for change.
Limitations
Limitations included the number of participating clusters; loss to follow-up of trial clusters, reducing statistical power and validity; incomplete audit and cost data contributing to outcome measures; participant self-selection; reporting; missing data related to additional staff activity generated in response to receiving feedback; and recall biases in the process evaluation interviews.
Conclusions
The enhanced feedback interventions were acceptable to recipients but were more costly and no more effective than standard feedback in reducing unnecessary use of blood components, and, therefore, should not be recommended on economic grounds.
Future work
We have demonstrated the feasibility of embedding ambitious large-scale rigorous research within national audit programmes. Further head-to-head comparisons of different feedback interventions are needed in these programmes to identify cost-effective ways of increasing the impact of the interventions.
CORE (COnnecting REpositories)
CORE (COnnecting REpositories)