Interventions to optimise the outputs of national clinical audits to improve the quality of health care: a multi-method study including RCT

Background

National clinical audit programmes aim to improve patient care by reviewing performance against explicit standards and directing action towards areas not meeting those standards. Their impact can be improved by (1) optimising feedback content and format, (2) strengthening audit cycles and (3) embedding randomised trials evaluating different ways of delivering feedback.

Objectives

The objectives were to (1) develop and evaluate the effects of modifications to feedback on recipient responses, (2) identify ways of strengthening feedback cycles for two national audits and (3) explore opportunities, costs and benefits of national audit participation in a programme of trials.

Design

An online fractional factorial screening experiment (objective 1) and qualitative interviews (objectives 2 and 3).

Setting and participants

Participants were clinicians and managers involved in five national clinical audits – the National Comparative Audit of Blood Transfusions, the Paediatric Intensive Care Audit Network, the Myocardial Ischaemia National Audit Project, the Trauma Audit & Research Network and the National Diabetes Audit – (objective 1); and clinicians, members of the public and researchers (objectives 2 and 3).

Interventions

We selected and developed six online feedback modifications through three rounds of user testing. We randomised participants to one of 32 combinations of the following recommended specific actions: comparators reinforcing desired behaviour change; multimodal feedback; minimised extraneous cognitive load for feedback recipients; short, actionable messages followed by optional detail; and incorporating ‘the patient voice’ (objective 1).

Main outcome measures

The outcomes were intended actions, including enactment of audit standards (primary outcome), comprehension, user experience and engagement (objective 1).

Results

For objective 1, the primary analysis included 638 randomised participants, of whom 566 completed the outcome questionnaire. No modification independently increased intended enactment of audit standards. Minimised cognitive load improved comprehension (+0.1; p = 0.014) and plans to bring audit findings to colleagues’ attention (+0.13, on a –3 to +3 scale; p = 0.016). We observed important cumulative synergistic and antagonistic interactions between modifications, participant role and national audit. The analysis in objective 2 included 19 interviews assessing the Trauma Audit Research Network and the National Diabetes Audit. The identified ways of strengthening audit cycles included making performance data easier to understand and guiding action planning. The analysis in objective 3 identified four conditions for effective collaboration from 31 interviews: compromise – recognising capacity and constraints; logistics – enabling data sharing, audit quality and funding; leadership – engaging local stakeholders; and relationships – agreeing shared priorities and needs. The perceived benefits of collaboration outweighed the risks.

Limitations

The online experiment assessed intended enactment as a predictor of actual clinical behaviour. Interviews and surveys were subject to social desirability bias.

Conclusions

National audit impacts may be enhanced by strengthening all aspects of feedback cycles, particularly effective feedback, and considering how different ways of reinforcing feedback act together.

Future work

Embedded randomised trials evaluating different ways of delivering feedback within national clinical audits are acceptable and may offer efficient, evidence-based and cumulative improvements in outcomes.

Trial registration

This trial is registered as ISRCTN41584028.

Funding details

This project was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme and will be published in full in Health and Social Care Delivery Research; Vol. 10, No. 15. See the NIHR Journals Library website for further project information.