Assessing fidelity of a community based psychosocial intervention for people with mild dementia within a large randomised controlled trial

Background

Understanding intervention delivery as intended, particularly in complex interventions, should be underpinned by good quality fidelity assessment. We present the findings from a fidelity assessment embedded as part of a trial of a complex community-based psychosocial intervention, Journeying through Dementia (JtD). The intervention was designed to equip individuals with the knowledge and skills to successfully self-manage, maintain independence, and live well with dementia and involves both group and individual sessions. The methodological challenges of developing a conceptual framework for fidelity assessment and creating and applying purposely designed measures derived from this framework are discussed to inform future studies.

Methods

A conceptual fidelity framework was created out of core components of the intervention (including the intervention manual and training for delivery), associated trial protocols and pre-defined fidelity standards and criteria against which intervention delivery and receipt could be measured. Fidelity data collection tools were designed and piloted for reliability and usability. Data collection in four selected sites (fidelity sites) was via non-participatory observations of the group aspect of the intervention, attendance registers and interventionist (facilitator and supervisor) self-report.

Results

Interventionists from all four fidelity sites attended intervention training. The majority of group participants at the four sites (71%) received the therapeutic dose of 10 out of 16 sessions. Weekly group meeting attendance (including at ‘out of venue’ sessions) was excellent at 80%. Additionally, all but one individual session was attended by the participants who completed the intervention. It proved feasible to create tools derived from the fidelity framework to assess in-venue group aspects of this complex intervention. Results of fidelity assessment of the observed groups were good with substantial inter-rater reliability between researchers KAPPA 0.68 95% CI (0.58–0.78). Self-report by interventionists concurred with researcher assessments.

Conclusions

There was good fidelity to training and delivery of the group aspect of the intervention at four sites. However, the methodological challenges of assessing all aspects of this complex intervention could not be overcome due to practicalities, assessment methods and ethical considerations. Questions remain regarding how we can assess fidelity in community-based complex interventions without impacting upon intervention or trial delivery.

Trial registration

ISRCTN17993825.