How well do participants understand the questions asked in the Online Personal Utility Functions (OPUF) approach? A cognitive debrief of the EQ-HWB-S (EQ Health and Wellbeing Short version) valuation

Purpose

Online elicitation of Personal Utility Functions (OPUF) is an innovative approach to valuing health states. OPUF uses a combination of ranking, swing-weighting, levels-rating and anchoring dead tasks. Little is known about how participants interpret and engage with OPUF tasks. This study aimed to address this gap.

Method

Cognitive debrief interviews, which included ‘think-aloud’ and probing questions, were undertaken in the UK to understand how members of the public engage with OPUF when used to value the EQ-HWB-S (EQ Health and Wellbeing Short version). Coding drew upon a Framework approach, with final codes including an assessment of how participants engaged with each of the five OPUF tasks based on whether (1) they completed as expected, (2) minor concerns were identified or (3) major concerns were identified. The presence of major concerns was judged to undermine the validity of responses.

Results

All 27 interviews were identified to have at least minor concern and 18 (67%) were identified as having major concerns. Major concerns were identified in four of the tasks: ranking (in 19% of interviews), swing-weighting (30%), levels-rating (56%), anchoring dead task (48%). Older participants were more likely to have major errors.

Conclusion

Think-aloud and probing interviews with 27 participants completing the OPUF identified multiple concerns, to the extent that their data is unlikely to be a valid reflection of their preferences. The extent of concerns identified here suggests the need for interviewer led data collection within OPUF to ensure data quality.