Test-retest reliability of the Online Elicitation of Personal Utility Functions (OPUF) approach for valuing the EQ-HWB-S

Introduction

The EQ Health and Wellbeing Short (EQ-HWB-S) is a new 9-item instrument designed to generate utility values. However, its length makes traditional preference elicitation challenging. The Online elicitation of Personal Utility Functions (OPUF) approach has been tested as a potential solution. This study aimed to assess the test-retest reliability of OPUF for valuing the EQ-HWB-S.

Methods

The OPUF survey was administered twice, two weeks apart, to 220 German participants, including 73 from the general population and 147 patients with diabetes or rheumatic disease. Test-retest reliability was evaluated at individual and aggregate levels, examining dimension rankings, swing weights, level weights, and anchoring factors. Continuous data were analysed using the intraclass correlation coefficient (ICC), and ranking data were compared using Spearman’s correlation coefficient. Individual and aggregate level utility decrements were assessed using ICC and t-tests.

Results

Approximately 36% of participants had significantly correlated dimension ranks, with 42% selecting the same top-ranked dimension. Poor agreement was shown in 70% of ICC values for individual dimension swing weights. For intermediate level weights, ICC values showed poor agreement in 70% and moderate agreement in 30% of responses. The kappa for individual pairwise comparison tasks was 0.64 (95% CI: 0.54–0.75) showing moderate agreement; however, the ICC for individual-level anchoring factors was 0.12 (p < 0.05), indicating poor agreement. Aggregate utility decrements across dimensions were similar.

Conclusion

The OPUF approach generates reliable aggregate value sets for the EQ-HWB-S, but further exploration is needed to understand and address the reasons behind inconsistencies at the individual level.