Health state valuation and mode of administration: head to head comparison of online and CAPI

Objectives: Health state valuation exercises can be conducted online but the quality of data generated is unclear. We report a study investigating whether responses to binary choice health state valuation questions differ by administration mode (online vs. face-to-face).

Methods: Identical surveys including demographic, self reported health and seven types of binary choice valuation questions were administered in online and Computer Assisted Personal Interview (CAPI) settings. Samples were recruited following procedures employed in typical online or CAPI studies. Analysis included descriptive comparisons of the distribution of responses across the binary options, and probit regression to explain the propensity to choose one option across modes of administration, controlling for background characteristics.

Results: Overall, 422 (221 online; 201 CAPI) respondents completed a survey. There were no overall age or gender differences. Online respondents were educated to a higher level than the CAPI sample and general population, and employment status differed. CAPI respondents reported significantly better general health, and health/life satisfaction. CAPI took significantly longer to complete. There was no effect of mode of administration on responses to the valuation questions, and this was replicated when demographic differences were controlled.

Conclusions: The findings suggest that both modes may be equally valid for health state valuation studies using binary choice methods (e.g. Discrete Choice Experiments).

There are some differences between the observable characteristics of the samples, and the groups may differ further in terms of unobservable characteristics. When designing health state valuation studies, the advantages and disadvantages of both approaches must be considered.