The comprehensibility continuum: a novel method for analysing comprehensibility of patient reported outcome measures

Purpose

Evidence of comprehensibility is frequently required during the development of patient reported outcome measures (PROMs); the respondent’s interpretation of PROM items needs to align with intended meanings. Cognitive interviews are recommended for investigating PROM comprehensibility, yet guidance for analysis is lacking. Consequently, the quality and trustworthiness of cognitive interview data and analysis is threatened, as there is no clear procedure detailing how analysts can systematically, and consistently, identify evidence that respondent interpretations align/misalign with intended meanings.

Methods

This paper presents a novel, structured approach to comprehensibility analysis - the ‘Comprehensibility Continuum’ – that builds upon existing cognitive interview guidance.

Results

The Comprehensibility Continuum comprises a structured rating scale to code depth of alignment between intended item meaning and respondent interpretation and consists of five main stages: before cognitive interviews are conducted, researchers must (1) Define intended meanings of PROM items; and (2) Determine comprehensibility thresholds for both participant- and item-level. After conducting interviews, they (3) Prepare data by transcribing interviews ‘intelligent’ verbatim; (4) Code transcripts using the Comprehensibility Continuum scale in iterative sets, assigning an overall code for each item at participant-level; and (5) Compare participant-level codes across all participants to determine overall item comprehensibility, such that decisions can be made to retain, modify, or remove items.

Conclusion

Quality in qualitative data analysis is achieved through rigorous methods that are clearly described and justified. Given insufficiency in guidelines, cognitive interviewers must reflect on how best to demonstrate PROM comprehensibility systematically and consistently from interview data, and the Comprehensibility Continuum method offers a potential solution.