Development of an inflammatory bowel disease (IBD) Patient‐Reported Experience Measure (PREM): A patient‐led consensus work and ‘think aloud’ study for a quality improvement programme

Abstract Background Patient‐Reported Experience Measures (PREMs) are key in improving healthcare quality, but no PREM exists for inflammatory bowel disease (IBD). This study aimed to co‐produce a PREM with IBD service users for IBD service evaluation and quality improvement programme. Methods A pool of 75 items was drawn from published survey instruments covering interactions with services and aspects of living with IBD. In Stage 1, during two workshops, eight expert service users reduced candidate items through a ranked‐choice voting exercise and suggested further items. During Stage 2, 18 previously uninvolved people with IBD assessed the face and content validity of the candidate items in ‘Think Aloud’ interviews. During two final workshops (Stage 3), the expert service users removed, modified and added items based on the interview findings to produce a final version of the PREM. Results Stage 1 generated a draft working PREM mapped to the following four domains: Patient‐Centred Care; Quality; Accessibility; Communication and Involvement. The PREM included a set of nine items created by the expert group which shifted the emphasis from ‘self‐management’ to ‘living with IBD’. Stage 2 interviews showed that comprehension of the PREM was very good, although there were concerns about the wording, IBD‐relevance and ambiguity of some items. During the final two workshops in Stage 3, the expert service users removed 7 items, modified 15 items and added seven new ones based on the interview findings, resulting in a 38‐item PREM. Conclusions This study demonstrates how extensive service user involvement can inform PREM development. Patient or Public Contribution Patients were involved as active members of the research team and as research participants to co‐produce and validate a PREM for IBD services. In Stage 1, eight expert service users (‘the expert group’) reduced candidate items for the PREM through a voting exercise and suggested new items. During Stage 2, 18 previously uninvolved people with IBD (the ‘think aloud’ participants) assessed the validity of the candidate items in ‘Think Aloud’ interviews as research participants. In Stage 3, the expert group removed, changed and added items based on the interview findings to produce a final version of the 38‐item PREM. This study shows how service user involvement can meaningfully inform PREM development.

geneous needs which are often unmet by healthcare services. The views of healthcare professionals and patients differ concerning care priorities and quality. 2 In 2021, IBD UK published a UK-wide survey of 10,222 people with IBD, in which 28% rated their quality of care as only fair or poor. 3 The report identified four areas for change: improvements in diagnosis and information provision; personalized care and support for self-management; faster access to specialist advice and treatment and effective multidisciplinary team working.
Many IBD quality improvement initiatives take it for granted that organizations must learn from patients. 4 Self-report survey instruments are increasingly used as quality indicators, 5 including in IBD, 6 but not all measures are considered useful or effective. 7 Patient satisfaction measures, which capture whether a patient received care that met their expectations are biased by previous experiences. 8 Patient expectations are influenced by health status, frequency of service interaction and level of dependency on healthcare providers. 7,9 Satisfaction measures lack sensitivity, fail to distinguish between good and bad care and often overrate satisfaction due to gratitude bias. 10 Simply put, high self-reported satisfaction may not correlate with a positive healthcare experience. 9 Patient-Reported Experience Measures (PREMs) represent healthcare quality more accurately. 11 PREMs capture 'what' happened in the care process, 'how' and 'how often'. 8,12 Aspects of patient experience can become targets for service development, 5 and are essential to quality improvement as they provide actionable data based on what matters to patients. 5,[7][8][9]12,13 Survey instrument development requires a conceptual framework-a set of interlinked ideas that provide an understanding of, or are used to represent a phenomenon. 14,15 Thematic analysis of PREMs from a recently published conceptual framework maps out eight domains of patient experience of services: patient-centred care; quality; integration; accessibility; involvement; communication; discomfort and environment and facilities. 16 These domains strongly align with NHS England, National Clinical Guidelines Centre (NICE) and Institute of Medicine (IOM) definitions of quality in health care, which advocate that care should be patient-centric, safe, effective, efficient and equitable. 17, 18 A scoping review identified a range of IBD-specific instruments that measure experience-related concepts, such as patient satisfaction, 6 patient knowledge, 19 patient concerns, 20 self-efficacy 21 and quality of care, 22 but no validated PREM. To fill this knowledge gap, this study aimed to develop a PREM for people with IBDs to support IBD service evaluation.
Patient involvement in the development of survey instruments is recommended by regulators 23 ; however, it is rarely well-evidenced, except in a 'cursory and poorly reported' fashion, 24 leading to differences in the understanding of survey items. 25 Patient-led approaches make the instrument development process more accountable and ensure that instruments are relevant, transparent and less subject to ambiguity. 26 We combined patient leadership and qualitative research to ensure patients felt the PREM covered the most important issues (content validity) 27 with a meaningful relationship between the items and what matters to them (face validity). 28 The PREM was intended for use in a service evaluation alongside the Patient-Activation Measure (PAM) of knowledge, skills and confidence in self-management. 29 Expert patients expressed concern that-for newly diagnosed patients, those on surgical pathways and those in a flare-some of the PAM's items inappropriately implied that disease management was wholly the patient's responsibility. Their response echoed the previous research 30 [19][20][21] although this term is not preferred by the expert group, and the TDF is often used to identify barriers to, and facilitators of, desirable self-management behaviour. 35 Our scoping review indicated that 339 items on 20 existing IBD measurement instruments were not symptom measures, nor were they measured constructs to do with capability (n = 213 items), opportunity (n = 99) and motivation (n = 87). The TDF includes a wider range of determinants for successful self-management than the PAM and is often used to identify targets for the improvement of supportive services. 36 Both patients and clinicians have therefore recognized the utility of the COM-B and TDF for areas of living well with IBD where the patient can have more agency in managing the condition.  quality; integration; accessibility; involvement; communication; discomfort and environment and facilities. 16 These candidate items and domains were used only as stimuli for discussion.
The expert group recommended the inclusion of items in the PREM about determinants (barriers and facilitators) of service user behaviour. The expert group did not have to accept any item, its wording, or any theme. In the event, through discussion amongst themselves, patients re-categorized items in domains most meaningful to them. The academic frameworks were retained to allow for comparison with patient-derived themes and their own 'seconddegree' constructs. 37 Based on individuals' availability, four expert group members participated in individual one-to-one sessions with the project team to adapt candidate PREM items or create new ones that mapped to the TDF, 33   (whether items were acceptable to people with IBD), the study used the 'think aloud' protocol, 39 in which participants were asked to say what came into their mind as they completed the survey instrument.
A brief unstructured interview followed in which participants were asked to clarify any matters arising during the 'think aloud' interview and to evaluate content validity (the extent to which candidate items cover aspects of care that are important to people with IBD). 27 This included items about how participants found the overall length of the questionnaire, the Likert scale and general formatting. Crohn's & Colitis UK identified a purposive sample of previously uninvolved people with IBD via social media and the charity's website using RECapproved standard advertisement text, inviting patients to opt-in by email. Eligible participants were adults (aged 16 years or over) with IBD (CD or UC) and the capacity to give fully informed consent.
Interviews of 30-40 min were conducted by telephone or videoconference. Participants were provided with a £20 shopping voucher as compensation for their time.
Interview transcripts from encrypted recordings were analysed by E. M. S., D. H. and K. R., in NVivo (QSR International) version 12, using the National Centre for Social Research 'Framework' qualitative data analysis method, which involves five stages: familiarization, identifying a thematic framework, indexing, mapping and interpretation. 40 Following Morgan, we understand codes as a system for marking up 'parts of the text that are of special interest' and themes as converting 'codes into core concepts that represent the most important aspects of the results'. 41 In this case, the aim of the study is to ensure that items on the final PREM would reflect the range and content of deductively and inductively derived themes. Our analysis of think-aloud data combined deductive coding (based on Streiner and Norman's criteria 38 for face validity and the conceptual frameworks 8,16,32,33 for content validity) with the inductive development of codes for 'parts of the text of special interest', 41 with content not already covered by the frameworks. In general, these new inductively derived codes were developed during the closing brief unstructured interview (see above) and involved IBD context-specific responses to the face validity of items drawn from other survey instruments. For instance, interviewees felt that some questions presupposed a more predictable disease course than was typical with IBD.
Following Francis et al., 42 we specified a priori that 12 interviews would be considered analysed before considering saturation, allowing for stopping after every two further interviews if two coders agreed that no new themes were identified. Interviewing ran ahead of analysis which, retrospectively, showed that data saturation was and Norman's 'readability' criterion. 38 Items with scores of nine or over were rewritten where possible. The Gunning Fog index is widely used in health research 45 and provides an easily available, free-to-use web tool which the project team and expert patient group used to experiment with alternative wordings and sentence lengths. The ideal score for readability with the Fog index is 7 or 8; which is the equivalent of Years 8-9 in the UK schooling system and the seventh and eighth grades in the US education system. 44 The project team allowed scores of 9 with exceptions that allowed for contextually specific words with which patients were likely to be comfortable, for example 'colitis' and 'hospital'. At the final workshop, voting exercise results and revisions were presented. Outstanding issues, for example, where items received no majority vote, were resolved through discussion.       Patients should be supported to make informed, shared decisions about their treatment and care to ensure these take their preferences and goals fully into account. Statement 5.4 Patients with IBD being considered for surgery should be provided with information in a format and language they can easily understand to support shared decision-making and informed consent and offered psychological support. Statement 7.1 A personalized care plan should be in place for every IBD patient, with access to an IBD nurse specialist and telephone/email advice line.

10
The Care Team understands what matters to me (in healthcare and beyond) Statement 7.1 A personalized care plan should be in place for every IBD patient, with access to an IBD nurse specialist and telephone/email advice line. A further limitation is that the health literacy levels of the 'think aloud' participants were not assessed. Future research will assess different forms of reliability and validity in more representative quality improvement cohorts of people with IBD, and investigate the face and content validity of the instrument in young adults.
IBD UK standards provide a consensus of how high-quality care is defined. 3 We have related experience to such quality standards in the mapping exercise, with 28 of the PREM items defined by patients mapping to one or more standards. As such, the PREM can give a clear description, from a patient's perspective, of the extent to which they are actually experiencing these standards in their care. Services might use the responses as robust, patient-reported evidence of meeting the quality standards. Ten items in the PREM are not represented in the IBD UK standards. These include items which cover important issues, including the ability of an individual to mentally cope with their IBD; that they are treated with dignity and respect; that they understand the information given to them; that their concerns are taken seriously; that they have the confidence to express their needs and that appointments are in a format that suits them. Future research and iterations of the IBD standards should consider whether such items should be included within the overall standards of care.
Positive patient experience is associated with higher levels of care quality and clinical effectiveness. 39 Experience measures are increasingly used to complement process, clinical and cost data as evidence of a service's compliance with top-down policy, 8 and used as a bottom-up method of identifying targets for improvement. 9 As such, PREMs have the potential to benefit patients as well as to provide system-wide benefits. However, clinical teams can find experience data removed from day-to-day concerns or difficult to translate into actionable improvements and the use of PREMs without structured staff training is not recommended. 5 The AWARE-IBD collaboration (doi:10.17605/OSF.IO/H7FCP) is currently collecting PREM data using a co-produced web-based application, allowing service-user completion from home. The purpose of using the instrument is to make the patient experience visible to healthcare professionals so that they can optimize care at an individual and service level. Future evaluations will look at how PREM data are used to structure clinical encounters. PREM data will also be used in time series analyses to understand the success of patient-led quality improvement efforts.

| CONCLUSIONS
This paper describes a patient-led process for the development and validation of a 38-item IBD PREM. We are confident that our sample was adequate to explore content and face validity across two major subpopulations of IBD given the strength of complementary public involvement. However, further validation is required to test the psychometric properties of the PREM and to determine how patient experience data can evaluate the effects of changes in service delivery, particularly for underrepresented patient groups in IBD. Clear information about IBD, the local IBD service and patient organizations should be accessible in outpatient clinics, wards, endoscopy and day care areas. Statement 7.1 A personalized care plan should be in place for every IBD patient, with access to an IBD nurse specialist and telephone/email advice line. Statement 7.2 Patients should be supported in self-management, as appropriate, through referral or signposting to education, groups and support.

37
I believe that my care and treatment will benefit me Statement 3.3 Patients should be supported to make informed, shared decisions about their treatment and care to ensure these take their preferences and goals fully into account.

38
I have a personalized written care plan Statement 7.1 A personalized care plan should be in place for every IBD patient, with access to an IBD nurse specialist and telephone/email advice line.