Background
Clinical guidelines commonly recommend preventative treatments for people above a risk threshold. Therefore, decision-makers must have faith in risk prediction tools and model-based cost-effectiveness analyses for people at different levels of risk. Two problems that arise are inadequate handling of competing risks of death and failing to account for direct treatment disutility (i.e. the hassle of taking treatments). We explored these issues using two case studies: primary prevention of cardiovascular disease using statins and osteoporotic fracture using bisphosphonates.
Objectives
Externally validate three risk prediction tools [QRISK®3, QRISK®-Lifetime, QFracture-2012 (ClinRisk Ltd, Leeds, UK)]; derive and internally validate new risk prediction tools for cardiovascular disease [competing mortality risk model with Charlson Comorbidity Index (CRISK-CCI)] and fracture (CFracture), accounting for competing-cause death; quantify direct treatment disutility for statins and bisphosphonates; and examine the effect of competing risks and direct treatment disutility on the cost-effectiveness of preventative treatments.
Design, participants, main outcome measures, data sources
Discrimination and calibration of risk prediction models (Clinical Practice Research Datalink participants: aged 25–84 years for cardiovascular disease and aged 30–99 years for fractures); direct treatment disutility was elicited in online stated-preference surveys (people with/people without experience of statins/bisphosphonates); costs and quality-adjusted life-years were determined from decision-analytic modelling (updated models used in National Institute for Health and Care Excellence decision-making).
Results
CRISK-CCI has excellent discrimination, similar to that of QRISK3 (Harrell’s c = 0.864 vs. 0.865, respectively, for women; and 0.819 vs. 0.834, respectively, for men). CRISK-CCI has systematically better calibration, although both models overpredict in high-risk subgroups. People recommended for treatment (10-year risk of ≥ 10%) are younger when using QRISK-Lifetime than when using QRISK3, and have fewer observed events in a 10-year follow-up (4.0% vs. 11.9%, respectively, for women; and 4.3% vs. 10.8%, respectively, for men). QFracture-2012 underpredicts fractures, owing to under-ascertainment of events in its derivation. However, there is major overprediction among people aged 85–99 years and/or with multiple long-term conditions. CFracture is better calibrated, although it also overpredicts among older people. In a time trade-off exercise (n = 879), statins exhibited direct treatment disutility of 0.034; for bisphosphonates, it was greater, at 0.067. Inconvenience also influenced preferences in best–worst scaling (n = 631). Updated cost-effectiveness analysis generates more quality-adjusted life-years among people with below-average cardiovascular risk and fewer among people with above-average risk. If people experience disutility when taking statins, the cardiovascular risk threshold at which benefits outweigh harms rises with age (≥ 8% 10-year risk at 40 years of age; ≥ 38% 10-year risk at 80 years of age). Assuming that everyone experiences population-average direct treatment disutility with oral bisphosphonates, treatment is net harmful at all levels of risk.
Limitations
Treating data as missing at random is a strong assumption in risk prediction model derivation. Disentangling the effect of statins from secular trends in cardiovascular disease in the previous two decades is challenging. Validating lifetime risk prediction is impossible without using very historical data. Respondents to our stated-preference survey may not be representative of the population. There is no consensus on which direct treatment disutilities should be used for cost-effectiveness analyses. Not all the inputs to the cost-effectiveness models could be updated.
Conclusions
Ignoring competing mortality in risk prediction overestimates the risk of cardiovascular events and fracture, especially among older people and those with multimorbidity. Adjustment for competing risk does not meaningfully alter cost-effectiveness of these preventative interventions, but direct treatment disutility is measurable and has the potential to alter the balance of benefits and harms. We argue that this is best addressed in individual-level shared decision-making.
Study registration
This study is registered as PROSPERO CRD42021249959.
Funding
This award was funded by the National Institute for Health and Care Research (NIHR) Health and Social Care Delivery Research programme (NIHR award ref: 15/12/22) and is published in full in Health and Social Care Delivery Research; Vol. 12, No. 4. See the NIHR Funding and Awards website for further award information.
CORE (COnnecting REpositories)
CORE (COnnecting REpositories)