Methods for expected value of information analysis in complex health economic models: developments on the health economics of beta interferon and glatiramer acetate for multiple sclerosis

Objectives: To develop methods for performing expected value of perfect information (EVPI) analysis in computationally expensive models and to report on the developments on the health economics of interferon-β and glatiramer acetate in the management of multiple sclerosis (MS) using this methodological framework.

Data sources: Electronic databases and Internet resources, reference lists of relevant articles.

Review methods: A methodological framework was developed for undertaking EVPI analysis for complex models. The framework identifies conditions whereby EVPI may be calculated numerically, where the onelevel algorithm sufficiently approximates the two-level algorithm, and whereby metamodelling techniques may accurately approximate the original simulation model. Metamodelling techniques, including linear regression, neural networks and Gaussian processes (GP), were systematically reviewed and critically appraised. Linear regression metamodelling, GP metamodelling and the one-level EVPI approximation were used to estimate partial EVPIs using the ScHARR MS cost-effectiveness model.

Results: The review of metamodelling approaches suggested that in general the simpler techniques such as linear regression may be easier to implement, as they require little specialist expertise although may provide only limited predictive accuracy. More complex methods such as Gaussian process metamodelling and neural networks tend to use less-restrictive assumptions concerning the relationship between the model inputs and net benefits, and therefore may permit greater accuracy in estimating EVPIs. Assuming independent treatment efficacy, the ‘per patient’ EVPI for all uncertainty parameters within the ScHARR MS model is £8855. This leads to a population EVPI of £86,208,936, which represents the upper estimate for the overall EVPI over 10 years. Assuming all treatment efficacies are perfectly correlated, the overall per patient EVPI is £4271. This leads to a population EVPI of £41,581,273, which represents the lower estimate for the overall EVPI over 10 years. The partial EVPI analysis, undertaken using both the linear regression metamodel and Gaussian process metamodel clearly, suggests that further research is indicated on the long-term impact of these therapies on disease progression, the proportion of patients dropping off therapy and the relationship between the EDSS, quality of life and costs of care.

Conclusions: The applied methodology points towards using more sophisticated metamodelling approaches in order to obtain greater accuracy in EVPI estimation. Programming requirements, software availability and statistical accuracy should be considered when choosing between metamodelling techniques. Simpler, more accessible techniques are open to greater predictive error, whilst sophisticated methodologies may enhance accuracy within non-linear models, but are considerably more difficult to implement and may require specialist expertise. These techniques have been applied in only a limited number of cases hence their suitability for use in EVPI analysis has not yet been demonstrated. A number of areas requiring further research have been highlighted. Further clinical research is required concerning the relationship between the EDSS, costs of care and health outcomes, the rates at which patients drop off therapy and in particular the impact of disease-modifying therapies on the progression of MS. Further methodological research is indicated concerning the inclusion of epidemiological population parameters within the sensitivity analysis; the development of criteria for selecting a metamodelling approach; the application of metamodelling techniques within health economic models and in the specific application to EVI analyses; and the use of metamodelling for EVSI and ENBS analysis.