Bethell, Daniel, Gerasimou, Simos and Calinescu, Radu orcid.org/0000-0002-2678-9260 (2024) Robust Uncertainty Quantification Using Conformalised Monte Carlo Prediction. In: Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24). Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24) .
Abstract
Deploying deep learning models in safety-critical applications remains a very challenging task, mandating the provision of assurances for the dependable operation of these models. Uncertainty quantification (UQ) methods estimate the model’s confidence per prediction, informing decision-making by considering the effect of randomness and model misspecification. Despite the advances of state-of-the-art UQ methods, they are computationally expensive or produce conservative prediction sets/intervals. We introduce MC-CP, a novel hybrid UQ method that combines a new adaptive Monte Carlo (MC) dropout method with conformal prediction (CP). MC-CP adaptively modulates the traditional MC dropout at runtime to save memory and computation resources, enabling predictions to be consumed by CP, yielding robust prediction sets/intervals. Throughout comprehensive experiments, we show that MC-CP delivers significant improvements over comparable UQ methods, like MC dropout, RAPS and CQR, both in classification and regression benchmarks. MC-CP can be easily added to existing models, making its deployment simple. The MC-CP code and replication package is available at https://github.com/team-daniel/MC-CP.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy. |
Keywords: | Uncertainty Estimation,deep learning,Monte Carlo |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Depositing User: | Pure (York) |
Date Deposited: | 17 Jan 2024 10:50 |
Last Modified: | 14 Nov 2024 06:25 |
Status: | Published |
Series Name: | Thirty-Eighth AAAI Conference on Artificial Intelligence (AAAI-24) |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:207689 |
Download
Filename: AAAI_MC_CP_Paper_Camera_Ready_Version_-1.pdf
Description: AAAI_MC_CP_Paper__Camera_Ready_Version_-1
Licence: CC-BY 2.5