Carannante, G., Bouaynaya, N., Mihaylova, L. orcid.org/0000-0001-5856-2223 et al. (1 more author) (2024) BaSIS-Net: From point estimate to predictive distribution in neural networks - a Bayesian sequential importance sampling framework. Transactions on Machine Learning Research. ISSN 2835-8856
Abstract
Data-driven Deep Learning (DL) models have revolutionized autonomous systems, but ensuring their safety and reliability necessitates the assessment of predictive confidence or uncertainty. Bayesian DL provides a principled approach to quantify uncertainty via probability density functions defined over model parameters. However, the exact solution is intractable for most DL models, and the approximation methods, often based on heuristics, suffer from scalability issues and stringent distribution assumptions and may lack theoretical guarantees. This work develops a Sequential Importance Sampling framework that approximates the posterior probability density function through weighted samples (or particles), which can be used to find the mean, variance, or higher-order moments of the posterior distribution. We demonstrate that propagating particles, which capture information about the higher-order moments, through the layers of the DL model results in increased robustness to natural and malicious noise (adversarial attacks). The variance computed from these particles effectively quantifies the model’s decision uncertainty, demonstrating well-calibrated and accurate predictive confidence.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2024 The Author(s). Licensed under a CC BY 4.0 licence. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Automatic Control and Systems Engineering (Sheffield) |
Funding Information: | Funder Grant number ENGINEERING AND PHYSICAL SCIENCE RESEARCH COUNCIL EP/T013265/1 Engineering and Physical Sciences Research Council EP/T013265/1 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 04 Jul 2024 15:44 |
Last Modified: | 18 Jul 2024 15:55 |
Published Version: | https://jmlr.org/tmlr/papers/ |
Status: | Published online |
Publisher: | OpenReview |
Refereed: | Yes |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:214386 |