Carannante, G., Dera, D., Rasool, G. et al. (2 more authors) (2020) Robust learning via ensemble density propagation in deep neural networks. In: 2020 IEEE 30th International Workshop on Machine Learning for Signal Processing (MLSP). MLSP 2020 : IEEE International Workshop on Machine Learning for Signal Processing, 21-24 Sep 2020, Espoo, Finland. IEEE ISBN 9781728166636
Abstract
Learning in uncertain, noisy, or adversarial environments is a challenging task for deep neural networks (DNNs). We propose a new theoretically grounded and efficient approach for robust learning that builds upon Bayesian estimation and Variational Inference. We formulate the problem of density propagation through layers of a DNN and solve it using an Ensemble Density Propagation (EnDP) scheme. The EnDP approach allows us to propagate moments of the variational probability distribution across the layers of a Bayesian DNN, enabling the estimation of the mean and covariance of the predictive distribution at the output of the model. Our experiments using MNIST and CIFAR-10 datasets show a significant improvement in the robustness of the trained models to random noise and adversarial attacks.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | Variational inference; Ensemble techniques; robustness; adversarial learning |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Automatic Control and Systems Engineering (Sheffield) |
Funding Information: | Funder Grant number Engineering and Physical Science Research Council EP/T013265/1 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 28 Jul 2020 08:13 |
Last Modified: | 20 Oct 2021 00:38 |
Status: | Published |
Publisher: | IEEE |
Refereed: | Yes |
Identification Number: | 10.1109/MLSP49062.2020.9231635 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:163807 |