Javed, M., Mihaylova, L. orcid.org/0000-0001-5856-2223 and Bouaynaya, N. (2021) Variance guided continual learning in a convolutional neural network Gaussian process single classifier approach for multiple tasks in noisy images. In: de Villiers, P., de Waal, A. and Gustafsson, F., (eds.) 2021 IEEE 24th International Conference on Information Fusion (FUSION). 24th International Conference on Information Fusion (Fusion 2021), 01-04 Nov 2021, Sun City, South Africa. Institute of Electrical and Electronics Engineers ISBN 9781665414272
Abstract
This work provides a continual learning solution in a single-classifier to multiple classification tasks with various data sets. A Gaussian process (GP) is combined with a Convolutional Neural Network (CNN) feature extractor architecture (CNNGP). Post softmax samples are used to estimate the variance. The variance is characterising the impact of uncertainties and is part of the update process for the learning rate parameters. Within the proposed framework two learning approaches are adopted: 1) in the first, the weights of the CNN are deterministic and only the GP learning rate is updated, 2) in the second setting, prior distributions are adopted for the CNN weights. Both the learning rates of the CNN and the GP are updated. The algorithm is trained on two variants of the MNIST dataset, split-MNIST and permuted-MNIST. Results are compared with the Uncertainty Guided Continual Bayesian Networks (UCB) multi-classifier approach [1]. The validation shows that the proposed algorithm in the Bayesian setting outperforms the UCB in tasks subject to Gaussian noise image noises and shows robustness.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2021 ISIF. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | deep learning; Bayesian learning; classification; artificial intelligence; machine learning; continual learning |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Automatic Control and Systems Engineering (Sheffield) |
Funding Information: | Funder Grant number Engineering and Physical Sciences Research Council EP/T013265/1 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 01 Sep 2021 10:24 |
Last Modified: | 02 Dec 2022 01:13 |
Published Version: | https://ieeexplore.ieee.org/document/9626907 |
Status: | Published |
Publisher: | Institute of Electrical and Electronics Engineers |
Refereed: | Yes |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:177517 |