Ye, Fei and Bors, Adrian Gheorghe orcid.org/0000-0001-7838-0021 (2022) Learning an Evolved Mixture Model for Task-Free Continual Learning. In: IEEE International Conference on Image Processing (ICIP). IEEE , Bordeaux, France , pp. 1936-1940.
Abstract
Recently, continual learning (CL) has gained significant interest because it enables deep learning models to acquire new knowledge without forgetting previously learnt information. However, most existing works require knowing the task identities and boundaries, which is not realistic in a real context. In this paper, we address a more challenging and realistic setting in CL, namely the Task-Free Continual Learning (TFCL) in which a model is trained on non-stationary data streams with no explicit task information. To address TFCL, we introduce an evolved mixture model whose network architecture is dynamically expanded to adapt to the data distribution shift. We implement this expansion mechanism by evaluating the probability distance between the knowledge stored in each mixture model component and the current memory buffer using the Hilbert Schmidt Independence Criterion (HSIC). We further introduce two simple dropout mechanisms to selectively remove stored examples in order to avoid memory overload while preserving memory diversity. Empirical results demonstrate that the proposed approach achieves excellent performance.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2022 IEEE. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Depositing User: | Pure (York) |
Date Deposited: | 09 Nov 2022 10:40 |
Last Modified: | 18 Dec 2024 00:40 |
Published Version: | https://doi.org/10.1109/ICIP46576.2022.9898047 |
Status: | Published |
Publisher: | IEEE |
Identification Number: | 10.1109/ICIP46576.2022.9898047 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:193134 |
Download
Filename: ICIP2022d.pdf
Description: Learning an Evolved Mixture Model for Task-Free Continual Learning