Ye, Fei and Bors, Adrian Gheorghe orcid.org/0000-0001-7838-0021 (2022) Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process. In: IEEE/CVF International Conference on Computer Vision (ICCV). IEEE , pp. 10695-10704.
Abstract
Recent research efforts in lifelong learning propose to grow a mixture of models to adapt to an increasing number of tasks. The proposed methodology shows promising results in overcoming catastrophic forgetting. However, the theory behind these successful models is still not well understood. In this paper, we perform the theoretical analysis for lifelong learning models by deriving the risk bounds based on the discrepancy distance between the probabilistic representation of data generated by the model and that corresponding to the target dataset. Inspired by the theoretical analysis, we introduce a new lifelong learning approach , namely the Lifelong Infinite Mixture (LIMix) model, which can automatically expand its network architectures or choose an appropriate component to adapt its parameters for learning a new task, while preserving its previously learnt information. We propose to incorporate the knowledge by means of Dirichlet processes by using a gat-ing mechanism which computes the dependence between the knowledge learnt previously and stored in each component , and a new set of data. Besides, we train a compact Student model which can accumulate cross-domain representations over time and make quick inferences. The code is available at https://github.com/dtuzi123/ Lifelong-infinite-mixture-model.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © IEEE, 2022. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Depositing User: | Pure (York) |
Date Deposited: | 09 Nov 2022 10:50 |
Last Modified: | 18 Dec 2024 00:40 |
Published Version: | https://doi.org/10.1109/ICCV48922.2021.01052 |
Status: | Published |
Publisher: | IEEE |
Identification Number: | 10.1109/ICCV48922.2021.01052 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:193136 |
Download
Filename: ICCV_2021.pdf
Description: Lifelong Infinite Mixture Model Based on Knowledge-Driven Dirichlet Process