SPOT: An efficient training-free task similarity quantification method for continual learning

Quantifying task similarity is crucial in continual learning, enabling models to better mitigate catastrophic forgetting and facilitate knowledge transfer across tasks. However, existing similarity measures often demand extensive data and computation resources, resulting in low efficiency and limiting practical applications. To address this challenge, we introduce SPOT(Single-batch Probe Of Task-similarity), a novel task similarity measure that requires only a single batch of data to quickly estimate task similarity before training on a new task. SPOT leverages the change in the empirical loss between old and new tasks to quantify task relationships, offering a low-cost, efficient solution for task similarity estimation in continual learning. Experiments on three public datasets and one real-world dataset show that 1) task similarity and forgetting are negatively correlated. 2) SPOT can efficiently predict the forgetting risk with one batch of new task data. 3) Forgetting is most severe for tasks with significant semantic distinctions. Our findings indicate that SPOT serves as a passive yet efficient tool to predict catastrophic forgetting risk before training, facilitating continual learning with minimal computational overhead. This research provides new insights into task similarity quantification and has strong potential for deployment in resource-constrained environments. Code is available at https://github.com/wang-xulong/SPOT