Zhou, M. orcid.org/0000-0001-8973-418X and Yang, P. orcid.org/0000-0002-8553-7127 (2023) Automatic temporal relation in multi-task learning. In: KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. KDD '23: The 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 06-10 Aug 2023, Long Beach, CA, USA. Association for Computing Machinery (ACM) , pp. 3570-3580. ISBN 9798400701030
Abstract
Multi-task learning with temporal relation is a common prediction method for modelling the evolution of a wide range of systems. Considering the inherent relations between multiple time points, many works apply multi-task learning to jointly analyse all time points, with each time point corresponding to a prediction task. The most difficult challenge is determining how to fully explore and thus exploit the shared valuable temporal information between tasks to improve the generalization performance and robustness of the model. Existing works are classified as temporal smoothness and mean temporal relations. Both approaches, however, utilize a predefined and symmetric task relation structure that is too rigid and insufficient to adequately capture the intricate temporal relations between tasks. Instead, we propose a novel mechanism named Automatic Temporal Relation (AutoTR) for directly and automatically learning the temporal relation from any given dataset. To solve the biconvex objective function, we adopt the alternating optimization and show that the two related sub-optimization problems are amenable to closed-form computation of the proximal operator. To solve the two problems efficiently, the accelerated proximal gradient method is used, which has the fastest convergence rate of any first-order method. We have preprocessed six public real-life datasets and conducted extensive experiments to fully demonstrate the superiority of AutoTR. The results show that AutoTR outperforms several baseline methods on almost all datasets with different training ratios, in terms of overall model performance and every individual task performance. Furthermore, our findings verify that the temporal relation between tasks is asymmetrical, which has not been considered in previous works. The implementation source can be found at https://github.com/menghui-zhou/AutoTR.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: | |
Copyright, Publisher and Additional Information: | © 2023 ACM. This is an author-produced version of a paper subsequently published in KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. Uploaded in accordance with the publisher's self-archiving policy. |
Keywords: | Information and Computing Sciences; Machine Learning |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 08 Feb 2024 12:32 |
Last Modified: | 08 Feb 2024 14:48 |
Status: | Published |
Publisher: | Association for Computing Machinery (ACM) |
Refereed: | Yes |
Identification Number: | 10.1145/3580305.3599261 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:208825 |