Li, J., Wang, G., Zheng, S. et al. (4 more authors) (2026) Towards mitigating modality bias in vision-language models for Temporal Action Localization. In: Liakata, M., Moreira, V.P., Zhang, J. and Jurgens, D., (eds.) Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 02-07 Jul 2026, San Diego, California, United States. Vol. 1. Association for Computational Linguistics, pp. 11087-11104. ISBN: 9798891763906. ISSN: 0736-587X. EISSN: 9061-6464.
Abstract
Temporal Action Localization (TAL) requires identifying both the boundaries and categories of actions in untrimmed videos. While vision-language models (VLMs) offer rich semantics to complement visual evidence, existing approaches tend to overemphasize linguistic priors at the expense of visual performance, leading to a pronounced modality bias. We propose ActionVLM, a vision-language aggregation framework that systematically mitigates modality bias in TAL. Our key insight is to preserve vision as the dominant signal while adaptively exploiting language only when beneficial. To this end, we introduce (i) a debiasing reweighting module that estimates the language advantage—the incremental benefit of language over vision-only predictions—and dynamically reweights language modality accordingly, and (ii) a residual aggregation strategy that treats language as a complementary refinement rather than the primary driver. This combination alleviates modality bias, reduces overconfidence from linguistic priors, and strengthens temporal reasoning. Experiments on THUMOS14 show that our model outperforms state-of-the-art by up to 3.2% mAP. Our code is available at https://github.com/JiaqiLi404/ActionVLM
Metadata
| Item Type: | Proceedings Paper |
|---|---|
| Authors/Creators: |
|
| Editors: |
|
| Copyright, Publisher and Additional Information: | © 2026 Association for Computational Linguistics (ACL). Licensed on a Creative Commons Attribution 4.0 International License. (https://creativecommons.org/licenses/by/4.0/) |
| Dates: |
|
| Institution: | The University of Sheffield |
| Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
| Date Deposited: | 11 Nov 2026 16:46 |
| Last Modified: | 11 Nov 2026 16:46 |
| Status: | Published |
| Publisher: | Association for Computational Linguistics |
| Refereed: | Yes |
| Identification Number: | 10.18653/v1/2026.acl-long.508 |
| Related URLs: | |
| Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:247725 |
Download
Filename: 2026.acl-long.508.pdf
Licence: CC-BY 4.0
CORE (COnnecting REpositories)
CORE (COnnecting REpositories)