Wang, Y. and Lin, C. (2025) Tougher text, smarter models: Raising the bar for adversarial defence benchmarks. In: Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Di Eugenio, B. and Schockaert, S., (eds.) Proceedings of the 31st International Conference on Computational Linguistics. 31st International Conference on Computational Linguistics (COLING), 19-24 Jan 2025, Abu Dhabi, UAE. Association for Computational Linguistics , pp. 6475-6491.
Abstract
Recent advancements in natural language processing have highlighted the vulnerability of deep learning models to adversarial attacks. While various defence mechanisms have been proposed, there is a lack of comprehensive benchmarks that evaluate these defences across diverse datasets, models, and tasks. In this work, we address this gap by presenting an extensive benchmark for textual adversarial defence that significantly expands upon previous work. Our benchmark incorporates a wide range of datasets, evaluates state-of-the-art defence mechanisms, and extends the assessment to include critical tasks such as single-sentence classification, similarity and paraphrase identification, natural language inference, and commonsense reasoning. This work not only serves as a valuable resource for researchers and practitioners in the field of adversarial robustness but also identifies key areas for future research in textual adversarial defence. By establishing a new standard for benchmarking in this domain, we aim to accelerate progress towards more robust and reliable natural language processing systems.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2025 Association for Computational Linguistics. Licensed on a Creative Commons Attribution 4.0 International License. (https://creativecommons.org/licenses/by/4.0/) |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 31 Jan 2025 17:11 |
Last Modified: | 31 Jan 2025 17:11 |
Published Version: | https://aclanthology.org/2025.coling-main.432/ |
Status: | Published |
Publisher: | Association for Computational Linguistics |
Refereed: | Yes |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:222336 |