Madhyastha, P., Wang, J.K. orcid.org/0000-0003-0048-3893 and Specia, L. (2018) Defoiling foiled image captions. In: Walker, M., Ji, H. and Stent, A., (eds.) Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. North American Chapter of the Association of Computational Linguistics: Human Language Technologies (NAACL HLT), 01-06 Jun 2018, New Orleans, Louisiana. Association for Computational Linguistics (ACL) , pp. 433-438. ISBN 9781948087292
Abstract
We address the task of detecting foiled image captions, i.e. identifying whether a caption contains a word that has been deliberately replaced by a semantically similar word, thus rendering it inaccurate with respect to the image being described. Solving this problem should in principle require a fine-grained understanding of images to detect linguistically valid perturbations in captions. In such contexts, encoding sufficiently descriptive image information becomes a key challenge. In this paper, we demonstrate that it is possible to solve this task using simple, interpretable yet powerful representations based on explicit object information. Our models achieve stateof-the-art performance on a standard dataset, with scores exceeding those achieved by humans on the task. We also measure the upperbound performance of our models using gold standard annotations. Our analysis reveals that the simpler model performs well even without image information, suggesting that the dataset contains strong linguistic bias.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2018 The Association for Computational Linguistics. Article distributed under the terms of the Creative Commons Attribution Licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Funding Information: | Funder Grant number European Commission - Horizon 2020 678017 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 24 Apr 2018 14:28 |
Last Modified: | 29 Jun 2020 13:32 |
Published Version: | https://www.aclweb.org/anthology/N18-2069 |
Status: | Published |
Publisher: | Association for Computational Linguistics (ACL) |
Refereed: | Yes |
Identification Number: | 10.18653/v1/N18-2069 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:129791 |