Chrysostomou, G. and Aletras, N. orcid.org/0000-0003-4285-1965 (Submitted: 2021) Variable instance-level explainability for text classification. arXiv. (Submitted)
Abstract
Despite the high accuracy of pretrained transformer networks in text classification, a persisting issue is their significant complexity that makes them hard to interpret. Recent research has focused on developing feature scoring methods for identifying which parts of the input are most important for the model to make a particular prediction and use it as an explanation (i.e. rationale). A limitation of these approaches is that they assume that a particular feature scoring method should be used across all instances in a dataset using a predefined fixed length, which might not be optimal across all instances. To address this, we propose a method for extracting variable-length explanations using a set of different feature scoring methods at instance-level. Our method is inspired by word erasure approaches which assume that the most faithful rationale for a prediction should be the one with the highest divergence between the model's output distribution using the full text and the text after removing the rationale for a particular instance. Evaluation on four standard text classification datasets shows that our method consistently provides more faithful explanations compared to previous fixed-length and fixed-feature scoring methods for rationale extraction.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2021 The Author(s). This is an Open Access pre-print distributed under the terms of the Creative Commons Attribution Licence (http://creativecommons.org/licenses/by/4.0). |
Keywords: | cs.CL; cs.CL; cs.AI |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Funding Information: | Funder Grant number Engineering and Physical Science Research Council EP/V055712/1 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 21 Apr 2021 12:44 |
Last Modified: | 21 Apr 2021 12:44 |
Published Version: | https://arxiv.org/abs/2104.08219v1 |
Status: | Submitted |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:173285 |