Phelps, D., Fan, X.-R., Gow-Smith, E. et al. (3 more authors) (2022) Sample efficient approaches for idiomaticity detection. In: Bhatia, A., Cook, P., Taslimipoor, S., Garcia, M. and Ramisch, C., (eds.) Proceedings of The 18th Workshop on Multiword Expressions @LREC2022. The 18th Workshop on Multiword Expressions @LREC2022, 20-25 Jun 2022, Marseille, France. European Language Resources Association (ELRA) , pp. 105-111. ISBN 9791095546900
Abstract
Deep neural models, in particular Transformer-based pre-trained language models, require a significant amount of data to train. This need for data tends to lead to problems when dealing with idiomatic multiword expressions (MWEs), which are inherently less frequent in natural text. As such, this work explores sample efficient methods of idiomaticity detection. In particular we study the impact of Pattern Exploit Training (PET), a few-shot method of classification, and BERTRAM, an efficient method of creating contextual embeddings, on the task of idiomaticity detection. In addition, to further explore generalisability, we focus on the identification of MWEs not present in the training data. Our experiments show that while these methods improve performance on English, they are much less effective on Portuguese and Galician, leading to an overall performance about on par with vanilla mBERT. Regardless, we believe sample efficient methods for both identifying and representing potentially idiomatic MWEs are very encouraging and hold significant potential for future exploration.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2022 European Language Resources Association (ELRA), published under a CC-BY-NC-4.0 license (http://creativecommons.org/licenses/by-nc/4.0/). |
Keywords: | Idiomaticity Detection; Sample Efficient MWE Detection; Pre-Trained Language Models |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 01 Sep 2022 13:57 |
Last Modified: | 01 Sep 2022 13:57 |
Published Version: | http://www.lrec-conf.org/proceedings/lrec2022/work... |
Status: | Published |
Publisher: | European Language Resources Association (ELRA) |
Refereed: | Yes |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:190380 |