CogNLP-Sheffield at CMCL 2021 Shared Task: Blending cognitively inspired features with transformer-based language models for predicting eye tracking patterns

Vickers, P., Wainwright, R., Tayyar Madabushi, H. et al. (1 more author) (2021) CogNLP-Sheffield at CMCL 2021 Shared Task: Blending cognitively inspired features with transformer-based language models for predicting eye tracking patterns. In: Proceedings of the Workshop on Cognitive Modeling and Computational Linguistics (CMCL 2021). Cognitive Modeling and Computational Linguistics (CMCL 2021), 10 Jun 2021, Mexico City, Mexico. Association for Computational Linguistics (ACL) , pp. 125-133. ISBN 9781954085350

Abstract

The CogNLP-Sheffield submissions to the CMCL 2021 Shared Task examine the value of a variety of cognitively and linguistically inspired features for predicting eye tracking patterns, as both standalone model inputs and as supplements to contextual word embeddings (XLNet). Surprisingly, the smaller pre-trained model (XLNet-base) outperforms the larger (XLNet-large), and despite evidence that multi-word expressions (MWEs) provide cognitive processing advantages, MWE features provide little benefit to either model.