Mirowski, P. and Vlachos, A. (2015) Dependency Recurrent Neural Language Models for Sentence Completion. In: ACL 2015: Proceedings of the Conference. 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 26-31 Jul 2015, Beijing, China. ACL Anthology: A Digital Archive of Research Papers in Computational Linguistics, 1 . The Association for Computational Linguistics (ACL) , pp. 511-517.
Abstract
Recent work on language modelling has shifted focus from count-based models to neural models. In these works, the words in each sentence are always considered in a left-to-right order. In this paper we show how we can improve the performance of the recurrent neural network (RNN) lan- guage model by incorporating the syntac- tic dependencies of a sentence, which have the effect of bringing relevant contexts closer to the word being predicted. We evaluate our approach on the Microsoft Research Sentence Completion Challenge and show that the dependency RNN pro- posed improves over the RNN by about 10 points in accuracy. Furthermore, we achieve results comparable with the state- of-the-art models on this task.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | Copyright 2015 Association for Computational Linguistics. Licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 License. Permission is granted to make copies for the purposes of teaching and research. |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 03 Dec 2015 15:45 |
Last Modified: | 19 Dec 2022 13:32 |
Published Version: | http://www.aclweb.org/anthology/P/P15/ |
Status: | Published |
Publisher: | The Association for Computational Linguistics (ACL) |
Series Name: | ACL Anthology: A Digital Archive of Research Papers in Computational Linguistics |
Refereed: | Yes |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:91376 |