Deena, S., Ng, R.W.M., Madhyashtha, P. et al. (2 more authors) (2018) Exploring the use of Acoustic Embeddings in Neural Machine Translation. In: Proceedings of IEEE Automatic Speech Recognition and Understanding Workshop. 2017 IEEE Automatic Speech Recognition and Understanding Workshop, December 16-20, 2017, Okinawa, Japan. IEEE ISBN 978-1-5090-4788-8
Abstract
Neural Machine Translation (NMT) has recently demonstrated improved performance over statistical machine translation and relies on an encoder-decoder framework for translating text from source to target. The structure of NMT makes it amenable to add auxiliary features, which can provide complementary information to that present in the source text. In this paper, auxiliary features derived from accompanying audio, are investigated for NMT and are compared and combined with text-derived features. These acoustic embeddings can help resolve ambiguity in the translation, thus improving the output. The following features are experimented with: Latent Dirichlet Allocation (LDA) topic vectors and GMM subspace i-vectors derived from audio. These are contrasted against: skip-gram/Word2Vec features and LDA features derived from text. The results are encouraging and show that acoustic information does help with NMT, leading to an overall 3.3% relative improvement in BLEU scores.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © IEEE 2018. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | Neural Machine Translation; LDA topics; Acoustic Embeddings |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Funding Information: | Funder Grant number EUROPEAN COMMISSION - HORIZON 2020 678017 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 29 Sep 2017 09:57 |
Last Modified: | 20 Aug 2018 11:08 |
Published Version: | https://doi.org/10.1109/ASRU.2017.8268971 |
Status: | Published |
Publisher: | IEEE |
Refereed: | Yes |
Identification Number: | 10.1109/ASRU.2017.8268971 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:121515 |