Li, R., Li, X., Lin, C. orcid.org/0000-0003-3454-2468 et al. (2 more authors) (2019) A stable variational autoencoder for text modelling. In: van Deemter, K., Lin, C. and Takamura, H., (eds.) Proceedings of the 12th International Conference on Natural Language Generation (INLG 2019). 12th International Conference on Natural Language Generation - INLG 2019, 29 Oct - 01 Nov 2020, Tokyo, Japan. Association for Computational Linguistics (ACL) , pp. 594-599. ISBN 9781950737949
Abstract
Variational Autoencoder (VAE) is a powerful method for learning representations of high-dimensional data. However, VAEs can suffer from an issue known as latent variable collapse (or KL term vanishing), where the posterior collapses to the prior and the model will ignore the latent codes in generative tasks. Such an issue is particularly prevalent when employing VAE-RNN architectures for text modelling (Bowman et al., 2016; Yang et al., 2017). In this paper, we present a new architecture called Full-Sampling-VAE-RNN, which can effectively avoid latent variable collapse. Compared to the general VAE-RNN architectures, we show that our model can achieve much more stable training process and can generate text with significantly better quality.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2019 Association for Computational Linguistics. This is an Open Access article distributed under the terms of the Creative Commons Attribution Licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 11 Jun 2020 09:39 |
Last Modified: | 11 Jun 2020 09:39 |
Published Version: | https://www.aclweb.org/anthology/W19-8673 |
Status: | Published |
Publisher: | Association for Computational Linguistics (ACL) |
Refereed: | Yes |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:161488 |