Broughton, S.J., Jalal, M.A. and Moore, R.K. orcid.org/0000-0003-0065-3311 (2021) Investigating deep neural structures and their interpretability in the domain of voice conversion. In: Heřmanský, H., Černocký, H., Burget, L., Lamel, L., Scharenborg, O. and Motlicek, P., (eds.) Interspeech 2021. Interspeech 2021, 30 Aug - 03 Sep 2021, Brno, Czechia. ISCA - International Speech Communication Association , pp. 806-810.
Abstract
Generative Adversarial Networks (GANs) are machine learning networks based around creating synthetic data. Voice Conversion (VC) is a subset of voice translation that involves translating the paralinguistic features of a source speaker to a target speaker while preserving the linguistic information. The aim of non-parallel conditional GANs for VC is to translate an acoustic speech feature sequence from one domain to another without the use of paired data. In the study reported here, we investigated the interpretability of state-of-the-art implementations of non-parallel GANs in the domain of VC. We show that the learned representations in the repeating layers of a particular GAN architecture remain close to their original random initialised parameters, demonstrating that it is the number of repeating layers that is more responsible for the quality of the output. We also analysed the learned representations of a model trained on one particular dataset when used during transfer learning on another dataset. This also showed high levels of similarity in the repeating layers. Together, these results provide new insight into how the learned representations of deep generative networks change during learning and the importance of the number of layers, which would help build better GAN-based speech conversion models.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2021 ISCA. This is an author-produced version of a paper subsequently published in Interspeech 2021 Proceedings. Uploaded in accordance with the publisher's self-archiving policy. |
Keywords: | voice conversion (VC); generative adversarial networks (GANs); canonical correlation analysis (CCA); SVCCA; transfer learning; non-parallel VC; multi-domain VC |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 25 Nov 2021 07:30 |
Last Modified: | 25 Nov 2021 08:59 |
Published Version: | https://www.isca-speech.org/archive/interspeech_20... |
Status: | Published |
Publisher: | ISCA - International Speech Communication Association |
Refereed: | Yes |
Identification Number: | 10.21437/interspeech.2021-1730 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:180841 |