Ollerenshaw, A. orcid.org/0000-0001-5779-1905, Jalal, M.A. and Hain, T. orcid.org/0000-0003-0939-3464 (2021) Insights on neural representations for end-to-end speech recognition. In: Heřmanský, H., Černocký, H., Burget, L., Lamel, L., Scharenborg, O. and Motlicek, P., (eds.) Interspeech 2021. Interspeech 2021, 30 Aug - 03 Sep 2021, Brno, Czechia. ISCA - International Speech Communication Association , pp. 4079-4083.
Abstract
End-to-end automatic speech recognition (ASR) models aim to learn a generalised speech representation. However, there are limited tools available to understand the internal functions and the effect of hierarchical dependencies within the model architecture. It is crucial to understand the correlations between the layer-wise representations, to derive insights on the relationship between neural representations and performance. Previous investigations of network similarities using correlation analysis techniques have not been explored for End-to-End ASR models. This paper analyses and explores the internal dynamics between layers during training with CNN, LSTM and Transformer based approaches using Canonical correlation analysis (CCA) and centered kernel alignment (CKA) for the experiments. It was found that neural representations within CNN layers exhibit hierarchical correlation dependencies as layer depth increases but this is mostly limited to cases where neural representation correlates more closely. This behaviour is not observed in LSTM architecture, however there is a bottom-up pattern observed across the training process, while Transformer encoder layers exhibit irregular coefficiency correlation as neural depth increases. Altogether, these results provide new insights into the role that neural architectures have upon speech recognition performance. More specifically, these techniques can be used as indicators to build better performing speech recognition models.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2021 ISCA. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | end-to-end; speech recognition; analysis |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 17 Jun 2022 13:24 |
Last Modified: | 17 Jun 2022 13:24 |
Status: | Published |
Publisher: | ISCA - International Speech Communication Association |
Refereed: | Yes |
Identification Number: | 10.21437/interspeech.2021-1516 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:187597 |