Farooq, M.U. and Hain, T. orcid.org/0000-0003-0939-3464 (2022) Investigating the impact of cross-lingual acoustic-phonetic similarities on multilingual speech recognition. In: Interspeech 2022 - 23rd Annual Conference of the International Speech Communication Association. Interspeech 2022 - Human and Humanizing Speech Technology, 18-22 Sep 2022, Incheon, Korea. International Speech Communication Association , pp. 3849-3853.
Abstract
Multilingual speech recognition systems mostly benefit low resource languages but suffer degradation in the performance of several languages relative to their monolingual counterparts. Limited studies have focused on understanding the languages behaviour in the multilingual speech recognition setups. In this paper, a novel data-driven approach is proposed to investigate the cross-lingual acoustic-phonetic similarities. This technique measures the similarities between posterior distributions from various monolingual acoustic models against a target speech signal. Deep neural networks are trained as mapping networks to transform the distributions from different acoustic models into a directly comparable form. The analysis observes that the languages ‘closeness' can not be truly estimated by the volume of overlapping phonemes set. Entropy analysis of the proposed mapping networks exhibits that a language with lesser overlap can be more amenable to cross-lingual transfer, and hence more beneficial in the multilingual setup. Finally, the proposed posterior transformation approach is leveraged to fuse monolingual models for a target language. A relative improvement of ∼8% over monolingual counterpart is achieved.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2022 ISCA. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | automatic speech recognition; multilingual; acoustic-phonetic similarities; model fusion |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 28 Jul 2022 12:45 |
Last Modified: | 01 Nov 2022 18:18 |
Status: | Published |
Publisher: | International Speech Communication Association |
Refereed: | Yes |
Identification Number: | 10.21437/Interspeech.2022-10916 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:189118 |