Malinin, A., Knill, K., Ragni, A. orcid.org/0000-0003-0634-4456 et al. (2 more authors) (2017) An attention based model for off-topic spontaneous spoken response detection : an initial study. In: Engwall, O. and Lopes, J.D., (eds.) 7th ISCA Workshop on Speech and Language Technology in Education (SLaTE). 7th ISCA Workshop on Speech and Language Technology in Education (SLaTE), 25-26 Aug 2017, Stockholm, Sweden. SLaTE Conference Proceedings . ISCA , pp. 144-149.
Abstract
Automatic spoken language assessment systems are gaining popularity due to the rising demand for English second language learning. Current systems primarily assess fluency and pronunciation, rather than semantic content and relevance of a candidate's response to a prompt. However, to increase reliability and robustness, relevance assessment and off-topic response detection are desirable, particularly for spontaneous spoken responses to open-ended prompts. Previously proposed approaches usually require prompt-response pairs for all prompts. This limits flexibility as example responses are required whenever a new test prompt is introduced. This paper presents a initial study of an attention based neural model which assesses the relevance of prompt-response pairs without the need to see them in training. This model uses a bidirectional Recurrent Neural Network (BiRNN) embedding of the prompt to compute attention over the hidden states of a BiRNN embedding of the response. The resulting fixed-length embedding is fed into a binary classifier to predict relevance of the response. Due to a lack of off-topic responses, negative examples for both training and evaluation are created by randomly shuffling prompts and responses. On spontaneous spoken data this system is able to assess relevance to both seen and unseen prompts.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2017 International Speech Communication Association (ISCA). Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | Spoken Language Assessment; Relevance Assessment; Deep Learning |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 11 Nov 2019 12:36 |
Last Modified: | 11 Nov 2019 12:36 |
Status: | Published |
Publisher: | ISCA |
Series Name: | SLaTE Conference Proceedings |
Refereed: | Yes |
Identification Number: | 10.21437/SLaTE.2017-25 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:152830 |