Aftab, Haris orcid.org/0000-0001-7981-1743, Gautam, Vibhu, Hawkins, Richard David orcid.org/0000-0001-7347-3413 et al. (2 more authors) (2022) Robust Intent Classification using Bayesian LSTM for Clinical Conversational Agents (CAs). In: MobiHealth 2021:Wireless Mobile Communication and Healthcare. 10th EAI International Conference on Wireless Mobile Communication and Healthcare, 13-14 Nov 2021 Springer , CHN , pp. 106-118.
Abstract
Conversational Agents (CAs) are software programs that replicate human conversations using machine learning (ML) and natural language processing (NLP). CAs are currently being utilised for diverse clinical applications such as symptom checking, health monitoring, medical triage and diagnosis. Intent classification (IC) is an essential task of understanding user utterance in CAs which makes use of modern deep learning (DL) methods. Because of the inherent model uncertainty associated with those methods, accuracy alone cannot be relied upon in clinical applications where certain errors may compromise patient safety. In this work, we employ Bayesian Long Short-Term Memory Networks (LSTMs) to calculate model uncertainty for IC, with a specific emphasis on symptom checker CAs. This method provides a certainty measure with IC prediction that can be utilised in assuring safe response from CAs. We evaluated our method on in-distribution (ID) and out-of-distribution (OOD) data and found mean uncer-tainty to be much higher for OOD data. These findings suggest that our method is robust to OOD utterances and can detect non-understanding errors in CAs.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2022 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details |
Keywords: | Conversational Agents (CAs),Machine Learning,Model Uncertainty,Out-of-Distribution (OOD),Healthcare,Patient Safety |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Depositing User: | Pure (York) |
Date Deposited: | 06 Jan 2022 09:40 |
Last Modified: | 02 Apr 2025 23:34 |
Published Version: | https://doi.org/10.1007/978-3-031-06368-8_8 |
Status: | Published |
Publisher: | Springer |
Identification Number: | 10.1007/978-3-031-06368-8_8 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:182119 |
Download
Filename: Robust_intent_classification_using_bayesian_lstm_for_clinical_conversational_agents.pdf
Description: Robust_intent_classification_using_bayesian_lstm_for_clinical_conversational_agents.pdf