Aftab, Haris orcid.org/0000-0001-7981-1743, Hammad Hussain Shah, Syed and Habli, Ibrahim orcid.org/0000-0003-2736-8238 (2021) Classification of Failures in the Perception of Conversational Agents (CAs) and their Implications on Patient Safety. In: Mantas, John, Stoicu-Tivadar, Lăcrămioara, Chronaki, Catherine Chronaki, Hasman, Arie, Weber, Patrick, Gallos, Parisis, Crişan-Vida, Mihaela, Zoulias, Emmanouil and Sorina Chirila, Oana, (eds.) Public Health and Informatics:Proceedings of MIE 2021. Studies in Health Technology and Informatics . IOS Press , pp. 659-663.
Abstract
The use of Conversational agents (CAs) in healthcare is an emerging field. These CAs seem to be effective in accomplishing administrative tasks, e.g. providing locations of care facilities and scheduling appointments. Modern CAs use machine learning (ML) to recognize, understand and generate a response. Given the criticality of many healthcare settings, ML and other component errors may result in CA failures and may cause adverse effects on patients. Therefore, in-depth assurance is required before the deployment of ML in critical clinical applications, e.g. management of medication dose or medical diagnosis. CA safety issues could arise due to diverse causes, e.g. related to user interactions, environmental factors and ML errors. In this paper, we classify failures of perception (recognition and understanding) of CAs and their sources. We also present a case study of a CA used for calculating insulin dose for gestational diabetes mellitus (GDM) patients. We then correlate identified perception failures of CAs to potential scenarios that might compromise patient safety.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Editors: |
|
Copyright, Publisher and Additional Information: | © 2021 European Federation for Medical Informatics (EFMI) and IOS Press |
Keywords: | conversational agents,Healthcare,Machine learning,PATIENT SAFETY |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Depositing User: | Pure (York) |
Date Deposited: | 02 Jun 2021 16:00 |
Last Modified: | 19 Dec 2024 00:26 |
Published Version: | https://doi.org/10.3233/shti210253 |
Status: | Published |
Publisher: | IOS Press |
Series Name: | Studies in Health Technology and Informatics |
Identification Number: | 10.3233/shti210253 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:174796 |
Download
Filename: SHTI_281_SHTI210253.pdf
Description: Classification of Failures in the Perception of Conversational Agents (CAs) and Their Implications on Patient Safety
Licence: CC-BY-NC 2.5