Ryan Conmy, Philippa Mary orcid.org/0000-0003-1307-5207, Ozturk, Berk, Habli, Ibrahim orcid.org/0000-0003-2736-8238 et al. (1 more author) (2023) The Impact of Training Data Shortfalls on Safety of AI-based Clinical Decision Support Systems. In: SAFECOMP 2023 (42nd International Conference on Computer Safety, Reliability and Security). International Conference on Computer Safety, Reliability and Security, 20-22 Sep 2023 , FRA
Abstract
Decision support systems with Artificial intelligence (AI) and specifically Machine Learning (ML) components present many challenges when assuring trust in operational performance, particularly in a safety-critical domain such as healthcare. During operation the Human in/on The Loop (HTL) may need assistance in determining when to trust the ML output and when to override it, particularly to prevent hazardous situations. In this paper, we consider how issues with training data shortfalls can cause varying safety performance in ML. We present a case study using an ML-based clinical decision support system for Type-2 diabetes related co-morbidity prediction (DCP). The DCP ML component is trained using real patient data, but the data was taken from a very large live database gathered over many years, and the records vary in distribution and completeness. Research developing similar clinical predictor systems describe different methods to compensate for training data shortfalls, but concentrate only on fixing the data to maximise the ML performance without considering a system safety perspective. This means the impact of the ML's varying performance is not fully understood at the system level. Further, methods such as data imputation can introduce a further risk of bias which is not addressed. This paper combines the use of ML data shortfall compensation measures with exploratory safety analysis to ensure all means of reducing risk are considered. We demonstrate that together these provide a richer picture allowing more effective identification and mitigation of risks from training data shortfalls.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | This is an author-produced version of the published paper. Uploaded in accordance with the University’s Research Publications and Open Access policy. |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Depositing User: | Pure (York) |
Date Deposited: | 07 Jun 2023 08:10 |
Last Modified: | 04 Feb 2025 00:04 |
Status: | Published |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:200068 |
Download
Filename: SAFECOMP_Diabetes_paper_2023_corrections.pdf
Description: SAFECOMP_Diabetes_paper_2023
Licence: CC-BY 2.5