Habli, Ibrahim orcid.org/0000-0003-2736-8238, Lawton, Tom and Porter, Zoe (2020) Artificial intelligence in health care: accountability and safety. Bulletin of the world health organization. pp. 251-256. ISSN 0042-9686
Abstract
The prospect of patient harm caused by the decisions made by an artificial intelligence-based clinical tool is something to which current practices of accountability and safety worldwide have not yet adjusted. We focus on two aspects of clinical artificial intelligence used for decision-making: moral accountability for harm to patients; and safety assurance to protect patients against such harm. Artificial intelligence-based tools are challenging the standard clinical practices of assigning blame and assuring safety. Human clinicians and safety engineers have weaker control over the decisions reached by artificial intelligence systems and less knowledge and understanding of precisely how the artificial intelligence systems reach their decisions. We illustrate this analysis by applying it to an example of an artificial intelligence-based system developed for use in the treatment of sepsis. The paper ends with practical suggestions for ways forward to mitigate these concerns. We argue for a need to include artificial intelligence developers and systems safety engineers in our assessments of moral accountability for patient harm. Meanwhile, none of the actors in the model robustly fulfil the traditional conditions of moral accountability for the decisions of an artificial intelligence system. We should therefore update our conceptions of moral accountability in this context. We also need to move from a static to a dynamic model of assurance, accepting that considerations of safety are not fully resolvable during the design of the artificial intelligence system before the system has been deployed.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2020 WHO. This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details. |
Keywords: | Safety,Artificial intelligence,Autonomous systems,Sepsis,ETHICS,Healthcare |
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) The University of York > Faculty of Arts and Humanities (York) > Philosophy (York) |
Depositing User: | Pure (York) |
Date Deposited: | 06 Apr 2020 15:40 |
Last Modified: | 23 Jan 2025 00:22 |
Published Version: | https://doi.org/10.2471/BLT.19.237487 |
Status: | Published |
Refereed: | Yes |
Identification Number: | 10.2471/BLT.19.237487 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:159102 |