Newman-Griffis, D., Rauchberg, J.S., Alharbi, R. et al. (2 more authors) (2023) Definition drives design: disability models and mechanisms of bias in AI technologies. First Monday, 28 (1). ISSN 1396-0466
Abstract
The increasing deployment of artificial intelligence (AI) tools to inform decision-making across diverse areas including healthcare, employment, social benefits, and government policy, presents a serious risk for disabled people, who have been shown to face bias in AI implementations. While there has been significant work on analysing and mitigating algorithmic bias, the broader mechanisms of how bias emerges in AI applications are not well understood, hampering efforts to address bias where it begins. In this article, we illustrate how bias in AI-assisted decision-making can arise from a range of specific design decisions, each of which may seem self-contained and non-biasing when considered separately. These design decisions include basic problem formulation, the data chosen for analysis, the use the AI technology is put to, and operational design elements in addition to the core algorithmic design. We draw on three historical models of disability common to different decision-making settings to demonstrate how differences in the definition of disability can lead to highly distinct decisions on each of these aspects of design, leading in turn to AI technologies with a variety of biases and downstream effects. We further show that the potential harms arising from inappropriate definitions of disability in fundamental design stages are further amplified by alack of transparency and disabled participation throughout the AI design process. Our analysis provides a framework for critically examining AI technologies in decision-making contexts and guiding the development of a design praxis for disability-related AI analytics. We put forth this article to provide key questions to facilitate disability-led design and participatory development to produce more fair and equitable AI technologies in disability-related contexts.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2022 First Monday. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike Licence (https://creativecommons.org/licenses/by-nc-sa/4.0/) |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Social Sciences (Sheffield) > Information School (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 17 Feb 2023 16:02 |
Last Modified: | 17 Feb 2023 16:02 |
Status: | Published |
Publisher: | University of Illinois Libraries |
Refereed: | Yes |
Identification Number: | 10.5210/fm.v28i1.12903 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:196472 |