(2023) Federated benchmarking of medical artificial intelligence with MedPerf. Nature Machine Intelligence. pp. 799-810. ISSN 2522-5839
Abstract
Medical artificial intelligence (AI) has tremendous potential to advance healthcare by supporting and contributing to the evidence-based practice of medicine, personalizing patient treatment, reducing costs, and improving both healthcare provider and patient experience. Unlocking this potential requires systematic, quantitative evaluation of the performance of medical AI models on large-scale, heterogeneous data capturing diverse patient populations. Here, to meet this need, we introduce MedPerf, an open platform for benchmarking AI models in the medical domain. MedPerf focuses on enabling federated evaluation of AI models, by securely distributing them to different facilities, such as healthcare organizations. This process of bringing the model to the data empowers each facility to assess and verify the performance of AI models in an efficient and human-supervised process, while prioritizing privacy. We describe the current challenges healthcare and AI communities face, the need for an open platform, the design philosophy of MedPerf, its current implementation status and real-world deployment, our roadmap and, importantly, the use of MedPerf with multiple international institutions within cloud-based technology and on-premises scenarios. Finally, we welcome new contributions by researchers and organizations to further strengthen MedPerf as an open benchmarking platform.
Metadata
Item Type: | Article |
---|---|
Dates: |
|
Institution: | The University of York |
Academic Units: | The University of York > Faculty of Sciences (York) > Computer Science (York) |
Depositing User: | Pure (York) |
Date Deposited: | 30 May 2024 11:10 |
Last Modified: | 25 Mar 2025 00:15 |
Published Version: | https://doi.org/10.1038/s42256-023-00652-2 |
Status: | Published |
Refereed: | Yes |
Identification Number: | 10.1038/s42256-023-00652-2 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:212963 |