Schobs, L. orcid.org/0000-0001-6942-1194, Swift, A.J. orcid.org/0000-0002-8772-409X and Lu, H. orcid.org/0000-0002-0349-2181 (2022) Uncertainty estimation for heatmap-based landmark localization. IEEE Transactions on Medical Imaging. ISSN 0278-0062
Abstract
Automatic anatomical landmark localization has made great strides by leveraging deep learning methods in recent years. The ability to quantify the uncertainty of these predictions is a vital component needed for these methods to be adopted in clinical settings, where it is imperative that erroneous predictions are caught and corrected. We propose Quantile Binning, a data-driven method to categorize predictions by uncertainty with estimated error bounds. Our framework can be applied to any continuous uncertainty measure, allowing straightforward identification of the best subset of predictions with accompanying estimated error bounds. We facilitate easy comparison between uncertainty measures by constructing two evaluation metrics derived from Quantile Binning. We compare and contrast three epistemic uncertainty measures (two baselines, and a proposed method combining aspects of the two), derived from two heatmap-based landmark localization model paradigms (U-Net and patch-based). We show results across three datasets, including a publicly available Cephalometric dataset. We illustrate how filtering out gross mispredictions caught in our Quantile Bins significantly improves the proportion of predictions under an acceptable error threshold. Finally, we demonstrate that Quantile Binning remains effective on landmarks with high aleatoric uncertainty caused by inherent landmark ambiguity, and offer recommendations on which uncertainty measure to use and how to use it. The code and data are available at https://github.com/schobs/qbin.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | confidence; heatmaps; landmark localization; Uncertainty estimation; U-Net |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) The University of Sheffield > Faculty of Medicine, Dentistry and Health (Sheffield) > Department of Infection, Immunity and Cardiovascular Disease |
Funding Information: | Funder Grant number WELLCOME TRUST (THE) 205188/Z/16/Z WELLCOME TRUST (THE) 215799/Z/19/Z Engineering and Physical Sciences Research Council 2274702 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 12 Jan 2023 16:55 |
Last Modified: | 16 Nov 2023 01:13 |
Status: | Published |
Publisher: | Institute of Electrical and Electronics Engineers (IEEE) |
Refereed: | Yes |
Identification Number: | 10.1109/tmi.2022.3222730 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:195227 |