Huang, R, Lin, Z, Dou, H orcid.org/0000-0001-8628-5489 et al. (10 more authors) (2021) AW3M: An auto-weighting and recovery framework for breast cancer diagnosis using multi-modal ultrasound. Medical Image Analysis, 72. 102137. ISSN 1361-8415
Abstract
Recently, more clinicians have realized the diagnostic value of multi-modal ultrasound in breast cancer identification and began to incorporate Doppler imaging and Elastography in the routine examination. However, accurately recognizing patterns of malignancy in different types of sonography requires expertise. Furthermore, an accurate and robust diagnosis requires proper weights of multi-modal information as well as the ability to process missing data in practice. These two aspects are often overlooked by existing computer-aided diagnosis (CAD) approaches. To overcome these challenges, we propose a novel framework (called AW3M) that utilizes four types of sonography (i.e. B-mode, Doppler, Shear-wave Elastography, and Strain Elastography) jointly to assist breast cancer diagnosis. It can extract both modality-specific and modality-invariant features using a multi-stream CNN model equipped with self-supervised consistency loss. Instead of assigning the weights of different streams empirically, AW3M automatically learns the optimal weights using reinforcement learning techniques. Furthermore, we design a light-weight recovery block that can be inserted to a trained model to handle different modality-missing scenarios. Experimental results on a large multi-modal dataset demonstrate that our method can achieve promising performance compared with state-of-the-art methods. The AW3M framework is also tested on another independent B-mode dataset to prove its efficacy in general settings. Results show that the proposed recovery block can learn from the joint distribution of multi-modal features to further boost the classification accuracy given single modality input during the test.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Keywords: | Multi-modality; Ultrasound; Breast cancer; Reinforcement learning; Missing data |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 08 Nov 2021 10:00 |
Last Modified: | 26 Nov 2021 09:48 |
Status: | Published |
Publisher: | Elsevier |
Identification Number: | 10.1016/j.media.2021.102137 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:180018 |