Suvon, M.N.I., Tripathi, P.C., Alabed, S. et al. (2 more authors) (2023) Multimodal learning for predicting mortality in patients with pulmonary arterial hypertension. In: 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). 2022 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 06-08 Dec 2022, Las Vegas, NV, USA. Institute of Electrical and Electronics Engineers (IEEE) , pp. 2704-2710. ISBN 9781665468206
Abstract
Pulmonary Arterial Hypertension (PAH) is a lifethreatening disorder. The prediction of mortality in PAH patients can play a crucial role in the clinical management of this disease. The prediction of mortality from one modality is a difficult task that may only provide limited performance. Therefore, we propose a multimodal learning approach in this work to predict one-year mortality in PAH patients. We have utilised three modalities, which include extracted numerical imaging features, echo report categorical features, and echo report text features from Electronic Health Records (EHRs) of patients. We have proposed a feature integration module to combine features from multiple modalities. The text features have been extracted from the echo reports using the Bidirectional Encoder Representations from Transformers (BERT). An attention mechanism and a weighted summation method are also adopted during the process of feature integration. We have performed different experiments to evaluate the performance of the proposed framework for mortality prediction. The experimental results indicate that we can achieve the best AUC score of 0.89 for predicting one-year mortality by combining all three modalities. The source code of this paper is available at https://github.com/Mdnaimulislam/MultimodalTab.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | Data integration; Mortality prediction; Multimodal learning; Pulmonary Arterial Hypertension |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Computer Science (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 20 Jan 2023 14:08 |
Last Modified: | 17 Jan 2024 12:23 |
Status: | Published |
Publisher: | Institute of Electrical and Electronics Engineers (IEEE) |
Refereed: | Yes |
Identification Number: | 10.1109/bibm55620.2022.9995597 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:195233 |