Quality of reporting in AI cardiac MRI segmentation studies – a systematic review and recommendations for future studies

Abstract

Background: There has been a rapid increase in the number of Artificial Intelligence (AI) studies of cardiac MRI (CMR) segmentation aiming to automate image analysis. However, advancement and clinical translation in this field depend on researchers presenting their work in a transparent and reproducible manner. This systematic review aimed to evaluate the quality of reporting in AI studies involving CMR segmentation.

Methods: MEDLINE and EMBASE were searched for AI CMR segmentation studies in April 2022. Any fully automated AI method for segmentation of cardiac chambers, myocardium or scar on CMR was considered for inclusion. For each study, compliance with the Checklist for Artificial Intelligence in Medical Imaging (CLAIM) was assessed. The CLAIM criteria were grouped into study, dataset, model and performance description domains.

Results: 209 studies published between 2012 and 2022 were included in the analysis. Studies were mainly published in technical journals (58%), with the majority (57%) published since 2019. Studies were from 37 different countries, with most from China (26%), the United States (18%) and the United Kingdom (11%). Short axis CMR images were most frequently used (70%), with the left ventricle the most commonly segmented cardiac structure (49%). Median compliance of studies with CLAIM was 67% (IQR 59–73%). Median compliance was highest for the model description domain (100%, IQR 80–100%) and lower for the study (71%, IQR 63–86%), dataset (63%, IQR 50–67%) and performance (60%, IQR 50–70%) description domains.

Conclusion: This systematic review highlights important gaps in the literature of CMR studies using AI. We identified key items missing—most strikingly poor description of patients included in the training and validation of AI models and inadequate model failure analysis—that limit the transparency, reproducibility and hence validity of published AI studies. This review may support closer adherence to established frameworks for reporting standards and presents recommendations for improving the quality of reporting in this field.

Systematic Review Registration: [www.crd.york.ac.uk/prospero/], identifier [CRD42022279214].

Metadata

Item Type:	Article
Authors/Creators:	Alabed, S. https://orcid.org/0000-0002-9960-7587 Maiter, A. Salehi, M. Mahmood, A. Daniel, S. Jenkins, S. Goodlad, M. Sharkey, M. https://orcid.org/0000-0001-9851-0014 Mamalakis, M. Rakocevic, V. Dwivedi, K. Assadi, H. Wild, J.M. https://orcid.org/0000-0002-7246-8660 Lu, H. O’Regan, D.P. van der Geest, R.J. Garg, P. Swift, A.J. https://orcid.org/0000-0002-8772-409X
Copyright, Publisher and Additional Information:	© 2022 Alabed, Maiter, Salehi, Mahmood, Daniel, Jenkins, Goodlad, Sharkey, Mamalakis, Rakocevic, Dwivedi, Assadi, Wild, Lu, O’Regan, van der Geest, Garg and Swift. This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Keywords:	artificial intelligence; machine learning; cardiac MRI; segmentation; systematic review; quality; reporting
Dates:	Published: 15 July 2022 Published (online): 15 July 2022 Accepted: 30 June 2022
Institution:	The University of Sheffield
Academic Units:	The University of Sheffield > Sheffield Teaching Hospitals
Funding Information:	Funder Grant number WELLCOME TRUST (THE) 205188/Z/16/Z National Institute for Health Research AI_AWARD01706 DEPARTMENT OF HEALTH AND SOCIAL CARE AI_AWARD01706 WELLCOME TRUST (THE) 215799/Z/19/Z
Depositing User:	Symplectic Sheffield
Date Deposited:	01 Aug 2022 15:05
Last Modified:	01 Aug 2022 15:05
Status:	Published
Publisher:	Frontiers Media
Refereed:	Yes
Identification Number:	10.3389/fcvm.2022.956811
Open Archives Initiative ID (OAI ID):	oai:eprints.whiterose.ac.uk:189566

Download

Published Version

Filename: fcvm-09-956811.pdf

Licence: CC-BY 4.0

CLICK TO DOWNLOAD

CORE (COnnecting REpositories)

Quality of reporting in AI cardiac MRI segmentation studies – a systematic review and recommendations for future studies

Abstract

Metadata

Download

Published Version

Export

Statistics