Shalom, E.S. orcid.org/0000-0001-8762-3726, Kim, H., van der Heijden, R.A. orcid.org/0000-0001-6964-2536 et al. (51 more authors) (2023) The ISMRM Open Science Initiative for Perfusion Imaging (OSIPI): Results from the OSIPI–Dynamic Contrast-Enhanced challenge. Magnetic Resonance in Medicine. ISSN 0740-3194
Abstract
Purpose: K trans has often been proposed as a quantitative imaging biomarker for diagnosis,prognosis,andtreatmentresponseassessmentforvarioustumors.Noneofthe many software tools for K trans quantification are standardized. The ISMRM OpenScience Initiative for Perfusion Imaging–Dynamic Contrast-Enhanced (OSIPI-DCE)challenge was designed to benchmark methods to better help the efforts to standardize K trans measurement.
Methods: A framework was created to evaluate K trans values produced by DCE-MRI analysis pipelines to enable benchmarking. The perfusion MRI community was invited to apply their pipelines for K trans quantification in glioblastoma from clinical and synthetic patients. Submissions were required to include the entrants’ K trans values, the applied software, and a standard operating procedure. These were evaluated using the proposed OSIPIgold score defined with accuracy, repeatability, and reproducibility components.
Results: Across the 10 received submissions, the OSIPIgold score ranged from28% to 78% with a 59% median. The accuracy, repeatability, and reproducibility scores ranged from 0.54 to 0.92, 0.64 to 0.86, and 0.65 to 1.00, respectively(0–1=lowest–highest). Manual arterial input function selection markedly affected the reproducibility and showed greater variability in K trans analysis than automated methods. Furthermore, provision of a detailed standard operating procedure was critical for higher reproducibility.
Conclusions: This study reports results from the OSIPI-DCE challenge and high-lights the high inter-software variability within K trans estimation, providing a framework for ongoing benchmarking against the scores presented. Through this challenge, the participating teams were ranked based on the performance of their software tools in the particular setting of this challenge. In a real-world clinical setting, many of these tools may perform differently with different benchmarking methodology.
Metadata
Authors/Creators: |
|
---|---|
Copyright, Publisher and Additional Information: | This is the peer reviewed version of the following article: Shalom ES, Kim H, van der Heijden RA, et al. The ISMRM Open Science Initiative for Perfusion Imaging (OSIPI): Results from the OSIPI–Dynamic Contrast-Enhanced challenge. Magn Reson Med. 2023; 1-19. doi: 10.1002/mrm.29909, which has been published in final form at https://doi.org/10.1002/mrm.29909. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions. This article may not be enhanced, enriched or otherwise transformed into a derivative work, without express permission from Wiley or by statutory rights under applicable legislation. Copyright notices must not be removed, obscured or modified. The article must be linked to Wiley’s version of record on Wiley Online Library and any embedding, framing or otherwise making available the article or pages thereof by third parties from platforms, services and websites other than Wiley Online Library must be prohibited. |
Keywords: | DCE-MRI; challenge; data analysis; glioblastoma; open-science; perfusion |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Physics and Astronomy (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 16 Jan 2024 14:34 |
Last Modified: | 16 Jan 2024 14:34 |
Published Version: | https://onlinelibrary.wiley.com/doi/10.1002/mrm.29... |
Status: | Published online |
Publisher: | Wiley |
Identification Number: | https://doi.org/10.1002/mrm.29909 |
Related URLs: |
Download
Filename: cleancopy_Manuscript_Revision2.pdf