Liemohn, M.W., McCollough, J.P., Jordanova, V.K. et al. (18 more authors) (2018) Model evaluation guidelines for geomagnetic index predictions. Space Weather, 16 (12). pp. 2079-2102. ISSN 1539-4956
Abstract
Geomagnetic indices are convenient quantities that distill the complicated physics of some region or aspect of near‐Earth space into a single parameter. Most of the best‐known indices are calculated from ground‐based magnetometer data sets, such as Dst, SYM‐H, Kp, AE, AL, and PC. Many models have been created that predict the values of these indices, often using solar wind measurements upstream from Earth as the input variables to the calculation. This document reviews the current state of models that predict geomagnetic indices and the methods used to assess their ability to reproduce the target index time series. These existing methods are synthesized into a baseline collection of metrics for benchmarking a new or updated geomagnetic index prediction model. These methods fall into two categories: (1) fit performance metrics such as root‐mean‐square error and mean absolute error that are applied to a time series comparison of model output and observations and (2) event detection performance metrics such as Heidke Skill Score and probability of detection that are derived from a contingency table that compares model and observation values exceeding (or not) a threshold value. A few examples of codes being used with this set of metrics are presented, and other aspects of metrics assessment best practices, limitations, and uncertainties are discussed, including several caveats to consider when using geomagnetic indices.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2018 American Geophysical Union. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | Space weather; geomagnetic indices; metrics; statistical analysis; forecasting; ROC curve |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Automatic Control and Systems Engineering (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 03 Jan 2019 13:38 |
Last Modified: | 06 May 2019 00:44 |
Published Version: | https://doi.org/10.1029/2018SW002067 |
Status: | Published |
Publisher: | American Geophysical Union |
Refereed: | Yes |
Identification Number: | 10.1029/2018SW002067 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:139033 |