Gavish, Y orcid.org/0000-0002-6025-5668, O'Connell, J and Benton, TG orcid.org/0000-0002-7448-1973 (2018) Quantifying and modelling decay in forecast proficiency indicates the limits of transferability in land-cover classification. Methods in Ecology and Evolution, 9 (2). pp. 235-244. ISSN 2041-210X
Abstract
1. The ability to provide reliable projections for the current and future distribution of land-cover is fundamental if we wish to protect and manage our diminishing natural resources. Two inter-related revolutions make map productions feasible at unprecedented resolutions - the availability of high-resolution remotely-sensed data and the development of machine-learning algorithms. However, ground-truthed data needed for training models is in most cases spatially and temporally clustered. Therefore, map production requires extrapolation of models from one place to another and the uncertainty cost of such extrapolation is rarely explored. In other words, the focus has mainly been on projections, and less on quantifying their reliability. 2. Using the concept of ‘forecast horizon’, we suggest that the predictability of land-cover classification models should be quantitatively explored as a continuum against distances measured along multiple dimensions – space, time, environmental and spectral. Focusing on ten agricultural sites from England and using models specifically designed to predict multivariate decay-curves, we ask: how does a model’s predictive performance decay with distance? More specifically, we explored if we could predict the proficiency (kappa statistics) of a model trained in one site when making predictions in another site based on the spatial, temporal, spectral and environmental distances between sites. 3. We found that model proficiency decays with distance between sites in each dimension. More importantly, we found for the first time, that it is possible to predict the performance a model transferred to or from a novel site will have, based on its distances from known sites. The spatial distance variables were the most important when predicting model transferability. 4. Exploring model transferability as a continuum may have multiple usages including predicting uncertainty values in space and time, prioritization of strategies for ground-truth data collection, and optimizing model characteristics for defined tasks.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2017 The Authors. Methods in Ecology and Evolution © 2017 British Ecological Society. This is the peer reviewed version of the following article: Gavish, Y., O'Connell, J. and Benton, T. G. (2017), Quantifying and modelling decay in forecast proficiency indicates the limits of transferability in land-cover classification. Methods in Ecology and Evolution., which has been published in final form at https://doi.org/10.1111/2041-210X.12870. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving. Uploaded in accordance with the publisher's self-archiving policy. |
Keywords: | community similarity; earth‐observation; forecast horizon; habitat mapping; predictive ecology; random forest; remote sensing; signature extension; species‐distribution models; uncertainty |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Biological Sciences (Leeds) > School of Biology (Leeds) The University of Leeds > Faculty of Environment (Leeds) > School of Geography (Leeds) > Ecology & Global Change (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 23 Aug 2017 11:13 |
Last Modified: | 17 Aug 2018 10:49 |
Status: | Published |
Publisher: | Wiley |
Identification Number: | 10.1111/2041-210X.12870 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:120509 |