Mai, G., Huang, W. orcid.org/0000-0002-3208-4208, Sun, J. et al. (11 more authors) (2024) On the Opportunities and Challenges of Foundation Models for GeoAI (Vision Paper). ACM Transactions on Spatial Algorithms and Systems, 10 (2). 11. ISSN 2374-0353
Abstract
Large pre-trained models, also known as foundation models (FMs), are trained in a task-agnostic manner on large-scale data and can be adapted to a wide range of downstream tasks by fine-tuning, few-shot, or even zero-shot learning. Despite their successes in language and vision tasks, we have not yet seen an attempt to develop foundation models for geospatial artificial intelligence (GeoAI). In this work, we explore the promises and challenges of developing multimodal foundation models for GeoAI. We first investigate the potential of many existing FMs by testing their performances on seven tasks across multiple geospatial domains, including Geospatial Semantics, Health Geography, Urban Geography, and Remote Sensing. Our results indicate that on several geospatial tasks that only involve text modality, such as toponym recognition, location description recognition, and US state-level/county-level dementia time series forecasting, the task-agnostic large learning models (LLMs) can outperform task-specific fully supervised models in a zero-shot or few-shot learning setting. However, on other geospatial tasks, especially tasks that involve multiple data modalities (e.g., POI-based urban function classification, street view image–based urban noise intensity classification, and remote sensing image scene classification), existing FMs still underperform task-specific models. Based on these observations, we propose that one of the major challenges of developing an FM for GeoAI is to address the multimodal nature of geospatial tasks. After discussing the distinct challenges of each geospatial data modality, we suggest the possibility of a multimodal FM that can reason over various types of geospatial data through geospatial alignments. We conclude this article by discussing the unique risks and challenges to developing such a model for GeoAI.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2024 Copyright held by the owner/author(s). This is an open access article under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited. |
Keywords: | Foundation models, geospatial artificial intelligence, multimodal learning |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Environment (Leeds) > School of Geography (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 13 Mar 2025 13:33 |
Last Modified: | 13 Mar 2025 13:33 |
Status: | Published |
Publisher: | Association for Computing Machinery (ACM) |
Identification Number: | 10.1145/3653070 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:224348 |