Azarmi, M., Rezaei, M. orcid.org/0000-0003-3892-421X and Wang, H. (Accepted: 2025) Pedestrian Intention Prediction via Vision-Language Foundation Models. In: IEEE Intelligent Vehicles Symposium Proceedings. IEEE Intelligent Vehicles Symposium 2025, 22-25 Jun 2025, Napoca, Romania. IEEE (In Press)
Abstract
Prediction of pedestrian crossing intention is a critical function in autonomous vehicles. Conventional vision based methods of crossing intention prediction often struggle with generalizability, context understanding, and causal reasoning. This study explores the potential of vision-language foundation models (VLFMs) for predicting pedestrian crossing intentions by integrating multimodal data through hierarchical prompt templates. The methodology incorporates contextual information, including visual frames, physical cues observations, and ego-vehicle dynamics, into systematically refined prompts to guide VLFMs effectively in intention prediction. Experiments were conducted on three common datasets—JAAD, PIE, and FU-PIP. Results demonstrate that incorporating vehicle speed, its variations over time, and time-conscious prompts significantly enhances the prediction accuracy up to 19.8%. Additionally, optimised prompts generated via an automatic prompt engineering framework yielded 12.5% further accuracy gains. These findings highlight the superior performance of VLFMs compared to conventional vision-based models, offering enhanced generalisation and contextual understanding for autonomous driving applications.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | This is an author produced version of a conference paper accepted for publication in IEEE Symposium on Intelligent Vehicle Proceedings made available under the terms of the Creative Commons Attribution License (CC-BY), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited. |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Environment (Leeds) > Institute for Transport Studies (Leeds) |
Funding Information: | Funder Grant number EU - European Union 101006664 |
Depositing User: | Symplectic Publications |
Date Deposited: | 29 Jul 2025 13:49 |
Last Modified: | 29 Jul 2025 13:49 |
Status: | In Press |
Publisher: | IEEE |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:229640 |