UrbanExpert: Task-Conditioned Multi-Modal Fusion via Semantic Expert Routing for Urban Socioeconomic Prediction

Predicting urban indicators from multi-modal sensing data requires fusing heterogeneous modalities, yet different prediction tasks rely on different feature combinations while semantically related tasks share common cues. Existing approaches either train task-specific models that cannot share knowledge across tasks, or learn a single task-agnostic representation that ignores inter-task semantic correlations. Both paradigms also assume complete modality observations, which rarely holds in practice. We propose UrbanExpert, a multi-task framework that explicitly conditions multi-modal fusion on task semantics while accommodating incomplete modality observations. UrbanExpert first encodes each available modality into a shared token sequence via pretrained encoders, and recovers missing modality representations through a cross-modal reconstruction module to handle incomplete observations. A task-conditioned Mixture-of-Experts module then routes each task to a relevant subset of experts based on the semantic affinity between the task description and learnable expert prototypes, producing task-conditioned region embeddings. Task-specific prediction heads map the embeddings to target indicators, and the model is jointly optimized with prediction, reconstruction, and routing regularization objectives. Experiments on two real-world urban datasets show that UrbanExpert consistently outperforms both single-task and multi-task baselines, and that the learned routing structure generalizes effectively to unseen tasks and cities.