A Sparse Bayesian Learning-Based OTFS Channel Estimation for 6G Wireless System

Orthogonal time-frequency space (OTFS) modulation is considered a promising candidate for 6G wireless systems due to its superior performance in high mobility scenarios and resilience to Doppler and multipath effects. Reliable channel estimation is crucial to fully realising the potential of OTFS, but most of the existing methods suffer from limited accuracy and high computational complexity. This letter proposes a channel estimation method exploiting generalized approximate message passing sparse Bayesian learning with geometric mean decomposition (GMD-GAMP-SBL). Unlike conventional methods relying on iterative matrix inversion, the proposed approach integrates GAMP and GMD to enhance estimation accuracy and computational efficiency. Specifically, GAMP reduces complexity by replacing expectation-maximization’s (EM) matrix inversion, while GMD preconditions the model to reduce the impact of sensing matrix correlations, which in turn improves the estimation accuracy. Simulation results demonstrate that the proposed GMD-GAMP-SBL achieves channel estimation accuracy that is nearly identical to that of the conventional SBL algorithm, while its computational complexity and runtime are substantially lower. This favorable trade-off positions it as a practical and efficient candidate for OTFS systems.