Uncertainty-aware deep kernel Gaussian process estimation for proximal policy optimisation control in mixed autonomy traffic systems

Autonomous vehicle (AV) control with reinforcement learning (RL) policy generation has been an active area of research. However, most state-of-the-art methods assume that data is available and sufficient for training, whereas in a practical traffic system, some data may be missing due to sensor failures or communication losses. This paper presents a probabilistic prediction approach leveraging deep kernel learning (DKL) to handle uncertainties arising from missing data during RL policy learning. The prediction result is induced into the policy learning of a proximal policy optimisation (PPO). DKL uses data from the initial episodes of policy learning to train a probabilistic prior before using it for the rest of the policy learning. The proposed framework is evaluated through simulations on a single-lane ring road in a mixed-autonomy environment. The results show that the approach retained approximately 98% of the cumulative reward achieved under the ’No Missing’ condition. It also outperforms the PPO with standard Gaussian processes (GPs) as its prediction method and linear extrapolation for handling the missing data.