Reinforcement learning-based output stabilization control for nonlinear systems with generalized disturbances

This paper proposes a new disturbance observer (DO)-based reinforcement learning (RL) control approach for nonlinear systems with unmatched (generalized) disturbances. While a nonlinear disturbance observer (NDO) is utilized to measure the plant uncertainties, disturbances can exist in the plant via distinct channels from those of the control signals; so-called mismatched disturbances are theoretically difficult to attenuate within the channel of the system's states. A generalized disturbance observer-based compensator is implemented to address the uncertainty cancellation problem by removing the influence of uncertainties from the output channels. Con-currently, a composite actor-critic RL scheme is utilized for approximating the optimal control policy as well as the ideal value function pertaining to the compensated system by solving a Hamilton-Jacobi-Bellman (HJB) equation for both online and offline iterations simultaneously. Stability analysis verifies the convergence of the proposed framework. Simulation results are included to illustrate the effectiveness of the proposed scheme.