Coordinated control of wind turbine and hybrid energy storage system based on multi-agent deep reinforcement learning for wind power smoothing

Due to the inherent fluctuation, wind power integration into the large-scale grid brings instability and other safety risks. In this study by using a multi-agent deep reinforcement learning, a new coordinated control strategy of a wind turbine (WT) and a hybrid energy storage system (HESS) is proposed for the purpose of wind power smoothing, where the HESS is combined with the rotor kinetic energy and pitch control of the wind turbine. Firstly, the wind power output is forecasted and decomposed into high, medium, and low-frequency components through an adaptive variational mode decomposition (VMD). The optimal secondary allocation of the reference power of the high-frequency and medium-frequency is then performed through a multi-agent twin-delay deep deterministic policy gradient algorithm (MATD3) to smooth the power output. To improve the exploration ability of the learning, a new type of α-“stable” Lévy noises is injected into the action space of the MATD3 and the noises are dynamically adjusted. Simulation and RT-LAB semi-physical real-time experimental results show that the proposed control strategy can make full use of the smoothing output power of the WT and HESS combined generation system reasonably, extend the life of the energy storage elements and reduce the wear of the WT.