Aerobatic tic-toc control of planar quadcopters via reinforcement learning

This letter studies aerobatic tic-toc control of quadcopters. Tic-toc control enables rotorcraft to fly almost in the vertical plane rather than the horizontal plane. It is one of the most challenging manoeuvrers to achieve autonomously. The problem has to our knowledge not yet been studied for quadcopters. Studying it could expand their flight envelope and improve their performance in extreme, aerobatic flight tasks. In this letter, we employ a deep deterministic gradient policy approach to train reinforcement learning (RL) controllers based on carefully designed rewards. The obtained RL controllers are shown to generate two flight modes, spin and tic-toc. We analyse the properties of these flight modes and screen out unfavourable RL controllers. The qualified RL controller is then enhanced by combining it with PID and LQR controllers which achieves better flight performance and enables the quadcopter to track a moving reference point and recover to hovering flight status. Physical simulations using Simscape are presented to verify the proposed approach.