Hierarchical reinforcement learning for cost-effective railway microgrid management

Railway electrification is emerging worldwide as a key enabler of high-capacity, low-carbon transport. However, this transition introduces substantially more inflexible traction loads while leaving much of the regenerative braking energy underutilized. This may further place additional strain on already stretched power grids, degrading traction voltage stability and the overall system energy performance. This paper presents a novel dynamic and economic energy management solution for railway power supply systems, which integrates a railway microgrid at the terminal station of a feeding zone of the traction network and coordinates on-site energy storage systems, renewable generation, and local utility grid resources to enhance the overall system efficiency while sustaining desired traction voltage levels. A hierarchical reinforcement learning framework is designed to determine the optimal strategies. In this framework, the upper-level agent provides time varying allowable voltage ranges to the lower-level agent, which serve as reference guidance for energy management. Subsequently, the lower-level agent treats these received voltage bands as operational limits, enabling it to coordinate energy scheduling while maintaining voltage regulation performance. Case studies conducted on a representative UK feeding zone demonstrate that the proposed solution can effectively improve the traction voltage level at a weak node in the traction network while cutting the grid electricity procurement costs by as much as 35.2%, highlighting its practical significance for railway electrification.