OREO: Open RAN Energy Optimisation via Deep Reinforcement Learning for 6G Networks

This paper introduces an intelligent energy optimisation framework designed to enable sustainable operation in 6G O-RAN heterogeneous networks while maintaining stringent QoS guarantees for both terrestrial and non-terrestrial users. The framework utilises a proximal policy optimisation (PPO) reinforcement learning (RL) model deployed as an rApp in the Non-RT RIC. It is further enhanced by a hierarchical rApp-xApp architecture for robust real-time execution and to jointly optimise radio unit activation states and user association policies based on dynamic network conditions. By learning from network interactions and adapting to time-varying user traffic patterns and channel conditions, the proposed framework minimises energy consumption while maintaining service quality, thereby ensuring efficient network operation. The performance is evaluated through a comprehensive simulation that integrates high-fidelity Sionna ray-tracing channel models and realistic graph-constrained mobility patterns, demonstrating significant improvements over traditional energy management schemes in energy efficiency (34.6% reduction), maintained service quality (0.89% outage), and network stability (reduced handovers).