MADDPG-based joint service placement and task offloading in MEC empowered air-ground integrated networks

Multi-access Edge Computing (MEC) empowered Air-Ground Integrated Networks (AGINs) hold great promise in delivering accessible computing services for users and Internet of Things (IoT) applications, such as forest fire monitoring, emergency rescue operations, etc. In this paper, we present a comprehensive air-ground integrated MEC framework, where edge servers carried by Unmanned Aerial Vehicles (UAVs) will provide efficient computation services to IoT devices and User Equipment (which are collectively referred to as UEs). We aim to minimize the long-term average weighted sum of task completion delay and economic expenditure for all the UEs. This objective is achieved through various strategies, including pre-installing new service instances into UAVs, removing idle service instances from UAVs, task offloading decision making, access control, selecting appropriate service instances for each offloaded service request, and resource allocation optimization. Considering the complexity of the problem and the dynamics of the system, we reformulate the problem as a Markov decision process (MDP) and present a Multi-Agent Deep Deterministic Policy Gradient (MADDPG)-based algorithm to enable low-complexity and real-time adaptive decision-making. Since our problem contains integer, binary and continuous variables, it is not straightforward to apply the MADDPG algorithm. Specifically, we first normalize the continuous variables, and then convert the continuous output generated by MADDPG into discrete variables, while ensuring the coupling constraints between different variables are preserved. The simulation results demonstrate the fast convergence of our proposed algorithm and its superior performance in minimizing costs compared with the baseline algorithms.