This is the latest version of this eprint.
Fontanesi, G., Zhu, A., Arvaneh, M. orcid.org/0000-0002-5124-3497 et al. (1 more author) (2023) A transfer learning approach for UAV path design with connectivity outage constraint. IEEE Internet of Things Journal, 10 (6). pp. 4998-5012. ISSN 2327-4662
Abstract
The connectivity-aware path design is crucial in the effective deployment of autonomous Unmanned Aerial Vehicles (UAVs). Recently, Reinforcement Learning (RL) algorithms have become the popular approach to solving this type of complex problem, but RL algorithms suffer slow convergence. In this paper, we propose a Transfer Learning (TL) approach, where we use a teacher policy previously trained in an old domain to boost the path learning of the agent in the new domain. As the exploration processes and the training continue, the agent refines the path design in the new domain based on the subsequent interactions with the environment. We evaluate our approach considering an old domain at sub-6 GHz and a new domain at millimeter Wave (mmWave). The teacher path policy, previously trained at sub-6 GHz path, is the solution to a connectivity-aware path problem that we formulate as a constrained Markov Decision Process (CMDP). We employ a Lyapunov-based model-free Deep Q-Network (DQN) to solve the path design at sub-6 GHz that guarantees connectivity constraint satisfaction. We empirically demonstrate the effectiveness of our approach for different urban environment scenarios. The results demonstrate that our proposed approach is capable of reducing the training time considerably at mmWave.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other users, including reprinting/ republishing this material for advertising or promotional purposes, creating new collective works for resale or redistribution to servers or lists, or reuse of any copyrighted components of this work in other works. Reproduced in accordance with the publisher's self-archiving policy. |
Keywords: | Cellular networks; deep reinforcement learning; path design; transfer learning; Unmanned Aerial Vehicle (UAV) |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Automatic Control and Systems Engineering (Sheffield) |
Funding Information: | Funder Grant number DAIWA ANGLO JAPANESE FOUNDATION UNSPECIFIED |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 11 Nov 2022 14:54 |
Last Modified: | 25 Sep 2024 16:15 |
Status: | Published |
Publisher: | Institute of Electrical and Electronics Engineers |
Refereed: | Yes |
Identification Number: | 10.1109/JIOT.2022.3220981 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:192904 |
Available Versions of this Item
-
A transfer learning approach for UAV path design with connectivity outage constraint. (deposited 05 Jul 2024 10:03)
- A transfer learning approach for UAV path design with connectivity outage constraint. (deposited 11 Nov 2022 14:54) [Currently Displayed]