Liu, X. orcid.org/0009-0000-7674-3183, Ma, L. orcid.org/0009-0006-8776-9112, Chen, Z. orcid.org/0009-0002-2151-7028 et al. (4 more authors) (2023) A Novel State Space Exploration Method for the Sparse-Reward Reinforcement Learning Environment. In: 43rd SGAI International Conference on Artificial Intelligence, AI 2023, 12-14 Dec 2023, Cambridge, UK.
Abstract
Sparse-reward reinforcement learning environments pose a particular challenge because the agent receives infrequent rewards, making it difficult to learn an optimal policy. In this paper, we propose NSSE, a novel approach that combines that stratified state space exploration with prioritised sweeping to enhance the informativeness of learning, thus enabling fast learning convergence. We evaluate NSSE on three typical Atari sparse reward environments. The results demonstrate that our state space exploration method exhibits strong performance compared to two baseline algorithms: Deep Q-Network (DQN) and noisy Deep Q-Network (Noisy DQN).
Metadata
Item Type: | Conference or Workshop Item |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG. This is an author produced version of a conference paper accepted for publication in Artificial Intelligence XL (SGAI 2023). Uploaded in accordance with the publisher's self-archiving policy. |
Keywords: | Sparse-reward; Replay Sub-buffers; DQN; Exploration; Reinforcement Learning |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Mechanical Engineering (Leeds) > Institute of Medical and Biological Engineering (iMBE) (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 16 Apr 2024 14:02 |
Last Modified: | 16 Apr 2024 14:02 |
Status: | Published |
Publisher: | Springer Nature Switzerland |
Identification Number: | 10.1007/978-3-031-47994-6_18 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:211536 |