A Self-Organising State Space Decoder for Reinforcement Learning

A self-organising architecture, loosely based upon a particular implementation of adaptive resonance theory ( ART) is used here as an alternative to the fixed decoder in the seminal implementation of reinforcement learning of Barto, Sutton and Anderson (BSA). The cart-pole problem is considered and the results are compared to those of the original study. The objective is to illustrate the possibility of controllers that partition state space through experience without the need for a priori knowledge. Input/output pattern pairs, desired state space regions and the network size/topology are not known in advance. Results show that, although learning is not smooth, the reinforcement learning implementation considered here is successful and learns a meaningful control mapping. The adaptive search element and the adaptive critic element of the original (BSA) study are retained.