Tang, Z., Rossiter, J.A. orcid.org/0000-0002-1336-0633 and Panoutsos, G. orcid.org/0000-0002-7395-8418 (2024) A reinforcement learning-based approach for optimal output tracking in uncertain nonlinear systems with mismatched disturbances. In: 2024 UKACC 14th International Conference on Control (CONTROL). CONTROL 2024: 14th United Kingdom Automatic Control Council (UKACC) International Conference on Control, 10-12 Apr 2024, Winchester, United Kingdom. Institute of Electrical and Electronics Engineers , pp. 169-174. ISBN 979-8-3503-7427-8
Abstract
In this paper, the optimal control problem of uncertain nonlinear systems is considered. A nonlinear disturbance observer (NDO) is proposed to measure the lumped uncertainties present in the system. Disturbances that do not enter the same channel as the control signal, so-called mismatched disturbances, are difficult to reject directly within the control channel. To overcome the challenge, a generalized disturbance observer-based compensator is implemented to address the uncertainty compensation problem by attenuating its influence on the output channel. In real time, by augmenting the system states with the output tracking error, we develop a composite actor-critic reinforcement learning (RL) scheme for approximating the optimal control policy as well as the ideal value function pertaining to the compensated system by solving the Hamilton-Jacobi-Bellman (HJB) equation. Concurrent learning is applied in this article by using the recorded data of the known model of the system, in order to enhance the robustness of the system by canceling the influence of the probing signal. Simulation results demonstrate the effectiveness of the proposed scheme, offering an optimal solution for the output tracking problem in a second-order model with mismatched disturbances.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2024 The author(s). Except as otherwise noted, this author-accepted version of a paper published in UKACC International Conference on Control (CONTROL) is made available via the University of Sheffield Research Publications and Copyright Policy under the terms of the Creative Commons Attribution 4.0 International License (CC-BY 4.0), which permits unrestricted use, distribution and reproduction in any medium, provided the original work is properly cited. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ |
Keywords: | Uncertainty; Simulation; Measurement uncertainty; Optimal control; Reinforcement learning; Mathematical models; Robustness |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > School of Electrical and Electronic Engineering |
Funding Information: | Funder Grant number Engineering and Physical Sciences Research Council EP/V051261/1 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 27 Nov 2024 09:32 |
Last Modified: | 02 Dec 2024 12:50 |
Status: | Published |
Publisher: | Institute of Electrical and Electronics Engineers |
Refereed: | Yes |
Identification Number: | 10.1109/control60310.2024.10532060 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:220143 |
Download
Filename: Zezhi_Tang__2023_UKACC_Full_Paper_final_short_version__9_.pdf
Licence: CC-BY 4.0