Atwya, M. and Panoutsos, G. orcid.org/0000-0002-7395-8418 (2022) Structure optimization of prior-knowledge-guided neural networks. Neurocomputing, 491. pp. 464-488. ISSN 0925-2312
Abstract
Prior-knowledge use in neural networks, for example, knowledge of a physical system, allows network training to be tailored to specific problems. Literature shows that prior-knowledge in neural network training enhances predictive performance. Research to date focuses on parametric optimization rather than structure optimization. We present a new framework to optimize the structure of a neural network using prior-knowledge. This is achieved through optimizing the number of hidden units via a line search and cross-validation using the empirical error to eliminate data-set/model-structure application dependency for prior-knowledge guided neural networks. In addition to using the prior-knowledge in the model training step, we propose utilizing the prior errors as part of the cross-validation performance index to improve generalization. Results demonstrate that the proposed training framework enhances the model’s prediction accuracy and prior-knowledge consistency for convex data sets with a unique minimum and non-convex multi-modal data sets. The presented results yield a new understanding of physics-guided neural networks in terms of their structural and parametric optimization.
Metadata
Authors/Creators: |
|
||||
---|---|---|---|---|---|
Copyright, Publisher and Additional Information: | © 2022 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). | ||||
Keywords: | Prior-guided neural network; Machine learning; Constrained optimization; Structure optimization | ||||
Dates: |
|
||||
Institution: | The University of Sheffield | ||||
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Automatic Control and Systems Engineering (Sheffield) | ||||
Funding Information: |
|
||||
Depositing User: | Symplectic Sheffield | ||||
Date Deposited: | 13 Jul 2022 13:23 | ||||
Last Modified: | 13 Jul 2022 13:23 | ||||
Status: | Published | ||||
Publisher: | Elsevier BV | ||||
Refereed: | Yes | ||||
Identification Number: | https://doi.org/10.1016/j.neucom.2022.03.008 |