Atwya, M. and Panoutsos, G. orcid.org/0000-0002-7395-8418 (2022) Structure optimization of prior-knowledge-guided neural networks. Neurocomputing, 491. pp. 464-488. ISSN 0925-2312
Abstract
Prior-knowledge use in neural networks, for example, knowledge of a physical system, allows network training to be tailored to specific problems. Literature shows that prior-knowledge in neural network training enhances predictive performance. Research to date focuses on parametric optimization rather than structure optimization. We present a new framework to optimize the structure of a neural network using prior-knowledge. This is achieved through optimizing the number of hidden units via a line search and cross-validation using the empirical error to eliminate data-set/model-structure application dependency for prior-knowledge guided neural networks. In addition to using the prior-knowledge in the model training step, we propose utilizing the prior errors as part of the cross-validation performance index to improve generalization. Results demonstrate that the proposed training framework enhances the model’s prediction accuracy and prior-knowledge consistency for convex data sets with a unique minimum and non-convex multi-modal data sets. The presented results yield a new understanding of physics-guided neural networks in terms of their structural and parametric optimization.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2022 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). |
Keywords: | Prior-guided neural network; Machine learning; Constrained optimization; Structure optimization |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Automatic Control and Systems Engineering (Sheffield) |
Funding Information: | Funder Grant number Engineering and Physical Sciences Research Council EP/P006566/1 |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 13 Jul 2022 13:23 |
Last Modified: | 13 Jul 2022 13:23 |
Status: | Published |
Publisher: | Elsevier BV |
Refereed: | Yes |
Identification Number: | 10.1016/j.neucom.2022.03.008 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:188859 |