Effects of Overparameterisation in Non-Linear System Identification and Neural Networks

The effects of overparametrisation in nonlinear systems identification and neural networks are often presented as overfitting into a particular data set. An over-parametrised model tends to interpolate the data set and can fail to capture the underlying dynamics represented by the data. Past studies have usually concentrated on overparamtrisation in the number of model terms or the number of hidden layer nodes. In the present study, the effects of several forms of overparamtrisation on the model dynamics are compared using qualitative validation methods. The dynamic properties of the identified model or the trained network are graphically displayed and compared with those of the real system. It is shown that overparametrisation in the number of time lags or input nodes has the worst effect on the dynamic properties of the resulting model.