Multiple-model approach to non-linear kernel-based adaptive filtering

Kernel methods now provide standard tools for the solution of function approximation and pattern classification problems. However, it is typically assumed that all data are available for training. More recently, various approaches have been proposed for extending kernel methods to sequential problems whereby the model is updated as each new data point arrives. Whilst these approaches have proven successful in estimating the basic parameters, the problem of estimating the hyperparameters which determine the overall model behaviour, remains essentially unsolved. In this paper a novel approach to the hyperparameters is presented based on a multiple model framework. An ensemble of models with different hyperparameters is trained in parallel, the outputs of which are subsequently combined based on a predictive performance measure. This new approach is sucessfully demonstrated in a standard benchmark time series problem.