Support Vector Neural Networks

The kernel Adatron support vector neural network (SVNN) is a new neural network alternative to support vector (SV) machines. It can learn large-margin decision functions in kernel feature spaces in an iterative "on-line" fashion which are identical to support vector machines. In contrast "conventional" support vector learning is batch learning and is strongly based on solving constrained quadratic programming problems. Quadratic programming is nontrivial to implement and can be subject to stability problems. The kernel Adatron algorithm (KA) has been introduced recently. So far it has been assumed that the bias parameter of the plane in feature space is always zero and that all patterns can be correctly classified by the learning machine. These assumptions cannot always be made. The kernel Adatron SVNN with bias and soft margin combines the speed and simplicity of neural networks with the predictive power of SV machines. However, the SVNN does not, unlike to SV machines, suffer from any problems related to quadratic programming and unlike to conventional neural networks the SVNN's cost function is always CONVEX. The support vector neural network is introduced, then experimental results using bench-marks and real data are presented which allow to compare the performance of SVNN's and SV machines.