Compact real-valued teaching-learning based optimization with the applications to neural network training

The majority of embedded systems are designed for specific applications, often associated with limited hardware resources in order to meet various and sometime conflicting requirements such as cost, speed, size and performance. Advanced intelligent heuristic optimization algorithms have been widely used in solving engineering problems. However, they might not be applicable to embedded systems, which often have extremely limited memory size. In this paper, a new compact teaching-learning based optimization method for solving global continuous problems is proposed, particularly aiming for neural network training in portable artificial intelligent (AI) devices. Comprehensive numerical experiments on benchmark problems and the training of two popular neural network systems verify that the new compact algorithm is capable of maintaining the high performance while the memory requirement is significantly reduced. It offers a promising tool for continuous optimization problems including the training of neural networks for intelligent embedded systems with limited memory resources.