Multilayer evolving fuzzy neural networks

It is widely recognised that learning systems have to go deeper to exchange for more powerful representation learning capabilities in order to precisely approximate nonlinear complex problems. However, the best known computational intelligence approaches with such characteristics, namely, deep neural networks, are often criticised for lacking transparency. In this paper, a novel multilayer evolving fuzzy neural network (MEFNN) with a transparent system structure is proposed. The proposed MEFNN is a meta-level stacking ensemble learning system composed of multiple cascading evolving neuro-fuzzy inference systems (ENFISs), processing input data layer-by-layer to automatically learn multi-level nonlinear distributed repre- sentations from data. Each ENFIS is an evolving fuzzy system capable of learning from new data sample by sample to self- organise a set of human-interpretable IF-THEN fuzzy rules that facilitate approximate reasoning. Adopting ENFIS as its ensemble component, the multilayer system structure of MEFNN is flexible and transparent, and its internal reasoning and decision-making mechanism can be explained and interpreted to/by humans. To facilitate information exchange between different layers and at- tain stronger representation learning capability, MEFNN utilises error backpropagation to self-update the consequent parameters of the IF-THEN rules of each ensemble component based on the approximation error propagated backward. To enhance the capability of MEFNN to handle complex problems, a nonlinear activation function is introduced to modelling the consequent parts of the IF-THEN rules of ENFISs, thereby empowering both the representation and the reflection of nonlinearity in the resulting fuzzy outputs. Numerical examples on a wide variety of challenging (benchmark and real-world) classification and regression problems demonstrate the superior practical performance of MEFNN, revealing the effectiveness and validity of the proposed approach.