Rodríguez-Salas, D, Ravikumar, N, Seuret, M et al. (1 more author) (2020) ForestNet – Automatic Design of Sparse Multilayer Perceptron Network Architectures Using Ensembles of Randomized Trees. In: Lecture Notes in Computer Science. ACPR 2019: Asian Conference on Pattern Recognition, 26-29 Nov 2019, Auckland, New Zealand. Springer Verlag , pp. 32-45. ISBN 9783030414030
Abstract
In this paper, we introduce a mechanism for designing the architecture of a Sparse Multi-Layer Perceptron network, for classification, called ForestNet. Networks built using our approach are capable of handling high-dimensional data and learning representations of both visual and non-visual data. The proposed approach first builds an ensemble of randomized trees in order to gather information on the hierarchy of features and their separability among the classes. Subsequently, such information is used to design the architecture of a sparse network, for a specific data set and application. The number of neurons is automatically adapted to the dataset. The proposed approach was evaluated using two non-visual and two visual datasets. For each dataset, 4 ensembles of randomized trees with different sizes were built. In turn, per ensemble, a sparse network architecture was designed using our approach and a fully connected network with same architecture was also constructed. The sparse networks defined using our approach consistently outperformed their respective tree ensembles, achieving statistically significant improvements in classification accuracy. While we do not beat state-of-art results with our network size and the lack of data augmentation techniques, our method exhibits very promising results, as the sparse networks performed similarly to their fully connected counterparts with a reduction of more than 98% of connections in the visual tasks.
Metadata
Item Type: | Proceedings Paper |
---|---|
Authors/Creators: |
|
Keywords: | Multilayer perceptron; Random forest; Randomized trees; Sparse neural networks; Network architecture |
Dates: |
|
Institution: | The University of Leeds |
Academic Units: | The University of Leeds > Faculty of Engineering & Physical Sciences (Leeds) > School of Computing (Leeds) |
Depositing User: | Symplectic Publications |
Date Deposited: | 16 Apr 2020 13:19 |
Last Modified: | 13 Jul 2020 15:28 |
Status: | Published |
Publisher: | Springer Verlag |
Identification Number: | 10.1007/978-3-030-41404-7_3 |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:159513 |