Cao, Y. and Rockett, P.I. (2015) The use of vicinal-risk minimization for training decision trees. Applied Soft Computing, 31. 185 - 195. ISSN 1568-4946
Abstract
We propose the use of Vapnik's vicinal risk minimization (VRM) for training decision trees to approximately maximize decision margins. We implement VRM by propagating uncertainties in the input attributes into the labeling decisions. In this way, we perform a global regularization over the decision tree structure. During a training phase, a decision tree is constructed to minimize the total probability of misclassifying the labeled training examples, a process which approximately maximizes the margins of the resulting classifier. We perform the necessary minimization using an appropriate meta-heuristic (genetic programming) and present results over a range of synthetic and benchmark real datasets. We demonstrate the statistical superiority of VRM training over conventional empirical risk minimization (ERM) and the well-known C4.5 algorithm, for a range of synthetic and real datasets. We also conclude that there is no statistical difference between trees trained by ERM and using C4.5. Training with VRM is shown to be more stable and repeatable than by ERM.
Metadata
Item Type: | Article |
---|---|
Authors/Creators: |
|
Copyright, Publisher and Additional Information: | © 2015 Elsevier B.V. All rights reserved. This is an author produced version of a paper subsequently published in Applied Soft Computing. Uploaded in accordance with the publisher's self-archiving policy. Article available under the terms of the CC-BY-NC-ND licence (https://creativecommons.org/licenses/by-nc-nd/4.0/) |
Keywords: | Decision trees; Vicinal-risk minimization; Decision trees; Classification |
Dates: |
|
Institution: | The University of Sheffield |
Academic Units: | The University of Sheffield > Faculty of Engineering (Sheffield) > Department of Electronic and Electrical Engineering (Sheffield) |
Depositing User: | Symplectic Sheffield |
Date Deposited: | 19 Nov 2015 17:13 |
Last Modified: | 30 Jul 2017 03:16 |
Published Version: | https://dx.doi.org/10.1016/j.asoc.2015.02.043 |
Status: | Published |
Publisher: | Elsevier |
Refereed: | Yes |
Identification Number: | 10.1016/j.asoc.2015.02.043 |
Related URLs: | |
Open Archives Initiative ID (OAI ID): | oai:eprints.whiterose.ac.uk:90870 |