Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Optimal Generalized Decision Trees via Integer Programming
(Submitted on 10 Dec 2016 (this version), latest version 13 Aug 2019 (v3))
Abstract: Decision trees have been a very popular class of predictive models for decades due to their interpretability and good performance on categorical features. However, they are not always robust and tend to overfit the data. Additionally, if allowed to grow large, they lose interpretability. In this paper, we present a novel mixed integer programming formulation to construct optimal decision trees of specified size. We take special structure of categorical features into account and allow combinatorial decisions (based on subsets of values of such a feature) at each node. We show that very good accuracy can be achieved with small trees using moderately-sized training sets. The optimization problems we solve are easily tractable with modern solvers.
Submission history
From: Katya Scheinberg [view email][v1] Sat, 10 Dec 2016 00:05:37 GMT (902kb,D)
[v2] Sun, 14 Jan 2018 20:56:14 GMT (49kb)
[v3] Tue, 13 Aug 2019 17:19:17 GMT (53kb)
Link back to: arXiv, form interface, contact.