References & Citations
Statistics > Machine Learning
Title: Binarsity: a penalization for one-hot encoded features in linear supervised learning
(Submitted on 24 Mar 2017 (v1), last revised 9 Jan 2019 (this version, v4))
Abstract: This paper deals with the problem of large-scale linear supervised learning in settings where a large number of continuous features are available. We propose to combine the well-known trick of one-hot encoding of continuous features with a new penalization called \emph{binarsity}. In each group of binary features coming from the one-hot encoding of a single raw continuous feature, this penalization uses total-variation regularization together with an extra linear constraint. This induces two interesting properties on the model weights of the one-hot encoded features: they are piecewise constant, and are eventually block sparse. Non-asymptotic oracle inequalities for generalized linear models are proposed. Moreover, under a sparse additive model assumption, we prove that our procedure matches the state-of-the-art in this setting. Numerical experiments illustrate the good performances of our approach on several datasets. It is also noteworthy that our method has a numerical complexity comparable to standard $\ell_1$ penalization.
Submission history
From: Simon Bussy [view email][v1] Fri, 24 Mar 2017 22:54:17 GMT (4738kb,D)
[v2] Sun, 7 May 2017 10:58:03 GMT (5035kb,D)
[v3] Sat, 25 Nov 2017 15:46:07 GMT (5548kb,D)
[v4] Wed, 9 Jan 2019 14:58:44 GMT (3500kb,D)
Link back to: arXiv, form interface, contact.