Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Fast Margin Maximization via Dual Acceleration
(Submitted on 1 Jul 2021 (v1), last revised 22 Aug 2021 (this version, v2))
Abstract: We present and analyze a momentum-based gradient method for training linear classifiers with an exponentially-tailed loss (e.g., the exponential or logistic loss), which maximizes the classification margin on separable data at a rate of $\widetilde{\mathcal{O}}(1/t^2)$. This contrasts with a rate of $\mathcal{O}(1/\log(t))$ for standard gradient descent, and $\mathcal{O}(1/t)$ for normalized gradient descent. This momentum-based method is derived via the convex dual of the maximum-margin problem, and specifically by applying Nesterov acceleration to this dual, which manages to result in a simple and intuitive method in the primal. This dual view can also be used to derive a stochastic variant, which performs adaptive non-uniform sampling via the dual variables.
Submission history
From: Ziwei Ji [view email][v1] Thu, 1 Jul 2021 16:36:39 GMT (1142kb,D)
[v2] Sun, 22 Aug 2021 01:44:00 GMT (1022kb,D)
Link back to: arXiv, form interface, contact.