We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Machine Learning

Title: Faster Algorithms for Learning Convex Functions

Abstract: The task of approximating an arbitrary convex function arises in several learning problems such as convex regression, learning with a difference of convex (DC) functions, and learning Bregman or $f$-divergences. In this paper, we develop and analyze an approach for solving a broad range of convex function learning problems that is faster than state-of-the-art approaches. Our approach is based on a 2-block ADMM method where each block can be computed in closed form. For the task of convex Lipschitz regression, we establish that our proposed algorithm converges with iteration complexity of $ O(n\sqrt{d}/\epsilon)$ for a dataset $\bm X \in \mathbb R^{n\times d}$ and $\epsilon > 0$. Combined with per-iteration computation complexity, our method converges with the rate $O(n^3 d^{1.5}/\epsilon+n^2 d^{2.5}/\epsilon+n d^3/\epsilon)$. This new rate improves the state of the art rate of $O(n^5d^2/\epsilon)$ if $d = o( n^4)$. Further we provide similar solvers for DC regression and Bregman divergence learning. Unlike previous approaches, our method is amenable to the use of GPUs. We demonstrate on regression and metric learning experiments that our approach is over 100 times faster than existing approaches on some data sets, and produces results that are comparable to state of the art.
Comments: 21 pages, 3 figures. Proceedings of the 39 th International Conference on Machine Learning, Baltimore, Maryland, USA, PMLR 162, 2022. Copy- right 2022 by the author(s)
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:2111.01348 [stat.ML]
  (or arXiv:2111.01348v4 [stat.ML] for this version)

Submission history

From: Ali Siahkamari [view email]
[v1] Tue, 2 Nov 2021 03:10:41 GMT (71kb,D)
[v2] Sat, 6 Nov 2021 17:49:39 GMT (208kb,D)
[v3] Mon, 29 Nov 2021 16:02:39 GMT (212kb,D)
[v4] Sun, 19 Jun 2022 15:42:34 GMT (99kb,D)

Link back to: arXiv, form interface, contact.