Current browse context:
math.PR
Change to browse by:
References & Citations
Mathematics > Probability
Title: A Concentration of Measure Framework to study convex problems and other implicit formulation problems in machine learning
(Submitted on 19 Oct 2020 (v1), last revised 14 Jan 2023 (this version, v2))
Abstract: This paper provides a framework to show the concentration of solutions $Y^*$ to convex minimizing problem where the objective function $\phi(X)(Y)$ depends on some random vector $X$ satisfying concentration of measure hypotheses. More precisely, the convex problem translates into a contractive fixed point equation that ensure the transmission of the concentration from $X$ to $Y^*$. This result is of central interest to characterize many machine learning algorithms which are defined through implicit equations (e.g., logistic regression, lasso, boosting, etc.). Based on our framework, we provide precise estimations for the first moments of the solution $Y^*$, when $X= (x_1,\ldots, x_n)$ is a data matrix of independent columns and $\phi(X)(y)$ writes as a sum $\frac{1}{n}\sum_{i=1}^n h_i(x_i^TY)$. That allows to describe the behavior and performance (e.g., generalization error) of a wide variety of machine learning classifiers.
Submission history
From: Cosme Louart [view email][v1] Mon, 19 Oct 2020 21:26:30 GMT (163kb)
[v2] Sat, 14 Jan 2023 09:17:57 GMT (116kb)
Link back to: arXiv, form interface, contact.