References & Citations
Computer Science > Machine Learning
Title: From exp-concavity to variance control: High probability O(1/n) rates and high probability online-to-batch conversion
(Submitted on 4 May 2016 (this version), latest version 14 Oct 2016 (v4))
Abstract: We present an algorithm for the statistical learning setting with a bounded exp-concave loss in $d$ dimensions that obtains excess risk $O(d / n)$ with high probability: the dependence on the confidence parameter $\delta$ is polylogarithmic in $1/\delta$. The core technique is to boost the confidence of recent $O(d / n)$ bounds, without sacrificing the rate, by leveraging a Bernstein-type condition which holds due to exp-concavity. This Bernstein-type condition implies that the variance of excess loss random variables are controlled in terms of their excess risk. Using this variance control, we further show that a regret bound for any online learner in this setting translates to a high probability excess risk bound for the corresponding online-to-batch conversion of the online learner.
Submission history
From: Nishant Mehta [view email][v1] Wed, 4 May 2016 14:22:59 GMT (15kb,D)
[v2] Mon, 23 May 2016 09:15:28 GMT (18kb,D)
[v3] Thu, 18 Aug 2016 13:52:47 GMT (18kb,D)
[v4] Fri, 14 Oct 2016 15:24:08 GMT (23kb,D)
Link back to: arXiv, form interface, contact.