### Current browse context:

cs.LG

### Change to browse by:

### References & Citations

# Computer Science > Machine Learning

# Title: Boosting as Frank-Wolfe

(Submitted on 22 Sep 2022 (v1), last revised 30 Sep 2022 (this version, v2))

Abstract: Some boosting algorithms, such as LPBoost, ERLPBoost, and C-ERLPBoost, aim to solve the soft margin optimization problem with the $\ell_1$-norm regularization. LPBoost rapidly converges to an $\epsilon$-approximate solution in practice, but it is known to take $\Omega(m)$ iterations in the worst case, where $m$ is the sample size. On the other hand, ERLPBoost and C-ERLPBoost are guaranteed to converge to an $\epsilon$-approximate solution in $O(\frac{1}{\epsilon^2} \ln \frac{m}{\nu})$ iterations. However, the computation per iteration is very high compared to LPBoost.

To address this issue, we propose a generic boosting scheme that combines the Frank-Wolfe algorithm and any secondary algorithm and switches one to the other iteratively. We show that the scheme retains the same convergence guarantee as ERLPBoost and C-ERLPBoost. One can incorporate any secondary algorithm to improve in practice. This scheme comes from a unified view of boosting algorithms for soft margin optimization. More specifically, we show that LPBoost, ERLPBoost, and C-ERLPBoost are instances of the Frank-Wolfe algorithm. In experiments on real datasets, one of the instances of our scheme exploits the better updates of the secondary algorithm and performs comparably with LPBoost.

## Submission history

From: Ryotaro Mitsuboshi [view email]**[v1]**Thu, 22 Sep 2022 07:36:55 GMT (634kb,D)

**[v2]**Fri, 30 Sep 2022 08:24:59 GMT (648kb,D)

Link back to: arXiv, form interface, contact.