We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ME

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo ScienceWISE logo

Statistics > Methodology

Title: Robust and Sparse Regression via $γ$-divergence

Abstract: In high-dimensional data, many sparse regression methods have been proposed. However, they may not be robust against outliers. Recently, the use of density power weight has been studied for robust parameter estimation and the corresponding divergences have been discussed. One of such divergences is the $\gamma$-divergence and the robust estimator using the $\gamma$-divergence is known for having a strong robustness. In this paper, we consider the robust and sparse regression based on $\gamma$-divergence. We extend the $\gamma$-divergence to the regression problem and show that it has a strong robustness under heavy contamination even when outliers are heterogeneous. The loss function is constructed by an empirical estimate of the $\gamma$-divergence with sparse regularization and the parameter estimate is defined as the minimizer of the loss function. To obtain the robust and sparse estimate, we propose an efficient update algorithm which has a monotone decreasing property of the loss function. Particularly, we discuss a linear regression problem with $L_1$ regularization in detail. In numerical experiments and real data analyses, we see that the proposed method outperforms past robust and sparse methods.
Comments: 25 pages
Subjects: Methodology (stat.ME); Machine Learning (stat.ML)
DOI: 10.3390/e19110608
Cite as: arXiv:1604.06637 [stat.ME]
  (or arXiv:1604.06637v3 [stat.ME] for this version)

Submission history

From: Takayuki Kawashima [view email]
[v1] Fri, 22 Apr 2016 12:53:27 GMT (16kb)
[v2] Thu, 28 Jul 2016 09:07:10 GMT (19kb)
[v3] Mon, 29 Aug 2016 06:52:11 GMT (19kb)

Link back to: arXiv, form interface, contact.