Current browse context:
stat.ME
Change to browse by:
References & Citations
Statistics > Methodology
Title: Regression-aware decompositions
(Submitted on 11 Oct 2017 (v1), last revised 12 Feb 2018 (this version, v2))
Abstract: Linear least-squares regression with a "design" matrix A approximates a given matrix B via minimization of the spectral- or Frobenius-norm discrepancy ||AX-B|| over every conformingly sized matrix X. Another popular approximation is low-rank approximation via principal component analysis (PCA) -- which is essentially singular value decomposition (SVD) -- or interpolative decomposition (ID). Classically, PCA/SVD and ID operate solely with the matrix B being approximated, not supervised by any auxiliary matrix A. However, linear least-squares regression models can inform the ID, yielding regression-aware ID. As a bonus, this provides an interpretation as regression-aware PCA for a kind of canonical correlation analysis between A and B. The regression-aware decompositions effectively enable supervision to inform classical dimensionality reduction, which classically has been totally unsupervised. The regression-aware decompositions reveal the structure inherent in B that is relevant to regression against A.
Submission history
From: Mark Tygert [view email][v1] Wed, 11 Oct 2017 18:06:25 GMT (6kb)
[v2] Mon, 12 Feb 2018 22:11:18 GMT (4904kb,D)
Link back to: arXiv, form interface, contact.