Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: Dimensionality reduction to maximize prediction generalization capability
(Submitted on 1 Mar 2020 (v1), last revised 20 Jan 2022 (this version, v2))
Abstract: Generalization of time series prediction remains an important open issue in machine learning, wherein earlier methods have either large generalization error or local minima. We develop an analytically solvable, unsupervised learning scheme that extracts the most informative components for predicting future inputs, termed predictive principal component analysis (PredPCA). Our scheme can effectively remove unpredictable noise and minimize test prediction error through convex optimization. Mathematical analyses demonstrate that, provided with sufficient training samples and sufficiently high-dimensional observations, PredPCA can asymptotically identify hidden states, system parameters, and dimensionalities of canonical nonlinear generative processes, with a global convergence guarantee. We demonstrate the performance of PredPCA using sequential visual inputs comprising hand-digits, rotating 3D objects, and natural scenes. It reliably estimates distinct hidden states and predicts future outcomes of previously unseen test input data, based exclusively on noisy observations. The simple architecture and low computational cost of PredPCA are highly desirable for neuromorphic hardware.
Submission history
From: Takuya Isomura [view email][v1] Sun, 1 Mar 2020 12:04:59 GMT (1418kb)
[v2] Thu, 20 Jan 2022 08:48:42 GMT (8841kb)
Link back to: arXiv, form interface, contact.