We gratefully acknowledge support from
the Simons Foundation and member institutions.
Full-text links:

Download:

Current browse context:

stat.ML

Change to browse by:

References & Citations

Bookmark

(what is this?)
CiteULike logo BibSonomy logo Mendeley logo del.icio.us logo Digg logo Reddit logo

Statistics > Machine Learning

Title: On Data-Dependent Random Features for Improved Generalization in Supervised Learning

Abstract: The randomized-feature approach has been successfully employed in large-scale kernel approximation and supervised learning. The distribution from which the random features are drawn impacts the number of features required to efficiently perform a learning task. Recently, it has been shown that employing data-dependent randomization improves the performance in terms of the required number of random features. In this paper, we are concerned with the randomized-feature approach in supervised learning for good generalizability. We propose the Energy-based Exploration of Random Features (EERF) algorithm based on a data-dependent score function that explores the set of possible features and exploits the promising regions. We prove that the proposed score function with high probability recovers the spectrum of the best fit within the model class. Our empirical results on several benchmark datasets further verify that our method requires smaller number of random features to achieve a certain generalization error compared to the state-of-the-art while introducing negligible pre-processing overhead. EERF can be implemented in a few lines of code and requires no additional tuning parameters.
Comments: 12 pages; (pages 1-8) to appear in Proc. of AAAI Conference on Artificial Intelligence (AAAI), 2018
Subjects: Machine Learning (stat.ML); Machine Learning (cs.LG)
Cite as: arXiv:1712.07102 [stat.ML]
  (or arXiv:1712.07102v1 [stat.ML] for this version)

Submission history

From: Shahin Shahrampour [view email]
[v1] Tue, 19 Dec 2017 18:40:36 GMT (61kb,D)

Link back to: arXiv, form interface, contact.