Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Generalized Min-Max Kernel and Generalized Consistent Weighted Sampling
(Submitted on 18 May 2016 (v1), revised 23 May 2016 (this version, v2), latest version 21 Feb 2017 (v4))
Abstract: We propose the "generalized min-max" (GMM) kernel as a measure of data similarity, where data vectors can have both positive and negative entries. GMM is positive definite as there is an associate hashing method named "generalized consistent weighted sampling" (GCWS) which linearizes this (nonlinear) kernel. A natural competitor of GMM is the radial basis function (RBF) kernel, whose corresponding hashing method is known as the "random Fourier features" (RFF). An extensive experimental study on classifications of \textbf{50} publicly available datasets demonstrates that both the GMM and RBF kernels can often substantially improve over linear classifiers. Furthermore, the GCWS hashing method typically requires substantially fewer samples than RFF in order to achieve similar classification accuracies.
To understand the property of random Fourier features (RFF), we derive the theoretical variance of RFF, which reveals that the variance of RFF has a term that does not vanish at any similarity. In comparison, the variance of GCWS approaches zero at certain similarities. Overall, the relative (to the expectation) variance of RFF is substantially larger than the relative variance of GCWS. This helps explain the superb empirical results of GCWS compared to RFF.
We expect that GMM and GCWS will be adopted in practice for large-scale statistical machine learning applications and efficient near neighbor search (as GMM generates discrete hash values).
Submission history
From: Ping Li [view email][v1] Wed, 18 May 2016 19:54:22 GMT (109kb)
[v2] Mon, 23 May 2016 19:51:39 GMT (148kb)
[v3] Thu, 3 Nov 2016 18:42:09 GMT (172kb)
[v4] Tue, 21 Feb 2017 17:11:48 GMT (303kb)
Link back to: arXiv, form interface, contact.