Current browse context:
stat.ML
Change to browse by:
References & Citations
Statistics > Machine Learning
Title: A Test of Relative Similarity For Model Selection in Generative Models
(Submitted on 14 Nov 2015 (v1), last revised 15 Feb 2016 (this version, v4))
Abstract: Probabilistic generative models provide a powerful framework for representing data that avoids the expense of manual annotation typically needed by discriminative approaches. Model selection in this generative setting can be challenging, however, particularly when likelihoods are not easily accessible. To address this issue, we introduce a statistical test of relative similarity, which is used to determine which of two models generates samples that are significantly closer to a real-world reference dataset of interest. We use as our test statistic the difference in maximum mean discrepancies (MMDs) between the reference dataset and each model dataset, and derive a powerful, low-variance test based on the joint asymptotic distribution of the MMDs between each reference-model pair. In experiments on deep generative models, including the variational auto-encoder and generative moment matching network, the tests provide a meaningful ranking of model performance as a function of parameter and training settings.
Submission history
From: Eugene Belilovsky [view email][v1] Sat, 14 Nov 2015 17:18:47 GMT (26kb)
[v2] Fri, 20 Nov 2015 11:12:05 GMT (1726kb,D)
[v3] Wed, 6 Jan 2016 15:35:53 GMT (1727kb,D)
[v4] Mon, 15 Feb 2016 15:12:44 GMT (1715kb,D)
Link back to: arXiv, form interface, contact.