Current browse context:
cs.LG
Change to browse by:
References & Citations
Computer Science > Machine Learning
Title: Information Theoretic Optimal Learning of Gaussian Graphical Models
(Submitted on 15 Mar 2017 (v1), last revised 18 Nov 2018 (this version, v3))
Abstract: What is the optimal number of independent observations from which a sparse Gaussian Graphical Model can be correctly recovered? Information-theoretic arguments provide a lower bound on the minimum number of samples necessary to perfectly identify the support of any multivariate normal distribution as a function of model parameters. For a model defined on a sparse graph with $p$ nodes, a maximum degree $d$ and minimum normalized edge strength $\kappa$, this necessary number of samples scales at least as $d \log p/\kappa^2$. The sample complexity requirements of existing methods for perfect graph reconstruction exhibit dependency on additional parameters that do not enter in the lower bound. The question of whether the lower bound is tight and achievable by a polynomial time algorithm remains open. In this paper, we constructively answer this question and propose an algorithm, termed DICE, whose sample complexity matches the information-theoretic lower bound up to a universal constant factor. We also propose a related algorithm SLICE that has a slightly higher sample complexity, but can be implemented as a mixed integer quadratic program which makes it attractive in practice. Importantly, SLICE retains a critical advantage of DICE in that its sample complexity only depends on quantities present in the information theoretic lower bound. We anticipate that this result will stimulate future search of computationally efficient sample-optimal algorithms.
Submission history
From: Marc Vuffray [view email][v1] Wed, 15 Mar 2017 02:25:31 GMT (106kb,D)
[v2] Mon, 26 Feb 2018 05:46:43 GMT (267kb,D)
[v3] Sun, 18 Nov 2018 04:32:34 GMT (169kb,D)
Link back to: arXiv, form interface, contact.