Information Theoretic Optimal Learning of Gaussian Graphical Models
  What is the optimal number of independent observations from which a sparse
Gaussian Graphical Model can be correctly recovered? Information-theoretic
arguments provide a lower bound on the minimum number of samples necessary to
perfectly identify the support of any multivariate normal distribution as a
function of model parameters. For a model defined on a sparse graph with $p$
nodes, a maximum degree $d$ and minimum normalized edge strength $\kappa$, this
necessary number of samples scales at least as $d \log p/\kappa^2$. The sample
complexity requirements of existing methods for perfect graph reconstruction
exhibit dependency on additional parameters that do not enter in the lower
bound. The question of whether the lower bound is tight and achievable by a
polynomial time algorithm remains open. In this paper, we constructively answer
this question and propose an algorithm, termed DICE, whose sample complexity
matches the information-theoretic lower bound up to a universal constant
factor. We also propose a related algorithm SLICE that has a slightly higher
sample complexity, but can be implemented as a mixed integer quadratic program
which makes it attractive in practice. Importantly, SLICE retains a critical
advantage of DICE in that its sample complexity only depends on quantities
present in the information theoretic lower bound. We anticipate that this
result will stimulate future search of computationally efficient sample-optimal
algorithms.
