Iterative Thresholding Algorithm for Sparse Inverse Covariance
  Estimation
  The L1-regularized maximum likelihood estimation problem has recently become
a topic of great interest within the machine learning, statistics, and
optimization communities as a method for producing sparse inverse covariance
estimators. In this paper, a proximal gradient method (G-ISTA) for performing
L1-regularized covariance matrix estimation is presented. Although numerous
algorithms have been proposed for solving this problem, this simple proximal
gradient method is found to have attractive theoretical and numerical
properties. G-ISTA has a linear rate of convergence, resulting in an O(log e)
iteration complexity to reach a tolerance of e. This paper gives eigenvalue
bounds for the G-ISTA iterates, providing a closed-form linear convergence
rate. The rate is shown to be closely related to the condition number of the
optimal point. Numerical convergence results and timing comparisons for the
proposed method are presented. G-ISTA is shown to perform very well, especially
when the optimal point is well-conditioned.
