Learning to Initialize Gradient Descent Using Gradient Descent
  Non-convex optimization problems are challenging to solve; the success and
computational expense of a gradient descent algorithm or variant depend heavily
on the initialization strategy. Often, either random initialization is used or
initialization rules are carefully designed by exploiting the nature of the
problem class. As a simple alternative to hand-crafted initialization rules, we
propose an approach for learning "good" initialization rules from previous
solutions. We provide theoretical guarantees that establish conditions that are
sufficient in all cases and also necessary in some under which our approach
performs better than random initialization. We apply our methodology to various
non-convex problems such as generating adversarial examples, generating post
hoc explanations for black-box machine learning models, and allocating
communication spectrum, and show consistent gains over other initialization
techniques.
