Learning Proximal Operators to Discover Multiple Optima
  Finding multiple solutions of non-convex optimization problems is a
ubiquitous yet challenging task. Typical existing solutions either apply
single-solution optimization methods from multiple random initial guesses or
search in the vicinity of found solutions using ad hoc heuristics. We present
an end-to-end method to learn the proximal operator across a family of
non-convex problems, which can then be used to recover multiple solutions for
unseen problems at test time. Our method only requires access to the objectives
without needing the supervision of ground truth solutions. Notably, the added
proximal regularization term elevates the convexity of our formulation: by
applying recent theoretical results, we show that for weakly-convex objectives
and under mild regularity conditions, training of the proximal operator
converges globally in the over-parameterized setting. We further present a
benchmark for multi-solution optimization including a wide range of
applications and evaluate our method to demonstrate its effectiveness.
