A H\"olderian backtracking method for min-max and min-min problems
  We present a new algorithm to solve min-max or min-min problems out of the
convex world. We use rigidity assumptions, ubiquitous in learning, making our
method applicable to many optimization problems. Our approach takes advantage
of hidden regularity properties and allows us to devise a simple algorithm of
ridge type. An original feature of our method is to come with automatic step
size adaptation which departs from the usual overly cautious backtracking
methods. In a general framework, we provide convergence theoretical guarantees
and rates. We apply our findings on simple GAN problems obtaining promising
numerical results.
