A Value-Function-based Interior-point Method for Non-convex Bi-level
  Optimization
  Bi-level optimization model is able to capture a wide range of complex
learning tasks with practical interest. Due to the witnessed efficiency in
solving bi-level programs, gradient-based methods have gained popularity in the
machine learning community. In this work, we propose a new gradient-based
solution scheme, namely, the Bi-level Value-Function-based Interior-point
Method (BVFIM). Following the main idea of the log-barrier interior-point
scheme, we penalize the regularized value function of the lower level problem
into the upper level objective. By further solving a sequence of differentiable
unconstrained approximation problems, we consequently derive a sequential
programming scheme. The numerical advantage of our scheme relies on the fact
that, when gradient methods are applied to solve the approximation problem, we
successfully avoid computing any expensive Hessian-vector or Jacobian-vector
product. We prove the convergence without requiring any convexity assumption on
either the upper level or the lower level objective. Experiments demonstrate
the efficiency of the proposed BVFIM on non-convex bi-level problems.
