Minimax Optimization: The Case of Convex-Submodular
  Minimax optimization has been central in addressing various applications in
machine learning, game theory, and control theory. Prior literature has thus
far mainly focused on studying such problems in the continuous domain, e.g.,
convex-concave minimax optimization is now understood to a significant extent.
Nevertheless, minimax problems extend far beyond the continuous domain to mixed
continuous-discrete domains or even fully discrete domains. In this paper, we
study mixed continuous-discrete minimax problems where the minimization is over
a continuous variable belonging to Euclidean space and the maximization is over
subsets of a given ground set. We introduce the class of convex-submodular
minimax problems, where the objective is convex with respect to the continuous
variable and submodular with respect to the discrete variable. Even though such
problems appear frequently in machine learning applications, little is known
about how to address them from algorithmic and theoretical perspectives. For
such problems, we first show that obtaining saddle points are hard up to any
approximation, and thus introduce new notions of (near-) optimality. We then
provide several algorithmic procedures for solving convex and
monotone-submodular minimax problems and characterize their convergence rates,
computational complexity, and quality of the final solution according to our
notions of optimally. Our proposed algorithms are iterative and combine tools
from both discrete and continuous optimization. Finally, we provide numerical
experiments to showcase the effectiveness of our purposed methods.
