Zeroth-Order Algorithms for Nonconvex Minimax Problems with Improved
  Complexities
  In this paper, we study zeroth-order algorithms for minimax optimization
problems that are nonconvex in one variable and strongly-concave in the other
variable. Such minimax optimization problems have attracted significant
attention lately due to their applications in modern machine learning tasks. We
first consider a deterministic version of the problem. We design and analyze
the Zeroth-Order Gradient Descent Ascent (\texttt{ZO-GDA}) algorithm, and
provide improved results compared to existing works, in terms of oracle
complexity. We also propose the Zeroth-Order Gradient Descent Multi-Step Ascent
(\texttt{ZO-GDMSA}) algorithm that significantly improves the oracle complexity
of \texttt{ZO-GDA}. We then consider stochastic versions of \texttt{ZO-GDA} and
\texttt{ZO-GDMSA}, to handle stochastic nonconvex minimax problems. For this
case, we provide oracle complexity results under two assumptions on the
stochastic gradient: (i) the uniformly bounded variance assumption, which is
common in traditional stochastic optimization, and (ii) the Strong Growth
Condition (SGC), which has been known to be satisfied by modern
over-parametrized machine learning models. We establish that under the SGC
assumption, the complexities of the stochastic algorithms match that of
deterministic algorithms. Numerical experiments are presented to support our
theoretical results.
