Near-Optimal Statistical Query Hardness of Learning Halfspaces with
  Massart Noise
  We study the problem of PAC learning halfspaces with Massart noise. Given
labeled samples $(x, y)$ from a distribution $D$ on $\mathbb{R}^{d} \times \{
\pm 1\}$ such that the marginal $D_x$ on the examples is arbitrary and the
label $y$ of example $x$ is generated from the target halfspace corrupted by a
Massart adversary with flipping probability $\eta(x) \leq \eta \leq 1/2$, the
goal is to compute a hypothesis with small misclassification error. The best
known $\mathrm{poly}(d, 1/\epsilon)$-time algorithms for this problem achieve
error of $\eta+\epsilon$, which can be far from the optimal bound of
$\mathrm{OPT}+\epsilon$, where $\mathrm{OPT} = \mathbf{E}_{x \sim D_x}
[\eta(x)]$. While it is known that achieving $\mathrm{OPT}+o(1)$ error requires
super-polynomial time in the Statistical Query model, a large gap remains
between known upper and lower bounds.
  In this work, we essentially characterize the efficient learnability of
Massart halfspaces in the Statistical Query (SQ) model. Specifically, we show
that no efficient SQ algorithm for learning Massart halfspaces on
$\mathbb{R}^d$ can achieve error better than $\Omega(\eta)$, even if
$\mathrm{OPT} = 2^{-\log^{c} (d)}$, for any universal constant $c \in (0, 1)$.
Furthermore, when the noise upper bound $\eta$ is close to $1/2$, our error
lower bound becomes $\eta - o_{\eta}(1)$, where the $o_{\eta}(1)$ term goes to
$0$ when $\eta$ approaches $1/2$. Our results provide strong evidence that
known learning algorithms for Massart halfspaces are nearly best possible,
thereby resolving a longstanding open problem in learning theory.
