Statistically Near-Optimal Hypothesis Selection
  Hypothesis Selection is a fundamental distribution learning problem where
given a comparator-class $Q=\{q_1,\ldots, q_n\}$ of distributions, and a
sampling access to an unknown target distribution $p$, the goal is to output a
distribution $q$ such that $\mathsf{TV}(p,q)$ is close to $opt$, where $opt =
\min_i\{\mathsf{TV}(p,q_i)\}$ and $\mathsf{TV}(\cdot, \cdot)$ denotes the
total-variation distance. Despite the fact that this problem has been studied
since the 19th century, its complexity in terms of basic resources, such as
number of samples and approximation guarantees, remains unsettled (this is
discussed, e.g., in the charming book by Devroye and Lugosi `00). This is in
stark contrast with other (younger) learning settings, such as PAC learning,
for which these complexities are well understood.
  We derive an optimal $2$-approximation learning strategy for the Hypothesis
Selection problem, outputting $q$ such that $\mathsf{TV}(p,q) \leq2 \cdot opt +
\eps$, with a (nearly) optimal sample complexity of~$\tilde O(\log
n/\epsilon^2)$. This is the first algorithm that simultaneously achieves the
best approximation factor and sample complexity: previously, Bousquet, Kane,
and Moran (COLT `19) gave a learner achieving the optimal $2$-approximation,
but with an exponentially worse sample complexity of $\tilde
O(\sqrt{n}/\epsilon^{2.5})$, and Yatracos~(Annals of Statistics `85) gave a
learner with optimal sample complexity of $O(\log n /\epsilon^2)$ but with a
sub-optimal approximation factor of $3$.
