The Optimal Approximation Factor in Density Estimation
  Consider the following problem: given two arbitrary densities $q_1,q_2$ and a
sample-access to an unknown target density $p$, find which of the $q_i$'s is
closer to $p$ in total variation.
  A remarkable result due to Yatracos shows that this problem is tractable in
the following sense: there exists an algorithm that uses $O(\epsilon^{-2})$
samples from $p$ and outputs~$q_i$ such that with high probability, $TV(q_i,p)
\leq 3\cdot\mathsf{opt} + \epsilon$, where $\mathsf{opt}=
\min\{TV(q_1,p),TV(q_2,p)\}$. Moreover, this result extends to any finite class
of densities $\mathcal{Q}$: there exists an algorithm that outputs the best
density in $\mathcal{Q}$ up to a multiplicative approximation factor of 3.
  We complement and extend this result by showing that: (i) the factor 3 can
not be improved if one restricts the algorithm to output a density from
$\mathcal{Q}$, and (ii) if one allows the algorithm to output arbitrary
densities (e.g.\ a mixture of densities from $\mathcal{Q}$), then the
approximation factor can be reduced to 2, which is optimal. In particular this
demonstrates an advantage of improper learning over proper in this setup.
  We develop two approaches to achieve the optimal approximation factor of 2:
an adaptive one and a static one. Both approaches are based on a geometric
point of view of the problem and rely on estimating surrogate metrics to the
total variation. Our sample complexity bounds exploit techniques from {\it
Adaptive Data Analysis}.
