Beyond the Low-Degree Algorithm: Mixtures of Subcubes and Their
  Applications
  We introduce the problem of learning mixtures of $k$ subcubes over
$\{0,1\}^n$, which contains many classic learning theory problems as a special
case (and is itself a special case of others). We give a surprising $n^{O(\log
k)}$-time learning algorithm based on higher-order multilinear moments. It is
not possible to learn the parameters because the same distribution can be
represented by quite different models. Instead, we develop a framework for
reasoning about how multilinear moments can pinpoint essential features of the
mixture, like the number of components.
  We also give applications of our algorithm to learning decision trees with
stochastic transitions (which also capture interesting scenarios where the
transitions are deterministic but there are latent variables). Using our
algorithm for learning mixtures of subcubes, we can approximate the Bayes
optimal classifier within additive error $\epsilon$ on $k$-leaf decision trees
with at most $s$ stochastic transitions on any root-to-leaf path in $n^{O(s +
\log k)}\cdot\text{poly}(1/\epsilon)$ time. In this stochastic setting, the
classic Occam algorithms for learning decision trees with zero stochastic
transitions break down, while the low-degree algorithm of Linial et al.
inherently has a quasipolynomial dependence on $1/\epsilon$.
  In contrast, as we will show, mixtures of $k$ subcubes are uniquely
determined by their degree $2 \log k$ moments and hence provide a useful
abstraction for simultaneously achieving the polynomial dependence on
$1/\epsilon$ of the classic Occam algorithms for decision trees and the
flexibility of the low-degree algorithm in being able to accommodate stochastic
transitions. Using our multilinear moment techniques, we also give the first
improved upper and lower bounds since the work of Feldman et al. for the
related but harder problem of learning mixtures of binary product
distributions.
