Learning Entangled Single-Sample Distributions via Iterative Trimming
  In the setting of entangled single-sample distributions, the goal is to
estimate some common parameter shared by a family of distributions, given one
\emph{single} sample from each distribution. We study mean estimation and
linear regression under general conditions, and analyze a simple and
computationally efficient method based on iteratively trimming samples and
re-estimating the parameter on the trimmed sample set. We show that the method
in logarithmic iterations outputs an estimation whose error only depends on the
noise level of the $\lceil \alpha n \rceil$-th noisiest data point where
$\alpha$ is a constant and $n$ is the sample size. This means it can tolerate a
constant fraction of high-noise points. These are the first such results for
the method under our general conditions. It also justifies the wide application
and empirical success of iterative trimming in practice. Our theoretical
results are complemented by experiments on synthetic data.
