Smoothed Analysis of Online and Differentially Private Learning
  Practical and pervasive needs for robustness and privacy in algorithms have
inspired the design of online adversarial and differentially private learning
algorithms. The primary quantity that characterizes learnability in these
settings is the Littlestone dimension of the class of hypotheses [Ben-David et
al., 2009, Alon et al., 2019]. This characterization is often interpreted as an
impossibility result because classes such as linear thresholds and neural
networks have infinite Littlestone dimension. In this paper, we apply the
framework of smoothed analysis [Spielman and Teng, 2004], in which
adversarially chosen inputs are perturbed slightly by nature. We show that
fundamentally stronger regret and error guarantees are possible with smoothed
adversaries than with worst-case adversaries. In particular, we obtain regret
and privacy error bounds that depend only on the VC dimension and the
bracketing number of a hypothesis class, and on the magnitudes of the
perturbations.
