Closure Properties for Private Classification and Online Prediction
  Let~$\cH$ be a class of boolean functions and consider a {\it composed class}
$\cH'$ that is derived from~$\cH$ using some arbitrary aggregation rule (for
example, $\cH'$ may be the class of all 3-wise majority-votes of functions in
$\cH$). We upper bound the Littlestone dimension of~$\cH'$ in terms of that
of~$\cH$. As a corollary, we derive closure properties for online learning and
private PAC learning.
  The derived bounds on the Littlestone dimension exhibit an undesirable
exponential dependence. For private learning, we prove close to optimal bounds
that circumvents this suboptimal dependency. The improved bounds on the sample
complexity of private learning are derived algorithmically via transforming a
private learner for the original class $\cH$ to a private learner for the
composed class~$\cH'$. Using the same ideas we show that any ({\em proper or
improper}) private algorithm that learns a class of functions $\cH$ in the
realizable case (i.e., when the examples are labeled by some function in the
class) can be transformed to a private algorithm that learns the class $\cH$ in
the agnostic case.
