A New Analysis of Differential Privacy's Generalization Guarantees
  We give a new proof of the "transfer theorem" underlying adaptive data
analysis: that any mechanism for answering adaptively chosen statistical
queries that is differentially private and sample-accurate is also accurate
out-of-sample. Our new proof is elementary and gives structural insights that
we expect will be useful elsewhere. We show: 1) that differential privacy
ensures that the expectation of any query on the posterior distribution on
datasets induced by the transcript of the interaction is close to its true
value on the data distribution, and 2) sample accuracy on its own ensures that
any query answer produced by the mechanism is close to its posterior
expectation with high probability. This second claim follows from a thought
experiment in which we imagine that the dataset is resampled from the posterior
distribution after the mechanism has committed to its answers. The transfer
theorem then follows by summing these two bounds, and in particular, avoids the
"monitor argument" used to derive high probability bounds in prior work. An
upshot of our new proof technique is that the concrete bounds we obtain are
substantially better than the best previously known bounds, even though the
improvements are in the constants, rather than the asymptotics (which are known
to be tight). As we show, our new bounds outperform the naive
"sample-splitting" baseline at dramatically smaller dataset sizes compared to
the previous state of the art, bringing techniques from this literature closer
to practicality.
