Adaptive Statistical Learning with Bayesian Differential Privacy
  In statistical learning, a dataset is often partitioned into two parts: the
training set and the holdout (i.e., testing) set. For instance, the training
set is used to learn a predictor, and then the holdout set is used for
estimating the accuracy of the predictor on the true distribution. However,
often in practice, the holdout dataset is reused and the estimates tested on
the holdout dataset are chosen adaptively based on the results of prior
estimates, leading to that the predictor may become dependent of the holdout
set. Hence, overfitting may occur, and the learned models may not generalize
well to the unseen datasets. Prior studies have established connections between
the stability of a learning algorithm and its ability to generalize, but the
traditional generalization is not robust to adaptive composition. Recently,
Dwork et al. in NIPS, STOC, and Science 2015 show that the holdout dataset from
i.i.d. data samples can be reused in adaptive statistical learning, if the
estimates are perturbed and coordinated using techniques developed for
differential privacy, which is a widely used notion to quantify privacy. Yet,
the results of Dwork et al. are applicable to only the case of i.i.d. samples.
In contrast, correlations between data samples exist because of various
behavioral, social, and genetic relationships between users. Our results in
adaptive statistical learning generalize the results of Dwork et al. for i.i.d.
data samples to arbitrarily correlated data. Specifically, we show that the
holdout dataset from correlated samples can be reused in adaptive statistical
learning, if the estimates are perturbed and coordinated using techniques
developed for Bayesian differential privacy, which is a privacy notion recently
introduced by Yang et al. in SIGMOD 2015 to broaden the application scenarios
of differential privacy when data records are correlated.
