Differentially Private Federated Bayesian Optimization with Distributed
  Exploration
  Bayesian optimization (BO) has recently been extended to the federated
learning (FL) setting by the federated Thompson sampling (FTS) algorithm, which
has promising applications such as federated hyperparameter tuning. However,
FTS is not equipped with a rigorous privacy guarantee which is an important
consideration in FL. Recent works have incorporated differential privacy (DP)
into the training of deep neural networks through a general framework for
adding DP to iterative algorithms. Following this general DP framework, our
work here integrates DP into FTS to preserve user-level privacy. We also
leverage the ability of this general DP framework to handle different parameter
vectors, as well as the technique of local modeling for BO, to further improve
the utility of our algorithm through distributed exploration (DE). The
resulting differentially private FTS with DE (DP-FTS-DE) algorithm is endowed
with theoretical guarantees for both the privacy and utility and is amenable to
interesting theoretical insights about the privacy-utility trade-off. We also
use real-world experiments to show that DP-FTS-DE achieves high utility
(competitive performance) with a strong privacy guarantee (small privacy loss)
and induces a trade-off between privacy and utility.
