FedAUXfdp: Differentially Private One-Shot Federated Distillation
  Federated learning suffers in the case of non-iid local datasets, i.e., when
the distributions of the clients' data are heterogeneous. One promising
approach to this challenge is the recently proposed method FedAUX, an
augmentation of federated distillation with robust results on even highly
heterogeneous client data. FedAUX is a partially $(\epsilon,
\delta)$-differentially private method, insofar as the clients' private data is
protected in only part of the training it takes part in. This work contributes
a fully differentially private extension, termed FedAUXfdp. In experiments with
deep networks on large-scale image datasets, FedAUXfdp with strong differential
privacy guarantees performs significantly better than other equally privatized
SOTA baselines on non-iid client data in just a single communication round.
Full privatization results in a negligible reduction in accuracy at all levels
of data heterogeneity.
