PRECODE - A Generic Model Extension to Prevent Deep Gradient Leakage
  Collaborative training of neural networks leverages distributed data by
exchanging gradient information between different clients. Although training
data entirely resides with the clients, recent work shows that training data
can be reconstructed from such exchanged gradient information. To enhance
privacy, gradient perturbation techniques have been proposed. However, they
come at the cost of reduced model performance, increased convergence time, or
increased data demand. In this paper, we introduce PRECODE, a PRivacy EnhanCing
mODulE that can be used as generic extension for arbitrary model architectures.
We propose a simple yet effective realization of PRECODE using variational
modeling. The stochastic sampling induced by variational modeling effectively
prevents privacy leakage from gradients and in turn preserves privacy of data
owners. We evaluate PRECODE using state of the art gradient inversion attacks
on two different model architectures trained on three datasets. In contrast to
commonly used defense mechanisms, we find that our proposed modification
consistently reduces the attack success rate to 0% while having almost no
negative impact on model training and final performance. As a result, PRECODE
reveals a promising path towards privacy enhancing model extensions.
