NoPeek: Information leakage reduction to share activations in
  distributed deep learning
  For distributed machine learning with sensitive data, we demonstrate how
minimizing distance correlation between raw data and intermediary
representations reduces leakage of sensitive raw data patterns across client
communications while maintaining model accuracy. Leakage (measured using
distance correlation between input and intermediate representations) is the
risk associated with the invertibility of raw data from intermediary
representations. This can prevent client entities that hold sensitive data from
using distributed deep learning services. We demonstrate that our method is
resilient to such reconstruction attacks and is based on reduction of distance
correlation between raw data and learned representations during training and
inference with image datasets. We prevent such reconstruction of raw data while
maintaining information required to sustain good classification accuracies.
