TOFU: Towards Obfuscated Federated Updates by Encoding Weight Updates
  into Gradients from Proxy Data
  Advances in Federated Learning and an abundance of user data have enabled
rich collaborative learning between multiple clients, without sharing user
data. This is done via a central server that aggregates learning in the form of
weight updates. However, this comes at the cost of repeated expensive
communication between the clients and the server, and concerns about
compromised user privacy. The inversion of gradients into the data that
generated them is termed data leakage. Encryption techniques can be used to
counter this leakage, but at added expense. To address these challenges of
communication efficiency and privacy, we propose TOFU, a novel algorithm which
generates proxy data that encodes the weight updates for each client in its
gradients. Instead of weight updates, this proxy data is now shared. Since
input data is far lower in dimensional complexity than weights, this encoding
allows us to send much lesser data per communication round. Additionally, the
proxy data resembles noise, and even perfect reconstruction from data leakage
attacks would invert the decoded gradients into unrecognizable noise, enhancing
privacy. We show that TOFU enables learning with less than 1% and 7% accuracy
drops on MNIST and on CIFAR-10 datasets, respectively. This drop can be
recovered via a few rounds of expensive encrypted gradient exchange. This
enables us to learn to near-full accuracy in a federated setup, while being 4x
and 6.6x more communication efficient than the standard Federated Averaging
algorithm on MNIST and CIFAR-10, respectively.
