Data Leakage in Federated Averaging
Recent attacks have shown that user data can be reconstructed from FedSGD
updates, thus breaking privacy. However, these attacks are of limited practical
relevance as federated learning typically uses the FedAvg algorithm. It is
generally accepted that reconstructing data from FedAvg updates is much harder
than FedSGD as: (i) there are unobserved intermediate weight updates, (ii) the
order of inputs matters, and (iii) the order of labels changes every epoch. In
this work, we propose a new optimization-based attack which successfully
attacks FedAvg by addressing the above challenges. First, we solve the
optimization problem using automatic differentiation that forces a simulation
of the client's update for the reconstructed labels and inputs so as to match
the received client update. Second, we address the unknown input order by
treating images at different epochs as independent during optimization, while
relating them with a permutation invariant prior. Third, we reconstruct the
labels by estimating the parameters of existing FedSGD attacks at every FedAvg
step. On the popular FEMNIST dataset, we demonstrate that on average we
successfully reconstruct >45% of the client's images from realistic FedAvg
updates computed on 10 local epochs of 10 batches each with 5 images, compared
to only <10% using the baseline. These findings indicate that many real-world
federated learning implementations based on FedAvg are vulnerable.