Exploring the Security Boundary of Data Reconstruction via Neuron
  Exclusivity Analysis
  Among existing privacy attacks on the gradient of neural networks, \emph{data
reconstruction attack}, which reverse engineers the training batch from the
gradient, poses a severe threat on the private training data. Despite its
empirical success on large architectures and small training batches, unstable
reconstruction accuracy is also observed when a smaller architecture or a
larger batch is under attack. Due to the weak interpretability of existing
learning-based attacks, there is little known on why, when and how data
reconstruction attack is feasible.
  In our work, we perform the first analytic study on the security boundary of
data reconstruction from gradient via a microcosmic view on neural networks
with rectified linear units (ReLUs), the most popular activation function in
practice. For the first time, we characterize the insecure/secure boundary of
data reconstruction attack in terms of the \emph{neuron exclusivity state} of a
training batch, indexed by the number of \emph{\textbf{Ex}clusively
\textbf{A}ctivated \textbf{N}eurons} (ExANs, i.e., a ReLU activated by only one
sample in a batch). Intuitively, we show a training batch with more ExANs are
more vulnerable to data reconstruction attack and vice versa. On the one hand,
we construct a novel deterministic attack algorithm which substantially
outperforms previous attacks for reconstructing training batches lying in the
insecure boundary of a neural network. Meanwhile, for training batches lying in
the secure boundary, we prove the impossibility of unique reconstruction, based
on which an exclusivity reduction strategy is devised to enlarge the secure
boundary for mitigation purposes.
