Opacus: User-Friendly Differential Privacy Library in PyTorch
  We introduce Opacus, a free, open-source PyTorch library for training deep
learning models with differential privacy (hosted at https://opacus.ai). Opacus
is designed for simplicity, flexibility, and speed. It provides a simple and
user-friendly API, and enables machine learning practitioners to make a
training pipeline private by adding as little as two lines to their code. It
supports a wide variety of layers, including multi-head attention, convolution,
LSTM, and embedding, right out of the box, and it also provides the means for
supporting other user-defined layers. Opacus computes batched per-sample
gradients, providing better efficiency compared to the traditional "micro
batch" approach. In this paper we present Opacus, detail the principles that
drove its implementation and unique features, and compare its performance
against other frameworks for differential privacy in ML.
