Learning Numeric Optimal Differentially Private Truncated Additive
  Mechanisms
  Differentially private (DP) mechanisms face the challenge of providing
accurate results while protecting their inputs: the privacy-utility trade-off.
A simple but powerful technique for DP adds noise to sensitivity-bounded query
outputs to blur the exact query output: additive mechanisms. While a vast body
of work considers infinitely wide noise distributions, some applications (e.g.,
real-time operating systems) require hard bounds on the deviations from the
real query, and only limited work on such mechanisms exist. An additive
mechanism with truncated noise (i.e., with bounded range) can offer such hard
bounds. We introduce a gradient-descent-based tool to learn truncated noise for
additive mechanisms with strong utility bounds while simultaneously optimizing
for differential privacy under sequential composition, i.e., scenarios where
multiple noisy queries on the same data are revealed. Our method can learn
discrete noise patterns and not only hyper-parameters of a predefined
probability distribution. For sensitivity bounded mechanisms, we show that it
is sufficient to consider symmetric and that\new{, for from the mean
monotonically falling noise,} ensuring privacy for a pair of representative
query outputs guarantees privacy for all pairs of inputs (that differ in one
element). We find that the utility-privacy trade-off curves of our generated
noise are remarkably close to truncated Gaussians and even replicate their
shape for $l_2$ utility-loss. For a low number of compositions, we also
improved DP-SGD (sub-sampling). Moreover, we extend Moments Accountant to
truncated distributions, allowing to incorporate mechanism output events with
varying input-dependent zero occurrence probability.
