Individual Privacy Accounting via a Renyi Filter
  We consider a sequential setting in which a single dataset of individuals is
used to perform adaptively-chosen analyses, while ensuring that the
differential privacy loss of each participant does not exceed a pre-specified
privacy budget. The standard approach to this problem relies on bounding a
worst-case estimate of the privacy loss over all individuals and all possible
values of their data, for every single analysis. Yet, in many scenarios this
approach is overly conservative, especially for "typical" data points which
incur little privacy loss by participation in most of the analyses. In this
work, we give a method for tighter privacy loss accounting based on the value
of a personalized privacy loss estimate for each individual in each analysis.
To implement the accounting method we design a filter for R\'enyi differential
privacy. A filter is a tool that ensures that the privacy parameter of a
composed sequence of algorithms with adaptively-chosen privacy parameters does
not exceed a pre-specified budget. Our filter is simpler and tighter than the
known filter for $(\epsilon,\delta)$-differential privacy by Rogers et al. We
apply our results to the analysis of noisy gradient descent and show that
personalized accounting can be practical, easy to implement, and can only make
the privacy-utility tradeoff tighter.
