Improved Rates for Differentially Private Stochastic Convex Optimization
  with Heavy-Tailed Data
  We study stochastic convex optimization with heavy-tailed data under the
constraint of differential privacy (DP). Most prior work on this problem is
restricted to the case where the loss function is Lipschitz. Instead, as
introduced by Wang, Xiao, Devadas, and Xu \cite{WangXDX20}, we study general
convex loss functions with the assumption that the distribution of gradients
has bounded $k$-th moments. We provide improved upper bounds on the excess
population risk under concentrated DP for convex and strongly convex loss
functions. Along the way, we derive new algorithms for private mean estimation
of heavy-tailed distributions, under both pure and concentrated DP. Finally, we
prove nearly-matching lower bounds for private stochastic convex optimization
with strongly convex losses and mean estimation, showing new separations
between pure and concentrated DP.
