Distributed Online Learning Algorithm With Differential Privacy Strategy
  for Convex Nondecomposable Global Objectives
  In this paper, we deal with a general distributed constrained online learning
problem with privacy over time-varying networks, where a class of
nondecomposable objective functions are considered. Under this setting, each
node only controls a part of the global decision variable, and the goal of all
nodes is to collaboratively minimize the global objective over a time horizon
$T$ while guarantees the security of the transmitted information. For such
problems, we first design a novel generic algorithm framework, named as DPSDA,
of differentially private distributed online learning using the Laplace
mechanism and the stochastic variants of dual averaging method. Then, we
propose two algorithms, named as DPSDA-C and DPSDA-PS, under this framework.
Theoretical results show that both algorithms attain an expected regret upper
bound in $\mathcal{O}( \sqrt{T} )$ when the objective function is convex, which
matches the best utility achievable by cutting-edge algorithms. Finally,
numerical experiment results on both real-world and randomly generated datasets
verify the effectiveness of our algorithms.
