Distributionally-Robust Machine Learning Using Locally
  Differentially-Private Data
  We consider machine learning, particularly regression, using
locally-differentially private datasets. The Wasserstein distance is used to
define an ambiguity set centered at the empirical distribution of the dataset
corrupted by local differential privacy noise. The ambiguity set is shown to
contain the probability distribution of unperturbed, clean data. The radius of
the ambiguity set is a function of the privacy budget, spread of the data, and
the size of the problem. Hence, machine learning with locally-differentially
private datasets can be rewritten as a distributionally-robust optimization.
For general distributions, the distributionally-robust optimization problem can
relaxed as a regularized machine learning problem with the Lipschitz constant
of the machine learning model as a regularizer. For linear and logistic
regression, this regularizer is the dual norm of the model parameters. For
Gaussian data, the distributionally-robust optimization problem can be solved
exactly to find an optimal regularizer. This approach results in an entirely
new regularizer for training linear regression models. Training with this novel
regularizer can be posed as a semi-definite program. Finally, the performance
of the proposed distributionally-robust machine learning training is
demonstrated on practical datasets.
