Interpreting Robust Optimization via Adversarial Influence Functions
  Robust optimization has been widely used in nowadays data science, especially
in adversarial training. However, little research has been done to quantify how
robust optimization changes the optimizers and the prediction losses comparing
to standard training. In this paper, inspired by the influence function in
robust statistics, we introduce the Adversarial Influence Function (AIF) as a
tool to investigate the solution produced by robust optimization. The proposed
AIF enjoys a closed-form and can be calculated efficiently. To illustrate the
usage of AIF, we apply it to study model sensitivity -- a quantity defined to
capture the change of prediction losses on the natural data after implementing
robust optimization. We use AIF to analyze how model complexity and randomized
smoothing affect the model sensitivity with respect to specific models. We
further derive AIF for kernel regressions, with a particular application to
neural tangent kernels, and experimentally demonstrate the effectiveness of the
proposed AIF. Lastly, the theories of AIF will be extended to distributional
robust optimization.
