Scaling Up Influence Functions
  We address efficient calculation of influence functions for tracking
predictions back to the training data. We propose and analyze a new approach to
speeding up the inverse Hessian calculation based on Arnoldi iteration. With
this improvement, we achieve, to the best of our knowledge, the first
successful implementation of influence functions that scales to full-size
(language and vision) Transformer models with several hundreds of millions of
parameters. We evaluate our approach on image classification and
sequence-to-sequence tasks with tens to a hundred of millions of training
examples. Our code will be available at
https://github.com/google-research/jax-influence.
