A Learning-Theoretic Framework for Certified Auditing of Machine
  Learning Models
  Responsible use of machine learning requires that models be audited for
undesirable properties. However, how to do principled auditing in a general
setting has remained ill-understood. In this paper, we propose a formal
learning-theoretic framework for auditing. We propose algorithms for auditing
linear classifiers for feature sensitivity using label queries as well as
different kinds of explanations, and provide performance guarantees. Our
results illustrate that while counterfactual explanations can be extremely
helpful for auditing, anchor explanations may not be as beneficial in the worst
case.
