PermuteAttack: Counterfactual Explanation of Machine Learning Credit
  Scorecards
  This paper is a note on new directions and methodologies for validation and
explanation of Machine Learning (ML) models employed for retail credit scoring
in finance. Our proposed framework draws motivation from the field of
Artificial Intelligence (AI) security and adversarial ML where the need for
certifying the performance of the ML algorithms in the face of their
overwhelming complexity poses a need for rethinking the traditional notions of
model architecture selection, sensitivity analysis and stress testing. Our
point of view is that the phenomenon of adversarial perturbations when detached
from the AI security domain, has purely algorithmic roots and fall within the
scope of model risk assessment. We propose a model criticism and explanation
framework based on adversarially generated counterfactual examples for tabular
data. A counterfactual example to a given instance in this context is defined
as a synthetically generated data point sampled from the estimated data
distribution which is treated differently by a model. The counterfactual
examples can be used to provide a black-box instance-level explanation of the
model behaviour as well as studying the regions in the input space where the
model performance deteriorates. Adversarial example generating algorithms are
extensively studied in the image and natural language processing (NLP) domains.
However, most financial data come in tabular format and naive application of
the existing techniques on this class of datasets generates unrealistic
samples. In this paper, we propose a counterfactual example generation method
capable of handling tabular data including discrete and categorical variables.
Our proposed algorithm uses a gradient-free optimization based on genetic
algorithms and therefore is applicable to any classification model.
