A Holistic Approach to Interpretability in Financial Lending: Models,
  Visualizations, and Summary-Explanations
  Lending decisions are usually made with proprietary models that provide
minimally acceptable explanations to users. In a future world without such
secrecy, what decision support tools would one want to use for justified
lending decisions? This question is timely, since the economy has dramatically
shifted due to a pandemic, and a massive number of new loans will be necessary
in the short term. We propose a framework for such decisions, including a
globally interpretable machine learning model, an interactive visualization of
it, and several types of summaries and explanations for any given decision. The
machine learning model is a two-layer additive risk model, which resembles a
two-layer neural network, but is decomposable into subscales. In this model,
each node in the first (hidden) layer represents a meaningful subscale model,
and all of the nonlinearities are transparent. Our online visualization tool
allows exploration of this model, showing precisely how it came to its
conclusion. We provide three types of explanations that are simpler than, but
consistent with, the global model: case-based reasoning explanations that use
neighboring past cases, a set of features that were the most important for the
model's prediction, and summary-explanations that provide a customized sparse
explanation for any particular lending decision made by the model. Our
framework earned the FICO recognition award for the Explainable Machine
Learning Challenge, which was the first public challenge in the domain of
explainable machine learning.
