An Interpretable Probabilistic Approach for Demystifying Black-box
  Predictive Models
  The use of sophisticated machine learning models for critical decision making
is faced with a challenge that these models are often applied as a "black-box".
This has led to an increased interest in interpretable machine learning, where
post hoc interpretation presents a useful mechanism for generating
interpretations of complex learning models. In this paper, we propose a novel
approach underpinned by an extended framework of Bayesian networks for
generating post hoc interpretations of a black-box predictive model. The
framework supports extracting a Bayesian network as an approximation of the
black-box model for a specific prediction. Compared to the existing post hoc
interpretation methods, the contribution of our approach is three-fold.
Firstly, the extracted Bayesian network, as a probabilistic graphical model,
can provide interpretations about not only what input features but also why
these features contributed to a prediction. Secondly, for complex decision
problems with many features, a Markov blanket can be generated from the
extracted Bayesian network to provide interpretations with a focused view on
those input features that directly contributed to a prediction. Thirdly, the
extracted Bayesian network enables the identification of four different rules
which can inform the decision-maker about the confidence level in a prediction,
thus helping the decision-maker assess the reliability of predictions learned
by a black-box model. We implemented the proposed approach, applied it in the
context of two well-known public datasets and analysed the results, which are
made available in an open-source repository.
