Interpretation and Simplification of Deep Forest
  This paper proposes a new method for interpreting and simplifying a black box
model of a deep random forest (RF) using a proposed rule elimination. In deep
RF, a large number of decision trees are connected to multiple layers, thereby
making an analysis difficult. It has a high performance similar to that of a
deep neural network (DNN), but achieves a better generalizability. Therefore,
in this study, we consider quantifying the feature contributions and frequency
of the fully trained deep RF in the form of a decision rule set. The feature
contributions provide a basis for determining how features affect the decision
process in a rule set. Model simplification is achieved by eliminating
unnecessary rules by measuring the feature contributions. Consequently, the
simplified model has fewer parameters and rules than before. Experiment results
have shown that a feature contribution analysis allows a black box model to be
decomposed for quantitatively interpreting a rule set. The proposed method was
successfully applied to various deep RF models and benchmark datasets while
maintaining a robust performance despite the elimination of a large number of
rules.
