Learning Interpretable Characteristic Kernels via Decision Forests
  Decision forests are popular tools for classification and regression. These
forests naturally produce proximity matrices measuring how often each pair of
observations lies in the same leaf node. It has been demonstrated that these
proximity matrices can be thought of as kernels, connecting the decision forest
literature to the extensive kernel machine literature. While other kernels are
known to have strong theoretical properties such as being characteristic, no
similar result is available for any decision forest based kernel. In this
manuscript,we prove that the decision forest induced proximity can be made
characteristic, which can be used to yield a universally consistent statistic
for testing independence. We demonstrate the performance of the induced kernel
on a suite of 20 high-dimensional independence test settings. We also show how
this learning kernel offers insights into relative feature importance. The
decision forest induced kernel typically achieves substantially higher testing
power than existing popular methods in statistical tests.
