Random Hinge Forest for Differentiable Learning
  We propose random hinge forests, a simple, efficient, and novel variant of
decision forests. Importantly, random hinge forests can be readily incorporated
as a general component within arbitrary computation graphs that are optimized
end-to-end with stochastic gradient descent or variants thereof. We derive
random hinge forest and ferns, focusing on their sparse and efficient nature,
their min-max margin property, strategies to initialize them for arbitrary
network architectures, and the class of optimizers most suitable for optimizing
random hinge forest. The performance and versatility of random hinge forests
are demonstrated by experiments incorporating a variety of of small and large
UCI machine learning data sets and also ones involving the MNIST, Letter, and
USPS image datasets. We compare random hinge forests with random forests and
the more recent backpropagating deep neural decision forests.
