Neural Random Forest Imitation
  We present Neural Random Forest Imitation - a novel approach for transforming
random forests into neural networks. Existing methods produce very inefficient
architectures and do not scale. In this paper, we introduce a new method for
generating data from a random forest and learning a neural network that
imitates it. Without any additional training data, this transformation creates
very efficient neural networks that learn the decision boundaries of a random
forest. The generated model is fully differentiable and can be combined with
the feature extraction in a single pipeline enabling further end-to-end
processing. Experiments on several real-world benchmark datasets demonstrate
outstanding performance in terms of scalability, accuracy, and learning with
very few training examples. Compared to state-of-the-art mappings, we
significantly reduce the network size while achieving the same or even improved
accuracy due to better generalization.
