A Neural Tangent Kernel Perspective of Infinite Tree Ensembles
  In practical situations, the tree ensemble is one of the most popular models
along with neural networks. A soft tree is a variant of a decision tree.
Instead of using a greedy method for searching splitting rules, the soft tree
is trained using a gradient method in which the entire splitting operation is
formulated in a differentiable form. Although ensembles of such soft trees have
been used increasingly in recent years, little theoretical work has been done
to understand their behavior. By considering an ensemble of infinite soft
trees, this paper introduces and studies the Tree Neural Tangent Kernel (TNTK),
which provides new insights into the behavior of the infinite ensemble of soft
trees. Using the TNTK, we theoretically identify several non-trivial
properties, such as global convergence of the training, the equivalence of the
oblivious tree structure, and the degeneracy of the TNTK induced by the
deepening of the trees.
