Functional Gradient Boosting based on Residual Network Perception
  Residual Networks (ResNets) have become state-of-the-art models in deep
learning and several theoretical studies have been devoted to understanding why
ResNet works so well. One attractive viewpoint on ResNet is that it is
optimizing the risk in a functional space by combining an ensemble of effective
features. In this paper, we adopt this viewpoint to construct a new gradient
boosting method, which is known to be very powerful in data analysis. To do so,
we formalize the gradient boosting perspective of ResNet mathematically using
the notion of functional gradients and propose a new method called ResFGB for
classification tasks by leveraging ResNet perception. Two types of
generalization guarantees are provided from the optimization perspective: one
is the margin bound and the other is the expected risk bound by the
sample-splitting technique. Experimental results show superior performance of
the proposed method over state-of-the-art methods such as LightGBM.
