Deep Hierarchical Machine: a Flexible Divide-and-Conquer Architecture
  We propose Deep Hierarchical Machine (DHM), a model inspired from the
divide-and-conquer strategy while emphasizing representation learning ability
and flexibility. A stochastic routing framework as used by recent deep neural
decision/regression forests is incorporated, but we remove the need to evaluate
unnecessary computation paths by utilizing a different topology and introducing
a probabilistic pruning technique. We also show a specified version of DHM
(DSHM) for efficiency, which inherits the sparse feature extraction process as
in traditional decision tree with pixel-difference feature. To achieve sparse
feature extraction, we propose to utilize sparse convolution operation in DSHM
and show one possibility of introducing sparse convolution kernels by using
local binary convolution layer. DHM can be applied to both classification and
regression problems, and we validate it on standard image classification and
face alignment tasks to show its advantages over past architectures.
