DHEN: A Deep and Hierarchical Ensemble Network for Large-Scale
  Click-Through Rate Prediction
  Learning feature interactions is important to the model performance of online
advertising services. As a result, extensive efforts have been devoted to
designing effective architectures to learn feature interactions. However, we
observe that the practical performance of those designs can vary from dataset
to dataset, even when the order of interactions claimed to be captured is the
same. That indicates different designs may have different advantages and the
interactions captured by them have non-overlapping information. Motivated by
this observation, we propose DHEN - a deep and hierarchical ensemble
architecture that can leverage strengths of heterogeneous interaction modules
and learn a hierarchy of the interactions under different orders. To overcome
the challenge brought by DHEN's deeper and multi-layer structure in training,
we propose a novel co-designed training system that can further improve the
training efficiency of DHEN. Experiments of DHEN on large-scale dataset from
CTR prediction tasks attained 0.27\% improvement on the Normalized Entropy (NE)
of prediction and 1.2x better training throughput than state-of-the-art
baseline, demonstrating their effectiveness in practice.
