Res-embedding for Deep Learning Based Click-Through Rate Prediction
  Modeling
  Recently, click-through rate (CTR) prediction models have evolved from
shallow methods to deep neural networks. Most deep CTR models follow an
Embedding\&MLP paradigm, that is, first mapping discrete id features, e.g. user
visited items, into low dimensional vectors with an embedding module, then
learn a multi-layer perception (MLP) to fit the target. In this way, embedding
module performs as the representative learning and plays a key role in the
model performance. However, in many real-world applications, deep CTR model
often suffers from poor generalization performance, which is mostly due to the
learning of embedding parameters. In this paper, we model user behavior using
an interest delay model, study carefully the embedding mechanism, and obtain
two important results: (i) We theoretically prove that small aggregation radius
of embedding vectors of items which belongs to a same user interest domain will
result in good generalization performance of deep CTR model. (ii) Following our
theoretical analysis, we design a new embedding structure named res-embedding.
In res-embedding module, embedding vector of each item is the sum of two
components: (i) a central embedding vector calculated from an item-based
interest graph (ii) a residual embedding vector with its scale to be relatively
small. Empirical evaluation on several public datasets demonstrates the
effectiveness of the proposed res-embedding structure, which brings significant
improvement on the model performance.
