Learnable Embedding Sizes for Recommender Systems
  The embedding-based representation learning is commonly used in deep learning
recommendation models to map the raw sparse features to dense vectors. The
traditional embedding manner that assigns a uniform size to all features has
two issues. First, the numerous features inevitably lead to a gigantic
embedding table that causes a high memory usage cost. Second, it is likely to
cause the over-fitting problem for those features that do not require too large
representation capacity. Existing works that try to address the problem always
cause a significant drop in recommendation performance or suffers from the
limitation of unaffordable training time cost. In this paper, we proposed a
novel approach, named PEP (short for Plug-in Embedding Pruning), to reduce the
size of the embedding table while avoiding the drop of recommendation accuracy.
PEP prunes embedding parameter where the pruning threshold(s) can be adaptively
learned from data. Therefore we can automatically obtain a mixed-dimension
embedding-scheme by pruning redundant parameters for each feature. PEP is a
general framework that can plug in various base recommendation models.
Extensive experiments demonstrate it can efficiently cut down embedding
parameters and boost the base model's performance. Specifically, it achieves
strong recommendation performance while reducing 97-99% parameters. As for the
computation cost, PEP only brings an additional 20-30% time cost compared with
base models. Codes are available at
https://github.com/ssui-liu/learnable-embed-sizes-for-RecSys.
