Lightweight Self-Attentive Sequential Recommendation
  Modern deep neural networks (DNNs) have greatly facilitated the development
of sequential recommender systems by achieving state-of-the-art recommendation
performance on various sequential recommendation tasks. Given a sequence of
interacted items, existing DNN-based sequential recommenders commonly embed
each item into a unique vector to support subsequent computations of the user
interest. However, due to the potentially large number of items, the
over-parameterised item embedding matrix of a sequential recommender has become
a memory bottleneck for efficient deployment in resource-constrained
environments, e.g., smartphones and other edge devices. Furthermore, we observe
that the widely-used multi-head self-attention, though being effective in
modelling sequential dependencies among items, heavily relies on redundant
attention units to fully capture both global and local item-item transition
patterns within a sequence.
  In this paper, we introduce a novel lightweight self-attentive network (LSAN)
for sequential recommendation. To aggressively compress the original embedding
matrix, LSAN leverages the notion of compositional embeddings, where each item
embedding is composed by merging a group of selected base embedding vectors
derived from substantially smaller embedding matrices. Meanwhile, to account
for the intrinsic dynamics of each item, we further propose a temporal
context-aware embedding composition scheme. Besides, we develop an innovative
twin-attention network that alleviates the redundancy of the traditional
multi-head self-attention while retaining full capacity for capturing long- and
short-term (i.e., global and local) item dependencies. Comprehensive
experiments demonstrate that LSAN significantly advances the accuracy and
memory efficiency of existing sequential recommenders.
