Dual Correction Strategy for Ranking Distillation in Top-N Recommender
  System
  Knowledge Distillation (KD), which transfers the knowledge of a well-trained
large model (teacher) to a small model (student), has become an important area
of research for practical deployment of recommender systems. Recently, Relaxed
Ranking Distillation (RRD) has shown that distilling the ranking information in
the recommendation list significantly improves the performance. However, the
method still has limitations in that 1) it does not fully utilize the
prediction errors of the student model, which makes the training not fully
efficient, and 2) it only distills the user-side ranking information, which
provides an insufficient view under the sparse implicit feedback. This paper
presents Dual Correction strategy for Distillation (DCD), which transfers the
ranking information from the teacher model to the student model in a more
efficient manner. Most importantly, DCD uses the discrepancy between the
teacher model and the student model predictions to decide which knowledge to be
distilled. By doing so, DCD essentially provides the learning guidance tailored
to "correcting" what the student model has failed to accurately predict. This
process is applied for transferring the ranking information from the user-side
as well as the item-side to address sparse implicit user feedback. Our
experiments show that the proposed method outperforms the state-of-the-art
baselines, and ablation studies validate the effectiveness of each component.
