MOI-Mixer: Improving MLP-Mixer with Multi Order Interactions in
  Sequential Recommendation
  Successful sequential recommendation systems rely on accurately capturing the
user's short-term and long-term interest. Although Transformer-based models
achieved state-of-the-art performance in the sequential recommendation task,
they generally require quadratic memory and time complexity to the sequence
length, making it difficult to extract the long-term interest of users. On the
other hand, Multi-Layer Perceptrons (MLP)-based models, renowned for their
linear memory and time complexity, have recently shown competitive results
compared to Transformer in various tasks. Given the availability of a massive
amount of the user's behavior history, the linear memory and time complexity of
MLP-based models make them a promising alternative to explore in the sequential
recommendation task. To this end, we adopted MLP-based models in sequential
recommendation but consistently observed that MLP-based methods obtain lower
performance than those of Transformer despite their computational benefits.
From experiments, we observed that introducing explicit high-order interactions
to MLP layers mitigates such performance gap. In response, we propose the
Multi-Order Interaction (MOI) layer, which is capable of expressing an
arbitrary order of interactions within the inputs while maintaining the memory
and time complexity of the MLP layer. By replacing the MLP layer with the MOI
layer, our model was able to achieve comparable performance with
Transformer-based models while retaining the MLP-based models' computational
benefits.
