Sequence-Aware Factorization Machines for Temporal Predictive Analytics
  In various web applications like targeted advertising and recommender
systems, the available categorical features (e.g., product type) are often of
great importance but sparse. As a widely adopted solution, models based on
Factorization Machines (FMs) are capable of modelling high-order interactions
among features for effective sparse predictive analytics. As the volume of
web-scale data grows exponentially over time, sparse predictive analytics
inevitably involves dynamic and sequential features. However, existing FM-based
models assume no temporal orders in the data, and are unable to capture the
sequential dependencies or patterns within the dynamic features, impeding the
performance and adaptivity of these methods. Hence, in this paper, we propose a
novel Sequence-Aware Factorization Machine (SeqFM) for temporal predictive
analytics, which models feature interactions by fully investigating the effect
of sequential dependencies. As static features (e.g., user gender) and dynamic
features (e.g., user interacted items) express different semantics, we
innovatively devise a multi-view self-attention scheme that separately models
the effect of static features, dynamic features and the mutual interactions
between static and dynamic features in three different views. In SeqFM, we
further map the learned representations of feature interactions to the desired
output with a shared residual network. To showcase the versatility and
generalizability of SeqFM, we test SeqFM in three popular application scenarios
for FM-based models, namely ranking, classification and regression tasks.
Extensive experimental results on six large-scale datasets demonstrate the
superior effectiveness and efficiency of SeqFM.
