Exploiting Behavioral Consistence for Universal User Representation
  User modeling is critical for developing personalized services in industry. A
common way for user modeling is to learn user representations that can be
distinguished by their interests or preferences. In this work, we focus on
developing universal user representation model. The obtained universal
representations are expected to contain rich information, and be applicable to
various downstream applications without further modifications (e.g., user
preference prediction and user profiling). Accordingly, we can be free from the
heavy work of training task-specific models for every downstream task as in
previous works. In specific, we propose Self-supervised User Modeling Network
(SUMN) to encode behavior data into the universal representation. It includes
two key components. The first one is a new learning objective, which guides the
model to fully identify and preserve valuable user information under a
self-supervised learning framework. The other one is a multi-hop aggregation
layer, which benefits the model capacity in aggregating diverse behaviors.
Extensive experiments on benchmark datasets show that our approach can
outperform state-of-the-art unsupervised representation methods, and even
compete with supervised ones.
