TCL: Transformer-based Dynamic Graph Modelling via Contrastive Learning
  Dynamic graph modeling has recently attracted much attention due to its
extensive applications in many real-world scenarios, such as recommendation
systems, financial transactions, and social networks. Although many works have
been proposed for dynamic graph modeling in recent years, effective and
scalable models are yet to be developed. In this paper, we propose a novel
graph neural network approach, called TCL, which deals with the
dynamically-evolving graph in a continuous-time fashion and enables effective
dynamic node representation learning that captures both the temporal and
topology information. Technically, our model contains three novel aspects.
First, we generalize the vanilla Transformer to temporal graph learning
scenarios and design a graph-topology-aware transformer. Secondly, on top of
the proposed graph transformer, we introduce a two-stream encoder that
separately extracts representations from temporal neighborhoods associated with
the two interaction nodes and then utilizes a co-attentional transformer to
model inter-dependencies at a semantic level. Lastly, we are inspired by the
recently developed contrastive learning and propose to optimize our model by
maximizing mutual information (MI) between the predictive representations of
two future interaction nodes. Benefiting from this, our dynamic representations
can preserve high-level (or global) semantics about interactions and thus is
robust to noisy interactions. To the best of our knowledge, this is the first
attempt to apply contrastive learning to representation learning on dynamic
graphs. We evaluate our model on four benchmark datasets for interaction
prediction and experiment results demonstrate the superiority of our model.
