Layer-stacked Attention for Heterogeneous Network Embedding
  The heterogeneous network is a robust data abstraction that can model
entities of different types interacting in various ways. Such heterogeneity
brings rich semantic information but presents nontrivial challenges in
aggregating the heterogeneous relationships between objects - especially those
of higher-order indirect relations. Recent graph neural network approaches for
representation learning on heterogeneous networks typically employ the
attention mechanism, which is often only optimized for predictions based on
direct links. Furthermore, even though most deep learning methods can aggregate
higher-order information by building deeper models, such a scheme can diminish
the degree of interpretability. To overcome these challenges, we explore an
architecture - Layer-stacked ATTention Embedding (LATTE) - that automatically
decomposes higher-order meta relations at each layer to extract the relevant
heterogeneous neighborhood structures for each node. Additionally, by
successively stacking layer representations, the learned node embedding offers
a more interpretable aggregation scheme for nodes of different types at
different neighborhood ranges. We conducted experiments on several benchmark
heterogeneous network datasets. In both transductive and inductive node
classification tasks, LATTE can achieve state-of-the-art performance compared
to existing approaches, all while offering a lightweight model. With extensive
experimental analyses and visualizations, the framework can demonstrate the
ability to extract informative insights on heterogeneous networks.
