Adaptive Label Smoothing To Regularize Large-Scale Graph Training
  Graph neural networks (GNNs), which learn the node representations by
recursively aggregating information from its neighbors, have become a
predominant computational tool in many domains. To handle large-scale graphs,
most of the existing methods partition the input graph into multiple sub-graphs
(e.g., through node clustering) and apply batch training to save memory cost.
However, such batch training will lead to label bias within each batch, and
then result in over-confidence in model predictions. Since the connected nodes
with positively related labels tend to be assigned together, the traditional
cross-entropy minimization process will attend on the predictions of biased
classes in the batch, and may intensify the overfitting issue. To overcome the
label bias problem, we propose the adaptive label smoothing (ALS) method to
replace the one-hot hard labels with smoothed ones, which learns to allocate
label confidences from the biased classes to the others. Specifically, ALS
propagates node labels to aggregate the neighborhood label distribution in a
pre-processing step, and then updates the optimal smoothed labels online to
adapt to specific graph structure. Experiments on the real-world datasets
demonstrate that ALS can be generally applied to the main scalable learning
frameworks to calibrate the biased labels and improve generalization
performances.
