A Lightweight and Accurate Spatial-Temporal Transformer for Traffic
  Forecasting
  We study the forecasting problem for traffic with dynamic, possibly
periodical, and joint spatial-temporal dependency between regions. Given the
aggregated inflow and outflow traffic of regions in a city from time slots 0 to
t-1, we predict the traffic at time t at any region. Prior arts in the area
often consider the spatial and temporal dependencies in a decoupled manner or
are rather computationally intensive in training with a large number of
hyper-parameters to tune. We propose ST-TIS, a novel, lightweight, and accurate
Spatial-Temporal Transformer with information fusion and region sampling for
traffic forecasting. ST-TIS extends the canonical Transformer with information
fusion and region sampling. The information fusion module captures the complex
spatial-temporal dependency between regions. The region sampling module is to
improve the efficiency and prediction accuracy, cutting the computation
complexity for dependency learning from $O(n^2)$ to $O(n\sqrt{n})$, where n is
the number of regions. With far fewer parameters than state-of-the-art models,
the offline training of our model is significantly faster in terms of tuning
and computation (with a reduction of up to $90\%$ on training time and network
parameters). Notwithstanding such training efficiency, extensive experiments
show that ST-TIS is substantially more accurate in online prediction than
state-of-the-art approaches (with an average improvement of up to $9.5\%$ on
RMSE, and $12.4\%$ on MAPE).
