SAITS: Self-Attention-based Imputation for Time Series
  Missing data in time series is a pervasive problem that puts obstacles in the
way of advanced analysis. A popular solution is imputation, where the
fundamental challenge is to determine what values should be filled in. This
paper proposes SAITS, a novel method based on the self-attention mechanism for
missing value imputation in multivariate time series. Trained by a
joint-optimization approach, SAITS learns missing values from a weighted
combination of two diagonally-masked self-attention (DMSA) blocks. DMSA
explicitly captures both the temporal dependencies and feature correlations
between time steps, which improves imputation accuracy and training speed.
Meanwhile, the weighted-combination design enables SAITS to dynamically assign
weights to the learned representations from two DMSA blocks according to the
attention map and the missingness information. Extensive experiments
quantitatively and qualitatively demonstrate that SAITS outperforms the
state-of-the-art methods on the time-series imputation task efficiently and
reveal SAITS' potential to improve the learning performance of pattern
recognition models on incomplete time-series data from the real world.
