RAIN: Reinforced Hybrid Attention Inference Network for Motion
  Forecasting
  Motion forecasting plays a significant role in various domains (e.g.,
autonomous driving, human-robot interaction), which aims to predict future
motion sequences given a set of historical observations. However, the observed
elements may be of different levels of importance. Some information may be
irrelevant or even distracting to the forecasting in certain situations. To
address this issue, we propose a generic motion forecasting framework (named
RAIN) with dynamic key information selection and ranking based on a hybrid
attention mechanism. The general framework is instantiated to handle
multi-agent trajectory prediction and human motion forecasting tasks,
respectively. In the former task, the model learns to recognize the relations
between agents with a graph representation and to determine their relative
significance. In the latter task, the model learns to capture the temporal
proximity and dependency in long-term human motions. We also propose an
effective double-stage training pipeline with an alternating training strategy
to optimize the parameters in different modules of the framework. We validate
the framework on both synthetic simulations and motion forecasting benchmarks
in different domains, demonstrating that our method not only achieves
state-of-the-art forecasting performance, but also provides interpretable and
reasonable hybrid attention weights.
