An Attentive Survey of Attention Models
  Attention Model has now become an important concept in neural networks that
has been researched within diverse application domains. This survey provides a
structured and comprehensive overview of the developments in modeling
attention. In particular, we propose a taxonomy which groups existing
techniques into coherent categories. We review salient neural architectures in
which attention has been incorporated, and discuss applications in which
modeling attention has shown a significant impact. We also describe how
attention has been used to improve the interpretability of neural networks.
Finally, we discuss some future research directions in attention. We hope this
survey will provide a succinct introduction to attention models and guide
practitioners while developing approaches for their applications.
