Attention mechanisms and deep learning for machine vision: A survey of
  the state of the art
  With the advent of state of the art nature-inspired pure attention based
models i.e. transformers, and their success in natural language processing
(NLP), their extension to machine vision (MV) tasks was inevitable and much
felt. Subsequently, vision transformers (ViTs) were introduced which are giving
quite a challenge to the established deep learning based machine vision
techniques. However, pure attention based models/architectures like
transformers require huge data, large training times and large computational
resources. Some recent works suggest that combinations of these two varied
fields can prove to build systems which have the advantages of both these
fields. Accordingly, this state of the art survey paper is introduced which
hopefully will help readers get useful information about this interesting and
potential research area. A gentle introduction to attention mechanisms is
given, followed by a discussion of the popular attention based deep
architectures. Subsequently, the major categories of the intersection of
attention mechanisms and deep learning for machine vision (MV) based are
discussed. Afterwards, the major algorithms, issues and trends within the scope
of the paper are discussed.
