Conditionally Learn to Pay Attention for Sequential Visual Task
  Sequential visual task usually requires to pay attention to its current
interested object conditional on its previous observations. Different from
popular soft attention mechanism, we propose a new attention framework by
introducing a novel conditional global feature which represents the weak
feature descriptor of the current focused object. Specifically, for a standard
CNN (Convolutional Neural Network) pipeline, the convolutional layers with
different receptive fields are used to produce the attention maps by measuring
how the convolutional features align to the conditional global feature. The
conditional global feature can be generated by different recurrent structure
according to different visual tasks, such as a simple recurrent neural network
for multiple objects recognition, or a moderate complex language model for
image caption. Experiments show that our proposed conditional attention model
achieves the best performance on the SVHN (Street View House Numbers) dataset
with / without extra bounding box; and for image caption, our attention model
generates better scores than the popular soft attention model.
