Understanding top-down attention using task-oriented ablation design
  Top-down attention allows neural networks, both artificial and biological, to
focus on the information most relevant for a given task. This is known to
enhance performance in visual perception. But it remains unclear how attention
brings about its perceptual boost, especially when it comes to naturalistic
settings like recognising an object in an everyday scene. What aspects of a
visual task does attention help to deal with? We aim to answer this with a
computational experiment based on a general framework called task-oriented
ablation design. First we define a broad range of visual tasks and identify six
factors that underlie task variability. Then on each task we compare the
performance of two neural networks, one with top-down attention and one
without. These comparisons reveal the task-dependence of attention's perceptual
boost, giving a clearer idea of the role attention plays. Whereas many existing
cognitive accounts link attention to stimulus-level variables, such as visual
clutter and object scale, we find greater explanatory power in system-level
variables that capture the interaction between the model, the distribution of
training data and the task format. This finding suggests a shift in how
attention is studied could be fruitful. We make publicly available our code and
results, along with statistics relevant to ImageNet-based experiments beyond
this one. Our contribution serves to support the development of more human-like
vision models and the design of more informative machine-learning experiments.
