A deep Q-learning method for optimizing visual search strategies in
  backgrounds of dynamic noise
  Humans process visual information with varying resolution (foveated visual
system) and explore images by orienting through eye movements the
high-resolution fovea to points of interest. The Bayesian ideal searcher (IS)
that employs complete knowledge of task-relevant information optimizes eye
movement strategy and achieves the optimal search performance. The IS can be
employed as an important tool to evaluate the optimality of human eye
movements, and potentially provide guidance to improve human observer visual
search strategies. Najemnik and Geisler (2005) derived an IS for backgrounds of
spatial 1/f noise. The corresponding template responses follow Gaussian
distributions and the optimal search strategy can be analytically determined.
However, the computation of the IS can be intractable when considering more
realistic and complex backgrounds such as medical images. Modern reinforcement
learning methods, successfully applied to obtain optimal policy for a variety
of tasks, do not require complete knowledge of the background generating
functions and can be potentially applied to anatomical backgrounds. An
important first step is to validate the optimality of the reinforcement
learning method. In this study, we investigate the ability of a reinforcement
learning method that employs Q-network to approximate the IS. We demonstrate
that the search strategy corresponding to the Q-network is consistent with the
IS search strategy. The findings show the potential of the reinforcement
learning with Q-network approach to estimate optimal eye movement planning with
real anatomical backgrounds.
