Cooperative Deep $Q$-learning Framework for Environments Providing Image
  Feedback
  In this paper, we address two key challenges in deep reinforcement learning
setting, sample inefficiency and slow learning, with a dual NN-driven learning
approach. In the proposed approach, we use two deep NNs with independent
initialization to robustly approximate the action-value function in the
presence of image inputs. In particular, we develop a temporal difference (TD)
error-driven learning approach, where we introduce a set of linear
transformations of the TD error to directly update the parameters of each layer
in the deep NN. We demonstrate theoretically that the cost minimized by the
error-driven learning (EDL) regime is an approximation of the empirical cost
and the approximation error reduces as learning progresses, irrespective of the
size of the network. Using simulation analysis, we show that the proposed
methods enables faster learning and convergence and requires reduced buffer
size (thereby increasing the sample efficiency).
