An Efficient Algorithm for Deep Stochastic Contextual Bandits
  In stochastic contextual bandit (SCB) problems, an agent selects an action
based on certain observed context to maximize the cumulative reward over
iterations. Recently there have been a few studies using a deep neural network
(DNN) to predict the expected reward for an action, and the DNN is trained by a
stochastic gradient based method. However, convergence analysis has been
greatly ignored to examine whether and where these methods converge. In this
work, we formulate the SCB that uses a DNN reward function as a non-convex
stochastic optimization problem, and design a stage-wise stochastic gradient
descent algorithm to optimize the problem and determine the action policy. We
prove that with high probability, the action sequence chosen by this algorithm
converges to a greedy action policy respecting a local optimal reward function.
Extensive experiments have been performed to demonstrate the effectiveness and
efficiency of the proposed algorithm on multiple real-world datasets.
