Provably More Efficient Q-Learning in the
  One-Sided-Feedback/Full-Feedback Settings
  Motivated by the episodic version of the classical inventory control problem,
we propose a new Q-learning-based algorithm, Elimination-Based Half-Q-Learning
(HQL), that enjoys improved efficiency over existing algorithms for a wide
variety of problems in the one-sided-feedback setting. We also provide a
simpler variant of the algorithm, Full-Q-Learning (FQL), for the full-feedback
setting. We establish that HQL incurs $ \tilde{\mathcal{O}}(H^3\sqrt{ T})$
regret and FQL incurs $\tilde{\mathcal{O}}(H^2\sqrt{ T})$ regret, where $H$ is
the length of each episode and $T$ is the total length of the horizon. The
regret bounds are not affected by the possibly huge state and action space. Our
numerical experiments demonstrate the superior efficiency of HQL and FQL, and
the potential to combine reinforcement learning with richer feedback models.
