Context Uncertainty in Contextual Bandits with Applications to
  Recommender Systems
  Recurrent neural networks have proven effective in modeling sequential user
feedbacks for recommender systems. However, they usually focus solely on item
relevance and fail to effectively explore diverse items for users, therefore
harming the system performance in the long run. To address this problem, we
propose a new type of recurrent neural networks, dubbed recurrent exploration
networks (REN), to jointly perform representation learning and effective
exploration in the latent space. REN tries to balance relevance and exploration
while taking into account the uncertainty in the representations. Our
theoretical analysis shows that REN can preserve the rate-optimal sublinear
regret even when there exists uncertainty in the learned representations. Our
empirical study demonstrates that REN can achieve satisfactory long-term
rewards on both synthetic and real-world recommendation datasets, outperforming
state-of-the-art models.
