Online Interactive Collaborative Filtering Using Multi-Armed Bandit with
  Dependent Arms
  Online interactive recommender systems strive to promptly suggest to
consumers appropriate items (e.g., movies, news articles) according to the
current context including both the consumer and item content information.
However, such context information is often unavailable in practice for the
recommendation, where only the users' interaction data on items can be
utilized. Moreover, the lack of interaction records, especially for new users
and items, worsens the performance of recommendation further. To address these
issues, collaborative filtering (CF), one of the recommendation techniques
relying on the interaction data only, as well as the online multi-armed bandit
mechanisms, capable of achieving the balance between exploitation and
exploration, are adopted in the online interactive recommendation settings, by
assuming independent items (i.e., arms). Nonetheless, the assumption rarely
holds in reality, since the real-world items tend to be correlated with each
other (e.g., two articles with similar topics). In this paper, we study online
interactive collaborative filtering problems by considering the dependencies
among items. We explicitly formulate the item dependencies as the clusters on
arms, where the arms within a single cluster share the similar latent topics.
In light of the topic modeling techniques, we come up with a generative model
to generate the items from their underlying topics. Furthermore, an efficient
online algorithm based on particle learning is developed for inferring both
latent parameters and states of our model. Additionally, our inferred model can
be naturally integrated with existing multi-armed selection strategies in the
online interactive collaborating setting. Empirical studies on two real-world
applications, online recommendations of movies and news, demonstrate both the
effectiveness and efficiency of the proposed approach.
