An MDP-based Recommender System
  Typical Recommender systems adopt a static view of the recommendation process
and treat it as a prediction problem. We argue that it is more appropriate to
view the problem of generating recommendations as a sequential decision problem
and, consequently, that Markov decision processes (MDP) provide a more
appropriate model for Recommender systems. MDPs introduce two benefits: they
take into account the long-term effects of each recommendation, and they take
into account the expected value of each recommendation. To succeed in practice,
an MDP-based Recommender system must employ a strong initial model; and the
bulk of this paper is concerned with the generation of such a model. In
particular, we suggest the use of an n-gram predictive model for generating the
initial MDP. Our n-gram model induces a Markov-chain model of user behavior
whose predictive accuracy is greater than that of existing predictive models.
We describe our predictive model in detail and evaluate its performance on real
data. In addition, we show how the model can be used in an MDP-based
Recommender system.
