Sequential Relevance Maximization with Binary Feedback
  Motivated by online settings where users can provide explicit feedback about
the relevance of products that are sequentially presented to them, we look at
the recommendation process as a problem of dynamically optimizing this
relevance feedback. Such an algorithm optimizes the fine tradeoff between
presenting the products that are most likely to be relevant, and learning the
preferences of the user so that more relevant recommendations can be made in
the future.
  We assume a standard predictive model inspired by collaborative filtering, in
which a user is sampled from a distribution over a set of possible types. For
every product category, each type has an associated relevance feedback that is
assumed to be binary: the category is either relevant or irrelevant. Assuming
that the user stays for each additional recommendation opportunity with
probability $\beta$ independent of the past, the problem is to find a policy
that maximizes the expected number of recommendations that are deemed relevant
in a session.
  We analyze this problem and prove key structural properties of the optimal
policy. Based on these properties, we first present an algorithm that strikes a
balance between recursion and dynamic programming to compute this policy. We
further propose and analyze two heuristic policies: a `farsighted' greedy
policy that attains at least $1-\beta$ factor of the optimal payoff, and a
naive greedy policy that attains at least $\frac{1-\beta}{1+\beta}$ factor of
the optimal payoff in the worst case. Extensive simulations show that these
heuristics are very close to optimal in practice.
