Stochastic Low-rank Tensor Bandits for Multi-dimensional Online Decision
  Making
  Multi-dimensional online decision making plays a crucial role in many real
applications such as online recommendation and digital marketing. In these
problems, a decision at each time is a combination of choices from different
types of entities. To solve it, we introduce stochastic low-rank tensor
bandits, a class of bandits whose mean rewards can be represented as a low-rank
tensor. We consider two settings, tensor bandits without context and tensor
bandits with context. In the first setting, the platform aims to find the
optimal decision with the highest expected reward, a.k.a, the largest entry of
true reward tensor. In the second setting, some modes of the tensor are
contexts and the rest modes are decisions, and the goal is to find the optimal
decision given the contextual information. We propose two learning algorithms
tensor elimination and tensor epoch-greedy for tensor bandits without context,
and derive finite-time regret bounds for them. Comparing with existing
competitive methods, tensor elimination has the best overall regret bound and
tensor epoch-greedy has a sharper dependency on dimensions of the reward
tensor. Furthermore, we develop a practically effective Bayesian algorithm
called tensor ensemble sampling for tensor bandits with context. Numerical
experiments back up our theoretical findings and show that our algorithms
outperform various state-of-the-art approaches that ignore the tensor low-rank
structure. In an online advertising application with contextual information,
our tensor ensemble sampling reduces the cumulative regret by 75% compared to
the benchmark method.
