Efficient Multivariate Bandit Algorithm with Path Planning
  In this paper, we solve the arms exponential exploding issue in multivariate
Multi-Armed Bandit (Multivariate-MAB) problem when the arm dimension hierarchy
is considered. We propose a framework called path planning (TS-PP) which
utilizes decision graph/trees to model arm reward success rate with m-way
dimension interaction, and adopts Thompson sampling (TS) for heuristic search
of arm selection. Naturally, it is quite straightforward to combat the curse of
dimensionality using a serial processes that operates sequentially by focusing
on one dimension per each process. For our best acknowledge, we are the first
to solve Multivariate-MAB problem using graph path planning strategy and
deploying alike Monte-Carlo tree search ideas. Our proposed method utilizing
tree models has advantages comparing with traditional models such as general
linear regression. Simulation studies validate our claim by achieving faster
convergence speed, better efficient optimal arm allocation and lower cumulative
regret.
