Self-Supervised Contextual Bandits in Computer Vision
  Contextual bandits are a common problem faced by machine learning
practitioners in domains as diverse as hypothesis testing to product
recommendations. There have been a lot of approaches in exploiting rich data
representations for contextual bandit problems with varying degree of success.
Self-supervised learning is a promising approach to find rich data
representations without explicit labels. In a typical self-supervised learning
scheme, the primary task is defined by the problem objective (e.g. clustering,
classification, embedding generation etc.) and the secondary task is defined by
the self-supervision objective (e.g. rotation prediction, words in
neighborhood, colorization, etc.). In the usual self-supervision, we learn
implicit labels from the training data for a secondary task. However, in the
contextual bandit setting, we don't have the advantage of getting implicit
labels due to lack of data in the initial phase of learning. We provide a novel
approach to tackle this issue by combining a contextual bandit objective with a
self supervision objective. By augmenting contextual bandit learning with
self-supervision we get a better cumulative reward. Our results on eight
popular computer vision datasets show substantial gains in cumulative reward.
We provide cases where the proposed scheme doesn't perform optimally and give
alternative methods for better learning in these cases.
