Contextual Bandits with Sparse Data in Web setting
  This paper is a scoping study to identify current methods used in handling
sparse data with contextual bandits in web settings. The area is highly current
and state of the art methods are identified. The years 2017-2020 are
investigated, and 19 method articles are identified, and two review articles.
Five categories of methods are described, making it easy to choose how to
address sparse data using contextual bandits with a method available for
modification in the specific setting of concern. In addition, each method has
multiple techniques to choose from for future evaluation. The problem areas are
also mentioned that each article covers. An overall updated understanding of
sparse data problems using contextual bandits in web settings is given. The
identified methods are policy evaluation (off-line and on-line) ,
hybrid-method, model representation (clusters and deep neural networks),
dimensionality reduction, and simulation.
