Content-Level Selective Offloading in Heterogeneous Networks:
  Multi-armed Bandit Optimization and Regret Bounds
  We consider content-level selective offloading of cellular downlink traffic
to a wireless infostation terminal which stores high data-rate content in its
cache memory. Cellular users in the vicinity of the infostation can directly
download the stored content from the infostation through a broadband connection
(e.g., WiFi), reducing the latency and load on the cellular network. The goal
of the infostation cache controller (CC) is to store the most popular content
in the cache memory such that the maximum amount of traffic is offloaded to the
infostation. In practice, the popularity profile of the files is not known by
the CC, which observes only the instantaneous demands for those contents stored
in the cache. Hence, the cache content placement is optimised based on the
demand history and on the cost associated to placing each content in the cache.
By refreshing the cache content at regular time intervals, the CC gradually
learns the popularity profile, while at the same time exploiting the limited
cache capacity in the best way possible. This is formulated as a multi-armed
bandit (MAB) problem with switching cost. Several algorithms are presented to
decide on the cache content over time. The performance is measured in terms of
cache efficiency, defined as the amount of net traffic that is offloaded to the
infostation. In addition to theoretical regret bounds, the proposed algorithms
are analysed through numerical simulations. In particular, the impact of system
parameters, such as the number of files, number of users, cache size, and
skewness of the popularity profile, on the performance is studied numerically.
It is shown that the proposed algorithms learn the popularity profile quickly
for a wide range of system parameters.
