Measurement-based Online Available Bandwidth Estimation employing
  Reinforcement Learning
  An accurate and fast estimation of the available bandwidth in a network with
varying cross-traffic is a challenging task. The accepted probing tools, based
on the fluid-flow model of a bottleneck link with first-in, first-out
multiplexing, estimate the available bandwidth by measuring packet dispersions.
The estimation becomes more difficult if packet dispersions deviate from the
assumptions of the fluid-flow model in the presence of non-fluid bursty
cross-traffic, multiple bottleneck links, and inaccurate time-stamping. This
motivates us to explore the use of machine learning tools for available
bandwidth estimation. Hence, we consider reinforcement learning and implement
the single-state multi-armed bandit technique, which follows the
$\epsilon$-greedy algorithm to find the available bandwidth. Our measurements
and tests reveal that our proposed method identifies the available bandwidth
with high precision. Furthermore, our method converges to the available
bandwidth under a variety of notoriously difficult conditions, such as heavy
traffic burstiness, different cross-traffic intensities, multiple bottleneck
links, and in networks where the tight link and the bottleneck link are not
same. Compared to the piece-wise linear network a model-based direct probing
technique that employs a Kalman filter, our method shows more accurate
estimates and faster convergence in certain network scenarios and does not
require measurement noise statistics.
