Waterfall Bandits: Learning to Sell Ads Online
  A popular approach to selling online advertising is by a waterfall, where a
publisher makes sequential price offers to ad networks for an inventory, and
chooses the winner in that order. The publisher picks the order and prices to
maximize her revenue. A traditional solution is to learn the demand model and
then subsequently solve the optimization problem for the given demand model.
This will incur a linear regret. We design an online learning algorithm for
solving this problem, which interleaves learning and optimization, and prove
that this algorithm has sublinear regret. We evaluate the algorithm on both
synthetic and real-world data, and show that it quickly learns high quality
pricing strategies. This is the first principled study of learning a waterfall
design online by sequential experimentation.
