Recurrent Neural Networks for Stochastic Control in Real-Time Bidding
  Bidding in real-time auctions can be a difficult stochastic control task;
especially if underdelivery incurs strong penalties and the market is very
uncertain. Most current works and implementations focus on optimally delivering
a campaign given a reasonable forecast of the market. Practical implementations
have a feedback loop to adjust and be robust to forecasting errors, but no
implementation, to the best of our knowledge, uses a model of market risk and
actively anticipates market shifts. Solving such stochastic control problems in
practice is actually very challenging. This paper proposes an approximate
solution based on a Recurrent Neural Network (RNN) architecture that is both
effective and practical for implementation in a production environment. The RNN
bidder provisions everything it needs to avoid missing its goal. It also
deliberately falls short of its goal when buying the missing impressions would
cost more than the penalty for not reaching it.
