Exploration in Online Advertising Systems with Deep Uncertainty-Aware
  Learning
  Modern online advertising systems inevitably rely on personalization methods,
such as click-through rate (CTR) prediction. Recent progress in CTR prediction
enjoys the rich representation capabilities of deep learning and achieves great
success in large-scale industrial applications. However, these methods can
suffer from lack of exploration. Another line of prior work addresses the
exploration-exploitation trade-off problem with contextual bandit methods,
which are recently less studied in the industry due to the difficulty in
extending their flexibility with deep models. In this paper, we propose a novel
Deep Uncertainty-Aware Learning (DUAL) method to learn CTR models based on
Gaussian processes, which can provide predictive uncertainty estimations while
maintaining the flexibility of deep neural networks. DUAL can be easily
implemented on existing models and deployed in real-time systems with minimal
extra computational overhead. By linking the predictive uncertainty estimation
ability of DUAL to well-known bandit algorithms, we further present DUAL-based
Ad-ranking strategies to boost up long-term utilities such as the social
welfare in advertising systems. Experimental results on several public datasets
demonstrate the effectiveness of our methods. Remarkably, an online A/B test
deployed in the Alibaba display advertising platform shows an 8.2% social
welfare improvement and an 8.0% revenue lift.
