Join Query Optimization with Deep Reinforcement Learning Algorithms
  Join query optimization is a complex task and is central to the performance
of query processing. In fact it belongs to the class of NP-hard problems.
Traditional query optimizers use dynamic programming (DP) methods combined with
a set of rules and restrictions to avoid exhaustive enumeration of all possible
join orders. However, DP methods are very resource intensive. Moreover, given
simplifying assumptions of attribute independence, traditional query optimizers
rely on erroneous cost estimations, which can lead to suboptimal query plans.
Recent success of deep reinforcement learning (DRL) creates new opportunities
for the field of query optimization to tackle the above-mentioned problems. In
this paper, we present our DRL-based Fully Observed Optimizer (FOOP) which is a
generic query optimization framework that enables plugging in different machine
learning algorithms. The main idea of FOOP is to use a data-adaptive learning
query optimizer that avoids exhaustive enumerations of join orders and is thus
significantly faster than traditional approaches based on dynamic programming.
In particular, we evaluate various DRL-algorithms and show that Proximal Policy
Optimization significantly outperforms Q-learning based algorithms. Finally we
demonstrate how ensemble learning techniques combined with DRL can further
improve the query optimizer.
