Reinforcement Learning via Gaussian Processes with Neural Network Dual
  Kernels
  While deep neural networks (DNNs) and Gaussian Processes (GPs) are both
popularly utilized to solve problems in reinforcement learning, both approaches
feature undesirable drawbacks for challenging problems. DNNs learn complex
nonlinear embeddings, but do not naturally quantify uncertainty and are often
data-inefficient to train. GPs infer posterior distributions over functions,
but popular kernels exhibit limited expressivity on complex and
high-dimensional data. Fortunately, recently discovered conjugate and neural
tangent kernel functions encode the behavior of overparameterized neural
networks in the kernel domain. We demonstrate that these kernels can be
efficiently applied to regression and reinforcement learning problems by
analyzing a baseline case study. We apply GPs with neural network dual kernels
to solve reinforcement learning tasks for the first time. We demonstrate, using
the well-understood mountain-car problem, that GPs empowered with dual kernels
perform at least as well as those using the conventional radial basis function
kernel. We conjecture that by inheriting the probabilistic rigor of GPs and the
powerful embedding properties of DNNs, GPs using NN dual kernels will empower
future reinforcement learning models on difficult domains.
