Q-learning with online random forests
  $Q$-learning is the most fundamental model-free reinforcement learning
algorithm. Deployment of $Q$-learning requires approximation of the
state-action value function (also known as the $Q$-function). In this work, we
provide online random forests as $Q$-function approximators and propose a novel
method wherein the random forest is grown as learning proceeds (through
expanding forests). We demonstrate improved performance of our methods over
state-of-the-art Deep $Q$-Networks in two OpenAI gyms (`blackjack' and
`inverted pendulum') but not in the `lunar lander' gym. We suspect that the
resilience to overfitting enjoyed by random forests recommends our method for
common tasks that do not require a strong representation of the problem domain.
We show that expanding forests (in which the number of trees increases as data
comes in) improve performance, suggesting that expanding forests are viable for
other applications of online random forests beyond the reinforcement learning
setting.
