QLBS: Q-Learner in the Black-Scholes(-Merton) Worlds
  This paper presents a discrete-time option pricing model that is rooted in
Reinforcement Learning (RL), and more specifically in the famous Q-Learning
method of RL. We construct a risk-adjusted Markov Decision Process for a
discrete-time version of the classical Black-Scholes-Merton (BSM) model, where
the option price is an optimal Q-function, while the optimal hedge is a second
argument of this optimal Q-function, so that both the price and hedge are parts
of the same formula. Pricing is done by learning to dynamically optimize
risk-adjusted returns for an option replicating portfolio, as in the Markowitz
portfolio theory. Using Q-Learning and related methods, once created in a
parametric setting, the model is able to go model-free and learn to price and
hedge an option directly from data, and without an explicit model of the world.
This suggests that RL may provide efficient data-driven and model-free methods
for optimal pricing and hedging of options, once we depart from the academic
continuous-time limit, and vice versa, option pricing methods developed in
Mathematical Finance may be viewed as special cases of model-based
Reinforcement Learning. Further, due to simplicity and tractability of our
model which only needs basic linear algebra (plus Monte Carlo simulation, if we
work with synthetic data), and its close relation to the original BSM model, we
suggest that our model could be used for benchmarking of different RL
algorithms for financial trading applications
