Reinforcement Learning for Portfolio Management
  In this thesis, we develop a comprehensive account of the expressive power,
modelling efficiency, and performance advantages of so-called trading agents
(i.e., Deep Soft Recurrent Q-Network (DSRQN) and Mixture of Score Machines
(MSM)), based on both traditional system identification (model-based approach)
as well as on context-independent agents (model-free approach). The analysis
provides conclusive support for the ability of model-free reinforcement
learning methods to act as universal trading agents, which are not only capable
of reducing the computational and memory complexity (owing to their linear
scaling with the size of the universe), but also serve as generalizing
strategies across assets and markets, regardless of the trading universe on
which they have been trained. The relatively low volume of daily returns in
financial market data is addressed via data augmentation (a generative
approach) and a choice of pre-training strategies, both of which are validated
against current state-of-the-art models. For rigour, a risk-sensitive framework
which includes transaction costs is considered, and its performance advantages
are demonstrated in a variety of scenarios, from synthetic time-series
(sinusoidal, sawtooth and chirp waves), simulated market series (surrogate data
based), through to real market data (S\&P 500 and EURO STOXX 50). The analysis
and simulations confirm the superiority of universal model-free reinforcement
learning agents over current portfolio management model in asset allocation
strategies, with the achieved performance advantage of as much as 9.2\% in
annualized cumulative returns and 13.4\% in annualized Sharpe Ratio.
