TradeR: Practical Deep Hierarchical Reinforcement Learning for Trade
  Execution
  Advances in Reinforcement Learning (RL) span a wide variety of applications
which motivate development in this area. While application tasks serve as
suitable benchmarks for real world problems, RL is seldomly used in practical
scenarios consisting of abrupt dynamics. This allows one to rethink the problem
setup in light of practical challenges. We present Trade Execution using
Reinforcement Learning (TradeR) which aims to address two such practical
challenges of catastrophy and surprise minimization by formulating trading as a
real-world hierarchical RL problem. Through this lens, TradeR makes use of
hierarchical RL to execute trade bids on high frequency real market experiences
comprising of abrupt price variations during the 2019 fiscal year COVID19 stock
market crash. The framework utilizes an energy-based scheme in conjunction with
surprise value function for estimating and minimizing surprise. In a
large-scale study of 35 stock symbols from the S&P500 index, TradeR
demonstrates robustness to abrupt price changes and catastrophic losses while
maintaining profitable outcomes. We hope that our work serves as a motivating
example for application of RL to practical problems.
