Deep Stock Trading: A Hierarchical Reinforcement Learning Framework for
  Portfolio Optimization and Order Execution
  Portfolio management via reinforcement learning is at the forefront of
fintech research, which explores how to optimally reallocate a fund into
different financial assets over the long term by trial-and-error. Existing
methods are impractical since they usually assume each reallocation can be
finished immediately and thus ignoring the price slippage as part of the
trading cost. To address these issues, we propose a hierarchical reinforced
stock trading system for portfolio management (HRPM). Concretely, we decompose
the trading process into a hierarchy of portfolio management over trade
execution and train the corresponding policies. The high-level policy gives
portfolio weights at a lower frequency to maximize the long term profit and
invokes the low-level policy to sell or buy the corresponding shares within a
short time window at a higher frequency to minimize the trading cost. We train
two levels of policies via pre-training scheme and iterative training scheme
for data efficiency. Extensive experimental results in the U.S. market and the
China market demonstrate that HRPM achieves significant improvement against
many state-of-the-art approaches.
