Reinforcement-Learning based Portfolio Management with Augmented Asset
  Movement Prediction States
  Portfolio management (PM) is a fundamental financial planning task that aims
to achieve investment goals such as maximal profits or minimal risks. Its
decision process involves continuous derivation of valuable information from
various data sources and sequential decision optimization, which is a
prospective research direction for reinforcement learning (RL). In this paper,
we propose SARL, a novel State-Augmented RL framework for PM. Our framework
aims to address two unique challenges in financial PM: (1) data heterogeneity
-- the collected information for each asset is usually diverse, noisy and
imbalanced (e.g., news articles); and (2) environment uncertainty -- the
financial market is versatile and non-stationary. To incorporate heterogeneous
data and enhance robustness against environment uncertainty, our SARL augments
the asset information with their price movement prediction as additional
states, where the prediction can be solely based on financial data (e.g., asset
prices) or derived from alternative sources such as news. Experiments on two
real-world datasets, (i) Bitcoin market and (ii) HighTech stock market with
7-year Reuters news articles, validate the effectiveness of SARL over existing
PM approaches, both in terms of accumulated profits and risk-adjusted profits.
Moreover, extensive simulations are conducted to demonstrate the importance of
our proposed state augmentation, providing new insights and boosting
performance significantly over standard RL-based PM method and other baselines.
