The Recurrent Reinforcement Learning Crypto Agent
  We demonstrate a novel application of online transfer learning for a digital
assets trading agent. This agent uses a powerful feature space representation
in the form of an echo state network, the output of which is made available to
a direct, recurrent reinforcement learning agent. The agent learns to trade the
XBTUSD (Bitcoin versus US Dollars) perpetual swap derivatives contract on
BitMEX on an intraday basis. By learning from the multiple sources of impact on
the quadratic risk-adjusted utility that it seeks to maximise, the agent avoids
excessive over-trading, captures a funding profit, and can predict the market's
direction. Overall, our crypto agent realises a total return of 350\%, net of
transaction costs, over roughly five years, 71\% of which is down to funding
profit. The annualised information ratio that it achieves is 1.46.
