Interpretability in Safety-Critical FinancialTrading Systems
  Sophisticated machine learning (ML) models to inform trading in the financial
sector create problems of interpretability and risk management. Seemingly
robust forecasting models may behave erroneously in out of distribution
settings. In 2020, some of the world's most sophisticated quant hedge funds
suffered losses as their ML models were first underhedged, and then
overcompensated. We implement a gradient-based approach for precisely
stress-testing how a trading model's forecasts can be manipulated, and their
effects on downstream tasks at the trading execution level. We construct inputs
-- whether in changes to sentiment or market variables -- that efficiently
affect changes in the return distribution. In an industry-standard trading
pipeline, we perturb model inputs for eight S&P 500 stocks. We find our
approach discovers seemingly in-sample input settings that result in large
negative shifts in return distributions. We provide the financial community
with mechanisms to interpret ML forecasts in trading systems. For the security
community, we provide a compelling application where studying ML robustness
necessitates that one capture an end-to-end system's performance rather than
study a ML model in isolation. Indeed, we show in our evaluation that errors in
the forecasting model's predictions alone are not sufficient for trading
decisions made based on these forecasts to yield a negative return.
