Forecasting Market Changes using Variational Inference
  Though various approaches have been considered, forecasting near-term market
changes of equities and similar market data remains quite difficult. In this
paper we introduce an approach to forecast near-term market changes for equity
indices as well as portfolios using variational inference (VI). VI is a machine
learning approach which uses optimization techniques to estimate complex
probability densities. In the proposed approach, clusters of explanatory
variables are identified and market changes are forecast based on
cluster-specific linear regression. Apart from the expected value of changes,
the proposed approach can also be used to obtain the distribution of possible
outcomes, which can be used to estimate confidence levels of forecasts and risk
measures such as VaR (Value at Risk) for the portfolio. Another advantage of
the proposed approach is the clear model interpretation, as clusters of
explanatory variables (or market regimes) are identified for which the future
changes follow similar relationships. Knowledge about such clusters can provide
useful insights about portfolio performance and identify the relative
importance of variables in different market regimes. Illustrative examples of
equity and bond indices are considered to demonstrate forecasts of the proposed
approach during Covid-related volatility in early 2020 and subsequent benign
market conditions. For the portfolios considered, it is shown that the proposed
approach provides useful forecasts in both normal and volatile markets even
with only a few explanatory variables. Additionally the predicted estimate and
distribution adapt quickly to changing market conditions and thus may also be
useful in obtaining better real-time estimates of risk measures such as VaR
compared to traditional approaches.
