DoubleEnsemble: A New Ensemble Method Based on Sample Reweighting and
  Feature Selection for Financial Data Analysis
  Modern machine learning models (such as deep neural networks and boosting
decision tree models) have become increasingly popular in financial market
prediction, due to their superior capacity to extract complex non-linear
patterns. However, since financial datasets have very low signal-to-noise ratio
and are non-stationary, complex models are often very prone to overfitting and
suffer from instability issues. Moreover, as various machine learning and data
mining tools become more widely used in quantitative trading, many trading
firms have been producing an increasing number of features (aka factors).
Therefore, how to automatically select effective features becomes an imminent
problem. To address these issues, we propose DoubleEnsemble, an ensemble
framework leveraging learning trajectory based sample reweighting and shuffling
based feature selection. Specifically, we identify the key samples based on the
training dynamics on each sample and elicit key features based on the ablation
impact of each feature via shuffling. Our model is applicable to a wide range
of base models, capable of extracting complex patterns, while mitigating the
overfitting and instability issues for financial market prediction. We conduct
extensive experiments, including price prediction for cryptocurrencies and
stock trading, using both DNN and gradient boosting decision tree as base
models. Our experiment results demonstrate that DoubleEnsemble achieves a
superior performance compared with several baseline methods.
