Tabular Data: Deep Learning is Not All You Need
  A key element in solving real-life data science problems is selecting the
types of models to use. Tree ensemble models (such as XGBoost) are usually
recommended for classification and regression problems with tabular data.
However, several deep learning models for tabular data have recently been
proposed, claiming to outperform XGBoost for some use cases. This paper
explores whether these deep models should be a recommended option for tabular
data by rigorously comparing the new deep models to XGBoost on various
datasets. In addition to systematically comparing their performance, we
consider the tuning and computation they require. Our study shows that XGBoost
outperforms these deep models across the datasets, including the datasets used
in the papers that proposed the deep models. We also demonstrate that XGBoost
requires much less tuning. On the positive side, we show that an ensemble of
deep models and XGBoost performs better on these datasets than XGBoost alone.
