More layers! End-to-end regression and uncertainty on tabular data with
  deep learning
  This paper attempts to analyze the effectiveness of deep learning for tabular
data processing. It is believed that decision trees and their ensembles is the
leading method in this domain, and deep neural networks must be content with
computer vision and so on. But the deep neural network is a framework for
building gradient-based hierarchical representations, and this key feature
should be able to provide the best processing of generic structured (tabular)
data, not just image matrices and audio spectrograms. This problem is
considered through the prism of the Weather Prediction track in the Yandex
Shifts challenge (in other words, the Yandex Shifts Weather task). This task is
a variant of the classical tabular data regression problem. It is also
connected with another important problem: generalization and uncertainty in
machine learning. This paper proposes an end-to-end algorithm for solving the
problem of regression with uncertainty on tabular data, which is based on the
combination of four ideas: 1) deep ensemble of self-normalizing neural
networks, 2) regression as parameter estimation of the Gaussian target error
distribution, 3) hierarchical multitask learning, and 4) simple data
preprocessing. Three modifications of the proposed algorithm form the top-3
leaderboard of the Yandex Shifts Weather challenge respectively. This paper
considers that this success has occurred due to the fundamental properties of
the deep learning algorithm, and tries to prove this.
