SubTab: Subsetting Features of Tabular Data for Self-Supervised
  Representation Learning
  Self-supervised learning has been shown to be very effective in learning
useful representations, and yet much of the success is achieved in data types
such as images, audio, and text. The success is mainly enabled by taking
advantage of spatial, temporal, or semantic structure in the data through
augmentation. However, such structure may not exist in tabular datasets
commonly used in fields such as healthcare, making it difficult to design an
effective augmentation method, and hindering a similar progress in tabular data
setting. In this paper, we introduce a new framework, Subsetting features of
Tabular data (SubTab), that turns the task of learning from tabular data into a
multi-view representation learning problem by dividing the input features to
multiple subsets. We argue that reconstructing the data from the subset of its
features rather than its corrupted version in an autoencoder setting can better
capture its underlying latent representation. In this framework, the joint
representation can be expressed as the aggregate of latent variables of the
subsets at test time, which we refer to as collaborative inference. Our
experiments show that the SubTab achieves the state of the art (SOTA)
performance of 98.31% on MNIST in tabular setting, on par with CNN-based SOTA
models, and surpasses existing baselines on three other real-world datasets by
a significant margin.
