DANets: Deep Abstract Networks for Tabular Data Classification and
  Regression
  Tabular data are ubiquitous in real world applications. Although many
commonly-used neural components (e.g., convolution) and extensible neural
networks (e.g., ResNet) have been developed by the machine learning community,
few of them were effective for tabular data and few designs were adequately
tailored for tabular data structures. In this paper, we propose a novel and
flexible neural component for tabular data, called Abstract Layer (AbstLay),
which learns to explicitly group correlative input features and generate
higher-level features for semantics abstraction. Also, we design a structure
re-parameterization method to compress the learned AbstLay, thus reducing the
computational complexity by a clear margin in the reference phase. A special
basic block is built using AbstLays, and we construct a family of Deep Abstract
Networks (DANets) for tabular data classification and regression by stacking
such blocks. In DANets, a special shortcut path is introduced to fetch
information from raw tabular features, assisting feature interactions across
different levels. Comprehensive experiments on seven real-world tabular
datasets show that our AbstLay and DANets are effective for tabular data
classification and regression, and the computational complexity is superior to
competitive methods. Besides, we evaluate the performance gains of DANet as it
goes deep, verifying the extendibility of our method. Our code is available at
https://github.com/WhatAShot/DANet.
