ARM-Net: Adaptive Relation Modeling Network for Structured Data
  Relational databases are the de facto standard for storing and querying
structured data, and extracting insights from structured data requires advanced
analytics. Deep neural networks (DNNs) have achieved super-human prediction
performance in particular data types, e.g., images. However, existing DNNs may
not produce meaningful results when applied to structured data. The reason is
that there are correlations and dependencies across combinations of attribute
values in a table, and these do not follow simple additive patterns that can be
easily mimicked by a DNN. The number of possible such cross features is
combinatorial, making them computationally prohibitive to model. Furthermore,
the deployment of learning models in real-world applications has also
highlighted the need for interpretability, especially for high-stakes
applications, which remains another issue of concern to DNNs.
  In this paper, we present ARM-Net, an adaptive relation modeling network
tailored for structured data, and a lightweight framework ARMOR based on
ARM-Net for relational data analytics. The key idea is to model feature
interactions with cross features selectively and dynamically, by first
transforming the input features into exponential space, and then determining
the interaction order and interaction weights adaptively for each cross
feature. We propose a novel sparse attention mechanism to dynamically generate
the interaction weights given the input tuple, so that we can explicitly model
cross features of arbitrary orders with noisy features filtered selectively.
Then during model inference, ARM-Net can specify the cross features being used
for each prediction for higher accuracy and better interpretability. Our
extensive experiments on real-world datasets demonstrate that ARM-Net
consistently outperforms existing models and provides more interpretable
predictions for data-driven decision making.
