Group-Connected Multilayer Perceptron Networks
  Despite the success of deep learning in domains such as image, voice, and
graphs, there has been little progress in deep representation learning for
domains without a known structure between features. For instance, a tabular
dataset of different demographic and clinical factors where the feature
interactions are not given as a prior. In this paper, we propose
Group-Connected Multilayer Perceptron (GMLP) networks to enable deep
representation learning in these domains. GMLP is based on the idea of learning
expressive feature combinations (groups) and exploiting them to reduce the
network complexity by defining local group-wise operations. During the training
phase, GMLP learns a sparse feature grouping matrix using temperature annealing
softmax with an added entropy loss term to encourage the sparsity. Furthermore,
an architecture is suggested which resembles binary trees, where group-wise
operations are followed by pooling operations to combine information; reducing
the number of groups as the network grows in depth. To evaluate the proposed
method, we conducted experiments on different real-world datasets covering
various application areas. Additionally, we provide visualizations on MNIST and
synthesized data. According to the results, GMLP is able to successfully learn
and exploit expressive feature combinations and achieve state-of-the-art
classification performance on different datasets.
