Distribution-Based Invariant Deep Networks for Learning Meta-Features
  Recent advances in deep learning from probability distributions successfully
achieve classification or regression from distribution samples, thus invariant
under permutation of the samples. The first contribution of the paper is to
extend these neural architectures to achieve invariance under permutation of
the features, too. The proposed architecture, called Dida, inherits the NN
properties of universal approximation, and its robustness w.r.t.
Lipschitz-bounded transformations of the input distribution is established. The
second contribution is to empirically and comparatively demonstrate the merits
of the approach on two tasks defined at the dataset level. On both tasks, Dida
learns meta-features supporting the characterization of a (labelled) dataset.
The first task consists of predicting whether two dataset patches are extracted
from the same initial dataset. The second task consists of predicting whether
the learning performance achieved by a hyper-parameter configuration under a
fixed algorithm (ranging in k-NN, SVM, logistic regression and linear
classifier with SGD) dominates that of another configuration, for a dataset
extracted from the OpenML benchmarking suite. On both tasks, Dida outperforms
the state of the art: DSS (Maron et al., 2020) and Dataset2Vec (Jomaa et al.,
2019) architectures, as well as the models based on the hand-crafted
meta-features of the literature.
