Learning Functions over Sets via Permutation Adversarial Networks
  In this paper, we consider the problem of learning functions over sets, i.e.,
functions that are invariant to permutations of input set items. Recent
approaches of pooling individual element embeddings can necessitate extremely
large embedding sizes for challenging functions. We address this challenge by
allowing standard neural networks like LSTMs to succinctly capture the function
over the set. However, to ensure invariance with respect to permutations of set
elements, we propose a novel architecture called SPAN that simultaneously
learns the function as well as adversarial or worst-case permutations for each
input set. The learning problem reduces to a min-max optimization problem that
is solved via a simple alternating block coordinate descent technique. We
conduct extensive experiments on a variety of set-learning tasks and
demonstrate that SPAN learns nearly permutation-invariant functions while still
ensuring accuracy on test data. On a variety of tasks sampled from the domains
of statistics, graph functions and linear algebra, we show that our method can
significantly outperform state-of-the-art methods such as DeepSets and Janossy
Pooling. Finally, we present a case study of how learning set-functions can
help extract powerful features for recommendation systems, and show that such a
method can be as much as 2% more accurate than carefully hand-tuned features on
a real-world recommendation system.
