Abelian Neural Networks
  We study the problem of modeling a binary operation that satisfies some
algebraic requirements. We first construct a neural network architecture for
Abelian group operations and derive a universal approximation property. Then,
we extend it to Abelian semigroup operations using the characterization of
associative symmetric polynomials. Both models take advantage of the analytic
invertibility of invertible neural networks. For each case, by repeating the
binary operations, we can represent a function for multiset input thanks to the
algebraic structure. Naturally, our multiset architecture has
size-generalization ability, which has not been obtained in existing methods.
Further, we present modeling the Abelian group operation itself is useful in a
word analogy task. We train our models over fixed word embeddings and
demonstrate improved performance over the original word2vec and another naive
learning method.
