Multi-output Polynomial Networks and Factorization Machines
  Factorization machines and polynomial networks are supervised polynomial
models based on an efficient low-rank decomposition. We extend these models to
the multi-output setting, i.e., for learning vector-valued functions, with
application to multi-class or multi-task problems. We cast this as the problem
of learning a 3-way tensor whose slices share a common basis and propose a
convex formulation of that problem. We then develop an efficient conditional
gradient algorithm and prove its global convergence, despite the fact that it
involves a non-convex basis selection step. On classification tasks, we show
that our algorithm achieves excellent accuracy with much sparser models than
existing methods. On recommendation system tasks, we show how to combine our
algorithm with a reduction from ordinal regression to multi-output
classification and show that the resulting algorithm outperforms simple
baselines in terms of ranking accuracy.
