DS-FACTO: Doubly Separable Factorization Machines
  Factorization Machines (FM) are powerful class of models that incorporate
higher-order interaction among features to add more expressive power to linear
models. They have been used successfully in several real-world tasks such as
click-prediction, ranking and recommender systems. Despite using a low-rank
representation for the pairwise features, the memory overheads of using
factorization machines on large-scale real-world datasets can be prohibitively
high. For instance on the criteo tera dataset, assuming a modest $128$
dimensional latent representation and $10^{9}$ features, the memory requirement
for the model is in the order of $1$ TB. In addition, the data itself occupies
$2.1$ TB. Traditional algorithms for FM which work on a single-machine are not
equipped to handle this scale and therefore, using a distributed algorithm to
parallelize the computation across a cluster is inevitable. In this work, we
propose a hybrid-parallel stochastic optimization algorithm DS-FACTO, which
partitions both the data as well as parameters of the factorization machine
simultaneously. Our solution is fully de-centralized and does not require the
use of any parameter servers. We present empirical results to analyze the
convergence behavior, predictive power and scalability of DS-FACTO.
