Leveraging Cross Feedback of User and Item Embeddings with Attention for
  Variational Autoencoder based Collaborative Filtering
  Matrix factorization (MF) has been widely applied to collaborative filtering
in recommendation systems. Its Bayesian variants can derive posterior
distributions of user and item embeddings, and are more robust to sparse
ratings. However, the Bayesian methods are restricted by their update rules for
the posterior parameters due to the conjugacy of the priors and the likelihood.
Variational autoencoders (VAE) can address this issue by capturing complex
mappings between the posterior parameters and the data. However, current
research on VAEs for collaborative filtering only considers the mappings based
on the explicit data information while the implicit embedding information is
overlooked. In this paper, we first derive evidence lower bounds (ELBO) for
Bayesian MF models from two viewpoints: user-oriented and item-oriented. Based
on the ELBOs, we propose a VAE-based Bayesian MF framework. It leverages not
only the data but also the embedding information to approximate the user-item
joint distribution. As suggested by the ELBOs, the approximation is iterative
with cross feedback of user and item embeddings into each other's encoders.
More specifically, user embeddings sampled at the previous iteration are fed to
the item-side encoders to estimate the posterior parameters for the item
embeddings at the current iteration, and vice versa. The estimation also
attends to the cross-fed embeddings to further exploit useful information. The
decoder then reconstructs the data via the matrix factorization over the
currently re-sampled user and item embeddings.
