Improving Relational Regularized Autoencoders with Spherical Sliced
  Fused Gromov Wasserstein
  Relational regularized autoencoder (RAE) is a framework to learn the
distribution of data by minimizing a reconstruction loss together with a
relational regularization on the latent space. A recent attempt to reduce the
inner discrepancy between the prior and aggregated posterior distributions is
to incorporate sliced fused Gromov-Wasserstein (SFG) between these
distributions. That approach has a weakness since it treats every slicing
direction similarly, meanwhile several directions are not useful for the
discriminative task. To improve the discrepancy and consequently the relational
regularization, we propose a new relational discrepancy, named spherical sliced
fused Gromov Wasserstein (SSFG), that can find an important area of projections
characterized by a von Mises-Fisher distribution. Then, we introduce two
variants of SSFG to improve its performance. The first variant, named mixture
spherical sliced fused Gromov Wasserstein (MSSFG), replaces the vMF
distribution by a mixture of von Mises-Fisher distributions to capture multiple
important areas of directions that are far from each other. The second variant,
named power spherical sliced fused Gromov Wasserstein (PSSFG), replaces the vMF
distribution by a power spherical distribution to improve the sampling time in
high dimension settings. We then apply the new discrepancies to the RAE
framework to achieve its new variants. Finally, we conduct extensive
experiments to show that the new proposed autoencoders have favorable
performance in learning latent manifold structure, image generation, and
reconstruction.
