Representation Learning via Adversarially-Contrastive Optimal Transport
  In this paper, we study the problem of learning compact (low-dimensional)
representations for sequential data that captures its implicit spatio-temporal
cues. To maximize extraction of such informative cues from the data, we set the
problem within the context of contrastive representation learning and to that
end propose a novel objective via optimal transport. Specifically, our
formulation seeks a low-dimensional subspace representation of the data that
jointly (i) maximizes the distance of the data (embedded in this subspace) from
an adversarial data distribution under the optimal transport, a.k.a. the
Wasserstein distance, (ii) captures the temporal order, and (iii) minimizes the
data distortion. To generate the adversarial distribution, we propose a novel
framework connecting Wasserstein GANs with a classifier, allowing a principled
mechanism for producing good negative distributions for contrastive learning,
which is currently a challenging problem. Our full objective is cast as a
subspace learning problem on the Grassmann manifold and solved via Riemannian
optimization. To empirically study our formulation, we provide experiments on
the task of human action recognition in video sequences. Our results
demonstrate competitive performance against challenging baselines.
