On Transportation of Mini-batches: A Hierarchical Approach
  Mini-batch optimal transport (m-OT) has been successfully used in practical
applications that involve probability measures with a very high number of
supports. The m-OT solves several smaller optimal transport problems and then
returns the average of their costs and transportation plans. Despite its
scalability advantage, the m-OT does not consider the relationship between
mini-batches which leads to undesirable estimation. Moreover, the m-OT does not
approximate a proper metric between probability measures since the identity
property is not satisfied. To address these problems, we propose a novel
mini-batch scheme for optimal transport, named Batch of Mini-batches Optimal
Transport (BoMb-OT), that finds the optimal coupling between mini-batches and
it can be seen as an approximation to a well-defined distance on the space of
probability measures. Furthermore, we show that the m-OT is a limit of the
entropic regularized version of the BoMb-OT when the regularized parameter goes
to infinity. Finally, we carry out experiments on various applications
including deep generative models, deep domain adaptation, approximate Bayesian
computation, color transfer, and gradient flow to show that the BoMb-OT can be
widely applied and performs well in various applications.
