Woodbury Transformations for Deep Generative Flows
  Normalizing flows are deep generative models that allow efficient likelihood
calculation and sampling. The core requirement for this advantage is that they
are constructed using functions that can be efficiently inverted and for which
the determinant of the function's Jacobian can be efficiently computed.
Researchers have introduced various such flow operations, but few of these
allow rich interactions among variables without incurring significant
computational costs. In this paper, we introduce Woodbury transformations,
which achieve efficient invertibility via the Woodbury matrix identity and
efficient determinant calculation via Sylvester's determinant identity. In
contrast with other operations used in state-of-the-art normalizing flows,
Woodbury transformations enable (1) high-dimensional interactions, (2)
efficient sampling, and (3) efficient likelihood evaluation. Other similar
operations, such as 1x1 convolutions, emerging convolutions, or periodic
convolutions allow at most two of these three advantages. In our experiments on
multiple image datasets, we find that Woodbury transformations allow learning
of higher-likelihood models than other flow architectures while still enjoying
their efficiency advantages.
