Deep Equilibrium Optical Flow Estimation
  Many recent state-of-the-art (SOTA) optical flow models use finite-step
recurrent update operations to emulate traditional algorithms by encouraging
iterative refinements toward a stable flow estimation. However, these RNNs
impose large computation and memory overheads, and are not directly trained to
model such stable estimation. They can converge poorly and thereby suffer from
performance degradation. To combat these drawbacks, we propose deep equilibrium
(DEQ) flow estimators, an approach that directly solves for the flow as the
infinite-level fixed point of an implicit layer (using any black-box solver),
and differentiates through this fixed point analytically (thus requiring $O(1)$
training memory). This implicit-depth approach is not predicated on any
specific model, and thus can be applied to a wide range of SOTA flow estimation
model designs. The use of these DEQ flow estimators allows us to compute the
flow faster using, e.g., fixed-point reuse and inexact gradients, consumes
$4\sim6\times$ times less training memory than the recurrent counterpart, and
achieves better results with the same computation budget. In addition, we
propose a novel, sparse fixed-point correction scheme to stabilize our DEQ flow
estimators, which addresses a longstanding challenge for DEQ models in general.
We test our approach in various realistic settings and show that it improves
SOTA methods on Sintel and KITTI datasets with substantially better
computational and memory efficiency.
