Self-Consistency of the Fokker-Planck Equation
The Fokker-Planck equation (FPE) is the partial differential equation that
governs the density evolution of the It\^o process and is of great importance
to the literature of statistical physics and machine learning. The FPE can be
regarded as a continuity equation where the change of the density is completely
determined by a time varying velocity field. Importantly, this velocity field
also depends on the current density function. As a result, the ground-truth
velocity field can be shown to be the solution of a fixed-point equation, a
property that we call self-consistency. In this paper, we exploit this concept
to design a potential function of the hypothesis velocity fields, and prove
that, if such a function diminishes to zero during the training procedure, the
trajectory of the densities generated by the hypothesis velocity fields
converges to the solution of the FPE in the Wasserstein-2 sense. The proposed
potential function is amenable to neural-network based parameterization as the
stochastic gradient with respect to the parameter can be efficiently computed.
Once a parameterized model, such as Neural Ordinary Differential Equation is
trained, we can generate the entire trajectory to the FPE.