{'fraction_non_alphanumeric': 0.05717207399832667, 'fraction_numerical': 0.011899228409407827, 'mean_word_length': 4.00139470013947, 'pattern_counts': {'":': 0, '<': 1, '<?xml version=': 0, '>': 3, 'https://': 0, 'lorem ipsum': 0, 'www.': 0, 'xml': 0}, 'pii_count': 0, 'substrings_counts': 0, 'word_list_counts': {'cursed_substrings.json': 4, 'profanity_word_list.json': 0, 'sexual_word_list.json': 0, 'zh_pornsignals.json': 0}}
Parametric Fokker-Planck equation

We derive the Fokker-Planck equation on the parametric space. It is the Wasserstein gradient flow of relative entropy on the statistical manifold. We pull back the PDE to a finite dimensional ODE on parameter space. Some analytical example and numerical examples are presented.


Introduction
Fokker-Planck equation, a linear evolution partial differential equation (PDE), plays a crucial role in stochastic calculus, statistical physics and modeling [13,16,18]. Recently, people also discover its importance in statistics and machine learning [11,15,17]. Fokker-Planck equation describes the evolution of density functions of the stochastic process driven by a stochastic differential equation (SDE).
There is another viewpoint of Fokker-Planck equation based on optimal transport theory. It treats the equation as the gradient flow of relative entropy on probability manifold equipped with Wasserstein metric [5,14]. Recently, the studies have been extended to information geometry [1,2,3], creating a new area known as Wasserstein information geometry [7,9,10]. Inspired by those studies, in this paper, we derive the metric tensor on parameter space by pulling back the Wasserstein metric via the parameterized pushforward map. Then we compute the Wasserstein gradient flow (an ODE system) of relative entropy defined on parameter space. This leads to a statistical manifold version of Fokker Planck equation, which can be viewed as an approximation of the original PDE.
Our work is motivated by two purposes, (1) reducing the evolution PDE to a finite dimensional ODE system on parameter space; (2) applying parameterized pushforward map to obtain an efficient sampling method to generate samples from SDE. This is different from Markov Chain Monte Carlo (MCMC) methods [12] or momentum methods [16]. In this brief presentation, we sketch the theoretical framework with illustrations on several examples. The complete results will be reported in an extended version [8].

Parametric Fokker-Planck equation
In this section, we briefly review the fact that Fokker-Planck equation is a Wasserstein gradient flow of relative entropy. We then introduce a Wasserstein arXiv:1903.10076v1 [math.OC] 24 Mar 2019 statistical manifold generated by parameterized mapping function. Based on it, we derive the parametric Fokker-Planck equation as the gradient flow of parameterized relative entropy.

Fokker-Planck equation
Consider the Fokker-Planck equation: Here ∇·, ∇ is the divergence and gradient operator in R d , ∇V is the drift function and β > 0 is a diffusion constant. There are several understandings for the equation (1). On the one hand, consider the stochastic differential equation: Here {B t } t≥0 is the standard Brownian motion. It is well known that the density function ρ(t, x) of stochastic process X t , i.e. X t ∼ ρ(t, x), satisfies the Fokker-Planck equation (1). On the other hand, equation (1) is the Wasserstein gradient flow of relative entropy. Denote the probability space supported on R d : Equipped with the Wasserstein metric [6,14], P is an infinite dimensional Riemmanian manifold. Denote Consider a specific ρ ∈ P andρ i ∈ T ρ P, i = 1, 2. The Wasserstein metric tensor g W is defined as: Here g W is a metric tensor, which is a positive definite bilinear form defined on tangent bundle T P = {(ρ,ρ) : ρ ∈ P,ρ ∈ T ρ P}. The Riemannian gradient in (P, g W ) is given as follows. Consider a smooth functional F : P → R, then where δ δρ(x) is the L 2 first variation at variable x ∈ R d . In particular, consider the relative entropy Then ∇ δF δρ = ∇V + β∇ log ρ, and (3) forms Notice ∇ log ρ = ∇ρ ρ , then ∇ · (ρ∇ log ρ) = ∇ · (∇ρ) = ∆ρ. The above equation is exactly Fokker-Planck equation (1).
From now on, we apply the above geometric gradient flow formulation and derive the Fokker-Planck equation (1) on parameter space.

Parameter space equipped with Wasserstein metric
We consider a parameter space Θ as an open set in R m . Denote the sample space we can also let T θ be a neural network with parameter θ. We further assume that T θ is invertible and smooth with respect to parameter θ and variable x.
Denote p ∈ P as a reference probability measure with positive density defined on M . For example, we can choose p as the standard Gaussian. We denote ρ θ as the density of T θ # p. 3 We further require: |T θ (x)| 2 dp(x) < ∞ holds for all θ ∈ Θ. Then ρ θ ∈ P for each θ ∈ Θ. Denote P Θ = {ρ θ = ρ(θ, x)|θ ∈ Θ}, then P Θ ⊂ P. Now the connection between P and Θ is the pushforward operation T # : Θ → P Θ ⊂ P, θ → ρ θ . In order to introduce the Wasserstein metric to parameter space Θ, we assume that T # is an isometric immersion from Θ to P. Under this assumption, the pullback (T # ) * g W of the Wasserstein metric g W by T # is the metric tensor on Θ. Let us denote G = (T # ) * g W . Then for each θ, G(θ) is a bilinear form on T θ Θ R m , thus G(θ) can be treated as an m × m matrix. Computation of G(θ) is illustrated in the following theorem: Then the metric tensor G(θ) at θ ∈ Θ is m×m non-negative definite symmetric matrix and can be computed as: Or in entry-wised form: Here Ψ = (ψ 1 , ...ψ m ) T and ∇Ψ is m × d Jacobian matrix of Ψ . For each k = 1, 2, ..., m, ψ k solves the following equation: Proof. Suppose ξ ∈ T Θ is a vector field on Θ, for a fixed θ ∈ Θ, we first compute the pushforward ( . To compute , we consider for any φ ∈ C ∞ 0 (M ): This weak formulation reveals that Now let us compute the metric tensor G. Since T # is isometric immersion from Θ to P, the pullback of g W by T # gives G, i.e. (T # ) * g W = G. By definition of pullback map, for any ξ ∈ T Θ and for any θ ∈ Θ, we have: To compute the right hand side of (8), recall (3), we need to solve for ϕ from: By (7), (9) is: We can straightforwardly check that ϕ(x) = Ψ T (x)ξ(θ) is the solution of (10). Then G(θ) is computed as: Thus we can verify that: Generally speaking, the metric tensor G doesn't have an explicit form when d ≥ 2; but for d = 1, G has an explicit form and can be computed directly.

Fokker-Planck equation on statistical manifold
Recall the relative entropy functional F defined in (4), we consider F = F • T # : Θ → R. Then: As in [1], the gradient flow of F on Wasserstein statistical manifold (Θ, G) sat- We call (13) parametric Fokker-Planck equation. The ODE (13) as the Wasserstein gradient flow on parameter space (Θ, G) is closely related to Fokker-Planck equation on probability submanifold P Θ . We have the following theorem, which is a natural result derived from submanifold geometry:

Example on Fokker-Planck equations with quadratic potential
The solution of Fokker-Planck equation on statistical manifold (13) can serve as an approximation to the solution of the original equation (1). However, in some special cases, ρ θt exactly solves (1). In this section, we demonstrate such examples. Let us consider Fokker-Planck equations with quadratic potentials whose initial conditions are Gaussian, i.e. Consider We define the parametric map as T θ (x) = Γ x + b. We choose the reference measure p = N (0, I).
Here is the lemma we have to use: Lemma 1. Let F be the relative entropy defined in (4) and F defined in (12).
For θ ∈ Θ, If the vector function ∇ δF δρ • T θ can be written as the linear , which is the Wasserstein gradient of F at θ.
Proof. The detailed proof is provided in [8]. Here is an intuitive explanation: ∇ δF δρ = ∇V + β∇ log ρ θ is the real vector field that moves the particles in Fokker-Planck equation; and ∂ θ T θ (T −1 θ (·))θ is the approximate vector field induced by the pushforward map T θ . If such approximate is perfect with zero error, i.e. exits ζ such that ∇ δF and the submanifold gradient agrees with entire manifold gradient. Now, let us come back to our example, we can compute Then we have: By 1) of Corollary 1, ζ = G(θ) −1 ∇ θ F (θ). Thus ODE (13) for our example is: By 2) of Corollary 1, we know gradF(ρ θ )| PΘ = gradF(ρ θ ) for all θ ∈ Θ. This indicates that there is no local error for our approximation, one can verify that the solution to the parametric Fokker-Planck equation also solves the original equation.
In addition to previous results, we have the following corollary:  (14) is Gaussian distribution for all t > 0.
Proof. If we denote {Γ t , b t } as the solutions to (15), (16), set θ t = (Γ t , b t ), then ρ t = T θt # p solves the Fokker Planck Equation (1) with conditions (14). Since the pushforward of Gaussian distribution p by an affine transform T θ is still a Gaussian, we conclude that for any t > 0, the solution ρ t = T θt # p is always Gaussian distribution. This is already a well known result about Fokker-Planck equation. We reprove it under our framework.

Numerical examples for 1D Fokker-Planck equation
Since the Wasserstein metric tensor G has an explicit solution when dimension d = 1, it is convenient to numerically compute ODE (13). For example, we can choose a series of basis functions {ϕ k } n k=1 . Each ϕ k can be chosen as a sinusoidal function or a piece-wise linear function defined on a certain interval [−l, l]. It is also beneficial to choose orthogonal or nearorthogonal basis functions because they will keep the metric tensor G far away from ill-posedness. We set T θ (x) = m k=1 θ k ϕ k (x) 4 . Then according to (11), we can compute G as The second part of F is the entropy of ρ θ , which can be computed by solving the following optimization problem [4]: We can solve (17) by parametrizing h. Suppose the optimal solution is h * . Then by envelope theorem, we know ∇ θ F (θ) can be computed as Notice that both the metric tensor G and ∇ θ F (θ) are written in forms of expectations, thus we can compute them by Monte Carlo simulations. And finally, (13) can be computed by forward Euler method.
Our numerical results are always demonstrated by sample points: For each time node t, we sample points {X 1 , ..., X N } from p, then {T θt (X 1 ), ..., T θt (X N )} are our numerical samples from distribution ρ t which solves the Fokker-Planck equation.
Here are several numerical results based on our method. We exhibit them in the form of histograms. Consider the potential V (x) = (x + 1) 2 (x − 1) 2 .

Discussion
We presented a new approach for approximating Fokker-Planck equations by parameterized push-forward mapping functions. Compared to the classical moment method and MCMC method, we propose a systemic way for obtaining a finite dimensional ODE on parameter space. The ODE represents the evolution of statistical information conveyed in the original Fokker-Planck equation. In the future, we will study its geometric and statistical properties, and derive practical numerical methods for applications in scientific computing and machine learning.