On the Convergence of Extended Variational Inference for Non-Gaussian
  Statistical Models
  Variational inference (VI) is a widely used framework in Bayesian estimation.
For most of the non-Gaussian statistical models, it is infeasible to find an
analytically tractable solution to estimate the posterior distributions of the
parameters. Recently, an improved framework, namely the extended variational
inference (EVI), has been introduced and applied to derive analytically
tractable solution by employing lower-bound approximation to the variational
objective function. Two conditions required for EVI implementation, namely the
weak condition and the strong condition, are discussed and compared in this
paper. In practical implementation, the convergence of the EVI depends on the
selection of the lower-bound approximation, no matter with the weak condition
or the strong condition. In general, two approximation strategies, the single
lower-bound (SLB) approximation and the multiple lower-bounds (MLB)
approximation, can be applied to carry out the lower-bound approximation. To
clarify the differences between the SLB and the MLB, we will also discuss the
convergence properties of the aforementioned two approximations. Extensive
comparisons are made based on some existing EVI-based non-Gaussian statistical
models. Theoretical analysis are conducted to demonstrate the differences
between the weak and the strong conditions. Qualitative and quantitative
experimental results are presented to show the advantages of the SLB
approximation.
